# patroni

The patroni plugin runs on each node of a Patroni Postgres cluster and reads the node's local REST API. It reports this node's role and state, every member's role, state and replication lag, and which member holds the leader. Turn it on for every Patroni node. Cluster placement `leader` depends on it, because it is how daemonitor learns which host holds the primary.

## Enable it

Add `patroni` to the `plugins` array on each node.

```json
{
  "plugins": ["patroni"],
  "patroni": {
    "refreshInterval": 30000,
    "url": "http://localhost:8008",
    "lagWarnBytes": 33554432,
    "lagCritBytes": 268435456
  }
}
```

| Option | Default | What it does |
|---|---|---|
| `refreshInterval` | `30000` | Milliseconds between reports. Shorter than most plugins because it bounds how long daemonitor trusts an old leader after a failover. |
| `url` | `"http://localhost:8008"` | Base URL of this node's Patroni REST API. |
| `username`, `password` | none | Basic auth, for nodes whose `restapi.authentication` covers GET requests. |
| `lagWarnBytes` | `33554432` (32 MB) | Replication lag that counts as a warning. Sent with each report and judged by the server. |
| `lagCritBytes` | `268435456` (256 MB) | Replication lag that counts as an error. |
| `requestTimeout` | `5000` | Milliseconds before each REST call gives up. |
| `uniqueId` | `"patroni"` | Unique id for this node's service. |

## What it reports

The plugin calls `GET /patroni` and `GET /cluster` in parallel. If one fails, it still reports the other and sets `partial: true`.

- `scope`, `member` (Patroni's name for this node), `role`, `isLeader`, `state`, `timeline`.
- `leader` (the leader's member name), `hasLeader`, `clusterUnlocked`.
- `paused` (maintenance mode, so failover is off) and `pendingRestart`.
- `lagBytes`, this node's lag behind the leader, plus `lagWarnBytes` and `lagCritBytes`.
- `members`, one entry per member with `name`, `role`, `state`, `host`, `port`, `timeline` and `lagBytes`. A lag of `unknown` from Patroni stays `null`, not 0.
- `memberCount`, `runningCount`, `serverVersion`, `patroniVersion`.

If both calls fail, the plugin sends `reachable: false` and an `error`, so a dead API looks different from a dead agent.

Roles `leader`, `master`, `primary` and `standby_leader` all count as leader. Patroni 4 says `primary`, older versions say `master` or `leader`.

## When it alerts

The server checks these in order and stops at the first match.

| Condition | Severity |
|---|---|
| `reachable` is false | error |
| This node's `state` is not `running` or `streaming` | error |
| A complete report shows no leader among one or more members | error |
| `paused` is true | warning |
| Lag at or above `lagCritBytes`, sustained | error |
| Lag at or above `lagWarnBytes`, sustained | warning |
| Another member's state is not `running` or `streaming` | warning |
| `pendingRestart` is true | warning |

Lag counts as sustained only when this node's reports from the last 5 minutes show all of the following:

- at least 3 samples,
- every sample at or above `lagWarnBytes`,
- the newest sample no smaller than the oldest.

A replica that is behind but catching up does not alert.

A new problem does not notify on the first bad report. The server holds it until a second consecutive report from the node is still bad. When the node reports healthy again, the server resolves the alert.

### Leader identity

Each report also updates the leader of every `patroni`-kind cluster this host belongs to. This happens separately from the node's own status.

- A node reporting `isLeader: true` sets itself as leader. No name matching is involved.
- A follower's `leader` name is matched against the `member` other nodes reported about themselves, then against system names.
- A complete report that sees members but no leader clears the leader.
- If no node reconfirms the leader within 180 seconds, `leader` placement falls back to "exactly one member".

## On the dashboard

One card per node, titled with the cluster `scope` and labelled `Patroni · <member>`. The badge is `leader` on the leader, otherwise the node's role.

| Reading | Source | Warn | Crit |
|---|---|---|---|
| ROLE | `role` | none | none |
| LEADER | `leader` | none | `NONE` when `hasLeader` is false |
| LAG | `lagBytes` | at or above `lagWarnBytes` | at or above `lagCritBytes` |
| MEMBERS | `runningCount/memberCount` | fewer running than total | none |
| FAILOVER | shown only when `paused` | `PAUSED` | none |
| RESTART | shown only when `pendingRestart` | `PENDING` | none |

Each member is a unit, red when its state is not `running` or `streaming`. An unreachable API shows a single red `API UNREACHABLE` metric.

The LAG metric colors on the current value. The alert waits for sustained lag, so a card can show amber lag that never notifies.

## Requirements

- Patroni's REST API reachable from the agent at `url`. Patroni only protects write requests by default, so the GETs need no credentials.
- To use the leader identity, the hosts must be members of a cluster of kind `patroni` in daemonitor.

## Gotchas

- Every member reports a leaderless cluster at once, so each node raises its own error. The one-report debounce and the cluster's grace window keep a normal election from paging.
- Patroni answers `/patroni` with HTTP 503 on a node that is up but unhealthy. The plugin reads the body anyway, because the body is the node's state.
- If `/patroni` fails but `/cluster` answers, the report has no state for this node. The server reads that as state `unknown` and raises an error.