Appearance
patroni
The patroni plugin runs on each node of a Patroni Postgres cluster and reads the node's local REST API. It reports this node's role and state, every member's role, state and replication lag, and which member holds the leader. Turn it on for every Patroni node. Cluster placement leader depends on it, because it is how daemonitor learns which host holds the primary.
Enable it
Add patroni to the plugins array on each node.
json
{
"plugins": ["patroni"],
"patroni": {
"refreshInterval": 30000,
"url": "http://localhost:8008",
"lagWarnBytes": 33554432,
"lagCritBytes": 268435456
}
}| Option | Default | What it does |
|---|---|---|
refreshInterval | 30000 | Milliseconds between reports. Shorter than most plugins because it bounds how long daemonitor trusts an old leader after a failover. |
url | "http://localhost:8008" | Base URL of this node's Patroni REST API. |
username, password | none | Basic auth, for nodes whose restapi.authentication covers GET requests. |
lagWarnBytes | 33554432 (32 MB) | Replication lag that counts as a warning. Sent with each report and judged by the server. |
lagCritBytes | 268435456 (256 MB) | Replication lag that counts as an error. |
requestTimeout | 5000 | Milliseconds before each REST call gives up. |
uniqueId | "patroni" | Unique id for this node's service. |
What it reports
The plugin calls GET /patroni and GET /cluster in parallel. If one fails, it still reports the other and sets partial: true.
scope,member(Patroni's name for this node),role,isLeader,state,timeline.leader(the leader's member name),hasLeader,clusterUnlocked.paused(maintenance mode, so failover is off) andpendingRestart.lagBytes, this node's lag behind the leader, pluslagWarnBytesandlagCritBytes.members, one entry per member withname,role,state,host,port,timelineandlagBytes. A lag ofunknownfrom Patroni staysnull, not 0.memberCount,runningCount,serverVersion,patroniVersion.
If both calls fail, the plugin sends reachable: false and an error, so a dead API looks different from a dead agent.
Roles leader, master, primary and standby_leader all count as leader. Patroni 4 says primary, older versions say master or leader.
When it alerts
The server checks these in order and stops at the first match.
| Condition | Severity |
|---|---|
reachable is false | error |
This node's state is not running or streaming | error |
| A complete report shows no leader among one or more members | error |
paused is true | warning |
Lag at or above lagCritBytes, sustained | error |
Lag at or above lagWarnBytes, sustained | warning |
Another member's state is not running or streaming | warning |
pendingRestart is true | warning |
Lag counts as sustained only when this node's reports from the last 5 minutes show all of the following:
- at least 3 samples,
- every sample at or above
lagWarnBytes, - the newest sample no smaller than the oldest.
A replica that is behind but catching up does not alert.
A new problem does not notify on the first bad report. The server holds it until a second consecutive report from the node is still bad. When the node reports healthy again, the server resolves the alert.
Leader identity
Each report also updates the leader of every patroni-kind cluster this host belongs to. This happens separately from the node's own status.
- A node reporting
isLeader: truesets itself as leader. No name matching is involved. - A follower's
leadername is matched against thememberother nodes reported about themselves, then against system names. - A complete report that sees members but no leader clears the leader.
- If no node reconfirms the leader within 180 seconds,
leaderplacement falls back to "exactly one member".
On the dashboard
One card per node, titled with the cluster scope and labelled Patroni · <member>. The badge is leader on the leader, otherwise the node's role.
| Reading | Source | Warn | Crit |
|---|---|---|---|
| ROLE | role | none | none |
| LEADER | leader | none | NONE when hasLeader is false |
| LAG | lagBytes | at or above lagWarnBytes | at or above lagCritBytes |
| MEMBERS | runningCount/memberCount | fewer running than total | none |
| FAILOVER | shown only when paused | PAUSED | none |
| RESTART | shown only when pendingRestart | PENDING | none |
Each member is a unit, red when its state is not running or streaming. An unreachable API shows a single red API UNREACHABLE metric.
The LAG metric colors on the current value. The alert waits for sustained lag, so a card can show amber lag that never notifies.
Requirements
- Patroni's REST API reachable from the agent at
url. Patroni only protects write requests by default, so the GETs need no credentials. - To use the leader identity, the hosts must be members of a cluster of kind
patroniin daemonitor.
Gotchas
- Every member reports a leaderless cluster at once, so each node raises its own error. The one-report debounce and the cluster's grace window keep a normal election from paging.
- Patroni answers
/patroniwith HTTP 503 on a node that is up but unhealthy. The plugin reads the body anyway, because the body is the node's state. - If
/patronifails but/clusteranswers, the report has no state for this node. The server reads that as stateunknownand raises an error.