Skip to content

patroni ​

The patroni plugin runs on each node of a Patroni Postgres cluster and reads the node's local REST API. It reports this node's role and state, every member's role, state and replication lag, and which member holds the leader. Turn it on for every Patroni node. Cluster placement leader depends on it, because it is how daemonitor learns which host holds the primary.

Enable it ​

Add patroni to the plugins array on each node.

json
{
  "plugins": ["patroni"],
  "patroni": {
    "refreshInterval": 30000,
    "url": "http://localhost:8008",
    "lagWarnBytes": 33554432,
    "lagCritBytes": 268435456
  }
}
OptionDefaultWhat it does
refreshInterval30000Milliseconds between reports. Shorter than most plugins because it bounds how long daemonitor trusts an old leader after a failover.
url"http://localhost:8008"Base URL of this node's Patroni REST API.
username, passwordnoneBasic auth, for nodes whose restapi.authentication covers GET requests.
lagWarnBytes33554432 (32 MB)Replication lag that counts as a warning. Sent with each report and judged by the server.
lagCritBytes268435456 (256 MB)Replication lag that counts as an error.
requestTimeout5000Milliseconds before each REST call gives up.
uniqueId"patroni"Unique id for this node's service.

What it reports ​

The plugin calls GET /patroni and GET /cluster in parallel. If one fails, it still reports the other and sets partial: true.

  • scope, member (Patroni's name for this node), role, isLeader, state, timeline.
  • leader (the leader's member name), hasLeader, clusterUnlocked.
  • paused (maintenance mode, so failover is off) and pendingRestart.
  • lagBytes, this node's lag behind the leader, plus lagWarnBytes and lagCritBytes.
  • members, one entry per member with name, role, state, host, port, timeline and lagBytes. A lag of unknown from Patroni stays null, not 0.
  • memberCount, runningCount, serverVersion, patroniVersion.

If both calls fail, the plugin sends reachable: false and an error, so a dead API looks different from a dead agent.

Roles leader, master, primary and standby_leader all count as leader. Patroni 4 says primary, older versions say master or leader.

When it alerts ​

The server checks these in order and stops at the first match.

ConditionSeverity
reachable is falseerror
This node's state is not running or streamingerror
A complete report shows no leader among one or more memberserror
paused is truewarning
Lag at or above lagCritBytes, sustainederror
Lag at or above lagWarnBytes, sustainedwarning
Another member's state is not running or streamingwarning
pendingRestart is truewarning

Lag counts as sustained only when this node's reports from the last 5 minutes show all of the following:

  • at least 3 samples,
  • every sample at or above lagWarnBytes,
  • the newest sample no smaller than the oldest.

A replica that is behind but catching up does not alert.

A new problem does not notify on the first bad report. The server holds it until a second consecutive report from the node is still bad. When the node reports healthy again, the server resolves the alert.

Leader identity ​

Each report also updates the leader of every patroni-kind cluster this host belongs to. This happens separately from the node's own status.

  • A node reporting isLeader: true sets itself as leader. No name matching is involved.
  • A follower's leader name is matched against the member other nodes reported about themselves, then against system names.
  • A complete report that sees members but no leader clears the leader.
  • If no node reconfirms the leader within 180 seconds, leader placement falls back to "exactly one member".

On the dashboard ​

One card per node, titled with the cluster scope and labelled Patroni · <member>. The badge is leader on the leader, otherwise the node's role.

ReadingSourceWarnCrit
ROLErolenonenone
LEADERleadernoneNONE when hasLeader is false
LAGlagBytesat or above lagWarnBytesat or above lagCritBytes
MEMBERSrunningCount/memberCountfewer running than totalnone
FAILOVERshown only when pausedPAUSEDnone
RESTARTshown only when pendingRestartPENDINGnone

Each member is a unit, red when its state is not running or streaming. An unreachable API shows a single red API UNREACHABLE metric.

The LAG metric colors on the current value. The alert waits for sustained lag, so a card can show amber lag that never notifies.

Requirements ​

  • Patroni's REST API reachable from the agent at url. Patroni only protects write requests by default, so the GETs need no credentials.
  • To use the leader identity, the hosts must be members of a cluster of kind patroni in daemonitor.

Gotchas ​

  • Every member reports a leaderless cluster at once, so each node raises its own error. The one-report debounce and the cluster's grace window keep a normal election from paging.
  • Patroni answers /patroni with HTTP 503 on a node that is up but unhealthy. The plugin reads the body anyway, because the body is the node's state.
  • If /patroni fails but /cluster answers, the report has no state for this node. The server reads that as state unknown and raises an error.