# Website checks

The `web` plugin fetches a URL on a schedule and checks the status code and the page body. Turn it on to catch a site that is down, a page that carries injected code, or a page whose expected content has vanished. It can also check the origin server directly, behind a CDN, so a CDN serving cached pages can't hide a dead backend.

Each endpoint becomes its own service on the dashboard, with the unique id `web-<name>` (or `web-<url>` when the endpoint has no name).

## Enable it

Add `web` to the `plugins` array. Endpoints can come from two places, and the plugin runs both at once.

### Endpoints in the config file

```json
{
  "plugins": ["web"],
  "web": {
    "refreshInterval": 60000,
    "endpoints": [
      {
        "name": "example.com",
        "url": "https://www.example.com/",
        "expectedStrings": ["Example Domain"],
        "unexpectedStrings": ["id=\"injected-script\"", "malware.example.net"],
        "timeout": 15000
      },
      {
        "name": "example.com xmlrpc (should be blocked)",
        "url": "https://www.example.com/xmlrpc.php",
        "expectedStatusCode": 403
      },
      {
        "name": "shop.example.com",
        "url": "https://shop.example.com/",
        "origin": {
          "url": "https://origin.example.com/",
          "host": "shop.example.com",
          "query": { "cf": "1" },
          "insecure": true
        }
      }
    ]
  }
}
```

Instead of `endpoints`, a single endpoint can sit at the top level of the `web` section (`name`, `url`, `expectedStrings`, `unexpectedStrings`, `expectedStatusCode`, `timeout`, `headers`, `origin`). The shorthand does not carry `originOnly`.

### Endpoints managed in the dashboard

Monitors can also live in the `web_monitors` table and be edited under Settings, Web Monitors. Each monitor belongs to one system. The client running that system pulls its enabled monitors from `GET /api/monitors/for-client`, authenticated by the `x-system-key` header.

The plugin only does this when the `web` section names both the endpoint and the key. The client does not copy the connector's key in for you, so repeat it here.

```json
{
  "plugins": ["web"],
  "web": {
    "refreshInterval": 60000,
    "configUrl": "https://daemonitor.example.com/api/monitors/for-client",
    "systemKey": "YOUR_SYSTEM_KEY"
  }
}
```

The plugin fetches the list at the start of every pass. When both sources define an endpoint with the same `name` and `url`, the dashboard copy wins. Endpoints that only exist in the config file keep running alongside. If a fetch fails, the plugin keeps running the last list it fetched. If the very first fetch fails, only the local endpoints run.

The settings form can set the name, URL, system, expected status code and the origin check (URL, Host override, `?cf=1`, ignore TLS errors). It has no fields for `expectedStrings`, `unexpectedStrings`, `timeout` or `originOnly`. The table's `config` column holds them and `for-client` passes them through, but saving the form rebuilds `config` from the form fields alone, so editing a monitor in the UI drops them.

### Options

Section options:

| Option | Default | What it does |
| --- | --- | --- |
| `refreshInterval` | `60000` | Milliseconds between passes. |
| `endpoints` | none | Array of endpoints, below. |
| `configUrl` | none | URL of `/api/monitors/for-client`. Needs `systemKey`. |
| `systemKey` | none | The system key sent as `x-system-key` to `configUrl`. |

Endpoint options:

| Option | Default | What it does |
| --- | --- | --- |
| `name` | the URL | Display name. Also used in the unique id and in alert titles. |
| `url` | required | The public URL. Redirects are followed. |
| `expectedStatusCode` | any 200 to 399 | Exact status the response must have. |
| `expectedStrings` | `[]` | Substrings that must appear in the body. A missing one is treated as possible defacement. |
| `unexpectedStrings` | `[]` | Substrings that must not appear. A match is treated as possible compromise. |
| `timeout` | `30000` | Milliseconds before the public request is aborted. Covers the body as well as the headers. |
| `headers` | none | Headers sent on the public request, for example a key a WAF skip rule matches. Config file only, `for-client` does not pass it. |
| `origin` | none | A second, direct request to the origin server, below. |
| `originOnly` | `false` | Skip the public request and report the origin check as the endpoint's health. Use it when a bot challenge blocks the checker on the public URL. Needs `origin.url`. |

Origin options:

| Option | Default | What it does |
| --- | --- | --- |
| `url` | required | An address that reaches the origin directly, by hostname or IP. |
| `host` | none | Host header to send, so the origin's vhost routes the request like public traffic. |
| `headers` | none | Extra headers for the origin request. |
| `query` | none | Query parameters added to the origin URL. |
| `insecure` | `false` | Skip TLS certificate checks on the origin request. |
| `timeout` | endpoint `timeout`, else `10000` | Milliseconds before the origin request is abandoned. |
| `expectedStrings`, `unexpectedStrings`, `expectedStatusCode` | the endpoint's values | Override how the origin response is judged. |

The `web_monitors` table also has an `interval_ms` column, 120000 by default. The plugin does not read it. `refreshInterval` sets the pace for every endpoint.

## What it reports

One payload per endpoint:

- `name`, `url`. After redirects, `url` is the final URL.
- `status`, the HTTP status, or `0` when the request failed.
- `duration` in milliseconds.
- `ok`, true when the status passed and the content checks passed. This is the public check only.
- `redirected`.
- `hasAllExpectedStrings`, `hasAnyUnexpectedStrings`, `missingExpected`, `matchedUnexpected`. The lists name the exact strings, so an alert can say which IOC matched.
- `error`, on a network failure or timeout.
- `origin`, when configured, with its own `url`, `host`, `status`, `duration`, `ok`, `error` and the same content fields.
- `originOnly: true` on origin-only endpoints. Their top-level fields are the origin result.

## When it alerts

Web checks use the findings model. Each problem is its own alert, keyed by a stable fingerprint, and it alerts on the first report that shows it. When a later report no longer contains that finding, the server resolves it. The same finding comes back as a new alert only after the earlier one has resolved, and a repeat within 30 minutes of the resolve is recorded without paging.

The server checks these rules in order and stops at the first match, so one report raises at most one finding:

| Finding | Severity | Trigger |
| --- | --- | --- |
| Unreachable | error | The payload has `error`, a connection failure or timeout. |
| Possible compromise | error | `hasAnyUnexpectedStrings` is true on the public check. The key includes the matched strings, so a new IOC alerts again even while an older one is open. |
| Possible defacement | error | `hasAllExpectedStrings` is false. The key includes the missing strings. |
| Check failed | error | `ok` is false for another reason, usually the wrong status code. |
| Possible compromise at origin | error | `origin.hasAnyUnexpectedStrings` is true while the public check passed. |
| Origin down | warning | `origin.ok` is false while the public check passed. Visitors are still served, probably from the CDN cache. |

One more rule sits outside the findings model. When nothing above matched and the response took more than 5000 ms, the service goes to warning with "slow". That alert goes through the normal path. It waits for a second bad report before notifying, and it resolves when the service reports online again.

## On the dashboard

The card is labelled "Web check", or "Web check · origin" when an origin check is configured. Its ping slot shows `duration`.

- STATUS shows the HTTP status. It turns red only when `ok` is false and the status is outside 200 to 399. A 200 page that fails a content check keeps a green STATUS, and the CHECKS list carries the failure.
- LATENCY shows `duration` and is never colored.
- ORIGIN and ORIGIN MS appear with an origin check. ORIGIN shows DOWN on an error and turns red whenever `origin.ok` is false.
- The CHECKS list has an HTTP row and an ORIGIN row. A CONTENT row appears when expected strings are missing, and an UNEXPECTED row when forbidden strings matched. Both are red. No content row appears on a passing check, because the payload can't tell "all strings found" from "no strings configured".

The card's overall color comes from the status the server stored. A web check with no report for 300 seconds shows OFFLINE.

## Requirements

- Network access from the client to each URL, and to each origin URL.
- For dashboard-managed monitors, `configUrl` and `systemKey` in the `web` section, and the monitor assigned to this client's system.

## Gotchas

- Matching is plain substring matching on the raw body. No regex, no case folding.
- A pass runs its endpoints one after another, and only one pass runs at a time. With many endpoints near their 30 second timeout, a pass can outlast `refreshInterval`, and the overlapping tick is skipped.
- When an origin is dialed by hostname, TLS validates against that hostname, not the `host` override. When dialed by IP, the certificate needs an IP SAN, or set `insecure`.
- An endpoint without a `name` uses its URL as the alert label. After a redirect the reported `url` changes, and so does the label.
- `client/README.md` lists endpoint options such as `method`, `body`, `interval` and `expectedResponseTime`. The plugin reads none of them.