# Incidents

An incident is your own record of something that happened. It has a title, a span of time, the things it affected, and a timeline you add to as you work. Alerts are raised by the machine, one per finding. An incident is what a person writes about a stretch of time, such as "what was going on between 02:10 and 03:40, and what did we do about it".

Incidents and alerts are kept apart. INBOX lists alerts. INCIDENTS lists incidents. Filing an incident doesn't touch any alert, and resolving one doesn't resolve the other.

## File an incident

There are three ways to open the incident dialog.

- **File incident** in a system's manage menu. The dialog opens with that system or service as the target.
- **New incident** on the INCIDENTS page.
- Drag across a chart on a system page. See [Drag to select](#drag-to-select).

The dialog has these fields.

| Field | Notes |
|---|---|
| Title | Required. |
| Severity | `info`, `warn` or `crit`. Default `warn`. |
| Status | `investigating`, `identified`, `monitoring` or `resolved`. Default `investigating`. |
| Start | Required. You pick it, because most incidents are filed after the fact and the point is to mark when it really began. |
| End | Leave **still happening** ticked for an incident that is ongoing. The dialog shows the running duration. |
| Targets | The systems, services and groups it affected. |
| Summary | One or two lines. The timeline carries the detail. |
| Client-visible | Off by default. See [Client visibility](#client-visibility). |

An incident with no end is still happening. There's no separate "open" flag. Resolving sets the end, and re-opening clears it.

## Targets

One incident can name several things, because one event often hits several at once. A database failover can affect three hosts and a website in the same minute. Search for systems, services or groups in the dialog and add each one.

Each target stores its name as it was when you filed the incident. If a container is later removed, the incident still reads correctly months on.

## Timeline

The incident page has a timeline you append to. Each entry has a kind.

| Kind | Use it for |
|---|---|
| `note` | What happened, what you tried. |
| `status` | Moving the incident to a new status. Adding one also updates the incident's status. |
| `link` | A URL, with an optional description. Dashboards, tickets, logs. |

Each entry records when it happened, not just when you typed it, so you can log a 02:10 restart at 03:00.

## Resolve and re-open

**Resolve** sets the end to now, sets the status to `resolved`, saves the solution text if you wrote one, and adds a `status` entry to the timeline. **Re-open** clears the end, sets the status back to `investigating`, and adds a "Re-opened." entry.

Deleting an incident deletes its timeline with it. Only the person who created it can delete it.

## Charts

### Incident spans on charts

Incidents show on the metric charts of the systems and services they name, as shaded spans behind the lines. The shading sits under the traces so it doesn't hide the spike it marks.

The incident page draws the charts of every target, with a margin before and after the span. The margin is a quarter of the span on each side, at least 15 minutes, so a three-minute blip still shows what the line was doing going in.

History is kept for 30 days. An older incident still reads in full, but its charts have no samples left, and the page says so instead of drawing an empty box.

### Drag to select

On a system page, drag across any metric row to select a span of time. The incident dialog opens with that start and end filled in and the service you are viewing as the target. One drag covers every metric row, so you can see the span across CPU, memory and the rest before you file.

## Who can see and edit

Incidents follow the same access as the fleet. You can read an incident if:

- you created it,
- its creator gave you a [whole-account grant](./sharing.md), or
- it names a system you can see.

The third rule means an incident on a shared system shows up for the person you shared it with.

Writing is narrower. Only the creator and anyone holding a manager grant on the creator's whole account can edit an incident or add to its timeline. A viewer on a shared system can read the record but can't change it.

## Client visibility

An incident can be published to customers through the [external API](./api.md#get-api-external-incidents). Nothing is published unless you opt in, at two levels.

1. The incident. Tick client-visible in the dialog. Off by default.
2. Each timeline entry. `status` entries are always client-visible, because they are the story a customer follows. Notes and links are internal unless you tick the box when adding them.

The export also leaves out who wrote each entry. It includes an incident only when the incident names at least one system. An incident with only group targets is never exported, because nothing ties it to a particular customer.