Timings
This page is the reference behind States: every state word, every freshness word, and every timing, retention period and certificate lifetime, with where each is set.
Dashboard states
A terminal has one of 5 states on the dashboard.
| State | Meaning | Colour |
|---|---|---|
| Healthy | The collector is connected, the terminal is on the Starlink connection, and latency and drop rate are at the baseline | green |
| Planned outage | The terminal is down inside a maintenance window. A user scheduled the window, or the fleet server opened one for a collector’s update restart | grey, hatched |
| Warning | Latency or drop rate is above the terminal’s baseline | blue |
| Degraded | Up, but not on the Starlink connection. The cause is on backup carrier or restricted by policy | amber |
| Outage | The connection has been down for the confirmation time, 3 minutes | red |
A terminal that is still reporting inside a maintenance window keeps its own state rather than Planned outage. A terminal that is still down when its window ends becomes an outage immediately, with the cause maintenance window overrun. Its state is Outage. Its card is badged Warning, so that a terminal expected back stands apart from the unexpected outages.
Stabilising is a badge shown while a recovery is being confirmed, not a sixth state. The state does not change. The terminal keeps its state word and colour, with stabilising appended.
Causes
Each cause has an internal name, which the webhook payload uses, and the words shown on the dashboard.
| Internal name | Shown as |
|---|---|
unreachable |
unreachable |
backup-carrying |
on backup carrier |
restricted-by-policy |
restricted by policy |
latency-drift |
latency above baseline |
drops-drift |
drop rate above baseline |
expected-window |
maintenance window opened |
expected-window-ended |
maintenance window ended |
missed-expected-return |
maintenance window overrun |
stream-clean-confirming |
stream stable · confirming recovery |
recovery-interrupted |
recovery interrupted · state held |
stopped-at-device |
stopped at the terminal |
returned-to-service, back-inside-own-normal |
returned to service |
Terminal states
The state word on a terminal’s own page comes from 2 things: the collector’s lifecycle status, and what the terminal’s readings say. The word shown is the first that applies, in the order given after the second table.
The lifecycle status is what the fleet server records about the collector. Each row gives when the collector enters that status and when it leaves it, and for what.
| Status | Shown as | Entered when | Left when |
|---|---|---|---|
enrolled |
Enrolled | The collector enrols | The collector first connects, to online. Or silence past the Stale threshold, to stale |
online |
Online | The collector connects | The connection drops, to offline |
offline |
Offline | The connection closes | The collector reconnects, to online. Or silence past the Stale threshold, to stale |
stale |
Stale | Nothing has been received for longer than the Stale threshold and nothing is connected | The collector reconnects, to online |
decommissioning |
Decommissioning | A user decommissions the collector | The collector confirms, or disconnects, or the Decommission timeout passes, to decommissioned |
decommissioned |
Decommissioned | The decommission completes and the certificate is added to the deny list | Never. Only a new enrolment creates a new record |
revoked |
Cert revoked | A user revokes the device certificate | Never. A late decommission confirmation cannot change it |
The second is what the terminal’s readings say. The word shown is the first of these that applies: Decommissioning, Offline, On backup, Restricted, Healthy.
| Word | Meaning |
|---|---|
| Healthy | Reporting, and at its baseline |
| Restricted | Starlink is limiting service |
| On backup | The Starlink connection dropped. Traffic is on the backup connection |
| Decommissioning | The collector is being decommissioned |
| Offline | The collector is not connected, or is stale |
| Provisioning is the word for a terminal whose collector has enrolled and | |
| never connected. The List, the Map and the search palette use it. |
Recovery adds 3 words: Stabilising while a recovery is being confirmed, Recovered once it is, and Unstable for a terminal whose connection keeps dropping and returning.
Freshness words
The Live view shows a freshness word beside the state word. It says how old the last reading is, not what the reading says. The thresholds are multiples of the terminal’s own reporting interval, not fixed seconds, so a terminal on a long interval is not called stale for keeping to it.
| Word | Shown when | With a 30 second interval |
|---|---|---|
| measuring | The terminal has just started reporting, and its interval is not known yet | any age |
| streaming | The last reading is no older than 1.5 intervals | up to 45 seconds |
| waiting | The last reading is older than 1.5 intervals and no older than 3 | 45 to 90 seconds |
| stale | The last reading is older than 3 intervals. The chart stops where the readings stop | over 90 seconds |
| offline | The collector is offline or being decommissioned, or has never reported | any age |
The interval is the typical gap between the terminal’s recent readings, so one long silence does not distort it.
The word idle appears in 2 places. The dashboard reads idle · no transitions yet or idle · last transition <age> ago once the last state change is over 10 minutes old. A group card counts a terminal as reporting but idle when its collector is connected and its last reading is over 2 minutes old.
Timings
Every value here is the default. The last column names where it is changed, or reads Fixed where it cannot be.
States
| Timing | Default | Where it is set |
|---|---|---|
| How long a fault must last before the state changes | 180 seconds | NF_STATUS_CONFIRM_SECS |
| How long a recovery must last before the state returns to healthy | 300 seconds | NF_STATUS_HOLD_SECS |
| How close together 2 faults count as one incident | 600 seconds | NF_STATUS_COALESCE_SECS |
| How often every terminal’s state is re-checked | 30 seconds | NF_STATUS_SWEEP_SECS |
| After a fleet server restart, how long a silent fleet is reported as a planned outage while collectors reconnect | 3600 seconds | NF_SETTLE_SECS. 0 turns it off |
| The same, after a collector restarts for an update | 600 seconds | Fixed |
| How often latency and drop rate are compared with the baseline | every 150 seconds | Fixed |
| How much history the baseline is taken from | the last 6 hours | Fixed |
| How much history there must be before the baseline is used | 1 hour | Fixed |
| How long a reading must stay above the baseline before it counts | 15 minutes | Fixed |
| How long state changes are kept | 30 days, and 2000 per terminal | Fixed |
Lifecycle
| Timing | Default | Where it is set |
|---|---|---|
| Stale threshold | 1 day | Settings, Terminals. 1 minute to 1 year |
| Decommission timeout | 10 minutes | Settings, Terminals. 1 minute to 1 day |
| How often the fleet server re-reads its settings | 30 seconds | Fixed, so no setting requires a restart |
| Silence before a connected collector’s readings count as stopped, for alerts | 3 intervals plus 30 seconds, and at least 120 seconds | Derived from NF_POLL_INTERVAL |
| Log heartbeat | 3600 seconds | NF_HEARTBEAT_SECS, limited to 60 to 31536000 |
| How long a collector waits before redialling the fleet server | 2, 5, 10, 30, 60 then 120 seconds, each waited between half the step and the step. A refused dial is retried after 1 second, twice, first | Fixed |
| Enrolment token | 1 use, 24 hours | --max-uses and --expires on create-token, or Add telemetry collector |
| How long a cancelled organisation’s data is kept | 7 days | The org_purge_grace_secs setting, which has no field on the page |
Alerts
| Timing | Default | Where it is set |
|---|---|---|
| Wait before an offline alert is sent, in case the terminal returns | 2 minutes | Fixed |
| Repeats of the same alert for the same terminal held for | 15 minutes | Fixed |
| Reminder while an alert stays open | every 4 hours | Fixed |
| State changes that count as flapping | 4 in 30 minutes | Fixed |
| Stable again after flapping | 30 minutes with no state change | Fixed |
| Most alerts sent per organisation | 20 an hour | Fixed |
| How often queued alerts are sent | every 30 seconds, up to 20 at a time | Fixed |
| Delivery retries | 24 attempts, after 1 minute, 5 minutes, 30 minutes, then hourly | Fixed |
| Delivery timeout | 10 seconds | Fixed |
| Longest silence | 30 days | The silence form |
Data retention
| Data | Default | Where it is set |
|---|---|---|
| Readings, on the collector | 7 days | NF_RAW_RETENTION_DAYS |
| Five-minute summaries, on the collector | 90 days | NF_FINE_RETENTION_DAYS |
| Hourly summaries, on the collector | 365 days | NF_COARSE_RETENTION_DAYS |
| Raw readings, on the fleet server | 7 days | Settings, Data. 1 to 3650 days |
| 5-minute summaries, on the fleet server | 90 days | Settings, Data. 1 to 3650 days |
| Percentile detail in a five-minute summary, on the fleet server | 30 days | Fixed |
| Hourly summaries, on the fleet server | 2 years | Settings, Data. 1 to 3650 days, and never less than the five-minute summaries |
| Audit log | 90 days | Settings, Data. 1 to 3650 days |
| Daily summaries | forever | Fixed |
| Audit entries for revoked certificates | forever | Fixed, whatever Audit log is set to |
| Event history per terminal | 1000 entries | Fixed |
| Capacity measurements, on Server health | 7 days, taken every minute | Fixed |
| Live readings kept in memory | 30 minutes per terminal and source | NF_LIVE_RING_MINUTES, 1 to 240 |
Certificate lifetimes
None of these has a field on the Settings page. They are set when the fleet server is deployed, and the ranges are what the fleet server accepts.
| Certificate | Default | Range |
|---|---|---|
| Device certificate | 1 year | 10 minutes to 3 years |
| Fleet Server Certificate | 1 year | 10 minutes to 3 years |
| Renewal starts | 30 days before expiry | 5 minutes to 90 days, and always less than half the certificate’s lifetime |
| CA certificate | 10 years | 1 hour to 20 years |
| Root key and its certificate | 30 years | Fixed |
| Each prepared CA certificate’s term | 547 days, about 18 months | Fixed |
| CA certificates kept ready, the signing cover | 15, about 6 years | The intermediate_queue_depth setting |
| Deny list time bound | 30 days | The deny_list_time_bound_secs setting |
Refresh cadences
None of these changes a state. They set the delay before a change is shown.
| Surface | Refreshes every |
|---|---|
| Attention cards | 2 seconds |
| The fleet wall, and the Map | 10 seconds |
| Feed and Groups | 5 seconds |
| The Live view | 1 second |
| A terminal’s other views | 30 seconds |
| Certificates, and Server health | 30 seconds |
| The certificate attention dot on the icon rail | 2 minutes |
| The licence chip | 5 minutes |
| The maintenance banner | 60 seconds |
See Server configuration and Collector configuration for the variables named here, and Alert rules and webhook for the alert rules.