Hardening

A Nexus Telemetry Fleet server starts listening on its own host only, with no encryption of its own. TLS for the dashboard comes from a reverse proxy in front of the fleet server, or from the fleet server itself with a certificate it is given. See Install for installing a fleet server.

Reverse proxy

The production compose file runs Caddy on ports 80 and 443, and keeps the dashboard port inside the compose network, so only Caddy answers from outside. Caddy obtains the dashboard’s certificate by ACME at the first start and renews it. Any other reverse proxy can serve the dashboard port in Caddy’s place.

Collectors connect to the fleet server on port 9443, separately from the dashboard. The compose file publishes that port straight from the host. It must never pass through Caddy or any other proxy that terminates TLS.

Warning: at enrolment a collector records the fleet server’s own private certificate chain, and from then on it accepts that chain only. A collector that is offered a publicly trusted certificate on port 9443 refuses it and never connects.

Your own certificate

To have the fleet server terminate TLS itself, put the certificate chain and its key where the service user can read them, set NF_DASHBOARD_TLS_CERT and NF_DASHBOARD_TLS_KEY to their paths, and restart.

bash
sudo systemctl restart nf-server

The startup log then reads Web server listening (HTTPS).

Both variables are paths to PEM files, and both must be set. With only one set, the dashboard stays on plain http://. Also set NF_SERVER_URL to the dashboard’s https:// address, so that the enrolment commands the dashboard generates use it. The fleet server does not reload a renewed certificate, so restart the service after each renewal.

A fleet server that listens on an address other than loopback, with neither variable set, warns at startup that anyone who can reach the dashboard has full control of the fleet.

Forwarded proto header

Where a proxy terminates TLS and the fleet server itself stays on loopback, set NF_TRUST_FORWARDED_PROTO=1. The fleet server then reads the proxy’s X-Forwarded-Proto header and marks the session cookie Secure.

Never set it on a fleet server whose dashboard port is reachable directly. The header comes from the client and can be forged, and the fleet server does not check that a proxy is in front of it.

Health checks

3 addresses answer without a sign-in.

Address Answers For
GET /health 200 while the process is running, whatever else is wrong The orchestrator or container health check
GET /ready 200 only when the database is open and the fleet server is not shutting down. 503 otherwise A load balancer, and a deployment waiting to send traffic
GET /healthz 200 with a small JSON body while the fleet server is healthy. 503 while it is degraded Monitoring from another host
bash
curl -s https://<your domain>/healthz

In the JSON body, status reads ok with a 200. It reads degraded with a 503 in the 2 cases the fleet server calls degraded: its database stops answering, or the hourly pass that summarises readings has not run for 3 hours. See Server health.

External probe

The monitor that calls /healthz belongs on a host outside the fleet server’s network, so that it still runs when the fleet server or its network is down. The External probe check on Server health only confirms that /healthz answers. It does not watch the fleet server from outside.

Logs and metrics

The fleet server writes its log to standard output, one JSON object per line, at the level set by NF_LOG. The default is info. NF_LOG_FORMAT=pretty writes readable lines instead of JSON.

bash
sudo journalctl -u nf-server -f

Under Docker the same lines come from docker logs. The fleet server writes no log file of its own.

GET /metrics answers in Prometheus format on the dashboard port. It gives fleet-wide figures: collectors enrolled and online, the database size, user sessions, certificate expiry times, and running totals of enrolments, renewals and revocations. It gives no terminal telemetry. It answers only to a request that sends NF_PROBE_TOKEN as a bearer token, and is not served while that variable is unset. The full /healthz body requires the same token.

Rate limits

Each limit is a count per minute. Once it is reached, further attempts in that minute are refused with HTTP 429.

Limit Per minute Variable
Enrolment attempts from one source address 30 NF_ENROL_MAX_PER_IP
Enrolment failures against one token 20 none
Enrolment failures across the fleet server 100 none
Sign-in failures from one address 20 none

The enrolment count uses the X-Forwarded-For header, so a public fleet server requires a proxy in front that sets that header from the real source address. The sign-in count uses the peer address, and reads that header only from a proxy on a loopback or private network address.

Dashboard access from other hosts

The dashboard listens on 127.0.0.1:3000 unless NF_LISTEN sets another address, so by default only the host it runs on can reach it. Before opening it to other hosts, put TLS or a proxy in front of it, and create the first user, because until a user exists the dashboard lets anyone in. See Install.

See Server configuration for every variable named here and its default.

← Install Backup and restore →