Best practice

The root key of a Nexus Telemetry Fleet organisation signs the organisation’s CA certificates in advance. How far ahead they are signed is the signing cover. While there is cover, the fleet server runs without the root key, so the key can be kept off the fleet server. See Root key.

The recommended practice is: back the key up, extend the cover, take the key offline, and restore the key only when the cover has to be extended again. The 4 actions are on the Root key card of the Certificates page, and users with the Owner or Admin role can run them.

# Backup

A backup of the root key is one .nfkey file containing the key and its certificate, encrypted with a passphrase chosen when the backup is made. The fleet server does not store the passphrase, and it cannot be recovered.

1. Press Download backup… in the Back up the root key row.
2. Type the same passphrase under Passphrase for this backup and Repeat the passphrase. The button is disabled until they match.
3. Press Download backup.

The browser saves org-root-bundle.nfkey. The key itself is not moved or changed.

Keep the file away from the fleet server, as 2 copies in 2 places, with the passphrase in a password manager and never with the file.

# Extend cover

While the key is on the fleet server, the fleet server extends the cover itself. While the key is offline, it cannot. So extend the cover by hand before taking the key offline, and each time the key is restored.

1. Press Extend cover… in the Extend signing cover row.
2. Check the preview. It shows the date the cover lasts until now, and the date it will last until after extending, with the number of certificates to be added.
3. Press Extend cover.

When there is nothing to add, the row reads Cover is already at its configured depth. and has no button.

Full cover is 15 CA certificates, about 6 years. Extending the cover changes neither the key nor a backup of it.

# Take offline

1. Press Remove key… in the Take the root key offline row.
2. Tick I have a backup of this key stored somewhere other than this server, and I know its passphrase. If the line under it reads No backup has ever been downloaded from this server., stop and make a backup first.
3. Type the organisation’s short name where asked, as confirmation. It is the name in the organisation’s web address. Press Remove root key from this server.

The badge reads Root key: offline and the Organisation tile reads Root key offline, backed up. Nothing changes for the collectors.

# Restore

1. Choose the .nfkey file under Backup file.
2. Type its passphrase under Passphrase of the backup.
3. Press Restore key.

The badge reads Root key: restored (take offline when done). The fleet server opens the backup on the fleet server itself, and the key does not leave it.

Do the task the key was restored for, then take it offline again. Extending the cover is the usual task. A backup restores onto the fleet server that made it, or onto a fleet server restored from that one’s data directory backup. It does not restore onto a fleet server with a different data directory.

# Custody state mismatch

When what the fleet server recorded about the key and the key file on disk disagree, the card reads The recorded custody state does not match the root key file on disk: investigate before running a ceremony. and the Organisation tile reads Key state and disk disagree.

Press nothing on the card. The usual cause is a data directory copied by hand. root-key status at the command line prints a Custody state line, what the fleet server recorded, and a Root key on disk line, what is on disk. If the recorded state says the key is on disk and the file is missing, restore the data directory from its most recent backup. If the file is present and the recorded state says it should not be, write to support.

# Quarterly check

Once a quarter, check that the key can be recovered. The check changes nothing.

1. Read the year in Queue covers to and the state on the badge.
2. Find the backup and prove it opens, with root-key verify.
3. Confirm that whoever would do this out of hours can reach the backup and the passphrase.
4. Read the Certificate Audit Log for root key entries that nobody can account for.

# Command line

The same actions run at the command line on the fleet server, as nf-server commands, with --org <id|slug> naming the organisation. Without it the command acts on the fleet server’s own transport root key. A passphrase is always asked for at a prompt, never given as a flag. The audit log records a command line action against the organisation’s short name and cli.

See Command line for the flags and what each command prints.