Key rotation

Rotate the JWT signing keys and the encryption key at rest without interruption: JWKS grace period, two-key re-encryption, schedule and compromise response.

Two families of keys have an operable rotation procedure, both designed to run without interrupting sign-ins or invalidating tokens still in flight.

Two keys, two procedures

Both commands are subcommands of the auth-service binary, invoked here through Compose.

Rotate the JWT signing key

Rotation has no impact on clients: the previous key is marked retired but stays published in the JWKS while tokens in flight expire, then gets purged.

docker compose -f compose.prod.yml -f compose.tls.yml --env-file .env.prod \
  exec auth-service /usr/local/bin/auth-service rotate-keys 48

One call does three things: it generates a new active key, retires the previous active key (still published), and purges any key retired for more than the grace period in hours (48 by default). Because verifiers select the public key by kid, tokens signed with the retired key keep validating until the purge.

Each rotation emits an oauth.signing_key.rotated audit event (and oauth.signing_key.purged when keys are purged), visible in the audit log.

Rotate the encryption key at rest

ENCRYPTION_KEY protects non-regenerable data (TOTP secrets), so the rotation must be lossless. The mechanism keeps the old key available for decryption only, while everything is re-encrypted under the new one.

1. Generate the new key: openssl rand -base64 32.
2. Reconfigure and restart: ENCRYPTION_KEY takes the new value, ENCRYPTION_KEY_OLD takes the previous one. Reads keep working (decryption tries the primary key, then each old key), writes now use the new key.
3. Re-encrypt the existing secrets under the primary key with reencrypt-secrets (idempotent).
4. Rotate the signing keys so the new active signing key is encrypted with the new ENCRYPTION_KEY: run rotate-keys.
5. After the grace period, purge the retired signing keys (still encrypted with the old key): run rotate-keys 48. Then run reencrypt-secrets again: it must report 0 re-encrypted before you go further.
6. Remove ENCRYPTION_KEY_OLD from the configuration, restart, and destroy the old key in the vault: no data depends on it anymore.

reencrypt-secrets is invoked like rotate-keys:

docker compose -f compose.prod.yml -f compose.tls.yml --env-file .env.prod \
  exec auth-service /usr/local/bin/auth-service reencrypt-secrets

The re-encryption emits a crypto.secrets.reencrypted audit event.

When to rotate

• Signing key: every 30 to 90 days, through an operator cron.
• Encryption key: on the schedule your security policy sets, and immediately when a compromise is suspected.
• On compromise: rotate at once. To invalidate tokens signed with a compromised key, purge it from the JWKS with a short grace period (for example rotate-keys 0), accepting that not-yet-expired tokens will be rejected: that is the point.

A monthly cron for the signing key:

0 3 1 * *  docker compose -f /opt/obexal/compose.prod.yml -f /opt/obexal/compose.tls.yml --env-file /opt/obexal/.env.prod exec -T auth-service /usr/local/bin/auth-service rotate-keys 48 >> /var/log/obexal/rotate.log 2>&1

Operational notes

• There is no in-process auto-rotation: rotations are explicit operator commands, run one invocation at a time to avoid races between replicas.
• Back up before every rotation, like before every migration.
• The stored format does not change across rotations: values remain base64(nonce || ciphertext || tag), and GCM authentication makes trying the wrong key fail fast.