Obexal Docs

Docs/Self-hosting/Backups and restore

Backups and restore

What to back up, how to encrypt and replicate the dumps, how to restore and test it, and how rollbacks work with forward-only migrations.

PostgreSQL carries critical, non-regenerable data: accounts, argon2id credentials, encrypted MFA secrets, encrypted OIDC private keys and the immutable audit log. The audit log's immutability (a trigger blocks UPDATE and DELETE) does not protect against disk loss: backups are indispensable.

What holds the state

  • PostgreSQL is the state. Everything durable lives there. It is the only component whose loss is unrecoverable without a backup.
  • Redis is volatile and reconstructible. It is deliberately run without persistence: a restart empties active sessions (users sign in again) but no durable data is touched.
  • Also protect: your .env.prod (in a secrets vault, because it holds ENCRYPTION_KEY), and the MinIO volume if you use it for tenant logos (re-uploadable, but a backup avoids the churn). Caddy certificates need no backup: ACME reissues them.

Back up PostgreSQL

The repository ships scripts/backup.sh (a pg_dump in custom format, with retention):

DATABASE_URL=postgres://obexal:$DB_PASSWORD@db-host:5432/obexal \
BACKUP_DIR=/var/backups/obexal RETENTION_DAYS=14 \
  ./scripts/backup.sh

Run it daily by cron, and always before a migration or a key rotation:

0 3 * * *  DATABASE_URL=... BACKUP_DIR=/var/backups/obexal /opt/obexal/scripts/backup.sh >> /var/log/obexal/backup.log 2>&1

Dumps are written with chmod 600: they contain personal data and encrypted secrets. Encrypt them at rest and replicate them to a separate EU object bucket. On a managed EU PostgreSQL, also enable point-in-time recovery (WAL archiving) on top of the logical dumps: it is the reference protection in production.

The encryption key is part of the backup

A database restore is only worth something if the matching encryption key is available: ENCRYPTION_KEY (and any ENCRYPTION_KEY_OLD still referenced) decrypts the TOTP secrets and the OIDC private keys inside the dump.

Danger

Store ENCRYPTION_KEY in a secrets vault, separately from the dumps. A dump and its key stored together defeat the encryption; a dump without its key is unreadable.

If you restore a dump older than a key rotation, provide the key of that era through ENCRYPTION_KEY_OLD, then run reencrypt-secrets.

Restore

scripts/restore.sh is destructive and requires explicit confirmation:

CONFIRM=yes DATABASE_URL=postgres://obexal:$DB_PASSWORD@target-host:5432/obexal \
  ./scripts/restore.sh /var/backups/obexal/obexal-20260701T030000Z.dump

Test your restores

An untested backup does not exist. Schedule a restore drill (quarterly is a good cadence), on a throwaway database:

  1. Restore the latest dump.
  2. Start the auth-service against that database.
  3. Verify: a password login, /.well-known/jwks.json (signing keys present), an MFA completion, and /v1/auth/me.
  4. Record the time it took: that is your real RTO.

Application rollback

Migrations are forward-only: there are no down migrations, by design. They are additive and compatible with version N-1, so in most cases a rollback is simply redeploying the previous application image, without touching the database. Only when a migration introduced an incompatible change do you restore the dump taken just before the deployment: which is why backing up before every update is non-negotiable.

GDPR and retention

Account deletion purges live data in cascade, but the backups remain subject to their retention window (RETENTION_DAYS, 14 days by default). Document that duration in your processing register and retention policy, and reduce it if your minimization requirements demand it. See GDPR.