Backup
Backups run on mocha as CronJobs defined in mocha/system/backup/. Each job writes a dump to an emptyDir, then a restic container pushes it to BunnyCDN S3 and prunes old snapshots (7 daily, 4 weekly, 6 monthly).
| CronJob | Namespace | Schedule | What | Repository |
|---|---|---|---|---|
authentik-pgdump |
authentik |
30 0 * * * |
pg_dump of Authentik |
mocha-coffee/authentik-postgres |
matomo-db-dump |
matomo |
0 1 * * * |
MariaDB dump | mocha-coffee/matomo-db |
tranquil-pds-db-dump |
tranquil-pds |
30 1 * * * |
database dump | mocha-coffee/tranquil-pds-db |
openbao-snapshot |
bank-vaults |
0 */6 * * * |
OpenBao raft snapshot | mocha-coffee/openbao-raft |
Repositories live under s3:https://de-s3.storage.bunnycdn.com/. A job initialises its repository on the first run.
Credentials
The S3 keys and the restic password are in Vault under kv/restic (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, RESTIC_PASSWORD). Each job has its own ExternalSecret (restic-<job>) that adds the RESTIC_REPOSITORY of that job.
Add a backup
Copy one of the blocks in logical-dumps.yaml (ExternalSecret + CronJob), change the namespace, the dump command, the repository path and the --tag, then commit.
Restore
export VAULT_CONTEXT=mocha-corium
export AWS_ACCESS_KEY_ID=$(mise run vault:kv get -field=AWS_ACCESS_KEY_ID kv/restic)
export AWS_SECRET_ACCESS_KEY=$(mise run vault:kv get -field=AWS_SECRET_ACCESS_KEY kv/restic)
export RESTIC_PASSWORD=$(mise run vault:kv get -field=RESTIC_PASSWORD kv/restic)
export RESTIC_REPOSITORY=s3:https://de-s3.storage.bunnycdn.com/mocha-coffee/matomo-db
restic snapshots
restic restore latest --target ./restore
PVC backups with VolSync
VolSync is installed (mocha/system/storage/volsync.yaml) but no PVC is backed up with it yet. knot.yaml holds the ReplicationSources for knot; it stays out of the kustomization until the knot PVCs move to openebs-lvmpv-thin, because openebs-lvm refuses the Direct copy method and only thin volumes can be snapshotted.