Backup Module

The Backup module (nimbus-module-backup) snapshots services, dedicated services, templates, controller config, state-sync canonical data, and the Nimbus database itself. Archives are written as tar.zst with an embedded MANIFEST.sha256 trailer and pruned via GFS retention.

Architecture

modules/backup/
└── src/main/kotlin/dev/nimbuspowered/nimbus/module/backup/
    ├── BackupModule.kt             # NimbusModule: wiring, event formatters
    ├── BackupManager.kt            # Orchestrator: target resolution, quiesce, archive runs
    ├── BackupArchiver.kt           # tar + zstd pipeline, verify()
    ├── BackupScheduler.kt          # Minute-tick cron + hourly prune loop
    ├── BackupRetention.kt          # GFS prune (hourly/daily/weekly/monthly/manual)
    ├── BackupConfig.kt             # BackupModuleConfig + ConfigManager (TOML)
    ├── BackupModels.kt             # Types, DTOs, TargetType, Status, RetentionClass
    ├── BackupEvents.kt             # ModuleEvent factories
    ├── BackupTables.kt             # Exposed: backups + backup_schedule_log
    ├── CronExpression.kt           # Hand-rolled 5-field POSIX evaluator
    ├── DatabaseBackupHelper.kt     # SQLite VACUUM INTO / mysqldump / pg_dump
    ├── commands/BackupCommand.kt   # Console `backup …`
    ├── migrations/BackupV1_Baseline.kt # Range 7000
    └── routes/BackupRoutes.kt      # REST: /api/backups/*

Schema-version range: 7000+.

All routes are admin-only (AuthLevel.ADMIN) — backups are sensitive.

Target model

BackupTargetType:

Archive pipeline

BackupArchiver.archive() streams the source tree through a single composite output chain:

FileOutputStream(<name>.tar.zst.tmp)
  └─► DigestingOutputStream (SHA-256 of archive bytes)
      └─► BufferedOutputStream (256 KiB)
          └─► ZstdOutputStream   (native multi-threaded compression)
              │   setLevel(compressionLevel)                // 1..22
              │   setWorkers(max(1, CPU/2) if workers <= 0)
              └─► TarArchiveOutputStream
                     ├─► per file: TarArchiveEntry + MessageDigest.update (SHA-256)
                     └─► trailing MANIFEST.sha256 entry

Why this beats tar --zstd by 3–5×:

1. zstd-jni's setWorkers(N) is native multi-threaded; coreutils tar pipes into single-threaded zstd by default.
2. No fork/exec per backup, no stdout pipe copy.
3. SHA-256 per file is computed in the same pass that writes the tar entry — one filesystem read instead of two.
4. 256 KiB upstream buffer keeps the compressor saturated on worlds with thousands of tiny region files.

On success, .tmp is atomically renamed to the final archive name.

MANIFEST.sha256

A trailing tar entry named MANIFEST.sha256, one line per file:

e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  worlds/world/level.dat
...

BackupArchiver.verify(archive) re-reads the archive, recomputes each entry's SHA-256, and compares against the manifest — catches silent on-disk corruption independent of the outer archive checksum.

Quiesce

For SERVICE / DEDICATED targets, the manager sends three commands to the live process via ServiceManager.executeCommand before archiving:

save-off              # Disable autosave
save-all flush        # Force a write of the world to disk
<wait quiesceWaitSeconds>
...archive source...
save-on               # Re-enable autosave

Skipped if the service isn't READY/STARTING, or lives on a remote node (nodeId != "local"), or [config].quiesceServices = false.

ServiceManager is resolved lazily via ModuleContext.service<ServiceManager>() after a 10s delay, since it's registered after module init().

Database dump

DatabaseBackupHelper.dump(stagingDir) returns Success | Skipped | Failed:

• SQLite → runs VACUUM INTO '<staging>/nimbus.db' (atomic, inside an implicit transaction — no open DB file concerns).
• MySQL → shells out to mysqldump. Missing binary → Skipped with a warning; the parent backup records PARTIAL and keeps going.
• PostgreSQL → shells out to pg_dump. Same missing-binary behaviour.

The staging directory is archived like any other source, then deleted.

GFS retention

BackupRetention.prune() groups rows by (targetType, targetName, scheduleClass). For each group, candidates filter to status IN ('SUCCESS', 'PARTIAL'); after sorting by startedAt DESC, anything beyond the per-class keep budget is deleted (archive file + DB row).

Defaults (from RetentionConfig):

FAILED rows are excluded from the per-class budget (a transient failure shouldn't cost a retained snapshot) but age-pruned after failedKeepDays days (default 7).

Retention budgets are read live from configManager.getConfig() via a provider lambda, so PUT /api/backups/config hot-reloads without a restart.

Scheduler

BackupScheduler has two coroutines on the module's scope:

1. Minute tick — after 15s startup delay, aligns to the next minute boundary and evaluates every ScheduleEntry.cron against LocalDateTime.now(). Matches fire runScheduledBackup(schedule) in a detached coroutine; the manager's semaphore caps concurrency.
2. Hourly prune — after 60s, calls retention.prune() every hour.

The scheduler updates backup_schedule_log with last_run_at and last_status (aggregated across the run's records: any FAILED → FAILED; else any PARTIAL → PARTIAL; else SUCCESS).

Cron evaluator

CronExpression is a hand-rolled 5-field POSIX evaluator with lists (1,5,9), ranges (1-5), steps (0-30/5, */15), and star (*). Days are 0–6 (Sun–Sat). nextAfter(now) computes the next fire time for describeSchedules().

Concurrency

BackupManager uses a volatile Semaphore(max(1, config.maxConcurrent)). Config changes rebuild the semaphore on the next currentSemaphore() call. In-flight jobs against the old semaphore still release into it — harmless; worst case during a reload is briefly up to old + new concurrent jobs.

Each scheduled "run" expands into one archive row per resolved target, so a schedule with targets = ["services", "database"] on a network of 5 services produces 6 rows.

Tables

backups

backup_schedule_log

One row per schedule_name (unique index) — tracks last_run_at, next_run_at, last_status.

REST API

All admin-only under /api/backups:

Events

Emitted as ModuleEvent("backup", type, data):

Restore safety

restore(id, overridePath, dryRun, force, triggeredBy) refuses to extract over a service's workdir while it is READY/STARTING/DRAINING unless the caller passes force = true. dryRun = true returns the file list without writing.

Config edits

BackupConfigManager reads config/modules/backup/backup.toml on init and atomically rewrites it on PUT /api/backups/config. A write watcher reloads the in-memory @Volatile config snapshot on disk changes, so scheduler, retention, and semaphore all pick up new values at their next iteration without a restart. Schema validation is provided by kotlinx.serialization deserialization — malformed payloads are rejected with HTTP 400.

Edge cases

• Source missing on disk → PARTIAL with reason source unavailable.
• Partial archive on failure → best-effort Files.deleteIfExists of the .tmp file on throw.
• Remote service → PARTIAL row with reason referencing the node ID.
• mysqldump / pg_dump missing → WARN log, parent run marked PARTIAL, other targets proceed.
• Verify on truncated archive → caught and returned as VerifyResult(valid = false, errors = ["Archive unreadable: …"]) — never surfaces as a 500.

Next steps

• Backup Guide — Operator workflows
• Backup Config Reference — Full backup.toml schema
• Custom Modules — Module API reference
• Events Reference — MODULE_EVENT envelope
• API Reference — /api/backups/*