Nimbusv1.0.0

Architecture

Internal architecture overview for developers who want to understand, contribute to, or build integrations with Nimbus.

This page covers Nimbus's internal architecture for developers who want to understand how the system works, contribute to the codebase, or build integrations.

Gradle subprojects

Nimbus is a multi-module Gradle build. The tree as of 0.9.x:

GroupSubprojectDirectoryPurpose
Core:nimbus-corenimbus-core/Controller entry point, REST/WS API, service lifecycle, cluster
Core:nimbus-agentnimbus-agent/Standalone node process (runs on remote machines)
Core:nimbus-clinimbus-cli/Remote JLine3 console that connects over REST + WS
Core:nimbus-protocolnimbus-protocol/Shared ClusterMessage + StateManifest types
Module API:nimbus-module-apimodules/api/Interfaces third-party modules build against
Controller modules:nimbus-module-permsmodules/perms/Permission groups, tracks, prefix/suffix
:nimbus-module-displaymodules/display/Sign + NPC configs
:nimbus-module-scalingmodules/scaling/Time-based schedules, predictive warmup
:nimbus-module-playersmodules/players/Centralized player tracking, session history
:nimbus-module-punishmentsmodules/punishments/Network-wide bans/mutes/kicks/warns
:nimbus-module-resourcepacksmodules/resourcepacks/Pack registry + scoped assignments
:nimbus-module-backupmodules/backup/Scheduled tar+zstd snapshots
Plugins:nimbus-sdkplugins/sdk/Paper/Spigot/Folia SDK deployed to backends
:nimbus-bridgeplugins/bridge/Velocity proxy plugin
:nimbus-permsplugins/perms/Permission provider plugin for backends
:nimbus-displayplugins/display/Sign + NPC renderer for backends
:nimbus-punishments-velocityplugins/punishments-velocity/Login/connect enforcement on proxies
:nimbus-punishments-backendplugins/punishments-backend/Chat-mute enforcement on backends
:nimbus-resourcepacksplugins/resourcepacks/Pack applier for backends

Controller package layout (nimbus-core/src/main/kotlin/dev/nimbuspowered/nimbus/)

Controller Packages
Nimbus.kt / NimbusKt.kt   → main() + bootstrap wiring
NimbusVersion.kt          → Version read from JAR manifest / gradle.properties
LogRotation.kt            → Rotate latest.log to dated archives
api/                      → Ktor REST + WebSocket server (NimbusApi, auth, routes/)
cluster/                  → NodeManager, ClusterServer (Netty TLS), ClusterWebSocketHandler,
                            RemoteServiceHandle, RemoteFileProxy, TlsHelper, placement/
config/                   → NimbusConfig, GroupConfig, DedicatedConfig, loaders
console/                  → JLine3 REPL, CommandDispatcher, commands/
database/                 → DatabaseManager, MigrationManager, Tables, MetricsCollector, AuditCollector
doctor/                   → Core doctor checks (cluster, config, ports, etc.)
event/                    → EventBus (SharedFlow) + sealed NimbusEvent
group/                    → ServerGroup runtime state, GroupManager
loadbalancer/             → TcpLoadBalancer, BackendHealthManager, strategies
module/                   → ModuleManager (URLClassLoader + ServiceLoader), ModuleContextImpl
proxy/                    → ProxySyncManager (tab list, MOTD, chat, maintenance)
scaling/                  → Core ScalingEngine + reactive scaling rules
service/                  → ServiceManager, ServiceFactory, ProcessHandle, PortAllocator,
                            StateSyncManager, WarmPoolManager, DedicatedServiceManager
setup/                    → First-run SetupWizard
stress/                   → StressTestManager (simulated load)
system/                   → OS-level helpers (memory reader, process inspector)
template/                 → TemplateManager, ConfigPatcher, SoftwareResolver, ServiceDeployer,
                            ModScanner
update/                   → UpdateChecker (GitHub Releases auto-updater)
velocity/                 → VelocityConfigGen (auto-manage proxy server list)

Module API (modules/api/src/main/kotlin/dev/nimbuspowered/nimbus/module/)

NimbusModule, ModuleContext, ModuleCommand, Migration, PluginDeployment, DoctorCheck, CommandOutput. Everything a third-party module needs sits here with no dependency on nimbus-core — see Custom Modules.

Agent package layout (nimbus-agent/src/main/kotlin/)

Agent.kt (main), AgentConfig, AgentRuntime (WS loop), AgentStateStore (JSON persistence), JavaResolver, LocalProcessHandle, LocalProcessManager, SetupWizard, TemplateDownloader, plus TlsHelper for fingerprint verification. See Agent Node for details.

Shared protocol (nimbus-protocol)

  • ClusterMessage — sealed class with every controller ↔ agent message type
  • StateManifest — per-service state-sync manifest (manifest-compare delta sync)
  • clusterJson — preconfigured kotlinx.serialization.json.Json

ServiceHandle lives in nimbus-core/service/ (not in protocol) — it abstracts over local ProcessHandle and RemoteServiceHandle.

Bootstrap flow

When Nimbus starts (Nimbus.ktnimbusMain()), components are initialized in this order:

Bootstrap Order
1. Log rotation           → Rotate latest.log to dated archives
2. Setup wizard           → First-run interactive setup (if needed)
3. Config loading         → Parse config/nimbus.toml + config/groups/*.toml
4. API token generation   → Auto-generate if missing
5. Directory creation     → Ensure templates/, services/, logs/, etc. exist
6. Plugin deployment      → Extract nimbus-bridge.jar, nimbus-sdk.jar
7. Component init         → EventBus, ServiceRegistry, PortAllocator,
                            TemplateManager, GroupManager, ProxySyncManager
8. Group loading          → Parse group configs into GroupManager
9. Module loading         → ModuleManager scans modules/*.jar, creates one
                            URLClassLoader per JAR, discovers NimbusModule via
                            module.properties (fallback: ServiceLoader), calls
                            init(). Modules register commands, routes, plugin
                            deployments, event formatters, doctor checks, and
                            migrations during init.
10. Migrations run        → MigrationManager applies every registered migration
                            in ascending version order (core 1–999, modules
                            1000+). Baseline migrations are marked applied
                            without executing.
11. ServiceManager        → Wire up all dependencies, register in ModuleContext
                            via registerService() so late-binding loops in modules
                            (e.g. backup, scaling) can resolve it.
12. Modules enable()      → Called on every loaded module after ServiceManager
                            is registered. State-sync manager for [group.sync]
                            groups is initialized here.
11. ScalingEngine         → Create (but DON'T start yet — deferred until after boot)
12. NimbusApi             → Create (but don't start) Ktor server
13. NodeManager           → If cluster.enabled: init cluster coordination
14. ClusterServer         → If cluster.enabled: separate Ktor server on agent_port for agent WebSocket connections
15. TcpLoadBalancer       → If loadbalancer.enabled: init Layer-4 TCP proxy
                            with BackendHealthManager (health checks, circuit breaker,
                            connection draining, idle timeout, connection limit)
16. Shutdown hook         → Register SIGTERM/SIGINT handler
17. NimbusConsole.init()  → Banner, event listener
18. Api.start()           → Start Ktor HTTP server (+ /cluster WS if cluster enabled)
19. LoadBalancer.start()  → If enabled: start TCP listener on LB port + health check loop
20. VelocityUpdater       → Start periodic update check (first check after 60s)
21. startMinimumInstances → Phased startup:
                            Phase 1: Start proxy groups, wait for READY (120s timeout)
                            Phase 2: Start backend groups
22. ScalingEngine.start() → Start periodic scaling loop (AFTER boot completes)
23. Console.start()       → JLine3 REPL (blocks until shutdown)

The shutdown hook and console REPL provide two paths to shutdown: external signals (SIGTERM) use the hook, while the shutdown command exits the REPL.

Coroutine architecture

Nimbus uses kotlinx-coroutines for all async work. No raw threads are created in the core module.

Coroutine Setup
// Root scope with SupervisorJob (child failures don't cancel siblings)
val scope = CoroutineScope(SupervisorJob() + Dispatchers.Default)

// EventBus uses the shared scope for event dispatch
val eventBus = EventBus(scope)

// Scaling engine launches a long-running coroutine
val scalingJob = scalingEngine.start()  // Returns a Job

Key patterns:

  • SupervisorJob -- A failing coroutine (e.g., scaling error) doesn't bring down the whole system
  • Dispatchers.Default -- CPU-bound work (most coordination)
  • Dispatchers.IO -- Used in ServiceManager for process I/O and file operations
  • MutableSharedFlow -- EventBus uses a shared flow with extraBufferCapacity = 64

Event bus

The event bus is the central communication backbone. All state changes are published as sealed class events.

EventBus Usage
// Emitting an event (suspend function)
eventBus.emit(NimbusEvent.ServiceReady(serviceName = "BedWars-1", groupName = "BedWars"))

// Subscribing to a specific event type
eventBus.on<NimbusEvent.ServiceReady> { event ->
    println("${event.serviceName} is ready!")
}

// Getting the raw SharedFlow
val flow: SharedFlow<NimbusEvent> = eventBus.subscribe()

Event types

Events are modeled as sealed class NimbusEvent in event/Events.kt:

CategoryEvents
Service lifecycleServiceStarting, ServiceReady, ServiceDraining, ServiceStopping, ServiceStopped, ServiceCrashed, ServiceRecovered, ServicePrepared, WarmPoolReplenished, ServiceDeployed
ScalingScaleUp, ScaleDown, PlacementBlocked
Custom stateServiceCustomStateChanged
State syncSyncCompleted, SyncFailed
PlayersPlayerConnected, PlayerDisconnected, PlayerServerSwitch
GroupsGroupCreated, GroupUpdated, GroupDeleted
MessagingServiceMessage
Modules (generic)ModuleEvent(moduleId, type, data) — the envelope used for every module-defined event (permissions, punishments, resource packs, backups, scaling)
Module lifecycleModuleLoaded, ModuleEnabled, ModuleDisabled
UpdatesProxyUpdateAvailable, ProxyUpdateApplied, NimbusUpdateAvailable, NimbusUpdateApplied
Proxy syncTabListUpdated, MotdUpdated, PlayerTabUpdated, ChatFormatUpdated
MaintenanceMaintenanceEnabled, MaintenanceDisabled
StressStressTestUpdated
DedicatedDedicatedCreated, DedicatedDeleted
ConfigConfigReloaded
APIApiStarted, ApiStopped, ApiWarning, ApiError
ClusterClusterStarted, NodeConnected, NodeDisconnected, NodeHeartbeat
Load balancerLoadBalancerStarted, LoadBalancerStopped, LoadBalancerBackendHealthChanged
CLI sessionsCliSessionConnected, CliSessionDisconnected

Events are broadcast via the REST API's WebSocket endpoint (/api/events) so external systems and plugins can react to them.

Module-defined events (e.g. PUNISHMENT_ISSUED, RESOURCE_PACK_CREATED, BACKUP_COMPLETED, SMART_SCHEDULE) are emitted as ModuleEvent(moduleId, type, data) and serialized on the wire as MODULE_EVENT frames. See reference/events for the envelope and per-module subtypes.

Process management

Each running service is wrapped in a ServiceHandle interface that abstracts process management across local and remote nodes:

  • ProcessHandle -- Local JVM subprocess started via ProcessBuilder with inherited environment
  • RemoteServiceHandle -- Proxy for services running on remote agent nodes, communicating via the cluster WebSocket protocol

Static services run on the controller by default. They have persistent data (worlds, configs) stored in services/static/. With [group.sync] enabled = true, static services can be placed on remote agent nodes — the controller's canonical store under services/state/<name>/ acts as the source of truth. Dynamic and proxy services can be distributed across any node without additional configuration.

Both implementations provide:

  • stdout/stderr -- Captured asynchronously via SharedFlow<String> for ready detection and logging
  • Ready detection -- Regex matching against stdout (default pattern: Done)
  • Graceful shutdown -- Sends stop command via stdin, then waits, then force-kills

State persistence & process recovery

The controller persists local service state to state/services.json via ControllerStateStore. On restart, ProcessHandle.adopt() re-attaches to running processes by PID. On Windows, it falls back to checking if the process is a Java process when commandLine() is unavailable. Recovered services are registered before temp directory cleanup to protect their working directories from being removed.

Service states

Service State Machine
PREPARING → PREPARED → STARTING → READY → DRAINING → STOPPING → STOPPED
                                    ↓                                 ↑
                                  CRASHED ── (restart?) ─────────────┘
StateDescription
PREPARINGTemplate being copied to service directory
PREPAREDTemplate copy complete; service is staged in the warm pool, waiting to be started
STARTINGJVM started, scanning stdout for ready pattern
READYServer accepting players
DRAININGLoad balancer draining existing connections; no new players routed here
STOPPINGGraceful shutdown in progress
STOPPEDClean shutdown complete
CRASHEDUnexpected exit (triggers restart if restart_on_crash = true)

Velocity config generation

The VelocityConfigGen class keeps the proxy's velocity.toml in sync with running backend services:

  1. Finds all READY backend services (non-VELOCITY groups)
  2. Builds the [servers] section with ServiceName = "127.0.0.1:port" entries
  3. Sets the try list to lobby servers (groups containing "lobby")
  4. Replaces the [servers] and [forced-hosts] sections in velocity.toml

This runs automatically whenever a backend service becomes READY or stops.

Plugin deployment

Templates are user-owned in 0.9.x+. Nimbus never writes managed plugin JARs into templates/global/plugins/ or templates/global_proxy/plugins/. Everything you drop into templates/ stays yours.

Every Nimbus-managed plugin (the SDK plus each module's companion plugin) is deployed at runtime on every service prepare by ServiceFactory.resolveModulePlugins(). A plugin registered by a module with ModuleContext.registerPluginDeployment(PluginDeployment(...)) is copied from the module JAR's resource path into the service's plugins/ directory with REPLACE_EXISTING — so deletions self-heal on next start and updates roll out automatically.

PluginRegistered byTarget
nimbus-sdk.jarCore (always)Backend services (Paper family, Folia)
nimbus-bridge.jarCore (Velocity)Proxy services
nimbus-perms.jarnimbus-module-permsBackend services
nimbus-display.jarnimbus-module-displayBackend services
nimbus-punishments.jarnimbus-module-punishmentsVelocity (login/connect enforcement)
nimbus-punishments-backend.jarnimbus-module-punishmentsBackend (chat-mute enforcement)
nimbus-resourcepacks.jarnimbus-module-resourcepacksBackend services
FancyNpcsOn-demand downloadOnly when the display module is used with NPCs

Each PluginDeployment can gate on minMinecraftVersion and declares a PluginTarget (BACKEND vs VELOCITY), so version-specific or proxy-only plugins are deployed only where they belong. See Custom Modules for the registration API.

Shutdown order

When Nimbus shuts down:

Shutdown Order
1. Cancel scaling engine job
2. Send graceful shutdown to all agents (if cluster enabled, wait up to 30s)
3. Stop TCP load balancer (if enabled)
4. Stop REST API server
5. Disconnect cluster nodes (if enabled)
6. Stop all local services (via ServiceManager.stopAll()):
   a. Dynamic (game) services first
   b. Static backend (lobby) services
   c. Proxy services last
7. Stop cluster WebSocket server (if enabled)
8. Cancel coroutine scope

This order ensures players are moved to lobbies before lobbies shut down, and proxies stay alive as long as possible to handle redirects.

Technology choices

ComponentTechnologyRationale
LanguageKotlin 2.1Coroutines, sealed classes, null safety
BuildGradle + ShadowFat JAR packaging with embedded plugins
ConfigktomlNative TOML parsing for Kotlin
ConsoleJLine 3Rich terminal with history, completion, ANSI colors
Asynckotlinx-coroutinesStructured concurrency, no callback hell
HTTP serverKtor (CIO)Lightweight, coroutine-native
HTTP clientKtor Client (CIO)Server JAR downloads
SDK clientJava HttpClientZero dependencies for Paper/Velocity plugins
Cluster protocolkotlinx-serialization JSONTyped messages between controller and agents
Load balancerJava NIO (ServerSocketChannel)Non-blocking TCP proxy with zero dependencies
Module systemServiceLoader + URLClassLoaderDynamic module loading without framework overhead

The core design principle is no frameworks -- no Spring, no dependency injection containers. Components are wired directly in Nimbus.kt, making the startup path explicit and debuggable.

Next steps