Architecture
Internal architecture overview for developers who want to understand, contribute to, or build integrations with Nimbus.
This page covers Nimbus's internal architecture for developers who want to understand how the system works, contribute to the codebase, or build integrations.
Gradle subprojects
Nimbus is a multi-module Gradle build. The tree as of 0.9.x:
| Group | Subproject | Directory | Purpose |
|---|---|---|---|
| Core | :nimbus-core | nimbus-core/ | Controller entry point, REST/WS API, service lifecycle, cluster |
| Core | :nimbus-agent | nimbus-agent/ | Standalone node process (runs on remote machines) |
| Core | :nimbus-cli | nimbus-cli/ | Remote JLine3 console that connects over REST + WS |
| Core | :nimbus-protocol | nimbus-protocol/ | Shared ClusterMessage + StateManifest types |
| Module API | :nimbus-module-api | modules/api/ | Interfaces third-party modules build against |
| Controller modules | :nimbus-module-perms | modules/perms/ | Permission groups, tracks, prefix/suffix |
:nimbus-module-display | modules/display/ | Sign + NPC configs | |
:nimbus-module-scaling | modules/scaling/ | Time-based schedules, predictive warmup | |
:nimbus-module-players | modules/players/ | Centralized player tracking, session history | |
:nimbus-module-punishments | modules/punishments/ | Network-wide bans/mutes/kicks/warns | |
:nimbus-module-resourcepacks | modules/resourcepacks/ | Pack registry + scoped assignments | |
:nimbus-module-backup | modules/backup/ | Scheduled tar+zstd snapshots | |
| Plugins | :nimbus-sdk | plugins/sdk/ | Paper/Spigot/Folia SDK deployed to backends |
:nimbus-bridge | plugins/bridge/ | Velocity proxy plugin | |
:nimbus-perms | plugins/perms/ | Permission provider plugin for backends | |
:nimbus-display | plugins/display/ | Sign + NPC renderer for backends | |
:nimbus-punishments-velocity | plugins/punishments-velocity/ | Login/connect enforcement on proxies | |
:nimbus-punishments-backend | plugins/punishments-backend/ | Chat-mute enforcement on backends | |
:nimbus-resourcepacks | plugins/resourcepacks/ | Pack applier for backends |
Controller package layout (nimbus-core/src/main/kotlin/dev/nimbuspowered/nimbus/)
Nimbus.kt / NimbusKt.kt → main() + bootstrap wiring
NimbusVersion.kt → Version read from JAR manifest / gradle.properties
LogRotation.kt → Rotate latest.log to dated archives
api/ → Ktor REST + WebSocket server (NimbusApi, auth, routes/)
cluster/ → NodeManager, ClusterServer (Netty TLS), ClusterWebSocketHandler,
RemoteServiceHandle, RemoteFileProxy, TlsHelper, placement/
config/ → NimbusConfig, GroupConfig, DedicatedConfig, loaders
console/ → JLine3 REPL, CommandDispatcher, commands/
database/ → DatabaseManager, MigrationManager, Tables, MetricsCollector, AuditCollector
doctor/ → Core doctor checks (cluster, config, ports, etc.)
event/ → EventBus (SharedFlow) + sealed NimbusEvent
group/ → ServerGroup runtime state, GroupManager
loadbalancer/ → TcpLoadBalancer, BackendHealthManager, strategies
module/ → ModuleManager (URLClassLoader + ServiceLoader), ModuleContextImpl
proxy/ → ProxySyncManager (tab list, MOTD, chat, maintenance)
scaling/ → Core ScalingEngine + reactive scaling rules
service/ → ServiceManager, ServiceFactory, ProcessHandle, PortAllocator,
StateSyncManager, WarmPoolManager, DedicatedServiceManager
setup/ → First-run SetupWizard
stress/ → StressTestManager (simulated load)
system/ → OS-level helpers (memory reader, process inspector)
template/ → TemplateManager, ConfigPatcher, SoftwareResolver, ServiceDeployer,
ModScanner
update/ → UpdateChecker (GitHub Releases auto-updater)
velocity/ → VelocityConfigGen (auto-manage proxy server list)Module API (modules/api/src/main/kotlin/dev/nimbuspowered/nimbus/module/)
NimbusModule, ModuleContext, ModuleCommand, Migration, PluginDeployment,
DoctorCheck, CommandOutput. Everything a third-party module needs sits here
with no dependency on nimbus-core — see Custom Modules.
Agent package layout (nimbus-agent/src/main/kotlin/)
Agent.kt (main), AgentConfig, AgentRuntime (WS loop), AgentStateStore
(JSON persistence), JavaResolver, LocalProcessHandle, LocalProcessManager,
SetupWizard, TemplateDownloader, plus TlsHelper for fingerprint verification.
See Agent Node for details.
Shared protocol (nimbus-protocol)
ClusterMessage— sealed class with every controller ↔ agent message typeStateManifest— per-service state-sync manifest (manifest-compare delta sync)clusterJson— preconfiguredkotlinx.serialization.json.Json
ServiceHandle lives in nimbus-core/service/ (not in protocol) — it abstracts
over local ProcessHandle and RemoteServiceHandle.
Bootstrap flow
When Nimbus starts (Nimbus.kt → nimbusMain()), components are initialized in this order:
1. Log rotation → Rotate latest.log to dated archives
2. Setup wizard → First-run interactive setup (if needed)
3. Config loading → Parse config/nimbus.toml + config/groups/*.toml
4. API token generation → Auto-generate if missing
5. Directory creation → Ensure templates/, services/, logs/, etc. exist
6. Plugin deployment → Extract nimbus-bridge.jar, nimbus-sdk.jar
7. Component init → EventBus, ServiceRegistry, PortAllocator,
TemplateManager, GroupManager, ProxySyncManager
8. Group loading → Parse group configs into GroupManager
9. Module loading → ModuleManager scans modules/*.jar, creates one
URLClassLoader per JAR, discovers NimbusModule via
module.properties (fallback: ServiceLoader), calls
init(). Modules register commands, routes, plugin
deployments, event formatters, doctor checks, and
migrations during init.
10. Migrations run → MigrationManager applies every registered migration
in ascending version order (core 1–999, modules
1000+). Baseline migrations are marked applied
without executing.
11. ServiceManager → Wire up all dependencies, register in ModuleContext
via registerService() so late-binding loops in modules
(e.g. backup, scaling) can resolve it.
12. Modules enable() → Called on every loaded module after ServiceManager
is registered. State-sync manager for [group.sync]
groups is initialized here.
11. ScalingEngine → Create (but DON'T start yet — deferred until after boot)
12. NimbusApi → Create (but don't start) Ktor server
13. NodeManager → If cluster.enabled: init cluster coordination
14. ClusterServer → If cluster.enabled: separate Ktor server on agent_port for agent WebSocket connections
15. TcpLoadBalancer → If loadbalancer.enabled: init Layer-4 TCP proxy
with BackendHealthManager (health checks, circuit breaker,
connection draining, idle timeout, connection limit)
16. Shutdown hook → Register SIGTERM/SIGINT handler
17. NimbusConsole.init() → Banner, event listener
18. Api.start() → Start Ktor HTTP server (+ /cluster WS if cluster enabled)
19. LoadBalancer.start() → If enabled: start TCP listener on LB port + health check loop
20. VelocityUpdater → Start periodic update check (first check after 60s)
21. startMinimumInstances → Phased startup:
Phase 1: Start proxy groups, wait for READY (120s timeout)
Phase 2: Start backend groups
22. ScalingEngine.start() → Start periodic scaling loop (AFTER boot completes)
23. Console.start() → JLine3 REPL (blocks until shutdown)The shutdown hook and console REPL provide two paths to shutdown: external signals (SIGTERM) use the hook, while the shutdown command exits the REPL.
Coroutine architecture
Nimbus uses kotlinx-coroutines for all async work. No raw threads are created in the core module.
// Root scope with SupervisorJob (child failures don't cancel siblings)
val scope = CoroutineScope(SupervisorJob() + Dispatchers.Default)
// EventBus uses the shared scope for event dispatch
val eventBus = EventBus(scope)
// Scaling engine launches a long-running coroutine
val scalingJob = scalingEngine.start() // Returns a JobKey patterns:
- SupervisorJob -- A failing coroutine (e.g., scaling error) doesn't bring down the whole system
- Dispatchers.Default -- CPU-bound work (most coordination)
- Dispatchers.IO -- Used in ServiceManager for process I/O and file operations
- MutableSharedFlow -- EventBus uses a shared flow with
extraBufferCapacity = 64
Event bus
The event bus is the central communication backbone. All state changes are published as sealed class events.
// Emitting an event (suspend function)
eventBus.emit(NimbusEvent.ServiceReady(serviceName = "BedWars-1", groupName = "BedWars"))
// Subscribing to a specific event type
eventBus.on<NimbusEvent.ServiceReady> { event ->
println("${event.serviceName} is ready!")
}
// Getting the raw SharedFlow
val flow: SharedFlow<NimbusEvent> = eventBus.subscribe()Event types
Events are modeled as sealed class NimbusEvent in event/Events.kt:
| Category | Events |
|---|---|
| Service lifecycle | ServiceStarting, ServiceReady, ServiceDraining, ServiceStopping, ServiceStopped, ServiceCrashed, ServiceRecovered, ServicePrepared, WarmPoolReplenished, ServiceDeployed |
| Scaling | ScaleUp, ScaleDown, PlacementBlocked |
| Custom state | ServiceCustomStateChanged |
| State sync | SyncCompleted, SyncFailed |
| Players | PlayerConnected, PlayerDisconnected, PlayerServerSwitch |
| Groups | GroupCreated, GroupUpdated, GroupDeleted |
| Messaging | ServiceMessage |
| Modules (generic) | ModuleEvent(moduleId, type, data) — the envelope used for every module-defined event (permissions, punishments, resource packs, backups, scaling) |
| Module lifecycle | ModuleLoaded, ModuleEnabled, ModuleDisabled |
| Updates | ProxyUpdateAvailable, ProxyUpdateApplied, NimbusUpdateAvailable, NimbusUpdateApplied |
| Proxy sync | TabListUpdated, MotdUpdated, PlayerTabUpdated, ChatFormatUpdated |
| Maintenance | MaintenanceEnabled, MaintenanceDisabled |
| Stress | StressTestUpdated |
| Dedicated | DedicatedCreated, DedicatedDeleted |
| Config | ConfigReloaded |
| API | ApiStarted, ApiStopped, ApiWarning, ApiError |
| Cluster | ClusterStarted, NodeConnected, NodeDisconnected, NodeHeartbeat |
| Load balancer | LoadBalancerStarted, LoadBalancerStopped, LoadBalancerBackendHealthChanged |
| CLI sessions | CliSessionConnected, CliSessionDisconnected |
Events are broadcast via the REST API's WebSocket endpoint (/api/events) so external systems and plugins can react to them.
Module-defined events (e.g. PUNISHMENT_ISSUED, RESOURCE_PACK_CREATED,
BACKUP_COMPLETED, SMART_SCHEDULE) are emitted as ModuleEvent(moduleId, type, data)
and serialized on the wire as MODULE_EVENT frames. See
reference/events for the envelope and per-module subtypes.
Process management
Each running service is wrapped in a ServiceHandle interface that abstracts process management across local and remote nodes:
ProcessHandle-- Local JVM subprocess started viaProcessBuilderwith inherited environmentRemoteServiceHandle-- Proxy for services running on remote agent nodes, communicating via the cluster WebSocket protocol
Static services run on the controller by default. They have persistent data (worlds, configs) stored in services/static/. With [group.sync] enabled = true, static services can be placed on remote agent nodes — the controller's canonical store under services/state/<name>/ acts as the source of truth. Dynamic and proxy services can be distributed across any node without additional configuration.
Both implementations provide:
- stdout/stderr -- Captured asynchronously via
SharedFlow<String>for ready detection and logging - Ready detection -- Regex matching against stdout (default pattern:
Done) - Graceful shutdown -- Sends
stopcommand via stdin, then waits, then force-kills
State persistence & process recovery
The controller persists local service state to state/services.json via ControllerStateStore. On restart, ProcessHandle.adopt() re-attaches to running processes by PID. On Windows, it falls back to checking if the process is a Java process when commandLine() is unavailable. Recovered services are registered before temp directory cleanup to protect their working directories from being removed.
Service states
PREPARING → PREPARED → STARTING → READY → DRAINING → STOPPING → STOPPED
↓ ↑
CRASHED ── (restart?) ─────────────┘| State | Description |
|---|---|
PREPARING | Template being copied to service directory |
PREPARED | Template copy complete; service is staged in the warm pool, waiting to be started |
STARTING | JVM started, scanning stdout for ready pattern |
READY | Server accepting players |
DRAINING | Load balancer draining existing connections; no new players routed here |
STOPPING | Graceful shutdown in progress |
STOPPED | Clean shutdown complete |
CRASHED | Unexpected exit (triggers restart if restart_on_crash = true) |
Velocity config generation
The VelocityConfigGen class keeps the proxy's velocity.toml in sync with running backend services:
- Finds all
READYbackend services (non-VELOCITY groups) - Builds the
[servers]section withServiceName = "127.0.0.1:port"entries - Sets the
trylist to lobby servers (groups containing "lobby") - Replaces the
[servers]and[forced-hosts]sections invelocity.toml
This runs automatically whenever a backend service becomes READY or stops.
Plugin deployment
Templates are user-owned in 0.9.x+. Nimbus never writes managed plugin JARs
into templates/global/plugins/ or templates/global_proxy/plugins/. Everything
you drop into templates/ stays yours.
Every Nimbus-managed plugin (the SDK plus each module's companion plugin) is
deployed at runtime on every service prepare by
ServiceFactory.resolveModulePlugins(). A plugin registered by a module with
ModuleContext.registerPluginDeployment(PluginDeployment(...)) is copied from
the module JAR's resource path into the service's plugins/ directory with
REPLACE_EXISTING — so deletions self-heal on next start and updates roll out
automatically.
| Plugin | Registered by | Target |
|---|---|---|
nimbus-sdk.jar | Core (always) | Backend services (Paper family, Folia) |
nimbus-bridge.jar | Core (Velocity) | Proxy services |
nimbus-perms.jar | nimbus-module-perms | Backend services |
nimbus-display.jar | nimbus-module-display | Backend services |
nimbus-punishments.jar | nimbus-module-punishments | Velocity (login/connect enforcement) |
nimbus-punishments-backend.jar | nimbus-module-punishments | Backend (chat-mute enforcement) |
nimbus-resourcepacks.jar | nimbus-module-resourcepacks | Backend services |
| FancyNpcs | On-demand download | Only when the display module is used with NPCs |
Each PluginDeployment can gate on minMinecraftVersion and declares a
PluginTarget (BACKEND vs VELOCITY), so version-specific or proxy-only
plugins are deployed only where they belong. See
Custom Modules for the registration
API.
Shutdown order
When Nimbus shuts down:
1. Cancel scaling engine job
2. Send graceful shutdown to all agents (if cluster enabled, wait up to 30s)
3. Stop TCP load balancer (if enabled)
4. Stop REST API server
5. Disconnect cluster nodes (if enabled)
6. Stop all local services (via ServiceManager.stopAll()):
a. Dynamic (game) services first
b. Static backend (lobby) services
c. Proxy services last
7. Stop cluster WebSocket server (if enabled)
8. Cancel coroutine scopeThis order ensures players are moved to lobbies before lobbies shut down, and proxies stay alive as long as possible to handle redirects.
Technology choices
| Component | Technology | Rationale |
|---|---|---|
| Language | Kotlin 2.1 | Coroutines, sealed classes, null safety |
| Build | Gradle + Shadow | Fat JAR packaging with embedded plugins |
| Config | ktoml | Native TOML parsing for Kotlin |
| Console | JLine 3 | Rich terminal with history, completion, ANSI colors |
| Async | kotlinx-coroutines | Structured concurrency, no callback hell |
| HTTP server | Ktor (CIO) | Lightweight, coroutine-native |
| HTTP client | Ktor Client (CIO) | Server JAR downloads |
| SDK client | Java HttpClient | Zero dependencies for Paper/Velocity plugins |
| Cluster protocol | kotlinx-serialization JSON | Typed messages between controller and agents |
| Load balancer | Java NIO (ServerSocketChannel) | Non-blocking TCP proxy with zero dependencies |
| Module system | ServiceLoader + URLClassLoader | Dynamic module loading without framework overhead |
The core design principle is no frameworks -- no Spring, no dependency injection containers. Components are wired directly in Nimbus.kt, making the startup path explicit and debuggable.
Next steps
- Custom Modules -- Building third-party controller modules
- Smart Scaling Module -- Time-based schedules + predictive warmup
- Players Module -- Centralized tracking, session history
- Punishments Module -- Bans/mutes/kicks + Bridge enforcement
- Resource Packs Module -- Registry, scoped assignments, hosting
- Backup Module -- tar+zstd pipeline, GFS retention, MANIFEST.sha256
- SDK -- Backend server plugin API
- Bridge Plugin -- Velocity proxy plugin
- Agent Node -- Remote node process
- Protocol -- Cluster message types
- WebSocket Reference -- Event stream protocol