diff --git a/PLAN.md b/PLAN.md index 0e94a7b..123d2c5 100644 --- a/PLAN.md +++ b/PLAN.md @@ -165,6 +165,7 @@ Jesli `/data/etc/microinit.json` nie istnieje: utworz katalog rekursywnie, zapis | startWaitSecs | int | 0 | dla daemon=true: ile sekund czekac po starcie; proces musi przezyc okno (wyjscie = Failed). Przy 0: krotki grace SysV, exit w successExitCodes = Running | | shutdownWaitSecs | int | 5 | po sygnale stop / stopCmd czekaj N s, potem SIGKILL | | background | bool | false | true=start rownolegle, nie blokuje boot | +| orderPriority | int | 100 | wsrod gotowych uslug (dependsOn OK): nizszy = wczesniejszy start; remis = alfabetycznie po name | | dependsOn | [string] | [] | uslugi, ktore musza osiagnac settled przed startem | | cmd | string? | null | bazowa komenda (fallback dla start/stop/restart) | | startCmd/stopCmd/restartCmd | string? | null | konkretne komendy | @@ -216,9 +217,11 @@ stateDiagram-v2 ## 7. Sortowanie i kolejnosc boot - Buduj DAG z `dependsOn`. Wykryj cykle -> blad na console + log. -- **Foreground** uslugi startuja w porzadku topologicznym, sekwencyjnie (kazda czeka na settled przed nastepna). -- **Background** uslugi startuja rownolegle w osobnych watkach, gdy ich `dependsOn` sa settled. +- Wsrod uslug gotowych (indegree 0) wybieraj zawsze min `(orderPriority, name)` — nizszy priority wczesniej; remis alfabetycznie. +- **Foreground** uslugi startuja w tej kolejnosci topologicznej, sekwencyjnie (kazda czeka na settled przed nastepna). +- **Background** uslugi startuja rownolegle (najpierw wszystkie background w kolejnosci topo, potem foreground). - To rozdziela szybkie uslugi (foreground: mount, network, sysctl) od dlugich demonow (background: grafana, bigfred). +- Shutdown: odwrotnosc kolejnosci startu. ## 8. IPC - socket uniksowy diff --git a/docs/README.md b/docs/README.md index 3c8a08a..9ed220a 100644 --- a/docs/README.md +++ b/docs/README.md @@ -5,7 +5,7 @@ Guides and references for running and integrating **microinit**. | Document | Audience | Contents | |----------|----------|----------| | [Operator guide](operator.md) | Linux admins / device operators | Everyday CLI, boot, quick config overview | -| [Configuration](configuration.md) | Administrators | JSON files, drop-ins in subfolders, hot reload, service fields | +| [Configuration](configuration.md) | Administrators | JSON files, drop-ins, service fields, **service ordering** | | [Service lifecycle](service-lifecycle.md) | Administrators | States over time; dependency when a service restarts at boot | | [Using as supervisord](using-as-supervisord.md) | Container / VM admins | `supervise` mode; PHP-FPM + NGINX with drop-ins and hot reload | | [Control socket API](api.md) | Integrators / UI / scripts | Unix socket framing, request/response JSON | diff --git a/docs/architecture.md b/docs/architecture.md index 73e0c07..e89b066 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -77,10 +77,13 @@ Each service has, among other fields: - whether it is a daemon (long-lived) or a one-shot job, - whether to restart on crash, - dependencies (`dependsOn`), +- start order among ready peers (`orderPriority`, default 100 — lower first), - wait times on start and on shutdown. **Dependencies:** when start is requested (boot or CLI) and a `dependsOn` service is not yet `Running`/`Succeeded`, the dependent enters **`waiting_for_dependency`** and stays there until every dependency is ready, then starts automatically. A manual `stop` (or disable) cancels that wait — satisfying the dependency later does **not** restart a stopped service. +**Ordering:** boot builds a topological order from `dependsOn`, always picking the ready service with the lowest `(orderPriority, name)`. See [Service ordering](configuration.md#service-ordering). + **Execution model:** one **monitor thread per service**. A shared main loop handles signals, zombie reaping, and socket commands. Child processes are collected by a central **reaper** (so multiple places do not race on `waitpid`). External control (CLI, UI) goes through a **Unix socket** (default `$DATA_DIR/run/microinit.sock`, hub `/data/run/microinit.sock`): start, stop, restart, enable/disable, list, logs. The CLI `--socket` flag sets the same path for both the daemon and clients. Parent directories for the socket are created automatically. diff --git a/docs/configuration.md b/docs/configuration.md index b05c9e1..b920a0d 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -102,6 +102,7 @@ Most operators only edit **`services`**. "restartBackoff": 2, "startWaitSecs": 1, "shutdownWaitSecs": 5, + "orderPriority": 100, "dependsOn": ["network"], "cmd": "/etc/init.d/myapp", "cwd": "/" @@ -126,6 +127,7 @@ Or explicit commands: "name": "network", "daemon": false, "restart": false, + "orderPriority": 30, "cmd": "/etc/init.d/network" } ``` @@ -144,6 +146,7 @@ Success → `succeeded`. Failure → `failed`. | `startWaitSecs` | After start, wait; if process dies in window → `failed` | | `shutdownWaitSecs` | After stop, wait then `SIGKILL` | | `background` | Parallel start at boot | +| `orderPriority` | Among ready services, lower starts earlier (default `100`; equal → alphabetical name) | | `dependsOn` | These must be `running` or `succeeded` first | | `livenessProbe` | Optional health check; failure triggers restart | @@ -212,6 +215,99 @@ Requires **microinit restart** (on PID 1 hosts: reboot): --- +## Service ordering + +Boot and shutdown order come from a topological sort of `dependsOn`, with +`orderPriority` as the tie-breaker among services that are currently ready. + +**Rules (in order):** + +1. `dependsOn` builds a hard DAG — a service cannot start before its + dependencies. +2. Among services with all dependencies satisfied (ready), pick the lowest + `orderPriority` first. +3. Equal `orderPriority` → alphabetical `name`. +4. That list is split into foreground / background for boot parallelism. +5. Shutdown uses the **reverse** of the start order. + +Default when the field is omitted: **`100`**. + +Do **not** confuse this with drop-in **merge** order (files sorted by path +alphabetically; later file wins for the same service name) — that only decides +which definition is kept, not boot order. + +### Example 1 — priority only (no deps) + +```json +[ + { "name": "cron", "orderPriority": 50 }, + { "name": "sysctl", "orderPriority": 10 }, + { "name": "watchdog", "orderPriority": 20 } +] +``` + +Start order: `sysctl` → `watchdog` → `cron`. + +### Example 2 — same priority → alphabetical + +```json +[ + { "name": "redis", "orderPriority": 100 }, + { "name": "alloy", "orderPriority": 100 }, + { "name": "microdns", "orderPriority": 100 } +] +``` + +Start order: `alloy` → `microdns` → `redis`. + +### Example 3 — `dependsOn` blocks; then priority among ready + +```json +[ + { "name": "network", "orderPriority": 30 }, + { "name": "app", "orderPriority": 10, "dependsOn": ["network"] }, + { "name": "cron", "orderPriority": 50 } +] +``` + +1. Ready at start: `network` (30), `cron` (50) → start **`network`**. +2. After `network`: ready `app` (10) and `cron` (50) → **`app`**, then **`cron`**. + +Start order: `network` → `app` → `cron`. +(`app` has a lower priority than `cron`, but cannot overtake `network`.) + +### Example 4 — shutdown = reverse + +For example 3: stop `cron` → `app` → `network`. + +### Example 5 — mini hub + +| name | orderPriority | dependsOn | +|------|---------------|-----------| +| sysctl | 10 | — | +| network | 30 | — | +| redis | 100 | network | +| bigfred | 300 | network, redis | +| grafana | 400 | — | + +Start: `sysctl` → `network` → `redis` → `bigfred` → `grafana`. +Shutdown: `grafana` → `bigfred` → `redis` → `network` → `sysctl`. + +(On a real hub image, `grafana` also `dependsOn` `victoriametrics`; the table +above is simplified.) + +### Example 6 — `background` vs `orderPriority` + +`orderPriority` only orders the topological list. At boot, microinit still +starts **all** `background: true` services first (fire-and-forget, in topo +order), then foreground services sequentially. A low `orderPriority` on a +foreground service does **not** make it start before background peers. + +See also [Operator guide](operator.md) (boot sequence) and +[Architecture](architecture.md). + +--- + ## Dependencies ```json @@ -227,6 +323,7 @@ microinit start --force myapp # debugging only ``` Boot example with restarts: [Service lifecycle](service-lifecycle.md). +Ordering of who gets `Start` first: [Service ordering](#service-ordering). --- diff --git a/docs/operator.md b/docs/operator.md index 5108347..8492a17 100644 --- a/docs/operator.md +++ b/docs/operator.md @@ -121,6 +121,7 @@ If `startCmd` is set, it is used instead of `cmd start`. Prefer **`exec` of the | `startWaitSecs` | After start, wait this long; if the process dies in that window → `failed`. Use `1` (or more) when the start command **stays** as the service process | | `shutdownWaitSecs` | After stop, wait then `SIGKILL` | | `background` | At boot, start in parallel (does not wait for the console `[ OK ]` sequence as long) | +| `orderPriority` | Among ready services, lower starts earlier (default `100`; equal → name A–Z). See [Service ordering](configuration.md#service-ordering) | | `dependsOn` | Other service names that must be `running` or `succeeded` first | | `env` / `cwd` | Extra environment and working directory | | `livenessProbe` | Optional periodic check. Exactly one of `cmd`, `httpUrl`, or `tcpAddr`. Shared: `interval` (default `60`), `timeout` (default `5`). `cmd` uses `successExitCodes` (default `[0]`); `httpUrl` uses `httpMethod` (default `GET`) and `httpAcceptedCodes` (default `[200]`); `tcpAddr` is `host:port`. Runs while `running` / `succeeded` / `failed`; failure re-runs start | @@ -281,11 +282,11 @@ microinit start --force redis # debugging only 1. Kernel starts `/sbin/init` (microinit). 2. **Early-boot** (mount `/data`, seed config, …). 3. Config loaded from disk. -4. Enabled services start (`dependsOn` order; `background: true` in parallel). +4. Enabled services start in topological order (`dependsOn` hard edges; among ready services lower `orderPriority` first, then name). `background: true` services are started first (in that order), then foreground sequentially. Details: [Service ordering](configuration.md#service-ordering). 5. Console `[ OK ]` / `[ FAIL ]`; getty. 6. IPC socket; JSON files watched for reload. -On shutdown in **`init`** mode (`shutdown -r`, IPC `shutdown`, SIGTERM, …): services stop in reverse dependency order, then the **unmount** script runs (unbind mounts / umount `/data`), then reboot or power-off. +On shutdown in **`init`** mode (`shutdown -r`, IPC `shutdown`, SIGTERM, …): services stop in **reverse** of that start order, then the **unmount** script runs (unbind mounts / umount `/data`), then reboot or power-off. In **`supervise`** mode there is no early-boot, getty, late unmount, or machine reboot — only the supervisor + socket (good for containers). On shutdown it stops services, syncs, and exits. diff --git a/docs/sdk/golang.md b/docs/sdk/golang.md index 7fe3dc0..d2be979 100644 --- a/docs/sdk/golang.md +++ b/docs/sdk/golang.md @@ -39,8 +39,11 @@ Service configs may include `labels` (`map[string]string`). Convention for embed ```go svc := config.WithCreatedBy(config.ServiceDef{ Name: "worker", StartCmd: "exec /usr/bin/worker", + OrderPriority: config.IntPtr(100), // optional; omitted → daemon default 100 }, "my-app") // writes labels: {"created-by":"my-app"} +// OrderPriority: among ready services, lower starts earlier (nil omits the field; +// microinit then applies default 100). Pointer to 0 is serialized as 0. // Filter a List() result: for _, s := range list { diff --git a/docs/service-lifecycle.md b/docs/service-lifecycle.md index a401887..717c50d 100644 --- a/docs/service-lifecycle.md +++ b/docs/service-lifecycle.md @@ -2,6 +2,8 @@ This page shows **what happens over time** — not just a list of states, but a realistic boot story: one service keeps crashing and restarting, and another waits until the first one is finally healthy. +Which service receives `Start` first at boot is decided by [Service ordering](configuration.md#service-ordering) (`dependsOn` + `orderPriority`). + --- ## States you will see diff --git a/examples/microinit.json.example b/examples/microinit.json.example index dc071d5..265fad5 100644 --- a/examples/microinit.json.example +++ b/examples/microinit.json.example @@ -28,7 +28,8 @@ "stopCmd": null, "restartCmd": null, "env": {}, - "cwd": "/" + "cwd": "/", + "orderPriority": 30 }, { "name": "redis", @@ -50,7 +51,8 @@ "stopCmd": null, "restartCmd": null, "env": {}, - "cwd": "/" + "cwd": "/", + "orderPriority": 100 }, { "name": "remote-icmp", @@ -78,7 +80,8 @@ "capabilities": [ "CAP_NET_RAW" ] - } + }, + "orderPriority": 210 } ] } diff --git a/go/config/types.go b/go/config/types.go index 957eed9..13c9ea7 100644 --- a/go/config/types.go +++ b/go/config/types.go @@ -10,6 +10,8 @@ type ServiceDef struct { RestartBackoff *int `json:"restartBackoff,omitempty"` StartWaitSecs *int `json:"startWaitSecs,omitempty"` ShutdownWaitSecs *int `json:"shutdownWaitSecs,omitempty"` + // OrderPriority: among ready services, lower starts earlier (default 100). + OrderPriority *int `json:"orderPriority,omitempty"` DependsOn []string `json:"dependsOn,omitempty"` StartCmd string `json:"startCmd,omitempty"` StopCmd string `json:"stopCmd,omitempty"` diff --git a/man/man5/microinit.json.5.mdoc b/man/man5/microinit.json.5.mdoc index f3bcafe..b6c00af 100644 --- a/man/man5/microinit.json.5.mdoc +++ b/man/man5/microinit.json.5.mdoc @@ -108,6 +108,13 @@ Seconds to wait after stop signal / stopCmd before sending .Pq default 5 .It Cm background Start in parallel; does not block boot (default false) +.It Cm orderPriority +Among currently ready services, lower values start earlier +.Pq default 100 . +Equal values fall back to alphabetical +.Cm name . +Does not override +.Cm dependsOn . .It Cm dependsOn Services that must be .Cm Running diff --git a/man/man8/microinit.8.mdoc b/man/man8/microinit.8.mdoc index d13ae43..d6513a4 100644 --- a/man/man8/microinit.8.mdoc +++ b/man/man8/microinit.8.mdoc @@ -83,7 +83,10 @@ and apply .Xr microinit.services.enabled-override.json 5 .It Topologically sort services by -.Cm dependsOn +.Cm dependsOn , +breaking ties with +.Cm orderPriority +.Pq lower first; equal → alphabetical name .It Start enabled services; print .Bq OK diff --git a/src/config.rs b/src/config.rs index 214a1b6..28d0340 100644 --- a/src/config.rs +++ b/src/config.rs @@ -245,6 +245,10 @@ pub struct ServiceConfig { pub shutdown_wait_secs: u64, #[serde(default)] pub background: bool, + /// Among currently ready services (`dependsOn` satisfied), lower values + /// start earlier. Equal values fall back to alphabetical name. Default 100. + #[serde(default = "default_order_priority")] + pub order_priority: u64, #[serde(default)] pub depends_on: Vec, #[serde(default)] @@ -318,6 +322,10 @@ fn default_cwd() -> String { "/".to_string() } +fn default_order_priority() -> u64 { + 100 +} + const LABEL_KEY_MAX: usize = 63; const LABEL_VALUE_MAX: usize = 253; @@ -855,6 +863,7 @@ pub fn example_config() -> Config { start_wait_secs: 0, shutdown_wait_secs: 5, background: false, + order_priority: 30, depends_on: vec![], cmd: Some("/etc/init.d/network".into()), start_cmd: None, @@ -887,6 +896,7 @@ pub fn example_config() -> Config { start_wait_secs: 0, shutdown_wait_secs: 5, background: false, + order_priority: 100, depends_on: vec!["network".into()], cmd: Some("/etc/init.d/redis".into()), start_cmd: None, @@ -910,6 +920,7 @@ pub fn example_config() -> Config { start_wait_secs: 0, shutdown_wait_secs: 5, background: true, + order_priority: 210, depends_on: vec!["network".into()], cmd: None, start_cmd: Some(format!( diff --git a/src/graph.rs b/src/graph.rs index 92cb9da..3f06cf0 100644 --- a/src/graph.rs +++ b/src/graph.rs @@ -1,13 +1,21 @@ //! Dependency graph: DAG build + topological sort. -use std::collections::{HashMap, HashSet, VecDeque}; +use std::collections::{BTreeSet, HashMap, HashSet}; use crate::config::ServiceConfig; use crate::error::{Error, Result}; /// Topologically sorted service names. Detects cycles. +/// +/// Among services that are currently ready (`indegree == 0`), always picks the +/// global minimum of `(order_priority, name)`. Lower `orderPriority` starts +/// earlier; equal priority falls back to alphabetical name. pub fn topological_sort(services: &[ServiceConfig]) -> Result> { let names: HashSet<&str> = services.iter().map(|s| s.name.as_str()).collect(); + let prio: HashMap<&str, u64> = services + .iter() + .map(|s| (s.name.as_str(), s.order_priority)) + .collect(); let mut indegree: HashMap<&str, usize> = HashMap::new(); let mut adj: HashMap<&str, Vec<&str>> = HashMap::new(); @@ -31,33 +39,25 @@ pub fn topological_sort(services: &[ServiceConfig]) -> Result> { } } - let mut queue: VecDeque<&str> = indegree + let mut ready: BTreeSet<(u64, &str)> = indegree .iter() .filter(|(_, &d)| d == 0) - .map(|(&n, _)| n) + .map(|(&n, _)| (*prio.get(n).unwrap_or(&100), n)) .collect(); - // Stable order among ready nodes: alphabetical among those with indegree 0 initially, - // then FIFO. For determinism, sort the initial queue. - let mut initial: Vec<&str> = queue.drain(..).collect(); - initial.sort_unstable(); - queue.extend(initial); let mut order = Vec::with_capacity(services.len()); - while let Some(n) = queue.pop_front() { + while let Some((_, n)) = ready.pop_first() { order.push(n.to_string()); if let Some(children) = adj.get(n) { - let mut next_ready = Vec::new(); for &c in children { let Some(d) = indegree.get_mut(c) else { continue; }; *d -= 1; if *d == 0 { - next_ready.push(c); + ready.insert((*prio.get(c).unwrap_or(&100), c)); } } - next_ready.sort_unstable(); - queue.extend(next_ready); } } diff --git a/src/liveness.rs b/src/liveness.rs index b09ccf2..d8d0ce0 100644 --- a/src/liveness.rs +++ b/src/liveness.rs @@ -145,6 +145,7 @@ mod tests { start_wait_secs: 0, shutdown_wait_secs: 5, background: false, + order_priority: 100, depends_on: vec![], cmd: None, start_cmd: Some("true".into()), diff --git a/tests/config_test.rs b/tests/config_test.rs index 2ae9578..ac05a05 100644 --- a/tests/config_test.rs +++ b/tests/config_test.rs @@ -17,6 +17,7 @@ fn minimal_svc(name: &str) -> ServiceConfig { start_wait_secs: 0, shutdown_wait_secs: 5, background: false, + order_priority: 100, depends_on: vec![], cmd: Some(format!("/bin/echo-{name}")), start_cmd: None, @@ -59,6 +60,7 @@ fn resolve_cmd_fallback() { start_wait_secs: 0, shutdown_wait_secs: 5, background: false, + order_priority: 100, depends_on: vec![], cmd: Some("/etc/init.d/redis".into()), start_cmd: None, @@ -89,6 +91,7 @@ fn resolve_explicit_cmds_prefer_over_cmd() { start_wait_secs: 0, shutdown_wait_secs: 5, background: false, + order_priority: 100, depends_on: vec![], cmd: Some("/ignored".into()), start_cmd: Some("start-me".into()), @@ -119,6 +122,7 @@ fn resolve_restart_falls_back_to_stop_and_start() { start_wait_secs: 0, shutdown_wait_secs: 5, background: false, + order_priority: 100, depends_on: vec![], cmd: None, start_cmd: Some("do-start".into()), @@ -147,6 +151,7 @@ fn resolve_start_errors_without_cmds() { start_wait_secs: 0, shutdown_wait_secs: 5, background: false, + order_priority: 100, depends_on: vec![], cmd: None, start_cmd: None, diff --git a/tests/graph_test.rs b/tests/graph_test.rs index 542c3e4..466a4af 100644 --- a/tests/graph_test.rs +++ b/tests/graph_test.rs @@ -7,6 +7,10 @@ use microinit::error::Error; use microinit::graph::*; fn svc(name: &str, deps: &[&str], bg: bool) -> ServiceConfig { + svc_prio(name, deps, bg, 100) +} + +fn svc_prio(name: &str, deps: &[&str], bg: bool, order_priority: u64) -> ServiceConfig { ServiceConfig { name: name.into(), enabled: true, @@ -17,6 +21,7 @@ fn svc(name: &str, deps: &[&str], bg: bool) -> ServiceConfig { start_wait_secs: 0, shutdown_wait_secs: 5, background: bg, + order_priority, depends_on: deps.iter().map(|s| (*s).to_string()).collect(), cmd: Some(format!("/bin/true-{name}")), start_cmd: None, @@ -103,3 +108,47 @@ fn independent_roots_sorted_alphabetically() { ]; assert_eq!(topological_sort(&services).unwrap(), vec!["a", "m", "z"]); } + +#[test] +fn independent_roots_sorted_by_order_priority() { + let services = vec![ + svc_prio("cron", &[], false, 50), + svc_prio("sysctl", &[], false, 10), + svc_prio("watchdog", &[], false, 20), + ]; + assert_eq!( + topological_sort(&services).unwrap(), + vec!["sysctl", "watchdog", "cron"] + ); +} + +#[test] +fn equal_priority_falls_back_to_name() { + let services = vec![ + svc_prio("redis", &[], false, 100), + svc_prio("alloy", &[], false, 100), + svc_prio("microdns", &[], false, 100), + ]; + assert_eq!( + topological_sort(&services).unwrap(), + vec!["alloy", "microdns", "redis"] + ); +} + +#[test] +fn depends_on_blocks_then_priority_wins_among_ready() { + // network(30) and cron(50) ready first → network; then app(10) and cron → app, cron. + let services = vec![ + svc_prio("network", &[], false, 30), + svc_prio("app", &["network"], false, 10), + svc_prio("cron", &[], false, 50), + ]; + assert_eq!( + topological_sort(&services).unwrap(), + vec!["network", "app", "cron"] + ); + assert_eq!( + shutdown_order(&services).unwrap(), + vec!["cron", "app", "network"] + ); +} diff --git a/tests/service_test.rs b/tests/service_test.rs index 34a3dcd..ba300f0 100644 --- a/tests/service_test.rs +++ b/tests/service_test.rs @@ -16,6 +16,7 @@ fn cfg() -> ServiceConfig { start_wait_secs: 0, shutdown_wait_secs: 5, background: false, + order_priority: 100, depends_on: vec![], cmd: None, start_cmd: Some("true".into()), diff --git a/tests/supervisor_test.rs b/tests/supervisor_test.rs index 24b5a27..e641cb7 100644 --- a/tests/supervisor_test.rs +++ b/tests/supervisor_test.rs @@ -37,6 +37,7 @@ fn job(name: &str, start: &str, deps: &[&str], enabled: bool) -> ServiceConfig { start_wait_secs: 0, shutdown_wait_secs: 5, background: false, + order_priority: 100, depends_on: deps.iter().map(|s| (*s).to_string()).collect(), cmd: None, start_cmd: Some(start.into()), @@ -63,6 +64,7 @@ fn daemon_cfg(name: &str, start: &str) -> ServiceConfig { start_wait_secs: 0, shutdown_wait_secs: 5, background: true, + order_priority: 100, depends_on: vec![], cmd: None, start_cmd: Some(start.into()),