Bootable USB installers: self-provisioning appliance images for the stack host and RigForge miner

[VijitSingh97]

Goal

Ship bootable USB installers so a user can go from bare server hardware → plug in a USB → a fully-provisioned machine, with little or no manual Linux setup. Two targets:

1. Stack host (this repo) — nodes + P2Pool + proxy + dashboard + Tor + Caddy.
2. RigForge miner (rigforge) — a tuned XMRig worker.

Each needs a provisioning system: read a user-supplied config off the USB and self-configure on first boot, so deployment is "edit one file, flash, boot."

Why the two targets need different shapes

They look similar ("boot a server, run our software") but are opposites under the hood:

	Stack host	RigForge miner
State	Stateful — 100s of GB of Monero/Tari chain data, Tor onion keys, generated secrets	Stateless — no wallet, no chain, nothing to persist
Workload	Docker Compose, 7 services	One process: tuned XMRig → stratum
Needs	apt ecosystem (docker, jq, openssl), GRUB/HugePages tuning, big disk	RandomX perf only: HugePages (1G/2M), MSR, NUMA, perf governor
Lifecycle	Long-lived, installed-to-disk, updated in place	Reflash-to-update is fine; could even run diskless/RAM-resident
Ideal form	Full distro installed to disk	Minimal/immutable appliance

This difference is the crux of the tooling choice.

Deployment options — the DIY path stays first-class

The installer/image work is additive, not a replacement. We support two ways to deploy, and neither deprecates the other:

1. Do-it-yourself (existing): bring your own Ubuntu/Debian, git clone, and run ./pithead setup (host) / ./rigforge.sh (miner). Full control over OS, partitioning, existing machines, and bespoke setups — this stays the documented default for people who want it.
2. Flash-and-go (new, this epic): a prebuilt bootable image you flash to a USB and boot — for people who want zero Linux setup.

Non-goal: removing or degrading the script-based install. pithead / rigforge.sh remain the single source of truth for provisioning logic; the image runs those same steps under the hood, so it's one codebase, two delivery methods (they can't drift apart).

Is Buildroot the right solution?

Not as a single answer. Buildroot fits the miner, not the stack host:

• Miner — yes, plausibly. Buildroot produces a tiny, immutable, fixed-function image, can boot diskless from USB into RAM, and reflash-to-update suits a stateless appliance. Cost: XMRig + all tuning must be packaged in-tree, cross-compile toolchain, no runtime package manager, and updates mean rebuilding/reflashing. Proven for real appliances (e.g. Home Assistant OS is Buildroot-based).
• Stack host — no. It wants Docker + a large stateful workload + apt-style flexibility, which all fight Buildroot's model (no package manager, reflash updates, heavy from-source rebuilds). Buildroot can run Docker but it's unconventional and high-maintenance here.

So: use different tools per repo. And for the miner, a lighter option than Buildroot may win (see Alpine diskless below).

Options evaluated (with research)

Stack host (install a real distro to disk):

• Ubuntu autoinstall (Subiquity + cloud-init) — ⭐ best MVP fit. We already target Ubuntu Server 24.04 and have an idempotent pithead. A standard Ubuntu installer ISO + a nocloud user-data/meta-data on the USB does an unattended install, then late-commands/first-boot runs pithead setup. Lowest effort, reuses everything.
• Debian preseed / debian-installer — same idea on Debian; viable, but no advantage over Ubuntu given our target.
• bootc / image mode (bootable containers) — build the OS from a Containerfile, install to disk via bootc-image-builder/Anaconda, update from a container registry. Very aligned (our workload is already containers) and synergises with Release & versioning structure (single-product releases via GHCR) #44 (GHCR releases). Newer, RHEL/Fedora-centric — a bigger shift from the Ubuntu+pithead model.
• Fedora CoreOS / Flatcar (Ignition) — purpose-built immutable container hosts, provisioned by a first-boot Ignition/Butane config. Great fit for "a server that runs containers," but rewrites our provisioning model and drops the pithead investment.
• mkosi — build a bespoke immutable Ubuntu/Debian/Fedora disk image (golden image). Good for a reproducible image; still need a flash/dd deploy path.
• Golden disk image + dd — simplest deploy (image is the installer), but hardware variance (disk size, NIC) needs cloud-init/growpart handling.

Miner (minimal appliance):

• Alpine diskless mode (apkovl) — ⭐ likely the miner sweet spot. Boots from USB into RAM, config persisted as a small apkovl overlay on the USB. Tiny and immutable like Buildroot but far less effort (no from-source OS build; XMRig via package or a build step).
• Buildroot — smallest/most locked-down; best if we want a true embedded appliance and accept the build complexity above.
• Yocto — industrial-grade, overkill here.

Recommendation — hybrid + phased

• Stack host → start with an Ubuntu autoinstall ISO (reuses pithead, fastest to ship). Keep bootc / image-mode on the roadmap as the immutable, registry-updated evolution (pairs with Release & versioning structure (single-product releases via GHCR) #44).
• Miner → ship the same Ubuntu-autoinstall approach first for speed (reuses rigforge.sh), then a true minimal appliance — Alpine diskless (preferred) or Buildroot — for the perf/immutability/diskless win.
• One config-injection UX shared by both.

Provisioning / config injection (the "provision system per each")

We support three ways to configure a freshly-booted appliance, leading with the easiest. The web form + config schema are shared with #33 (the day-2 config editor) — build the config UI/API once, invoke it in two contexts (first-boot vs running stack).

1. First-boot web setup wizard (primary — the dumbed-down path). On boot, if no valid config exists, the appliance serves a small router-style setup page. The user browses to it (printed URL / http://<ip> / *.local), fills a short form (wallets, monero.mode, pool, hostname) → it writes config.json and runs pithead setup, then hands off to the real dashboard. This is the Stack config editor: change any setting from the dashboard, applied via pithead (incl. P2Pool mode hot-swap) #33 config form served in "unprovisioned" mode.
2. config.json on the USB (pre-seed — power users / fleets). Drop a config.json on the USB config partition (FAT32, editable on any OS); first boot consumes it and skips the wizard. Ideal for flashing many identical rigs.
3. curl one-liner (headless / automation). A documented curl that POSTs the filled-in config to the setup endpoint — browser-less, scriptable.

Mechanics & considerations:

• The wizard runs before the full stack is up (no valid config yet) — a minimal first-boot provisioner mode (a sub-mode of the dashboard, or a tiny separate service) that retires once provisioned. It's a web front-end to pithead's existing ensure_config_exists + setup.
• Secure the setup window: the page is LAN-reachable and collects wallet addresses — gate it (only-when-unprovisioned + a token printed to console/serial, or a temporary setup AP).
• Miner (rigforge#1) needs ~one field (the stack endpoint): same pattern with a one-field form, or skip the UI via mDNS auto-discovery.
• Per-base hooks still run the provisioner at first boot: cloud-init (Ubuntu) · Ignition (CoreOS/Flatcar) · kickstart (bootc) · a first-boot systemd unit (Buildroot/Alpine).
• Both pithead and rigforge.sh are already idempotent and config-file driven, so the glue stays thin.

Cross-cutting concerns

• UEFI vs legacy BIOS, and Secure Boot (signed shim/bootloader) support.
• Destructive disk selection — installing wipes a disk; need a safe target-selection / confirmation story.
• AVX2 requirement (both stack mining prep and the miner) — detect and warn/abort.
• Networking — DHCP by default; static-IP option for the stack host (the dashboard hostname).
• Secrets on the USB — the stack config carries wallet addresses; consider an encrypted config partition or a post-boot prompt.
• Updates — stack: apt + pithead upgrade (or image/registry updates if we go bootc); miner: reflash or in-place.
• Build pipeline & artifacts — where ISOs/images are built (GH Actions can build Ubuntu ISOs; Buildroot builds are heavy/long), where they're hosted, plus checksums + signing (ties into Release & versioning structure (single-product releases via GHCR) #44).

Phasing

1. MVP — stack host Ubuntu autoinstall ISO. Config-injected, runs pithead setup at first boot. Manual build, documented.
2. Miner autoinstall ISO — same pipeline + rigforge.sh.
3. Minimal miner appliance — Alpine diskless (or Buildroot) for perf/immutability/diskless.
4. CI-built, versioned, checksummed image artifacts per release — wire into Release & versioning structure (single-product releases via GHCR) #44; validate on the Comprehensive end-to-end infrastructure test matrix on a real Ubuntu server (full Monero + Tari node) #54 real-server matrix.
5. (Stretch) Immutable stack host via bootc / image mode with registry updates.

Acceptance criteria

• Booting a prepared USB on bare metal yields a reachable, fully-provisioned stack host (or miner) with only a pre-edited config file and no interactive Linux install.
• Documented build → flash → configure → boot steps for each target.
• Released image artifacts are checksummed (and ideally signed).
• Miner: a documented reflash/update path.
• AVX2 / unsupported-hardware is detected and surfaced rather than silently failing.

Open questions

• Scope split: keep this as one umbrella epic, and mirror the miner-specific work into a rigforge issue (consistent with Extract the worker provisioning kit (worker/) into its own repository #71's repo separation)?
• Long-term host direction: stay Ubuntu+pithead, or move to an immutable container-OS (bootc / CoreOS / Flatcar)?
• Miner appliance: Alpine diskless vs Buildroot?
• Secrets: how to protect wallet addresses on a removable USB.
• Update model: image-based (reflash / registry) vs in-place (apt + pithead upgrade).

Related

• Release & versioning structure (single-product releases via GHCR) #44 — release & versioning structure; ISOs/images become release artifacts (and a registry-update path if we go bootc/image-mode).
• Comprehensive end-to-end infrastructure test matrix on a real Ubuntu server (full Monero + Tari node) #54 — E2E infra test matrix on a real server; installer images should be validated there.
• Extract the worker provisioning kit (worker/) into its own repository #71 — RigForge is now its own repo; the miner-appliance implementation likely belongs there.
• Add Kubernetes support #17 — Kubernetes support; an alternative (cluster) deploy target, complementary to bare-metal USB.

Research sources

• Ubuntu autoinstall (Subiquity/cloud-init): https://canonical-subiquity.readthedocs-hosted.com/en/latest/intro-to-autoinstall.html
• Buildroot manual (rootfs overlay, post-build): https://buildroot.org/downloads/manual/manual.html
• Buildroot real-world appliance (Home Assistant OS): https://developers.home-assistant.io/docs/operating-system/getting-started/
• Alpine diskless mode (apkovl): https://wiki.alpinelinux.org/wiki/Diskless_Mode
• Read-only rootfs + /etc overlay (Bootlin): https://bootlin.com/blog/systemd-read-only-rootfs-and-overlay-file-system-over-etc/
• Flatcar Ignition provisioning: https://www.flatcar.org/docs/latest/provisioning/ignition/
• Immutable distro comparison (FCOS/Flatcar/Talos/Bottlerocket): https://dev.to/matheus_releaserun/container-optimized-linux-distributions-compared-flatcar-bottlerocket-talos-and-fedora-coreos-4fj2
• mkosi: https://mkosi.systemd.io/
• bootc install to bare metal: https://bootc.dev/bootc/bootc-install.html
• bootc-image-builder (ISO/disk outputs): https://github.com/osbuild/bootc-image-builder

[VijitSingh97]

Miner counterpart filed: p2pool-starter-stack/rigforge#1 — Immutable, flashable miner image (ISO).

Resolves the "scope split" open question: this issue (#77) stays the umbrella + stack-host installer; the miner appliance lives in the RigForge repo (consistent with #71's separation).

Per maintainer steer, the miner track targets an immutable flashable image (Alpine diskless → Buildroot) rather than a scripted install — see rigforge#1. The same immutable direction is the long-term goal for the stack host too (bootc / image-mode, phase 5 above).

[VijitSingh97]

Runtime decision (appliance & deploy boundary)

Decided: the bootable appliance runs the stack on a single-host container runtime — Docker Compose today, and possibly Podman + Quadlet if/when we go full image-mode (evaluating in #78). It does not embed Kubernetes.

Rationale: the stack host is a single-instance workload — one monerod, one p2pool, one tari, one dashboard, nothing that scales horizontally. Orchestration across nodes/replicas (the reason k8s exists) adds control-plane overhead, extra state, boot fragility, and a networking rework on the appliance for zero benefit on one box; Compose restart: policies + the existing healthchecks already cover single-node self-heal. The miner (rigforge#1) is bare-metal-only and never a k8s workload.

Where Kubernetes fits: a bring-your-own-cluster target for people who already run k8s — layered on the #44 published images, not embedded in the appliance. The thing we share across deploy methods is the images (#44), not the orchestrator.

Consequence: closing #17 (Kubernetes) as not-planned — see its thread. It can return later as community-contributed manifests/Helm over the #44 images if real demand shows up.

[VijitSingh97]

First-boot configuration ties into #33 — reuse one config UI/API

Per maintainer steer: the appliance's first-boot configuration should reuse the same config form + config API as #33 (don't build two). #33 is the day-2 "edit the stack from a web page" capability; first-boot is the same form served in "unprovisioned" mode. I've reframed #33 accordingly and updated the Provisioning / config injection section above to lead with the web wizard.

Three input paths, leading with the easiest (for a bootable image we want maximally dumbed-down):

1. First-boot web setup wizard (primary). Boot → if no valid config, the appliance serves a small router-style setup page → user browses to it (printed URL / http://<ip> / *.local), fills a short form (wallets, mode, pool) → writes config.json + runs stack.sh setup → hands off to the real dashboard. This is the Stack config editor: change any setting from the dashboard, applied via pithead (incl. P2Pool mode hot-swap) #33 form in unprovisioned mode.
2. config.json on the USB (pre-seed). Power users / fleets drop a config.json on the USB config partition; first boot consumes it and skips the wizard. Best for flashing many identical rigs.
3. curl one-liner (headless/automation). A documented curl that POSTs the filled-in config to the setup endpoint — browser-less, scriptable.

Support all three; wizard-first for the appliance, config.json/curl kept for power users (consistent with "the DIY path stays first-class").

Considerations:

• The wizard runs before the full stack is up (no valid config yet) → a minimal first-boot provisioner mode (a sub-mode of the dashboard, or a tiny separate service) that retires once provisioned. It's effectively a web front-end to stack.sh's existing ensure_config_exists + setup.
• Secure the setup window — it's LAN-reachable and collects wallet addresses: gate it (only-when-unprovisioned + a token printed to console/serial, or a temporary setup AP).
• Miner (rigforge#1) needs ~one field (the stack endpoint) → same pattern with a one-field form, or skip the UI entirely via mDNS auto-discovery.