Skip to content

docs(architecture): ADR 0018 — Hermit raw NIC access strategy (Proposed) - #4

Closed
LeTuR wants to merge 1 commit into
mainfrom
docs/adr-0018-hermit-raw-nic-access
Closed

docs(architecture): ADR 0018 — Hermit raw NIC access strategy (Proposed)#4
LeTuR wants to merge 1 commit into
mainfrom
docs/adr-0018-hermit-raw-nic-access

Conversation

@LeTuR

@LeTuR LeTuR commented May 25, 2026

Copy link
Copy Markdown
Owner

Summary

Phase-1 PR B research surfaced a real architectural blocker: upstream hermit-os/kernel does not expose any public API for raw L2 frame access. Every relevant type (NetworkDriver trait, concrete drivers, the NIC static holding the smoltcp Interface) is pub(crate). The application-facing hermit-abi surface is strictly POSIX sockets — no AF_PACKET, no SOCK_RAW, no FFI hatch.

This is incompatible with ADR 0017's "smoltcp wire layer only, app owns the data path" without an additional decision.

This PR records the finding and proposes the path forward as a Proposed ADR. Implementation (PR B/C) is on hold until this lands as Accepted.

The recommendation

Fork hermit-os/kernel at v0.13.2 with a minimal patch (~50 lines):

  • Flip three pub(crate) → pub (the NetworkDriver trait and the device statics)
  • Add a pub fn take_nics() -> Vec<Box<dyn NetworkDriver>> accessor
  • Pin via [patch.crates-io] in Cargo.toml

This preserves every reason ADR 0017 was made (Rust-native, single toolchain, kernel-space app, smoltcp::wire available, QEMU/KVM + Firecracker deployment targets) while unblocking phase-1 PR B/C.

Alternatives weighed and rejected

Option Why rejected
Revert to Unikraft We don't consume its C library catalog (which was its value-add over a thin Rust unikernel); ADR 0017's memory-safety reasoning still applies
Custom no_std + virtio-pmd Months of work for table-stakes infra Hermit already gives us
Wait for upstream PR Puts thurward's roadmap on the upstream maintainer's schedule; upstream may reasonably reject "expose my whole networking stack". (Will pursue in parallel as a courtesy, not the critical path.)
Hermit + std::net sockets A firewall is a forwarding device; the socket layer terminates flows — fundamentally can't pass them through
Linux userspace + AF_PACKET/DPDK Violates ADR 0017's substrate decision wholesale; defeats the "unikernel firewall" thesis

Open question for deciders

The ADR ends with three confirmation questions:

  1. Acceptable to add a forks/hermit-kernel/ subtree + a [patch.crates-io] line pinning a thurward-controlled fork?
  2. Comfortable with the rebase-against-upstream cost? (~½ day per Hermit release, ~monthly cadence)
  3. Should the upstream PR (option C) block PR B/C, or run in parallel as a courtesy?

Once accepted, ADR 0017's "Consequences" section gets a pointer here and ADR 0010's supply-chain section documents the new versions.lock field.

Test plan

  • CI green (markdown lint + link check on the new ADR)
  • Reviewer agrees with the rejection reasoning for the four alternatives
  • Decision recorded on the three open questions in the ADR

@LeTuR
docs(architecture): adr 0018 — hermit raw NIC access strategy (proposed)
82240e3 
Phase-1 PR B research (against the live `hermit-os/kernel` HEAD,
2026-05-25) confirmed that upstream Hermit exposes no public API for
raw L2 frame access — every relevant type (`NetworkDriver` trait,
concrete drivers, the `NIC` static holding the smoltcp `Interface`)
is `pub(crate)`, and the application-facing `hermit-abi` surface is
strictly POSIX sockets (no `AF_PACKET`, no `SOCK_RAW`).

This is incompatible with ADR 0017's "smoltcp wire layer only, app
owns the data path" without an additional decision: do we fork
upstream, switch substrates, or relax the architecture?

ADR 0018 records the research finding and proposes the path forward.

The recommendation is **a minimal fork of `hermit-os/kernel`** that
flips three `pub(crate)` items to `pub` and adds a `take_nics()`
accessor — ~50 lines of delta — pinned via `[patch.crates-io]`. This
preserves every reason ADR 0017 was made (Rust-native, single
toolchain, kernel-space app, smoltcp::wire available) while
unblocking phase-1 PR B/C.

Four alternatives are weighed and rejected: revert to Unikraft,
custom no_std + virtio-pmd, wait for upstream PR, or relax to
Hermit's std::net (which fundamentally can't satisfy a forwarding
device). Each rejection cites the specific architecture
requirement(s) it would violate.

ADR 0017 gets a forward-pointer to 0018 in its header so readers see
the open question alongside the original decision.

Status: Proposed — the ADR ends with three confirmation questions
for the deciders. Until those are answered, phase-1 PRs B and C are
on hold.
@LeTuR

LeTuR commented May 25, 2026

Copy link
Copy Markdown
Owner Author

Closing — superseded by a substrate-pivot ADR (Hermit's lack of public raw-frame access is being treated as a substrate mismatch, not a patch-able gap). New ADR coming on a fresh branch.

@LeTuR LeTuR closed this May 25, 2026
@LeTuR LeTuR mentioned this pull request May 25, 2026
Merged
3 tasks
LeTuR added a commit that referenced this pull request May 25, 2026
@LeTuR
docs(architecture): adr 0018 — substrate pivot to unikraft (proposed)
9ccd17c 
Phase-1 PR B research found that upstream `hermit-os/kernel` does not
expose a public raw-frame API — every relevant layer is `pub(crate)`
and the application-facing `hermit-abi` is POSIX sockets only. A
firewall is a forwarding device; the socket layer terminates flows
and cannot pass them through.

The previous draft of ADR 0018 (closed PR #4) proposed forking
hermit-os/kernel. That's been rejected on reconsideration: a fork
papers over a substrate mismatch (Hermit's model is "app uses
network stack"; ours is "app IS network stack") rather than fixing
the underlying choice.

This ADR pivots the substrate back to **Unikraft** (the original
ADR 0001 choice) while keeping Rust as the application language
and smoltcp::wire as the parser library. Both of those clauses
from ADR 0017 stay live; only the Hermit substrate clause is
superseded.

The two reasons ADR 0017 cited for choosing Hermit over Unikraft
don't survive contact with the implementation:
- "Rust-native single toolchain" was true but the Rust we'd write
  has to fight Hermit's design intent.
- "Unikraft's value-add (C library catalog) is mostly libs we no
  longer use" was wrong: we DO want one specific piece —
  `lib-uknetdev` — which is exactly the raw-frame device API the
  thurward data path needs per ADR 0013 and chapter 02.

Four alternatives weighed and rejected: fork hermit-os/kernel,
distroless Linux + Rust + AF_PACKET, wait for upstream Hermit PR,
MirageOS. Each rejection cites the architecture requirement(s) it
would violate or the asymmetric cost.

Status: Proposed. The ADR ends with three confirmation questions
for the deciders. On acceptance, a follow-up PR updates ADR 0017
(status note), ADR 0001 (status note), the chapter docs that name
Hermit specifically, ADR 0010's supply-chain section, and the
in-progress `versions.lock` (currently lists Hermit TBDs from PR
#3).
LeTuR added a commit that referenced this pull request May 25, 2026
@LeTuR
docs(architecture): adr 0018 — substrate pivot to unikraft (proposed)
93a305b 
Phase-1 PR B research found that upstream `hermit-os/kernel` does not
expose a public raw-frame API — every relevant layer is `pub(crate)`
and the application-facing `hermit-abi` is POSIX sockets only. A
firewall is a forwarding device; the socket layer terminates flows
and cannot pass them through.

The previous draft of ADR 0018 (closed PR #4) proposed forking
hermit-os/kernel. That's been rejected on reconsideration: a fork
papers over a substrate mismatch (Hermit's model is "app uses
network stack"; ours is "app IS network stack") rather than fixing
the underlying choice.

This ADR pivots the substrate back to **Unikraft** (the original
ADR 0001 choice) while keeping Rust as the application language
and smoltcp::wire as the parser library. Both of those clauses
from ADR 0017 stay live; only the Hermit substrate clause is
superseded.

The two reasons ADR 0017 cited for choosing Hermit over Unikraft
don't survive contact with the implementation:
- "Rust-native single toolchain" was true but the Rust we'd write
  has to fight Hermit's design intent.
- "Unikraft's value-add (C library catalog) is mostly libs we no
  longer use" was wrong: we DO want one specific piece —
  `lib-uknetdev` — which is exactly the raw-frame device API the
  thurward data path needs per ADR 0013 and chapter 02.

Four alternatives weighed and rejected: fork hermit-os/kernel,
distroless Linux + Rust + AF_PACKET, wait for upstream Hermit PR,
MirageOS. Each rejection cites the architecture requirement(s) it
would violate or the asymmetric cost.

Status: Proposed. The ADR ends with three confirmation questions
for the deciders. On acceptance, a follow-up PR updates ADR 0017
(status note), ADR 0001 (status note), the chapter docs that name
Hermit specifically, ADR 0010's supply-chain section, and the
in-progress `versions.lock` (currently lists Hermit TBDs from PR
#3).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@LeTuR