Strengthen the position: runtime strength as a set, a portable conformance suite, and legible denials

[Lutherwaves]

openblox answers "how is a sandbox isolated" and declines "where does it run". That line has held up well. What it does not yet answer is the question every evaluator asks first, in one of two forms:

• Why not write this myself? It's a docker run invocation.
• Why not use wazero / Kata / bubblewrap / a microVM instead?

Both are fair, and neither is answered anywhere in the repository today. This issue proposes treating that as a gap in the engineering, not in the marketing — the answers are testable claims, and the work below is mostly about making them testable.

1. Runtime strength is modelled as a singleton, and that is now wrong

pkg/sandbox/options.go documents WithRuntime as:

Setting this to the host's default runtime trades away the isolation openblox exists to provide.

That is correct for runc. It is incorrect for Kata Containers, which is an OCI runtime registered in Docker exactly as runsc is, and which provides a stronger boundary than gVisor — a separate kernel per sandbox rather than a user-space one.

So the current model is binary (runsc = safe, everything else = unsafe) where the reality is ordered:

Boundary	Kernel surface reached by guest
namespaces only (runc)	the host kernel, in full
gVisor (runsc)	the Sentry; host kernel only past B1 and B2
microVM (Kata)	a separate guest kernel

THREAT_MODEL.md §B1/B2 is written specifically against gVisor, which is right, but it means a user who has deliberately chosen a stronger runtime currently reads documentation telling them they have weakened their deployment.

Proposal: make the approved set {runsc, kata} rather than the singleton runsc, keep refusing everything else for untrusted profiles, and say in SECURITY.md which boundary each provides. ErrRuntimeUnavailable semantics do not change — failing closed is the property worth keeping, and it is orthogonal to how many runtimes are approved.

2. Run the adversarial suite against Kata in CI

The payoff here is evidence, not support. pkg/docker/adversarial_integration_test.go asserts 21 named properties — metadata-address reachability, capability sets, noexec/nosuid mounts, path traversal, output flooding, setsid escape — and the suite is currently coupled to one runtime.

Running it against two independent boundaries is substantially more persuasive than running it against one, and it converts a claim about gVisor into a claim about openblox's defaults. It would also catch the case where a property holds only because of a gVisor implementation detail rather than because openblox configured it.

3. Extract the suite so it can run against any backend

This is the largest item and the one with the most leverage.

The suite currently tests pkg/docker. If it instead ran against anything satisfying sandbox.Backend, it could be pointed at:

• a runc-based configuration, to demonstrate what is lost
• Kata, per §2
• someone else's implementation, including a hand-rolled one

That last case is the interesting one. The honest answer to "why not write my own?" is "you can, and at ~3.3k lines you should feel free — but here is the suite, tell me how it scores." An implementation is copyable in a week. A shared set of properties that implementations are measured against is not, and it is more useful to the ecosystem than another library.

This also gives the project a defensible position that does not depend on being the only option, which matters for something MIT-licensed and deliberately small.

4. Write down why not to become a multi-backend abstraction

A recurring suggestion will be to support wazero (or any in-process WASM runtime) behind the same interface. There is a good case for WASM on its merits — no host syscalls at all, genuinely a stronger boundary than gVisor for workloads that fit it — but it is a poor fit for this interface:

• no POSIX filesystem, so Files has no meaning
• no subprocesses, so StartProcess has no meaning
• no pip install, no apt, no arbitrary userland — which is most of what the target workload does
• no ports to proxy, so preview links have no meaning

An interface where several methods return ErrUnsupported depending on configuration is a signal the abstraction is wrong. More importantly, the guarantee degrades from "gVisor, and it never silently falls back" to "depends which backend was configured" — which reintroduces, one level up, precisely the visible-vs-unreachable distinction openbloxd exists to eliminate.

Kata is a different case and belongs in scope, because it is the same integration point (an OCI runtime under Docker) and the same shape of guarantee. WASM is a different product that happens to share a problem statement.

Proposal: a short ADR recording this, so the question is answered once rather than in every thread. A written "no" with reasoning is more useful to a prospective contributor than silence.

5. Make denials structured, and machine-readable

This is the item with no prior art that I can find, and the one most specific to the workload openblox is actually for.

When sandboxed code hits a policy today, it fails the way the kernel fails: a DNS lookup hangs because there is no resolver, a write returns bare EROFS, a process dies at a memory ceiling with no indication which one. That is correct behaviour and terrible feedback.

For human operators it means debugging by inference. For generated code it is worse: a model that receives errno 30 has no way to know it hit a deliberate policy rather than a bug, so it retries the same operation, or "fixes" it by rewriting working code. The denial is the single most informative event in the sandbox's life and it is currently the least legible.

Emitting denials as structured records the caller can surface —

{"denied": "network", "attempted": "example.com:443",
 "because": "egress=none", "profile": "code-exec"}

— turns the most frustrating property of sandboxing into a usable signal. A caller can return it to the model, which can then adapt rather than loop; an operator gets an answer instead of a hang.

The plumbing is not free (gVisor's netstack and the container's own events are the sources, and not all denials are observable from outside the guest), so this likely wants a spike before an estimate. Worth noting that this is the kind of requirement that surfaces from running the thing in anger rather than from reading the spec — it comes out of Blox using openblox as its sandbox backend, where the loop-on-denial behaviour is observable in practice.

Why these five together

§1 and §2 fix a correctness gap and buy evidence cheaply. §3 converts the existing test suite from internal quality into something the ecosystem can use. §4 keeps the scope honest while those happen. §5 is the piece that is specific to running generated code rather than untrusted code generally, and is where the project has something to contribute that is not already solved elsewhere.

None of them require openblox to become larger in the way that matters — §1, §2 and §4 are close to net-zero on the line count that the README's central claim depends on.

Happy to split this into five issues if that tracks better; filed as one because the argument only holds together.

[Lutherwaves]

Picking up §1 and §4. Two corrections to §1 as filed, both of which make it smaller.

There is no approved-runtime set in the code. §1 proposes "make the approved set {runsc, kata} rather than the singleton runsc", but nothing enforces a singleton. Backend.assertRuntime (pkg/docker/backend.go:265) only asks Docker whether a runtime of that name is registered:

if _, ok := info.Runtimes[runtime]; !ok {
	return fmt.Errorf("%w: runtime %q is not registered with the docker daemon", sandbox.ErrRuntimeUnavailable, runtime)
}

Any string passes. Spec has no runtime validation, and internal/daemon/config.go:295 forwards a profile's runtime verbatim. Kata already works today — WithRuntime("kata-runtime") on a host with Kata registered creates a sandbox under a separate guest kernel and always has. So §1 carries no code change and no ErrRuntimeUnavailable change to preserve: failing closed is a property of "is it registered", which is orthogonal to strength and stays exactly as it is.

The doc comment §1 leads with is not the wrong one. pkg/sandbox/options.go:210 reads:

WithRuntime overrides the isolation runtime. The default, runsc, is gVisor. Setting this to the host's default runtime trades away the isolation openblox exists to provide.

That names runc, and for runc it is true. I quoted it in the issue as saying "any non-runsc runtime", which is a misreading on my part — it is imprecise (it implies the alternatives are all downward) but not false.

The claims that are false are both in SECURITY.md:

• L90, the security-sensitive-configuration table: "runtime / WithRuntime | runsc (default) | Any other runtime runs untrusted code on the host kernel."
• L107, under What you must not do: "Do not set a runtime other than runsc for untrusted code."

Under Kata neither holds — guest syscalls reach a separate guest kernel, not the host's. A user who has chosen the stronger boundary is told by the project's security document that they have broken their deployment, and the second line tells them to undo it.

So §1 reduces to: correct those two claims, sharpen the options.go comment to describe the ordering rather than a binary, and put the boundary table somewhere normative. README.md:146 ("never falls back to a weaker boundary") and docs/production.md:52 ("never falls back to runc") are both accurate as written and I am leaving them alone.

Keeping this as one issue rather than splitting, per the argument in the body. §1 + §4 going up as a single PR.

[Lutherwaves]

Reopened: #62 closed this through its merge message, but only §2 is done. Status: §1 and §4 in #58, §3 in #61, §2 in #62. Still open: §5 (structured denials). Follow-up from §2: crashed-sandbox-recovers-through-create assumes a guest can kill its own init, which only gVisor allows (see THREAT_MODEL.md, Under Kata).