diff --git a/.claude/skills/emu-dev/SKILL.md b/.claude/skills/emu-dev/SKILL.md new file mode 100644 index 000000000..e91769b31 --- /dev/null +++ b/.claude/skills/emu-dev/SKILL.md @@ -0,0 +1,83 @@ +--- +name: emu-dev +description: Write C code for the emulator (emu) correctly — the kproc kill path and blocking rules, CONF/mkdevlist device wiring, the incumbent-driver rule, and driver/policy layering. Use when adding or modifying emu devices, host drivers, or anything under emu// or emu/port/. +--- + +# Writing emulator C code + +The `limbo-dev` skill covers *building* this layer (platform scripts, +C-library rebuild order); this one covers writing it. The porting +history is in `docs/LESSONS-LEARNED.md`; read the incumbent code before +any of it. + +## Blocking: the kill path only interrupts syscalls + +Inferno kills a hosted proc via `oshostintr()` — on macOS, +`pthread_kill(SIGUSR1)` (`emu/MacOSX/os.c:413`); the signal handler +makes blocking *syscalls* return `EINTR`. It does **nothing** to a +`pthread_cond_wait`, a mutex wait, or any other in-process blocking: the +wait resumes as if nothing happened. A driver that parks in a condvar +wait is unkillable from inside Inferno. + +Rules, each learned from a real deadlock: + +- **Host blocking goes inside `osenter()`/`osleave()`** so the proc is + in the right state to be interrupted and the kill is delivered as + `EINTR`. `emu/FreeBSD/audio.c` is the exemplar — every blocking read + and write is wrapped. +- **Prefer the emu `Rendez`** (`emu/port/dat.h`) for waits the Inferno + side must be able to break; it participates in the proc model. +- **Never hold a `QLock` across a blocking wait.** The waiter blocks + with the lock held, the closer blocks on the lock, and the device is + wedged until the emulator dies. Take the lock, update state, release, + *then* wait. +- Design for the hostile posture first: the device that never delivers + (mic permission denied, disconnected hardware) is the case your + blocking model must survive. + +## A driver not named in the CONF is dead code + +Devices are wired by the platform CONF file (e.g. `emu/MacOSX/emu`): +each line names the device and its source files (`audio audio-sdl3`), +and `mkdevlist` generates the device table, `$DEVS`, and `$LIBS` from +it at build time. Consequences: + +- Adding a `.c` file without a CONF entry ships dead code — and if its + link flags or tests land anyway, the tree carries live assertions + about a driver that never runs. **Wire the CONF in the same PR or + don't ship the driver**; its SYSLIBS additions and driver-specific + test assertions go with it, both ways. +- One platform, one driver per device. If you must replace an + incumbent, flip the CONF in the same PR and delete or explicitly + fence the loser — the tree does not carry two rival drivers for one + device. + +## The incumbent rule + +Before writing a rival to an existing driver, read the incumbent — +especially its header comments — for the regressions it already fixed. +Those fixes are behavioral contracts (drain-on-close semantics, ctl +verbs callers already write, buffer-size negotiation), and a rewrite +that silently re-loses them is a regression factory. You own not +re-losing them: name each inherited fix in your PR description and say +where your version preserves it. + +## Drivers deliver events; policy lives in the window system + +No keyboard chords, gesture recognition, or UI bindings in a device +driver or the portable SDL3 layer — a chord swallowed at the driver +level is swallowed for every platform, every layout, and every hosted +app (modifier masks like `SDL_KMOD_ALT` include right-Alt, which is +AltGr on European layouts). Deliver the raw events; implement bindings +in the window manager, where focus, layout, and app context exist. +Matching smell row in `docs/DESIGN-PRINCIPLES.md`. + +## Hygiene + +- C here is Plan 9/Inferno style: tabs, K&R, match the surrounding + file — not a generic modern house style. +- Error paths release what they acquired: a partial device-start that + leaks its queues or locks turns the *next* open into the deadlock. +- After changing `libinterp/`, `libsec/`, or keyring C, rebuild those + libraries before relinking emu (`limbo-dev` skill has the order) — + a stale archive looks exactly like a VM bug. diff --git a/.claude/skills/limbo-dev/SKILL.md b/.claude/skills/limbo-dev/SKILL.md index 9b750f5a7..c420f8a9a 100644 --- a/.claude/skills/limbo-dev/SKILL.md +++ b/.claude/skills/limbo-dev/SKILL.md @@ -62,7 +62,17 @@ fine. Rebuild the directories that compile against the changed interface Plan 9 `mk`, not GNU make. No `&&` in recipes — use `;` or separate rules. `mk install` copies output into the tracked runtime tree `dis/`; `mk nuke` cleans. Never commit `.dis` from `appl/` or `tests/` (gitignored); -`dis/` changes must come from `mk install`, not manual copies. +`dis/` changes — including `dis/tests/`, which is tracked like the rest of +the runtime tree — must come from `mk install`, not manual copies. + +## Before opening a PR + +Verify every modified file's pre-image matches master tip (rebase, then +`git diff master...HEAD` and check the base blobs). A branch cut from — or +contaminated by — another unmerged branch can carry someone else's change +through a textually clean merge, silently. Then run +`tools/verify-dis-paths.sh` and fill the PR template's design-principles +checklist honestly. ## Running what you built diff --git a/.claude/skills/limbo-test/SKILL.md b/.claude/skills/limbo-test/SKILL.md index d9584e7e0..30790b304 100644 --- a/.claude/skills/limbo-test/SKILL.md +++ b/.claude/skills/limbo-test/SKILL.md @@ -42,6 +42,9 @@ Conventions the docs used to omit: - `mk install` places `.dis` both in `tests/` (where `runner.dis` scans) and `dis/tests/` (what `run-tests.sh` invokes). Compiling manually to only one location makes the test invisible to the other entry point. + **Commit the `dis/tests/` copy** — the runtime tree tracks test + bytecode like the rest of `dis/`; only the `tests/` and `appl/` + intermediates stay untracked. - **CI compiles tests per-file with plain `limbo -I$ROOT/module -o ...`, not via `tests/mkfile`** — a test that needs extra include paths (a special mkfile rule) will compile locally and fail CI's compile gate. @@ -93,6 +96,21 @@ Note: several `tests/host/*_test.sh` are static source-greps (regression pins on C/Limbo source patterns), not runtime tests — read the test before assuming it exercises behavior. +## Norms for new tests + +- **Assert behavior, not source text.** A grep over source is a + regression *pin* — label it as one, and never let it be the only + coverage of a runnable artifact. A grep-shape test cannot notice + that a script invokes a binary the sh path can't resolve; a contract + test that runs the script can. +- **A suite that skips in CI guards nothing in CI.** If a test exits + 77 whenever its environment is absent, say in its header where it + actually runs — and prefer a fake-helper/fake-backend it can drive + hermetically over an environment-gated skip. +- **The contract test is the durable form of "I ran it."** Every + shipped script and every served interface gets one, at the cheapest + tier that can observe it. + ## What CI actually does - Compile failures in the test tree **do** fail the build. diff --git a/.claude/skills/ninep-server/SKILL.md b/.claude/skills/ninep-server/SKILL.md index 4fdc2f2d3..14ed11c8e 100644 --- a/.claude/skills/ninep-server/SKILL.md +++ b/.claude/skills/ninep-server/SKILL.md @@ -85,6 +85,30 @@ Limbo-served mounts. See `docs/postmortems/2026-05-17-newns-vm-lock-deadlock.md` before touching namespace calls near server startup. +## The fid is the session + +Per-client state — busy flags, parked reads, a helper fd, anything that +exists because *this* client is using the service — hangs off the fid, +never off a global. 9P guarantees the server hears about client death: +the mount driver clunks a dead process's fids, and a connection hangup +clunks everything on it. So: + +- Tear session state down in the **Clunk** handler; cancel the + outstanding parked request in **Flush**. If cleanup only happens along + one client's happy-path exit, any other client (a crash, a stray + `cat`) wedges the service permanently. +- A resource that admits one holder at a time is expressed as + **exclusive-open** (`DMEXCL` in the file's mode) so a second open + fails honestly at open time — not as a first-come global busy flag + that only the winner can clear. +- `chatsrv.b` shows the pending-request/Flush-cancel idiom; `gpusrv.b` + shows per-session state behind `clone`. + +And on the client side of any 9P interface: **a write can fail — check +the reply.** A ctl write that gets `Rmsg.Error` and is merely logged to +stderr has not happened; callers that drop data on a failed write turn +a server-side validation into silent data loss. + ## Veltro tool modules (agent-callable tools) The contract is `appl/veltro/tool.m`: five functions — diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 4c92358f7..f4145bd50 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -29,3 +29,4 @@ Design principles (see [docs/DESIGN-PRINCIPLES.md](https://github.com/infernode- - [ ] New service/tool followed a namespace-sketch proposal issue (for non-trivial interfaces) - [ ] Scripts that run inside Inferno are rc-style sh (no `&&`/`||`) and committed executable - [ ] Irreversible/credentialed actions emit audit records; agent-facing effects have provenance emitters +- [ ] Anything this PR downloads or executes from outside the tree is pinned (commit SHA / revision URL) and checksum-verified diff --git a/.github/workflows/style-gate.yml b/.github/workflows/style-gate.yml index 22ab2a596..0aa852b60 100644 --- a/.github/workflows/style-gate.yml +++ b/.github/workflows/style-gate.yml @@ -22,3 +22,20 @@ jobs: persist-credentials: false - name: Run advisory style gate run: sh tools/style-gate.sh + - name: Proposal-link check (soft) + env: + PR_BODY: ${{ github.event.pull_request.body }} + BASE_SHA: ${{ github.event.pull_request.base.sha }} + run: | + git fetch -q --depth=1 origin "$BASE_SHA" + added=$(git diff --name-only --diff-filter=A "$BASE_SHA" HEAD -- \ + 'appl/*9p.b' 'appl/**/*9p.b' 'module/*.m' | tr '\n' ' ') + if [ -z "$added" ]; then + echo "proposal-check: no new file-interface sources" + exit 0 + fi + if printf '%s' "$PR_BODY" | grep -qiE 'proposal|#[0-9]+'; then + echo "proposal-check: new interface sources ($added) and the description references an issue" + else + echo "::warning::This PR adds file-interface sources ($added) but its description links no proposal issue. Non-trivial file interfaces start with a namespace-sketch proposal — see docs/DESIGN-PRINCIPLES.md, 'Proposing a new service or tool', and the 'New Service / Tool Proposal' issue template." + fi diff --git a/.gitignore b/.gitignore index 7eeec7a1c..a4894b557 100644 --- a/.gitignore +++ b/.gitignore @@ -79,8 +79,8 @@ services/httpd/httpd.log /*.sbl # Debug symbols in runtime dis tree /dis/**/*.sbl -# Compiled test bytecode in dis tree -/dis/tests/*.dis +# dis/tests/*.dis is TRACKED (runtime tree ships prebuilt, tests included — +# maintainer ruling 2026-08-21); only this stray lib-tree test artifact is not. /dis/lib/testing_test.dis # CI/local test output test-output.txt diff --git a/AGENTS.md b/AGENTS.md index f4012bdcd..b237f0cc2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -12,7 +12,7 @@ Before designing any new service, tool, or interface, read [docs/DESIGN-PRINCIPL A new service or tool starts with a namespace sketch — the file tree it serves, each file's read/write behavior, an example shell session — proposed in an issue *before* implementation ("Proposing a new service or tool" in DESIGN-PRINCIPLES.md). The file interface is the design; review happens there first. -Task-specific playbooks (compiling Limbo correctly, writing tests, the headless GUI harness, authoring a 9P file server) live in `.claude/skills/*/SKILL.md`. They are plain markdown — useful to any agent or human, not only Claude. +Task-specific playbooks (compiling Limbo correctly, writing tests, the headless GUI harness, authoring a 9P file server, writing emulator C) live in `.claude/skills/*/SKILL.md`. They are plain markdown — useful to any agent or human, not only Claude. ## Build, Test, and Development Commands diff --git a/CLAUDE.md b/CLAUDE.md index 6852deb5d..4e86be33f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -11,7 +11,7 @@ from this tree. New services start with a namespace sketch in an issue, not with code. Limbo is close to Go: [docs/LIMBO-FOR-GO-PROGRAMMERS.md](docs/LIMBO-FOR-GO-PROGRAMMERS.md). Task playbooks (compile loop, tests, headless GUI harness, authoring 9P -servers) are project skills under `.claude/skills/`. +servers, emulator C) are project skills under `.claude/skills/`. ## JIT Compiler Availability @@ -54,9 +54,8 @@ The `dis/` directory (the Inferno runtime tree) **is tracked in git**. This is i However, **build artifacts in source directories are not tracked**: - `appl/**/*.dis` — intermediate build outputs (`.gitignore`d) - `tests/**/*.dis` — compiled tests (`.gitignore`d) -- `dis/tests/*.dis` — test bytecode in the runtime tree (`.gitignore`d) -This means: the runtime tree ships pre-built, but you never commit `.dis` files from `appl/` or `tests/`. +This means: the runtime tree ships pre-built — including `dis/tests/`, whose test bytecode is tracked and updated via `mk install` like the rest of `dis/` — but you never commit `.dis` files from the `appl/` or `tests/` source directories. **The stale bytecode problem:** When a `.m` interface file changes (e.g. `module/widget.m`), every `.dis` compiled against the old interface becomes stale. The Dis VM rejects stale modules at load time with `link typecheck` errors — apps show blank tabs, commands fail to load, and everything looks broken even though the source is fine. This is the most common class of post-pull breakage. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ff71f20ee..0d1f37315 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -36,6 +36,9 @@ To make that painless rather than painful: behavior, and an example shell session — before writing code. A sketch is reviewed in minutes; most design feedback happens there, where changing course is free. Use the "New service / tool proposal" issue template. + This is policy, not a suggestion: PRs that introduce a non-trivial file + interface without a linked proposal issue will be returned for the + sketch first. - **New to Limbo?** It's close to Go, by ancestry: [docs/LIMBO-FOR-GO-PROGRAMMERS.md](docs/LIMBO-FOR-GO-PROGRAMMERS.md) maps the concepts and lists the gotchas. diff --git a/docs/DESIGN-PRINCIPLES.md b/docs/DESIGN-PRINCIPLES.md index 683b256b9..6802aca81 100644 --- a/docs/DESIGN-PRINCIPLES.md +++ b/docs/DESIGN-PRINCIPLES.md @@ -201,6 +201,14 @@ parses at startup or an API it exposes. Where a write changes a value that can also be read, the write format matches the read format. +**Writes are RPCs — on both sides.** The server rejects a bad +request with an error reply, so failures land at the writer. The +caller's dual: a 9P write can fail, and its error is the server +talking to you — a caller that logs the error and continues has +not handled it. Hardening lands as new error replies; validation +you passed last month may reject you today, and your caller must +surface that, never swallow it. + **Sessions via the clone pattern.** When clients need per-session state, serve a `clone` file: reading it allocates a session and returns its id, and a directory of that name appears with the @@ -413,6 +421,41 @@ manifests rot. Remember its scope: nsaudit is advisory; the namespace is still what enforces. +## The host boundary + +The principles above govern the world inside the namespace. A feature +also touches the host — installers, downloads, boot scripts, release +artifacts — and that boundary has its own rules, each of which exists +because a real contribution violated it: + +**Anything fetched is pinned and verified.** An installer that clones +a third-party repository at default-branch HEAD and executes the build +hands code execution on every user's machine to whoever controls that +repository. Clones pin a commit SHA; model and blob downloads pin a +revision URL (not a mutable branch ref); and files are verified +against a SHA256 manifest committed in this tree before they are +installed. A version pin without a hash pin still floats — the hash is +the guarantee, the version is documentation. + +**An installer never builds-and-executes unpinned HEAD.** Same rule, +stated for the case that hides it: "clone, compile, run" is execution +of unreviewed code even though no binary was downloaded. + +**Boot never executes files authored outside the tree.** +Configuration is data: the tree's code parses `key value` lines, or +the running service accepts ctl writes. A host-writable script that +boot runs verbatim is a persistence hook for anything that can write +the file — however convenient it was to generate at install time. + +**Placement is shipping.** The release copy loop packs `dis lib fonts +module services locale usr mnt` into every tarball, .app, and .zip. A +"dev-only" helper placed under `lib/` is in every user's install — its +placement *is* a release decision. Development-rig material lives +outside those directories; `tests/agent-harness/` and its CI +ring-fence are the precedent for keeping something out of artifacts +deliberately. + + ## The honest boundaries A philosophy is trustworthy only if it states its own limits. @@ -453,11 +496,14 @@ any code is written: | JSON crossing a 9P interface | Why isn't the hierarchy the schema? ([9p-data-conventions.md](9p-data-conventions.md)) | | A policy check on a path the caller can name | Why is the path nameable at all? | | A config file a service parses at startup | Why not a `ctl` file — or a mount? | +| Boot (or a service) *executes* a file written outside the tree | Configuration is data: parse `key value` lines or accept ctl writes. A host-writable script run at boot is a persistence hook, not a config. | | A client library other programs must link | Why isn't `open`/`read`/`write` enough? | | A daemon with a bespoke socket protocol | Why not a 9P server? | | A central registry/manager/bus | What existing mechanism composes instead? | | "Access denied" errors reachable by a confined process | Denial should be absence. | | A second copy of a security predicate | Share the function. | +| A global "busy"/"in-use" flag any client can wedge | Per-fid session state, torn down at clunk; exclusivity is `DMEXCL`. (See "The fid is the session" in the ninep-server skill.) | +| UI policy (chords, bindings, gestures) inside a device driver | Drivers deliver events; policy lives in the window system. | | An effectful file an agent can reach, guarded by a flag | Proposal/commit split; put the commit in a namespace the agent doesn't have. | | `&&`, `\|\|`, POSIX loops in Inferno-side scripts | Inferno `sh` is rc-style; see [LIMBO-FOR-GO-PROGRAMMERS.md](LIMBO-FOR-GO-PROGRAMMERS.md#the-shell). | | GNU make / `limbo -o` by hand | `mk` and `tools/compile-limbo.sh`; the module's `PATH` constant decides the target. | diff --git a/docs/LIMBO-FOR-GO-PROGRAMMERS.md b/docs/LIMBO-FOR-GO-PROGRAMMERS.md index bb53bef3e..750cc5d91 100644 --- a/docs/LIMBO-FOR-GO-PROGRAMMERS.md +++ b/docs/LIMBO-FOR-GO-PROGRAMMERS.md @@ -129,6 +129,8 @@ POSIX habits are the most common review comment we make: | `. script.sh` | `run script.sh` | | `VAR=v cmd` | `VAR=v; cmd` | | exit status tests | `raise 'fail:reason'` / `raise 'skip:reason'` | +| `"$var/path"` (empty var → `/path`) | `$var^/path` **raises** `null list in concatenation` when `$var` is empty — guard with `if {! ~ $#var 0}` before any `^` on command-substitution output | +| `cmd > $f` failing quietly in an `if` | a failed redirection **raises past `if {...}`**, aborting the script — to assert "writing $f fails", let the command open it: `cp /dev/null $f` | Scripts begin `#!/dis/sh.dis` and usually `load std`. Quoting is single-quote based, with `''` to embed a quote. A backgrounded @@ -136,6 +138,14 @@ single-quote based, with `''` to embed a quote. A backgrounded remember that in test scripts. `doc/sh.ms` is the full paper; `man/1/sh` the reference. +The last two table rows are semantic, not stylistic — rc-legal +scripts that raise at runtime. Both have shipped: an unguarded +null-list concatenation in a boot probe aborts the entire boot for +every user, and the redirection rule is why the tutorial's contract +test asserts write-denial with `cp` (see +[TUTORIAL-9P-SERVICE.md](TUTORIAL-9P-SERVICE.md), step 5). The +style gate cannot catch these — it checks style, not semantics. + ## Where tests go diff --git a/docs/TUTORIAL-9P-SERVICE.md b/docs/TUTORIAL-9P-SERVICE.md index 6bfdfc9cd..fe122a05f 100644 --- a/docs/TUTORIAL-9P-SERVICE.md +++ b/docs/TUTORIAL-9P-SERVICE.md @@ -195,6 +195,14 @@ On a booted system with `mntgen` serving `/mnt`, you would mount at `/mnt/count` directly. Note the last line: the bad verb failed *at the writer*, with the server's error text. +That cuts both ways. When you are the *caller* of a 9P interface, a +write is an RPC and its error reply is the server talking to you — +a caller that logs the error to stderr and carries on has not +handled it, and if it already consumed the data it meant to write, +it has silently lost it. Servers get hardened over time; validation +your write passed last month may reject it today. Check the reply, +surface the failure, and never discard the payload on error. + ## Step 5 — The contract test @@ -252,7 +260,11 @@ ones: `appl/cmd/webfs.b`. - **Blocking reads / events** → hold the `Tmsg.Read` and reply when data arrives; cancel it on `Flush`. `appl/cmd/chatsrv.b` shows - the pending-request idiom in ~30 lines. + the pending-request idiom in ~30 lines. The moment you hold + per-client state — a parked read, a busy flag, a helper fd — the + fid is the session: key the state to the fid and tear it down in + `Clunk`, which 9P guarantees you receive even when the client + dies. See "The fid is the session" in the `ninep-server` skill. - **Irreversible actions** → emit an audit record: one write to `/mnt/audit/log`, no-op if absent (`man/4/auditfs`). - **Agent access** → decide which files are grantable and which are diff --git a/tools/style-gate.sh b/tools/style-gate.sh index ad0415a2d..9f168ed56 100755 --- a/tools/style-gate.sh +++ b/tools/style-gate.sh @@ -6,6 +6,11 @@ # Set STYLE_GATE_STRICT=1 to make findings fail (exit 1) once the # tree is clean and the project decides to enforce. # +# Scope: this gate checks STYLE, not semantics. An rc-legal script can +# still raise at runtime (null-list concatenation, failed redirections +# escaping if-blocks) — those live in the shell table of +# docs/LIMBO-FOR-GO-PROGRAMMERS.md, which is documentation's job. +# # Checks: # 1. POSIX-isms (&& / ||) in Inferno-side shell scripts. Inferno sh # is rc-style; && and || do not exist and scripts containing them