Conversation
Every consumer of state/session.jsonl scanned it its own way: ResumeFold for counters, the recovery tail scan for crash classification, and the controller's ingest enum. Add crucible_contract::LoopState with one exhaustive apply, a classify() that reproduces the recovery precedence, and a resume_view() that matches ResumeFold, so a new event kind is taught to one place. Additive wire changes, WIRE_VERSION unchanged: an Unknown catch-all so a newer writer never makes an older reader refuse a log, an optional envelope ts, task, source and park on approval_wait, trace_id, by and source on approval_resolved, a suspended shutdown outcome, and the run-workspace and approval-waits artifact kinds. CONTRACT_VERSION 1.3.0. The old folds stay in place for now. A parity module folds every recovery and resume test log, and the run6 fixture, through both and asserts they agree. Assisted-by: Claude
No domain pack sets [search], and every piece the wide round adds is a second copy of something the deep loop already has: a second runner over Row, a second phase to classify, a second filter in every fold, and a third TaskKind dispatch site. The event-sourced loop refactor would have to carry all of it. Delete WideConfig, WideOutcome, run_wide_tournament, wide_template, WideRunner, wide_propose, render_wide_prompt, the [search] manifest table and its validation, --wide and --wide-keep, and the engine.measure_diff operation. The died_in_wide_round classification arm goes with them; a plan admitted before any iteration phase is an open plan like any other. The wire stays readable: phase "wide" and died_in_wide_round remain legal tokens because old logs carry them, and readers keep such rows out of the deep loop's baseline and best as the resume fold always did. ADR-0024 records the decision and supersedes ADR-0010; RFC-0001 C-SEARCH is deprecated and the [search] rule leaves C-MANIFEST. Assisted-by: Claude
The loop's decisions were threaded through one 800-line function together with every effect they cause, and a resume rebuilt state through two more scanners of the same log. Add crucible::machine: LoopCfg, RunState, Segment, LoopExit and a Machine whose methods answer the head-of-iteration checks, fold an iteration's outcome into a row, and do the keep/discard and budget bookkeeping from plain inputs, with no I/O. loop_driver keeps the host: it performs every effect in the order it always has and hands the results to the machine, so the session log a run writes is unchanged. Resume and crash classification now read the contract's LoopState fold; ResumeFold and the recovery tail scan are gone, and the parity oracle that proved the fold matched them goes with them. The plan lane gains its resume fold: execute_from seeds a graph with the results an earlier process settled, and a run that died inside a plan hands the resumed iteration the passing pack tasks under the same plan, so they are reported rather than re-dispatched (RFC-0002 C-PLAYBOOK-RESUME). Engine operations always run again, since the world they produced did not survive. Assisted-by: Claude
MAX_DEAD_TURN_ATTEMPTS bounds the attempts an iteration gets when a turn never starts, but the counter lived in Machine and began at zero on every construction, so it bounded attempts per process rather than per iteration. A pod that burned two attempts and died handed its successor three more, and a harness that cannot start never reaches the stall. LoopState::dead_turns reads the streak off the log: trailing rows in the infra phase, skipping a distressed row, which annotates the turn before it rather than reporting one, so it neither ends the streak nor counts toward it. The count travels on ResumeView into RunState, beside the rest of the state a resume restores. No wire change; those rows already carry phase "infra". Assisted-by: Claude
A pack can now say "run these tasks, then wait for someone to approve, then deploy". approve() is a task kind with dependents, a verdict contribution and a name, rather than the scored loop's one global provisioning wait that only an operator at a socket could end. A gate names what may resolve it. An operator on the run always can, over the control bridge or the controller; a gate may additionally name a pull request or a Jira issue, and whichever arrives first wins. Resolution is keyed by a trace id derived from the run and the task and is idempotent under it, so a retrying resolver, a second source, and a resumed process replaying a decision converge on one recorded outcome. Granted settles the gate passing, denied and timeout settle it failing; the lane's existing verdict rule does the rest, so there is no new task status. A run at a gate parks or suspends. Suspending writes the workspace and state dir to the controller's drop-box, reports outcome=suspended and exits zero; a later process restores them through a resume-restore init container and is handed the resolutions the controller settled meanwhile. Parked time does not count against the wall clock. The pod gets a preStop that writes the stop file and a 300s grace period, because ctrlc handles SIGINT only: without it a delete SIGKILLs a parked run mid-gate. ParkPolicy moves to the lib and CRUCIBLE_RESUME_OF to the contract so the renderer can name what the engine parses. The wire stays at version 1: every addition is an optional field or a new token. ADR-0025 records the decision; RFC-0002 gains C-PLAYBOOK-APPROVAL, and C-PLAYBOOK-LANE, C-PLAYBOOK-RESUME, C-PLAYBOOK-SHAPE and RFC-0001 C-WIRE are amended to match. Assisted-by: Claude
A command task runs sh -c in the loop pod with whatever toolchain the loop image baked, so a pack needing another one either grows the shared image or vendors a build. Nothing new is needed to fix that. Isolation is a one-variant enum, the sandbox module already moves a workdir in and named files out for agent turns, ComputeDriver already picks nested podman or a sibling pod, and emits/emits_files already fix the only channel out of a task, which is what makes where it ran a substrate detail. The proposal is an optional image on command and evaluate, dispatched as a second isolation variant. Proposed, not decided, and sequenced after the gate stack: it wants the pod-dispatch path the controller half is about to change. Assisted-by: Claude
|
Docs preview for this PR is built and attached as the Download Rebuilt for 00d5f9f. |
CI runs clippy from a cold cache with -D warnings; a warm local cache had been replaying stale clean results, so four disallowed anyhow::bail sites, a needless clone, and a large_enum_variant went unseen. The gate's failures become thiserror variants like every other module's: NoBridgeReply, SuspendNeedsManifest, StoppedWhileParked, and the escaping snapshot entry, the failed bundle, and the failed git run on SuspendError. The approve task kind grew Task past the point where holding one inline in the Starlark value enum was reasonable, so that variant is boxed, the same way the loop's decided step already carries its payload. Assisted-by: Claude
The loop's decisions became named types when they moved out of the driver, so the state chart is now derivable rather than something to keep redrawing by hand. `crucible loop-reference` prints it, scripts/loop-docs.sh writes docs/loop-machine.md, and a pre-commit hook regenerates the page when machine.rs changes, the same shape the DSL reference already uses. Drift is a compile error, not a stale diagram. Adding a LoopExit breaks shutdown_reason, adding an IterStep breaks step_label, and the coverage tests then hold the new variant to reaching the chart. A third test walks the edges from Head so a state cannot be drawn without a way to reach it. The exits on the chart are the shutdown tokens the session log carries, so a reader can match a node to a run's last line. Assisted-by: Claude
Assisted-by: Claude
Move gate events, suspension, resume-state recovery, and compression onto Host so callers invoke behavior through the type that owns gate-host state. Document the domain-method convention and remove the attribution requirement.
The head's order of operations existed only as the reading order of run_loop_body: park on a pending approval, drain a re-scope, drain a denial, park on distress, check the interrupt, check the caps. Nothing named that sequence, so the state chart had to assert it by hand and a reordering would have falsified the page without failing anything. machine::HEAD is now that sequence, as data. The host walks it and dispatches each HeadCheck through one function that returns HeadFlow rather than breaking a loop, which is what lets the order live outside the code that runs it. Each check names its own chart edges in an exhaustive match, so the head of the diagram is generated from the same list the host walks, and a new check compiles only once it is described and dispatched. run_loop_body goes from 814 lines to 689. The head's thirteen threaded arguments become one HeadCtx. No behavior change: the checks run in the same order, do the same things, and the 616 engine tests are untouched. The iteration half of the chart is still a written table. IterStep and Settle already name those transitions, but the host performs them inline; moving them behind the same dispatch is the remaining half of this. Assisted-by: Claude
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A pack can now write
approve()as a task: run these tasks, wait for someone to approve, then deploy. Today the only way to ask a human for something is the scored loop's provisioning ask, which is one global wait that only an operator at a socket can end. A gate is a node with dependents, a verdict contribution, and a name.A gate names what may resolve it. An operator on the run always can, over the control bridge; a gate may additionally name a pull request or a Jira issue, and whichever arrives first wins. Resolution is keyed by a trace id derived from the run and the task and is idempotent under it, so a retrying resolver, a second source, and a resumed process replaying a decision all converge on one recorded outcome. Granted settles the gate passing, denied and timeout settle it failing, and the lane's existing verdict rule does the rest — no new task status.
A run at a gate parks or suspends. Suspending writes the workspace and state dir to the ingest drop-box, reports
outcome=suspendedand exits zero; a later process restores them through aresume-restoreinit container and is handed the resolutions that were settled meanwhile. Parked time does not count against the wall clock.Getting there needed the loop's state to be one thing. Three folds of the session log had grown up independently and a gate adds events all of them would have to learn, so the fold moves into the contract as one exhaustive
apply, and the loop's decisions move out of the driver into a state machine with no I/O.Reading order
The commits are meant to be read one at a time:
a5243f6contract fold — additive wire, no behavior change, parity-tested against the old folds509c98dretire the wide tournament — pure deletion, −985 lines, nothing uses[search]166b7c4machine + host — the loop's decisions, testable with no fakes2575038dead-turn streak across a resume — a bound that had been counting per processdba1c74the gate itself, plus the pod render and the governance7b83eacan unrelated proposal (container isolation for deterministic tasks), proposed only, droppablec341800draw the loop's state machine from the machine's own vocabulary00d5f9fdispatch the head from a list the machine owns, so the chart's head is that listThe last two are why
docs/loop-machine.mdexists and why it cannot drift:machine::HEADis the order the host walks, each check names its own edges in an exhaustive match, and the page regenerates fromcrucible loop-referenceunder a pre-commit hook. Adding aLoopExit, anIterStep, or aHeadCheckfails to compile until it is described, and then fails a test until it reaches the chart.Scope
This is the engine side only. A suspended run needs a consumer that understands
outcome=suspended, holds the pending decisions, and redispatches with--resume; until that exists, a run can suspend but nothing brings it back. Consumers should not adopt this contract version before they handle that outcome explicitly — the shutdown vocabulary is additive, and a consumer that falls through to its success path would mark a run that is merely waiting as finished.Verification
cargo fmt --check,cargo clippy --all --benches --tests --examples --all-features,cargo test --workspace, andgovctl checkare clean. The gate has a three-node functional test (open a PR, gate on it, deploy) covering granted, denied, timeout, and an approve over a real control bridge.