From 118cb0aad8e554eaac6a56302f8cb5783971d4da Mon Sep 17 00:00:00 2001 From: Will Eaton Date: Tue, 22 Sep 2026 16:56:04 -0400 Subject: [PATCH] Gate the scored loop behind an autoresearch feature The engine binary shipped the optimization loop, scoping, grounded ranking and PR watching to every user, though a playbook run touches none of it. With the new `autoresearch` cargo feature off (the default) the binary is the playbook workflow engine: plan, check, deploy, build, flow and fetch. A bare `crucible --manifest` refuses with a message naming the feature instead of running. Shared machinery moves out of runloop so the plan lane no longer reaches into it: iteration_template to plan::template, the gate self-test to cli::selftest (crucible check runs it), and fetch_object with the S3/file URI helpers to object_store (the controller shells crucible fetch). The control bridge moves to control::bridge and the Reporter layer to report::reporter, so each gate is one cfg on a module. Release binaries, the runtime images and CI build with the feature on, since the controller runs crucible scope and rank-grounded in pods. CI also clippies and tests the engine on default features so neither set rots. The library the controller links is unchanged. Assisted-by: Claude --- .github/workflows/ci.yml | 6 + .github/workflows/docker.yml | 3 +- .github/workflows/release.yml | 2 +- Containerfile.runtime-selfcontained | 1 + README.md | 12 +- crucible/Cargo.toml | 3 + crucible/src/agent/agent_session.rs | 3 + crucible/src/agent/engine.rs | 19 + crucible/src/agent/event.rs | 2 + crucible/src/agent/mod.rs | 5 + crucible/src/args.rs | 15 + crucible/src/cli/check.rs | 2 +- crucible/src/cli/mod.rs | 12 +- crucible/src/cli/run.rs | 545 +------- crucible/src/cli/scored.rs | 520 +++++++ crucible/src/{runloop => cli}/selftest.rs | 36 +- crucible/src/control/bridge.rs | 1507 ++++++++++++++++++++ crucible/src/control/escalation.rs | 2 +- crucible/src/control/mod.rs | 1518 +-------------------- crucible/src/control/pr_watch.rs | 6 +- crucible/src/main.rs | 10 +- crucible/src/object_store.rs | 134 ++ crucible/src/plan/cli.rs | 3 +- crucible/src/plan/harness.rs | 14 +- crucible/src/plan/template.rs | 189 +++ crucible/src/process.rs | 2 + crucible/src/report/console.rs | 6 +- crucible/src/report/mod.rs | 269 +--- crucible/src/report/reporter.rs | 263 ++++ crucible/src/report/session.rs | 6 +- crucible/src/report/stream.rs | 4 +- crucible/src/runloop/driver.rs | 30 +- crucible/src/runloop/graph.rs | 201 +-- crucible/src/runloop/machine.rs | 14 +- crucible/src/runloop/mod.rs | 1 - crucible/src/runloop/preflight.rs | 4 +- crucible/src/runloop/publish.rs | 115 +- crucible/src/runloop/step.rs | 8 +- crucible/src/scope/pipeline.rs | 2 +- crucible/src/scope/refine.rs | 39 +- docs/getting-started.md | 4 +- justfile | 6 +- scripts/state-docs.sh | 2 +- 43 files changed, 2870 insertions(+), 2675 deletions(-) create mode 100644 crucible/src/cli/scored.rs rename crucible/src/{runloop => cli}/selftest.rs (93%) create mode 100644 crucible/src/control/bridge.rs create mode 100644 crucible/src/object_store.rs create mode 100644 crucible/src/plan/template.rs create mode 100644 crucible/src/report/reporter.rs diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ec705c36..3ac52b59 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -99,6 +99,9 @@ jobs: - name: Clippy run: cargo clippy --workspace --all-targets --all-features --locked --no-deps -- -D warnings + - name: Clippy (engine, default features) + run: cargo clippy -p crucible --all-targets --locked --no-deps -- -D warnings + # The controller's `#[sqlx::test]` suite needs a live Postgres at DATABASE_URL; each test # creates (and drops) its own database on it. An ephemeral host port so concurrent jobs on # a shared self-hosted runner never collide. Compilation stays on the checked-in `.sqlx/` @@ -191,6 +194,9 @@ jobs: - name: Test run: cargo nextest run --workspace --all-features --locked + - name: Test (engine, default features) + run: cargo nextest run -p crucible --locked + # Nextest deliberately excludes ignored tests; this one proves an idle controller tick # opens no span while an ingest still traces. - name: Test ignored tracing regression diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.yml index a90769c2..430d8517 100644 --- a/.github/workflows/docker.yml +++ b/.github/workflows/docker.yml @@ -72,7 +72,8 @@ jobs: # the loop pod runs) + forge (engine-side build/deploy bins). - name: Build release binaries run: | - cargo build --release --locked -p crucible -p crucible-broker -p forge --bins + cargo build --release --locked -p crucible -p crucible-broker -p forge --bins \ + --features crucible/autoresearch mkdir -p prebuilt/bin find target/release -maxdepth 1 -type f -executable -exec cp {} prebuilt/bin/ \; echo "staged binaries:" && ls -la prebuilt/bin/ diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 591e6e55..c9101524 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -65,7 +65,7 @@ jobs: env: CRUCIBLE_GIT_SHA: ${{ github.sha }} SQLX_OFFLINE: "true" - run: cargo build --release --locked -p crucible -p forge -p crucible-controller -p crux --bins + run: cargo build --release --locked -p crucible -p forge -p crucible-controller -p crux --bins --features crucible/autoresearch # Collect every top-level executable from target/release (same pattern the Containerfile # uses to stage /art/bin) into one tarball + a no-tooling sha256 integrity file. diff --git a/Containerfile.runtime-selfcontained b/Containerfile.runtime-selfcontained index 737e3c33..6e4c57a0 100644 --- a/Containerfile.runtime-selfcontained +++ b/Containerfile.runtime-selfcontained @@ -56,6 +56,7 @@ WORKDIR /src COPY . . # The same bins, flags and staging docker.yml uses, so the packaged output matches CI's. RUN cargo build --release --locked -p crucible -p crucible-broker -p forge --bins \ + --features crucible/autoresearch \ && mkdir -p /prebuilt/bin \ && find target/release -maxdepth 1 -type f -executable -exec cp {} /prebuilt/bin/ \; \ && ls -la /prebuilt/bin/ diff --git a/README.md b/README.md index 7df15a49..1a341580 100644 --- a/README.md +++ b/README.md @@ -105,10 +105,15 @@ Build and install from source: ```bash git clone https://github.com/neuralmagic/crucible.git cd crucible -cargo build --release -p crucible +cargo build --release -p crucible --features autoresearch install -m 755 target/release/crucible ~/.local/bin/crucible ``` +Without `--features autoresearch` the binary is the playbook workflow engine alone: `plan`, +`check`, `deploy`, `build`, `flow` and `fetch` work, and the optimization loop, `scope`, +`rank-grounded`, `watch-pr` and `loop-states` are not built. Release binaries and images carry +the feature. + Place the destination directory on `PATH`. Published binaries, when available for a platform, are listed on the [GitHub releases page](https://github.com/neuralmagic/crucible/releases). @@ -121,7 +126,7 @@ and Git history without a model or cluster. Its `command` backend increments the From a source checkout: ```bash -cargo run -p crucible -- \ +cargo run -p crucible --features autoresearch -- \ --manifest examples/counter/crucible.toml \ --iterations 6 ``` @@ -336,7 +341,8 @@ intended for deterministic tests and integrations that provide their own propose ## CLI reference Running `crucible` without a subcommand starts an optimization loop and requires -`--manifest`. The principal subcommands are: +`--manifest`. The loop, `scope`, `watch-pr`, `rank-grounded` and `loop-states` need a build with +`--features autoresearch`. The principal subcommands are: | Command | Function | | --- | --- | diff --git a/crucible/Cargo.toml b/crucible/Cargo.toml index 31400b1a..037b3fda 100644 --- a/crucible/Cargo.toml +++ b/crucible/Cargo.toml @@ -19,6 +19,9 @@ default = ["publish"] # manifests, plans, flow) needs none of it, and the SDK carries its own hyper/rustls generation, # so a linked consumer turns this off. publish = ["dep:aws-config", "dep:aws-sdk-s3"] +# The scored optimization loop (keep/discard over a judge), scoping, grounded ranking and PR +# watching. Off by default: without it the binary is the playbook workflow engine. +autoresearch = ["publish"] [dependencies] anyhow.workspace = true diff --git a/crucible/src/agent/agent_session.rs b/crucible/src/agent/agent_session.rs index 9133e596..59c26ece 100644 --- a/crucible/src/agent/agent_session.rs +++ b/crucible/src/agent/agent_session.rs @@ -13,6 +13,7 @@ use nix::fcntl::{Flock, FlockArg}; use serde::{Deserialize, Serialize}; use uuid::Uuid; +#[cfg(feature = "autoresearch")] use crate::report::session::SessionAction; const LEDGER_FILE: &str = "agent-sessions.json"; @@ -42,6 +43,7 @@ impl SessionTurn { self.completed_turns > 0 } + #[cfg(feature = "autoresearch")] pub(crate) fn action(&self) -> SessionAction { if self.is_resume() { SessionAction::Resumed @@ -155,6 +157,7 @@ pub(crate) fn commit_if_ok( .map(|e| format!("committing agent session failed: {e:#}")) } +#[cfg(feature = "autoresearch")] /// A resumed turn gets the follow-up prompt when the caller has one. pub(crate) fn effective_prompt<'a>( prepared: Option<&SessionTurn>, diff --git a/crucible/src/agent/engine.rs b/crucible/src/agent/engine.rs index ad67d8d1..0fad8a3a 100644 --- a/crucible/src/agent/engine.rs +++ b/crucible/src/agent/engine.rs @@ -87,6 +87,7 @@ pub(crate) fn handle() -> Result<&'static Handle> { ) } +#[cfg(feature = "autoresearch")] /// Best-effort span flush, for the run paths that end in `std::process::exit` (which skips /// [`EngineCtx`]'s `Drop`). Bounded on a scratch thread so a hung collector cannot wedge exit; /// a no-op when no exporter is installed. @@ -109,6 +110,7 @@ pub(crate) fn flush() { let _ = rx.recv_timeout(Duration::from_secs(3)); } +#[cfg(feature = "autoresearch")] /// Why the run is being torn down, and the exit code that reports it. `128 + signo`, the shell /// convention, so a killed loop is distinguishable from one that chose its own exit code. #[derive(Debug, Clone, Copy, PartialEq, Eq)] @@ -119,6 +121,7 @@ pub(crate) enum Termination { Interrupted, } +#[cfg(feature = "autoresearch")] impl Termination { fn exit_code(self) -> i32 { match self { @@ -135,6 +138,7 @@ impl Termination { } } +#[cfg(feature = "autoresearch")] /// Resolves on SIGTERM or ctrl-c, naming which arrived. Mirrors `crucible-broker`'s /// `telemetry::shutdown_signal`; the broker feeds it to axum's graceful shutdown, the engine has no /// server to drain so it acts on it directly in [`abort_on_signal`]. @@ -161,6 +165,7 @@ async fn termination_signal() -> Termination { } } +#[cfg(feature = "autoresearch")] /// End `span` as aborted: mark it, then close the OTel side explicitly. /// /// Dropping the `tracing::Span` is not enough here. The signal handler holds a CLONE while the run @@ -183,6 +188,7 @@ fn end_span_aborted(span: &tracing::Span, reason: &str) { otel.end(); } +#[cfg(feature = "autoresearch")] /// Close the run out on SIGTERM/ctrl-c instead of dying mid-batch. /// /// The engine has no graceful-shutdown path of its own: `run` ends in `std::process::exit`, so a @@ -215,12 +221,15 @@ pub(crate) fn abort_on_signal(run_span: Option) { }); } +#[cfg(feature = "autoresearch")] /// The W3C env vars the controller's dispatches (loop run, scope turn, rank turn) inject; the /// engine adopts them as its trace parent. Distinct from the `OTEL_*` exporter config, so they /// never collide with it. const TRACEPARENT_ENV: &str = "TRACEPARENT"; +#[cfg(feature = "autoresearch")] const TRACESTATE_ENV: &str = "TRACESTATE"; +#[cfg(feature = "autoresearch")] /// The controller-injected W3C parent from the process env, or `None` when `TRACEPARENT` is absent /// (the normal local/uninstrumented invocation). A present-but-unparseable value is a /// warn-and-ignore (never fail the work); the adopting span just roots itself. @@ -237,6 +246,7 @@ fn dispatch_parent() -> Option { parent } +#[cfg(feature = "autoresearch")] /// The long-lived `run` root span for a loop. When the controller dispatched this pod the span is /// parented to that dispatch, so the backend shows one tree (controller → run → turn → RPCs); a /// standalone run (no controller, so no `TRACEPARENT`) gets the same span self-rooted. Returns @@ -278,6 +288,7 @@ pub(crate) fn run_span(workspace: &str, run_id: &str) -> Option { Some(span) } +#[cfg(feature = "autoresearch")] /// Which controller-dispatched agent turn is adopting the dispatch as its trace parent. pub(crate) enum TurnSpanKind { /// `crucible scope --propose`, pairs with the controller's `dispatch_scope` PRODUCER. @@ -286,6 +297,7 @@ pub(crate) enum TurnSpanKind { RankGrounded, } +#[cfg(feature = "autoresearch")] /// The CONSUMER root span for one controller-dispatched agent turn (scope-propose / grounded-rank), /// parented to the controller's PRODUCER dispatch span exactly like [`run_span`], so the turn's /// `openshell_turn` span nests under the dispatch instead of floating as an orphaned trace. Same @@ -593,6 +605,7 @@ fn trace_env_from_context(cx: &opentelemetry::Context) -> Option<(String, Option Some((traceparent, tracestate)) } +#[cfg(feature = "autoresearch")] /// Extract the controller's remote parent context from the W3C carrier values, or `None` when /// `traceparent` is absent/blank or serializes to an invalid (all-zeros) span context. A local /// `TraceContextPropagator` mirrors the controller's injection side. @@ -823,6 +836,7 @@ mod tests { assert_eq!(resolve_logs_endpoint(Some(" ".into()), None), None); } + #[cfg(feature = "autoresearch")] #[test] fn extract_parent_round_trips_the_controller_traceparent() { use opentelemetry::trace::TraceContextExt as _; @@ -843,6 +857,7 @@ mod tests { ); } + #[cfg(feature = "autoresearch")] #[test] fn extract_parent_ignores_absent_or_garbage_traceparent() { // Absent: the normal local run (turn spans root themselves). @@ -861,6 +876,7 @@ mod tests { ); } + #[cfg(feature = "autoresearch")] #[test] fn trace_env_formats_and_round_trips_through_extract() { use opentelemetry::trace::{ @@ -1060,6 +1076,7 @@ mod tests { ); } + #[cfg(feature = "autoresearch")] #[test] fn termination_reports_the_shell_signal_codes() { // 128 + signo, so a killed loop is distinguishable from a chosen exit code. 143 is the one @@ -1070,12 +1087,14 @@ mod tests { assert_eq!(Termination::Interrupted.reason(), "SIGINT"); } + #[cfg(feature = "autoresearch")] #[test] fn abort_on_signal_without_a_span_is_a_noop() { // Telemetry off means no run span, and installing a handler must not require a runtime. abort_on_signal(None); } + #[cfg(feature = "autoresearch")] #[test] fn end_span_aborted_ignores_an_unsampled_span() { // No exporter installed in the test process, so the span carries an invalid SpanContext and diff --git a/crucible/src/agent/event.rs b/crucible/src/agent/event.rs index 23978a83..5c57f31b 100644 --- a/crucible/src/agent/event.rs +++ b/crucible/src/agent/event.rs @@ -77,6 +77,7 @@ fn openai_prices(m: &str) -> Option<(f64, f64)> { } } +#[cfg(feature = "autoresearch")] /// The turn's provisional cost from one mid-turn token sample: the OTEL number when /// telemetry stamped one, otherwise the pricing-table estimate. Reconciled by the /// authoritative turn-end cost, so streaming this keeps the budget line moving @@ -173,6 +174,7 @@ mod tests { assert_eq!(openai_prices("gpt-5.2-codex"), Some((1.75, 14.0))); } + #[cfg(feature = "autoresearch")] #[test] fn provisional_prefers_the_authoritative_sample_cost() { let mut t = sample(); diff --git a/crucible/src/agent/mod.rs b/crucible/src/agent/mod.rs index 50383dad..ea3ef782 100644 --- a/crucible/src/agent/mod.rs +++ b/crucible/src/agent/mod.rs @@ -17,6 +17,7 @@ //! - [`AgentSource::Command`]: run a deterministic shell command in the workspace for //! examples/tests; native `AgentEvent` JSON lines are decoded, everything else is raw. +#[cfg(feature = "autoresearch")] pub(crate) mod activity; pub(crate) mod agent_session; pub(crate) mod engine; @@ -30,6 +31,7 @@ pub(crate) mod turn; use crate::agent::event::{AgentEvent, RawStream, Tokens, estimate_cost}; use crate::agent::harness::{HarnessRuntime, StreamDecoder}; use crate::args::{Args, Paths}; +#[cfg(feature = "autoresearch")] use crate::manifest::AgentBackend; use crucible_harness::OtelCollector; use std::io::{BufRead, BufReader, Read}; @@ -59,10 +61,12 @@ pub enum AgentSource { Command(String), } +#[cfg(feature = "autoresearch")] pub(crate) fn supports_persistent_sessions(args: &Args) -> bool { backend_supports_persistent_sessions(args.agent_backend, args.harness()) } +#[cfg(feature = "autoresearch")] /// Capability predicate used by runtime admission and scope preview. pub(crate) fn backend_supports_persistent_sessions( backend: AgentBackend, @@ -216,6 +220,7 @@ fn spawn_local( }) } +#[cfg(feature = "autoresearch")] /// Run one agent turn against the source resolved from `args`. `sink(raw_line, stream, /// event)` is called per output line; returns the turn's [`TurnOutcome`]: the highest cost the /// agent reported (0 if none) plus the transport failure that stopped it, if any. diff --git a/crucible/src/args.rs b/crucible/src/args.rs index 4bdbda8c..443a5d72 100644 --- a/crucible/src/args.rs +++ b/crucible/src/args.rs @@ -1,11 +1,14 @@ //! The shared vocabulary of a run: its CLI options ([`Args`]), the paths everything anchors //! off ([`Paths`]), and the inputs resolved once before the loop starts ([`Prepared`]). +#[cfg(feature = "autoresearch")] use crate::duration::parse_duration; +#[cfg(feature = "autoresearch")] use crate::identity; use crate::manifest; use crate::openshell; use std::path::{Path, PathBuf}; +#[cfg(feature = "autoresearch")] use std::time::Duration; /// Which front-end to drive the loop with. @@ -166,16 +169,20 @@ pub(crate) struct Args { /// Publish-on-keep (composite only): per-component `(name, owner/repo)` fork map, populated from the /// composite manifest's `[[component]].pr_repo`, not a CLI flag. Each touched component opens one /// cross-linked draft PR against its fork. + #[cfg_attr(not(feature = "autoresearch"), allow(dead_code))] #[arg(skip)] pub component_pr_repos: Vec<(String, String)>, /// Declared pipeline artifacts (from `[[workspace.artifact]]`), for the publish layer: /// each `embed` match lands in the PR body and the S3 run record. No CLI flag. + #[cfg_attr(not(feature = "autoresearch"), allow(dead_code))] #[arg(skip)] pub artifacts: Vec, /// Wide-round search config (from `[search]`). No CLI flag, set by `run_from_manifest`. + #[cfg_attr(not(feature = "autoresearch"), allow(dead_code))] #[arg(skip)] pub search: Option, /// Manifest-only authored workflow. + #[cfg_attr(not(feature = "autoresearch"), allow(dead_code))] #[arg(skip)] pub workflow: Option, /// Manifest injects restored in each task workspace. @@ -203,11 +210,13 @@ impl Args { ::try_parse_from(["crucible"]).map(|f| f.run) } + #[cfg(feature = "autoresearch")] /// Parse `--max-time` (e.g. `30m`) into a duration; None when unset/invalid. pub(crate) fn max_time(&self) -> Option { parse_duration(&self.max_time) } + #[cfg(feature = "autoresearch")] /// Parse `--max-park` into a duration; None = wait on an approval indefinitely. pub(crate) fn max_park(&self) -> Option { parse_duration(&self.max_park) @@ -238,21 +247,26 @@ pub(crate) struct Paths { /// Toolbox source dir (`[agent].toolbox_dir`, manifest-relative); its subdirs are copied /// into `/.claude/skills` each run. `None` when the manifest sets no toolbox. pub skills: Option, + #[cfg_attr(not(feature = "autoresearch"), allow(dead_code))] pub steer: PathBuf, /// Cross-process state dir (gitignored): the session log + control file live here. pub state: PathBuf, /// Append-only NDJSON event log the headless loop emits for external tailers. pub session_log: PathBuf, /// Cross-process stop signal written by the `stop` tool. + #[cfg_attr(not(feature = "autoresearch"), allow(dead_code))] pub control: PathBuf, /// Escalation marker the agent's `escalate` tool writes in its workspace; the loop detects it /// after a turn, restores the world, and halts for human review. + #[cfg_attr(not(feature = "autoresearch"), allow(dead_code))] pub escalation: PathBuf, /// Pending-provisioning marker the agent writes when it has an open approval to wait on; the loop /// detects it after a turn and parks or continues per its `mode`. + #[cfg_attr(not(feature = "autoresearch"), allow(dead_code))] pub provisioning: PathBuf, /// Append-only NDJSON record of every external input, authoritative over the session /// log for what an operator asked for; a resume replays it. + #[cfg_attr(not(feature = "autoresearch"), allow(dead_code))] pub admissions: PathBuf, } @@ -298,6 +312,7 @@ impl Paths { } /// Inputs resolved once before the loop (and before any UI takes the screen). +#[cfg(feature = "autoresearch")] #[derive(Clone)] pub(crate) struct Prepared { pub goal: String, diff --git a/crucible/src/cli/check.rs b/crucible/src/cli/check.rs index fa8947cd..dc5b8449 100644 --- a/crucible/src/cli/check.rs +++ b/crucible/src/cli/check.rs @@ -4,10 +4,10 @@ //! (docs/crucible-contract.md §3), and warn when the gate is reachable by the agent's own edits (frozen- //! judge wall, applied here as a lint rather than a block). +use crate::cli::selftest::{self, SelftestReport}; use crate::manifest::{self, AgentCfg, CompositeManifest, Manifest, WorldCfg}; use crate::openshell; use crate::plan::ir::TaskKind; -use crate::runloop::selftest::{self, SelftestReport}; use anyhow::Result; use crucible::crucible::Direction; use std::collections::BTreeSet; diff --git a/crucible/src/cli/mod.rs b/crucible/src/cli/mod.rs index bafc2c83..7869e607 100644 --- a/crucible/src/cli/mod.rs +++ b/crucible/src/cli/mod.rs @@ -1,17 +1,23 @@ -//! The command line: the default (no subcommand) runs the loop. +//! The command line: the default (no subcommand) runs the scored loop, when it is built. pub(crate) mod build; pub(crate) mod check; pub(crate) mod init; pub(crate) mod ps; pub(crate) mod run; +#[cfg(feature = "autoresearch")] +pub(crate) mod scored; +pub(crate) mod selftest; pub(crate) mod setup; pub(crate) mod workspace; use crate::args::Args; +#[cfg(feature = "autoresearch")] use crate::control::pr_watch; use crate::openshell; +#[cfg(feature = "autoresearch")] use crate::scope; +#[cfg(feature = "autoresearch")] use crate::scope::rank_grounded; use clap::Parser; use std::path::PathBuf; @@ -68,6 +74,7 @@ pub(crate) enum Cmd { /// recording the goal source, the check outcome, and the pack's `RunIdentity` digest. No /// isolation preflight (S3), no draft-PR approval (S4), the freeze report names those as /// pending. + #[cfg(feature = "autoresearch")] Scope(scope::ScopeArgs), /// List every crucible loop pod in the cluster (kube-native): NAME, NAMESPACE, PHASE, AGE, /// RESTARTS, and a best-effort ITER (ships as `-` for now, see `ps.rs`'s module doc). Selects @@ -96,6 +103,7 @@ pub(crate) enum Cmd { /// Print the loop's control states: the transition table the driver runs on, as the /// reference page (`docs/loop-states.md` is generated from it), as Graphviz dot (the /// page's diagram), or as mermaid. + #[cfg(feature = "autoresearch")] LoopStates { #[arg(long, default_value = "markdown")] format: StatesFormat, @@ -105,6 +113,7 @@ pub(crate) enum Cmd { /// or appended to a reseed file that the next run's first turn reads, exactly one of /// `--control-addr`/`--reseed` is required. A kept composite candidate is a SET of linked PRs /// (one per component fork); pass `--pr` more than once to watch them all in one process. + #[cfg(feature = "autoresearch")] WatchPr { /// The PR to watch, e.g. `https://github.com/owner/repo/pull/42` (repeatable, a composite /// candidate opens one linked PR per component). @@ -151,6 +160,7 @@ pub(crate) enum Cmd { /// cheap text-only ranker escalates to this when it is unsure; the turn is read-only (a /// throwaway worktree contains any write). The caller owns `--workspace`, this command never /// clones or mutates it. + #[cfg(feature = "autoresearch")] RankGrounded(rank_grounded::RankGroundedArgs), /// Dispatch a named `[build.]` from the domain manifest, wait for it, and print the /// digest-pinned ref. The cluster backend renders a detached rootless-buildah Job; the diff --git a/crucible/src/cli/run.rs b/crucible/src/cli/run.rs index 8514c8b0..5032fc3e 100644 --- a/crucible/src/cli/run.rs +++ b/crucible/src/cli/run.rs @@ -1,57 +1,30 @@ -//! Run setup and command dispatch: the glue between the parsed CLI and the loop. -//! -//! [`dispatch`] routes the subcommands and otherwise hands off to [`run_from_manifest`], the one -//! run path: a `crucible.toml` builds the [`World`] + [`Judge`], anchors every path, picks a -//! front-end, and calls [`crate::runloop::driver::run_loop`]. +//! Command dispatch: the glue between the parsed CLI and each command. With no subcommand the +//! scored loop runs ([`crate::cli::scored::run_from_manifest`]), when it is built. -use crate::agent::harness::HarnessRuntime; -use crate::args::{Args, Paths, Prepared, Ui}; use crate::cli::check; use crate::cli::init; use crate::cli::{Cli, Cmd, FlowArgs}; -use crate::control; -use crate::control::recovery::{RecoveryPlan, ResumeRecovery, classify_session, plan_recovery}; use crate::deploy; -use crate::errors::FileError; use crate::flow; use crate::manifest; -use crate::process::STOP; -use crate::report; -use crate::report::console; -use crate::report::stream; -use crate::runloop::driver::{LoopRuntime, run_loop}; -use crate::runloop::publish; -use crate::scope; use anyhow::{Context, Result}; -use crucible::crucible::{Judge, World}; -use crucible_vcs::vcs; use std::path::{Path, PathBuf}; use std::sync::Arc; -use std::sync::atomic::Ordering; -/// The run layer's own failures: CLI-flag combinations the parser can't express, workspace -/// setup, and the manifest fields a run needs. Causes hang off `source()`; the top-level -/// dispatch turns these into anyhow errors so the CLI prints the whole chain. +/// Dispatch-level failures. The top-level dispatch turns these into anyhow errors so the CLI +/// prints the whole chain. #[derive(Debug, thiserror::Error)] pub(crate) enum RunError { - #[error("watch-pr needs exactly one of --control-addr or --reseed")] - WatchPrNoSink, - #[error("watch-pr takes exactly one of --control-addr or --reseed, not both")] - WatchPrTwoSinks, #[error( "a playbook needs a positive --max-cost; its source may not declare a limit its operator set" )] PlaybookNeedsBudget, - #[error("manifest [agent] needs `goal` or `goal_file` (or pass --goal)")] - NoGoal, - #[error("--control-port requires --ui stream (or --resume)")] - ControlPortNeedsStream, - #[error("resume: {message}")] - ResumeRefused { message: String }, - #[error(transparent)] - File(#[from] FileError), - #[error(transparent)] - Workspace(#[from] crate::cli::workspace::WorkspaceError), + #[cfg(not(feature = "autoresearch"))] + #[error( + "this crucible was built without the scored loop; rebuild with `--features autoresearch`, \ + or run a playbook with `crucible plan run`" + )] + ScoredLoopNotBuilt, } /// Route the parsed CLI: subcommands run standalone; everything else is a manifest run. @@ -100,15 +73,15 @@ pub(crate) fn dispatch(cli: Cli) -> Result<()> { std::process::exit(1); } + #[cfg(feature = "autoresearch")] if let Some(Cmd::Scope(args)) = cli.command { // Constructing the engine runtime publishes the handle a `--propose` openshell turn // reaches; held for the duration of `scope::run`. let _engine = crate::agent::engine::EngineCtx::new()?; - return scope::run(args); + return crate::scope::run(args); } - // Watch a draft PR's review comments and steer a live run (publish-on-keep closed loop). No - // workspace/loop, just poll the forge and write to the run's control bridge until interrupted. + #[cfg(feature = "autoresearch")] if let Some(Cmd::WatchPr { pr, control_addr, @@ -119,22 +92,15 @@ pub(crate) fn dispatch(cli: Cli) -> Result<()> { once, }) = cli.command { - let sink = match (control_addr, reseed) { - (Some(addr), None) => crate::control::pr_watch::Sink::Steer(addr), - (None, Some(path)) => crate::control::pr_watch::Sink::Reseed(path), - (None, None) => return Err(RunError::WatchPrNoSink.into()), - (Some(_), Some(_)) => return Err(RunError::WatchPrTwoSinks.into()), - }; - let opts = crate::control::pr_watch::WatchOpts { - poll: std::time::Duration::from_secs(poll_secs), + return crate::cli::scored::watch_pr( + &pr, + control_addr, + reseed, bot_user, - authz: crate::control::pr_watch::Authz { - allow_users: allow_user, - ..Default::default() - }, + allow_user, + poll_secs, once, - }; - return crate::control::pr_watch::watch_and_steer(&pr, &sink, &opts); + ); } if let Some(Cmd::Ps { namespace, json }) = cli.command { @@ -151,14 +117,15 @@ pub(crate) fn dispatch(cli: Cli) -> Result<()> { return flow_cmd(args); } if let Some(Cmd::Fetch { uri, out }) = &cli.command { - // The engine runtime the S3 GetObject block_ons on (published for `publish::fetch_object`). + // The engine runtime the S3 GetObject block_ons on (for `object_store::fetch_object`). let _engine = crate::agent::engine::EngineCtx::new()?; - return publish::fetch_object(uri, out); + return crate::object_store::fetch_object(uri, out); } // One code-grounded ranking turn over an existing checkout. A cheap, checkout-backed agent turn // that gates scope spend by confirming an API-tier verdict. Prints verdict JSON. The controller // shells this from its escalation arm. + #[cfg(feature = "autoresearch")] if let Some(Cmd::LoopStates { format }) = &cli.command { print!( "{}", @@ -171,6 +138,7 @@ pub(crate) fn dispatch(cli: Cli) -> Result<()> { return Ok(()); } + #[cfg(feature = "autoresearch")] if let Some(Cmd::RankGrounded(args)) = cli.command { // Constructing the engine runtime publishes the handle an openshell grounded turn reaches. let _engine = crate::agent::engine::EngineCtx::new()?; @@ -372,8 +340,16 @@ pub(crate) fn dispatch(cli: Cli) -> Result<()> { // The engine runtime is created here, once; constructing it publishes the handle every async call site reaches // (the openshell turns, the swept S3 publish calls). Held to the end of the process (the loop // exits via `process::exit`), so the runtime stays alive under the loop. - let _engine = crate::agent::engine::EngineCtx::new()?; - run_from_manifest(cli.run) + #[cfg(feature = "autoresearch")] + { + let _engine = crate::agent::engine::EngineCtx::new()?; + crate::cli::scored::run_from_manifest(cli.run) + } + #[cfg(not(feature = "autoresearch"))] + { + let _ = cli.run; + Err(RunError::ScoredLoopNotBuilt.into()) + } } /// `crucible flow`: gather the inputs (the session log, the span export from a file or Datadog), @@ -428,460 +404,9 @@ fn playbook_launch(args: &crate::cli::DeployArgs) -> Result Result<()> { - let manifest_path = args.manifest.clone().context( - "crucible needs a manifest: pass --manifest (see docs/crucible-contract.md)", - )?; - // A composite domain has a top-level `[composite]` table and a different shape; it runs - // multiple component workspaces under one base, so it takes the dedicated path. - if manifest::is_composite(&manifest_path) { - return run_composite(args, manifest_path); - } - let mut m = manifest::Manifest::load_frozen(&manifest_path)?; - // `parent()` of a bare `crucible.toml` is `Some("")`, which is not a usable cwd, treat - // an empty parent as the current directory. - let manifest_dir = manifest_path - .parent() - .filter(|p| !p.as_os_str().is_empty()) - .map(Path::to_path_buf) - .unwrap_or_else(|| PathBuf::from(".")); - m.resolve_workflow(&manifest_dir)?; - let workspace = manifest_dir.join(&m.workspace.dir); - let state = args - .state_dir - .clone() - .unwrap_or_else(|| manifest_dir.join("state")); - let skills = m.agent.toolbox_dir.as_ref().map(|d| manifest_dir.join(d)); - let p = Paths::for_manifest(workspace.clone(), state, &manifest_dir, skills); - - if !workspace.exists() { - crate::cli::workspace::manifest_setup(&m, &manifest_dir, &workspace)?; - // Inject baked judge/fixture files into the fresh clone (frozen judges + one-time fixtures). - // Frozen ones are also re-copied before each measure; the initial copy gives the agent a - // present, compilable harness from turn one. - for (src, dst, _frozen) in m.resolved_injects(&manifest_dir, &workspace)? { - manifest::apply_inject(&src, &dst) - .context("applying [workspace].inject after setup")?; - } - } - vcs::ensure_repo(&workspace).context("ensuring workspace is a git repo")?; - std::fs::create_dir_all(&p.state) - .with_context(|| format!("creating state dir {}", p.state.display()))?; - // The toolbox lands where the resolved harness discovers skills. - let harness = crate::cli::setup::pin_agent(&mut args, &m.agent)?; - crate::cli::workspace::install_toolbox( - &p, - &m.agent.toolbox_exclude, - harness.spec().skills_dir, - )?; - - // Fold the manifest's [agent] config onto Args (+ spawn the broker for openshell). - let frozen = crate::cli::setup::frozen_projection( - &m, - m.publish - .as_ref() - .and_then(|p| p.pr_repo.as_deref()) - .or(Some(args.pr_repo.as_str())), - &std::collections::BTreeMap::new(), - &p.session_log, - )?; - crate::cli::setup::apply_agent_cfg(&mut args, &m.agent, &m.secrets, &p.workspace, &frozen)?; - // Single-repo publish target: a `[publish] pr_repo` in the manifest wins over any `--pr-repo` the - // caller passed (the controller passes its per-repo default via the flag; a pack that names its - // own fork overrides it). Absent → keep the flag value (empty by default, so no PR opens). - if let Some(pr_repo) = m.publish.as_ref().and_then(|p| p.pr_repo.clone()) { - args.pr_repo = pr_repo; - } - // Declared pipeline artifacts, for the publish layer (PR-body embed + S3 upload). - args.artifacts = m.workspace.artifact.clone(); - - let (goal, template) = resolve_goal_template(&args, &m.agent, &manifest_dir)?; - // Cross-run memory: seed the prior run's tried-ideas ledger for this goal from S3 (best-effort), - // so a fresh run (or a future harness version) inherits history instead of re-walking dead ends. - let prior = publish::fetch_prior_results(&args.results_bucket, &goal).unwrap_or_default(); - if !prior.is_empty() { - let n = prior.lines().count(); - eprintln!("seeded {n} prior tried-idea row(s) from S3 (cross-run memory)"); - } - if m.is_task() { - eprintln!("task mode: no [judge] — every completed turn is kept and published unscored"); - } - // The world's comparability key, computed once the workspace has a HEAD - // to pin against. - let identity = crate::identity::for_manifest(&manifest_path, &manifest_dir, &workspace, &m) - .context("computing run identity")?; - let prep = Prepared { - run_id: publish::run_id(&goal), - prior, - goal, - template, - identity, - skip_baseline: m.is_task() || m.judge.as_ref().is_some_and(|j| j.skip_baseline), - preflight: m.preflight.clone(), - preflight_modes: m - .measure - .as_ref() - .and_then(crate::manifest::MeasureCfg::build_modes) - .unwrap_or_default(), - seed_diff: read_seed_diff(&manifest_dir, m.agent.seed_diff.as_deref())?, - }; - - // Frozen injects (the gate's own files) go to the judge so it re-establishes them before each - // scored measure, the agent can't edit the harness/test to game the gate. Resolve before the - // workspace move. - let frozen_injects: Vec<(PathBuf, PathBuf)> = m - .resolved_injects(&manifest_dir, &workspace)? - .into_iter() - .filter(|(_, _, frozen)| *frozen) - .map(|(src, dst, _)| (src, dst)) - .collect(); - args.search = m.search.clone(); - args.workflow = m.workflow.clone(); - args.workflow_frozen_injects = m.frozen_inject_pairs(&manifest_dir)?; - args.workflow_toolbox_exclude = m.agent.toolbox_exclude.clone(); - let world = m.build_world(workspace.clone()); - let judge = m.build_judge(workspace, frozen_injects)?; - - drive_loop(args, p, prep, world, judge) -} - -/// Read the `[agent].seed_diff` content for iteration 1's prompt. The identity build hashes the -/// same file; a declared seed that can't be read errors there first, this context is a backstop. -fn read_seed_diff(manifest_dir: &Path, seed_diff: Option<&str>) -> Result> { - seed_diff - .map(|rel| { - let path = manifest_dir.join(rel); - std::fs::read_to_string(&path) - .with_context(|| format!("reading [agent].seed_diff {}", path.display())) - }) - .transpose() -} - -/// Run a composite domain: set up each component's checkout under one base workspace, build -/// the multi-workspace [`CompositeWorld`] + the combined gate, and drive the same loop. The components -/// co-locate under the base so the agent has one cwd / one sandbox upload tree spanning both repos. -fn run_composite(mut args: Args, manifest_path: PathBuf) -> Result<()> { - let m = manifest::CompositeManifest::load_frozen(&manifest_path)?; - let manifest_dir = manifest_path - .parent() - .filter(|p| !p.as_os_str().is_empty()) - .map(Path::to_path_buf) - .unwrap_or_else(|| PathBuf::from(".")); - let base = m.base_dir(&manifest_dir); - let components = m.resolve_components(&manifest_dir)?; - eprintln!( - "composite `{}`: {} components — {}", - m.composite.name, - components.len(), - components - .iter() - .map(|c| format!("{} ({})", c.name, c.domain_dir.display())) - .collect::>() - .join(", ") - ); - - // Check out each component into /, then make each its own git repo (the per-component - // overlay CompositeWorld commits). The combined external deployment setup is a follow-up. - for c in &components { - if !c.workspace.exists() { - let repo = &c.manifest.repo; - let src = repo - .url - .clone() - .or_else(|| repo.path.clone()) - .with_context(|| format!("component `{}` [repo] needs url or path", c.name))?; - crate::cli::workspace::clone_repo(&src, repo.git_ref.as_deref(), &c.workspace) - .with_context(|| format!("cloning component `{}`", c.name))?; - } - vcs::ensure_repo(&c.workspace) - .with_context(|| format!("ensuring component `{}` is a git repo", c.name))?; - } - - // The world's comparability key: one component entry per checkout, all - // pinned now that every workspace has a HEAD. - let identity = crate::identity::for_composite(&manifest_path, &base, &components, &m) - .context("computing run identity")?; - - let state = args - .state_dir - .clone() - .unwrap_or_else(|| manifest_dir.join("state")); - let skills = m.agent.toolbox_dir.as_ref().map(|d| manifest_dir.join(d)); - // The agent's cwd is the base (it sees every component checkout as a subdir). - let p = Paths::for_manifest(base, state, &manifest_dir, skills); - std::fs::create_dir_all(&p.state) - .with_context(|| format!("creating state dir {}", p.state.display()))?; - let harness = crate::cli::setup::pin_agent(&mut args, &m.agent)?; - crate::cli::workspace::install_toolbox( - &p, - &m.agent.toolbox_exclude, - harness.spec().skills_dir, - )?; - - // A composite has no single-repo [publish]; its forks are per component. - let bounds = crate::cli::setup::run_bounds( - &m.outputs, - &m.build, - Some(args.pr_repo.as_str()), - &std::collections::BTreeMap::new(), - ); - let frozen = crate::cli::setup::FrozenProjection { - broker_env: crate::cli::setup::broker_bounds_env(&bounds, &p.session_log)?, - disclosure: Some(crate::exposure::covered_from( - crate::exposure::composite_capabilities(&m.agent, &m.capabilities), - )), - bounds: Some(bounds), - }; - crate::cli::setup::apply_agent_cfg(&mut args, &m.agent, &m.secrets, &p.workspace, &frozen)?; - // The per-component fork map for publish-on-keep, manifest-owned via [[component]].pr_repo. - args.component_pr_repos = m.component_pr_repos(); - let (goal, template) = resolve_goal_template(&args, &m.agent, &manifest_dir)?; - let prior = publish::fetch_prior_results(&args.results_bucket, &goal).unwrap_or_default(); - let prep = Prepared { - run_id: publish::run_id(&goal), - prior, - goal, - template, - identity, - skip_baseline: m.judge.skip_baseline, - preflight: m.preflight.clone(), - preflight_modes: m - .measure - .as_ref() - .and_then(crate::manifest::MeasureCfg::build_modes) - .unwrap_or_default(), - seed_diff: read_seed_diff(&manifest_dir, m.agent.seed_diff.as_deref())?, - }; - - args.search = m.search.clone(); - args.workflow = m.workflow.clone(); - args.workflow_frozen_injects = Vec::new(); - args.workflow_toolbox_exclude = m.agent.toolbox_exclude.clone(); - let world = m.build_world(&manifest_dir)?; - let judge = m.build_judge(&manifest_dir)?; - drive_loop(args, p, prep, world, judge) -} - -/// Resolve the run's goal + method-prompt template. `--goal`/`--goal-file` override the manifest (the -/// forge trigger injects a per-issue goal); otherwise the manifest's inline `goal` / `goal_file`. -fn resolve_goal_template( - args: &Args, - agent: &manifest::AgentCfg, - manifest_dir: &Path, -) -> Result<(String, String), RunError> { - let goal = if let Some(g) = &args.goal { - g.clone() - } else if let Some(f) = &args.goal_file { - std::fs::read_to_string(f).map_err(FileError::at("reading --goal-file", f))? - } else { - match (&agent.goal, &agent.goal_file) { - (Some(g), _) => g.clone(), - (None, Some(f)) => { - let path = manifest_dir.join(f); - std::fs::read_to_string(&path).map_err(FileError::at("reading goal_file", &path))? - } - (None, None) => return Err(RunError::NoGoal), - } - }; - let template = match &agent.method_prompt { - Some(mp) => { - let path = manifest_dir.join(mp); - std::fs::read_to_string(&path).map_err(FileError::at("reading method_prompt", &path))? - } - None => "{{GOAL}}\n\nStatus: {{STATUS}}\n{{STEER}}".to_string(), - }; - Ok((goal, template)) -} - -/// The shared loop tail: install Ctrl+C, then pick the front-end (resume / jsonl / stream / -/// console) and drive [`run_loop`]. Single-domain and composite runs both end here. -fn drive_loop( - args: Args, - p: Paths, - prep: Prepared, - world: Arc, - judge: Arc, -) -> Result<()> { - install_ctrlc()?; - if args.control_port.is_some() && !args.resume && args.ui != Ui::Stream { - return Err(RunError::ControlPortNeedsStream.into()); - } - - // When the controller dispatched this loop pod, adopt its dispatch span as the run's trace - // parent so Tempo shows controller → run → turn in one tree; the openshell turn spans nest under - // this span because they're created on this same thread. `None` (a local run, or telemetry off) - // leaves the turn spans rooting themselves independently. Held across the whole loop, then - // dropped below so the span closes and the OTLP layer batches it before `flush`. - let run_span = crate::agent::engine::run_span(&p.workspace.to_string_lossy(), &prep.run_id); - // A signal would otherwise kill the process with this span still open and the batch unflushed, - // so every rolled loop pod loses its run span. Installed here, where the span exists, rather - // than behind a static. - crate::agent::engine::abort_on_signal(run_span.clone()); - - // The liveness beat runs for the whole loop, parented to the run span so its beats hang off the - // run in Tempo. Declared AFTER `run_span` so the guard's Drop runs first and the beat is joined - // before the span it holds goes away, on the error returns below as well as the success path. - let heartbeat = crate::control::heartbeat::period_from_env() - .map(|period| crate::control::heartbeat::start(period, run_span.clone())); - let beat = heartbeat - .as_ref() - .map(crate::control::heartbeat::BeatGuard::beat); - - let outcome = { - let _run_guard = run_span.as_ref().map(tracing::Span::enter); - if args.resume { - // Replay the parked log, then continue in append mode. A NoOp exits 0 - // WITHOUT re-running the finish path (replaying finish re-published the - // kept candidate each crash-loop lap); Refuse keeps exit code 2's meaning. - let recovered = classify_session(&p.session_log)?; - match plan_recovery(&recovered, args.iterations, args.max_cost) { - RecoveryPlan::NoOp { message } => { - eprintln!("resume: {message}"); - return Ok(()); - } - RecoveryPlan::Refuse { message } => { - return Err(RunError::ResumeRefused { message }.into()); - } - RecoveryPlan::Continue { - repark, - pending_regime, - } => { - let recovery = ResumeRecovery { - class: recovered.classification.class(), - iter: recovered.classification.iter(), - detail: recovered.classification.detail(), - repark, - pending_regime, - }; - let meta = report::RunMeta::from_args(&args); - let r = stream::SessionReporter::resume(&p, meta)?; - // Fold the prior run's admissions before the bridge is up, so no - // inbound command can land on a half-built index. - let ledger = open_admission_ledger(&p, forge::ndjson::Open::Fold)?; - let control = start_control_bridge(&args, &p, &ledger)?; - let (_reporter, outcome) = run_loop( - &args, - &p, - &prep, - r, - &world, - &judge, - LoopRuntime { - control: control.clone(), - resume: Some(recovered.resume), - recovery: Some(recovery), - ledger: Some(ledger), - heartbeat: beat.clone(), - }, - ); - outcome? - } - } - } else { - let meta = report::RunMeta::from_args(&args); - match args.ui { - Ui::Jsonl => { - let r = stream::SessionReporter::stdout(meta); - let (_reporter, outcome) = run_loop( - &args, - &p, - &prep, - r, - &world, - &judge, - LoopRuntime { - heartbeat: beat.clone(), - ..LoopRuntime::default() - }, - ); - outcome? - } - Ui::Stream => { - let r = stream::SessionReporter::stream(&p, meta)?; - // A fresh run must not inherit the last run's un-drained inputs. - let ledger = open_admission_ledger(&p, forge::ndjson::Open::Truncate)?; - let control = start_control_bridge(&args, &p, &ledger)?; - let (_reporter, outcome) = run_loop( - &args, - &p, - &prep, - r, - &world, - &judge, - LoopRuntime { - control: control.clone(), - ledger: Some(ledger), - heartbeat: beat.clone(), - ..LoopRuntime::default() - }, - ); - outcome? - } - _ => { - let r = console::ConsoleReporter; - let (_reporter, outcome) = run_loop( - &args, - &p, - &prep, - r, - &world, - &judge, - LoopRuntime { - heartbeat: beat.clone(), - ..LoopRuntime::default() - }, - ); - outcome? - } - } - } - }; - crate::report::ingest_client::deliver_run_evidence(&p); - // Explicit because this path ends in `process::exit`, which runs no destructors: the beat - // thread holds a clone of the run span, so a live beat would keep it open past the flush below. - drop(heartbeat); - // Close the run span (drop it, now that its guard is gone) so the OTLP layer batches it, THEN - // flush: the loop exits via process::exit, which skips EngineCtx::Drop. - drop(run_span); - crate::agent::engine::flush(); - std::process::exit(outcome.exit_code()); -} - -/// Ctrl+C stops cleanly at the next checkpoint. -fn install_ctrlc() -> Result<()> { - ctrlc::set_handler(|| { - STOP.store(true, Ordering::SeqCst); - crate::process::pid_registry::kill_all(); - eprintln!("\n[crucible] interrupt received — wrapping up the current step…"); - }) - .context("installing Ctrl+C handler") -} - -fn start_control_bridge( - args: &Args, - p: &Paths, - ledger: &std::sync::Arc, -) -> Result>> { - args.control_port - .map(|port| control::spawn_bridge(port, p.clone(), ledger.clone())) - .transpose() -} - -/// Every external input is recorded here before it takes effect, so this must exist -/// before anything can deliver one. -fn open_admission_ledger( - p: &Paths, - mode: forge::ndjson::Open, -) -> Result> { - crate::control::admission::AdmissionLedger::open(&p.admissions, mode).map(std::sync::Arc::new) -} - #[cfg(test)] mod tests { - use super::*; + use crate::cli::run::*; use crate::testing::args_from; use clap::Parser; diff --git a/crucible/src/cli/scored.rs b/crucible/src/cli/scored.rs new file mode 100644 index 00000000..a91e1637 --- /dev/null +++ b/crucible/src/cli/scored.rs @@ -0,0 +1,520 @@ +//! The scored loop's run path: a `crucible.toml` builds the [`World`] + [`Judge`], anchors every +//! path, picks a front-end, and calls [`crate::runloop::driver::run_loop`]. + +use crate::agent::harness::HarnessRuntime; +use crate::args::{Args, Paths, Prepared, Ui}; +use crate::control; +use crate::control::recovery::{RecoveryPlan, ResumeRecovery, classify_session, plan_recovery}; +use crate::errors::FileError; +use crate::manifest; +use crate::process::STOP; +use crate::report; +use crate::report::console; +use crate::report::stream; +use crate::runloop::driver::{LoopRuntime, run_loop}; +use crate::runloop::publish; +use anyhow::{Context, Result}; +use crucible::crucible::{Judge, World}; +use crucible_vcs::vcs; +use std::path::{Path, PathBuf}; +use std::sync::Arc; +use std::sync::atomic::Ordering; + +/// The scored run's own failures: CLI-flag combinations the parser can't express, workspace +/// setup, and the manifest fields a run needs. +#[derive(Debug, thiserror::Error)] +pub(crate) enum ScoredError { + #[error("watch-pr needs exactly one of --control-addr or --reseed")] + WatchPrNoSink, + #[error("watch-pr takes exactly one of --control-addr or --reseed, not both")] + WatchPrTwoSinks, + #[error("manifest [agent] needs `goal` or `goal_file` (or pass --goal)")] + NoGoal, + #[error("--control-port requires --ui stream (or --resume)")] + ControlPortNeedsStream, + #[error("resume: {message}")] + ResumeRefused { message: String }, + #[error(transparent)] + File(#[from] FileError), + #[error(transparent)] + Workspace(#[from] crate::cli::workspace::WorkspaceError), +} + +/// `crucible watch-pr`: poll draft PRs' review comments and steer a live run or reseed the next. +pub(crate) fn watch_pr( + pr: &[String], + control_addr: Option, + reseed: Option, + bot_user: String, + allow_user: Vec, + poll_secs: u64, + once: bool, +) -> Result<()> { + let sink = match (control_addr, reseed) { + (Some(addr), None) => crate::control::pr_watch::Sink::Steer(addr), + (None, Some(path)) => crate::control::pr_watch::Sink::Reseed(path), + (None, None) => return Err(ScoredError::WatchPrNoSink.into()), + (Some(_), Some(_)) => return Err(ScoredError::WatchPrTwoSinks.into()), + }; + let opts = crate::control::pr_watch::WatchOpts { + poll: std::time::Duration::from_secs(poll_secs), + bot_user, + authz: crate::control::pr_watch::Authz { + allow_users: allow_user, + ..Default::default() + }, + once, + }; + crate::control::pr_watch::watch_and_steer(pr, &sink, &opts) +} + +/// Load a `crucible.toml`, build the World + Judge from it, and drive the loop. The one run +/// path: every domain flows through here. Front-ends: headless / jsonl / stream, +/// plus `--resume`. +pub(crate) fn run_from_manifest(mut args: Args) -> Result<()> { + let manifest_path = args.manifest.clone().context( + "crucible needs a manifest: pass --manifest (see docs/crucible-contract.md)", + )?; + // A composite domain has a top-level `[composite]` table and a different shape; it runs + // multiple component workspaces under one base, so it takes the dedicated path. + if manifest::is_composite(&manifest_path) { + return run_composite(args, manifest_path); + } + let mut m = manifest::Manifest::load_frozen(&manifest_path)?; + // `parent()` of a bare `crucible.toml` is `Some("")`, which is not a usable cwd, treat + // an empty parent as the current directory. + let manifest_dir = manifest_path + .parent() + .filter(|p| !p.as_os_str().is_empty()) + .map(Path::to_path_buf) + .unwrap_or_else(|| PathBuf::from(".")); + m.resolve_workflow(&manifest_dir)?; + let workspace = manifest_dir.join(&m.workspace.dir); + let state = args + .state_dir + .clone() + .unwrap_or_else(|| manifest_dir.join("state")); + let skills = m.agent.toolbox_dir.as_ref().map(|d| manifest_dir.join(d)); + let p = Paths::for_manifest(workspace.clone(), state, &manifest_dir, skills); + + if !workspace.exists() { + crate::cli::workspace::manifest_setup(&m, &manifest_dir, &workspace)?; + // Inject baked judge/fixture files into the fresh clone (frozen judges + one-time fixtures). + // Frozen ones are also re-copied before each measure; the initial copy gives the agent a + // present, compilable harness from turn one. + for (src, dst, _frozen) in m.resolved_injects(&manifest_dir, &workspace)? { + manifest::apply_inject(&src, &dst) + .context("applying [workspace].inject after setup")?; + } + } + vcs::ensure_repo(&workspace).context("ensuring workspace is a git repo")?; + std::fs::create_dir_all(&p.state) + .with_context(|| format!("creating state dir {}", p.state.display()))?; + // The toolbox lands where the resolved harness discovers skills. + let harness = crate::cli::setup::pin_agent(&mut args, &m.agent)?; + crate::cli::workspace::install_toolbox( + &p, + &m.agent.toolbox_exclude, + harness.spec().skills_dir, + )?; + + // Fold the manifest's [agent] config onto Args (+ spawn the broker for openshell). + let frozen = crate::cli::setup::frozen_projection( + &m, + m.publish + .as_ref() + .and_then(|p| p.pr_repo.as_deref()) + .or(Some(args.pr_repo.as_str())), + &std::collections::BTreeMap::new(), + &p.session_log, + )?; + crate::cli::setup::apply_agent_cfg(&mut args, &m.agent, &m.secrets, &p.workspace, &frozen)?; + // Single-repo publish target: a `[publish] pr_repo` in the manifest wins over any `--pr-repo` the + // caller passed (the controller passes its per-repo default via the flag; a pack that names its + // own fork overrides it). Absent → keep the flag value (empty by default, so no PR opens). + if let Some(pr_repo) = m.publish.as_ref().and_then(|p| p.pr_repo.clone()) { + args.pr_repo = pr_repo; + } + // Declared pipeline artifacts, for the publish layer (PR-body embed + S3 upload). + args.artifacts = m.workspace.artifact.clone(); + + let (goal, template) = resolve_goal_template(&args, &m.agent, &manifest_dir)?; + // Cross-run memory: seed the prior run's tried-ideas ledger for this goal from S3 (best-effort), + // so a fresh run (or a future harness version) inherits history instead of re-walking dead ends. + let prior = publish::fetch_prior_results(&args.results_bucket, &goal).unwrap_or_default(); + if !prior.is_empty() { + let n = prior.lines().count(); + eprintln!("seeded {n} prior tried-idea row(s) from S3 (cross-run memory)"); + } + if m.is_task() { + eprintln!("task mode: no [judge] — every completed turn is kept and published unscored"); + } + // The world's comparability key, computed once the workspace has a HEAD + // to pin against. + let identity = crate::identity::for_manifest(&manifest_path, &manifest_dir, &workspace, &m) + .context("computing run identity")?; + let prep = Prepared { + run_id: publish::run_id(&goal), + prior, + goal, + template, + identity, + skip_baseline: m.is_task() || m.judge.as_ref().is_some_and(|j| j.skip_baseline), + preflight: m.preflight.clone(), + preflight_modes: m + .measure + .as_ref() + .and_then(crate::manifest::MeasureCfg::build_modes) + .unwrap_or_default(), + seed_diff: read_seed_diff(&manifest_dir, m.agent.seed_diff.as_deref())?, + }; + + // Frozen injects (the gate's own files) go to the judge so it re-establishes them before each + // scored measure, the agent can't edit the harness/test to game the gate. Resolve before the + // workspace move. + let frozen_injects: Vec<(PathBuf, PathBuf)> = m + .resolved_injects(&manifest_dir, &workspace)? + .into_iter() + .filter(|(_, _, frozen)| *frozen) + .map(|(src, dst, _)| (src, dst)) + .collect(); + args.search = m.search.clone(); + args.workflow = m.workflow.clone(); + args.workflow_frozen_injects = m.frozen_inject_pairs(&manifest_dir)?; + args.workflow_toolbox_exclude = m.agent.toolbox_exclude.clone(); + let world = m.build_world(workspace.clone()); + let judge = m.build_judge(workspace, frozen_injects)?; + + drive_loop(args, p, prep, world, judge) +} + +/// Read the `[agent].seed_diff` content for iteration 1's prompt. The identity build hashes the +/// same file; a declared seed that can't be read errors there first, this context is a backstop. +fn read_seed_diff(manifest_dir: &Path, seed_diff: Option<&str>) -> Result> { + seed_diff + .map(|rel| { + let path = manifest_dir.join(rel); + std::fs::read_to_string(&path) + .with_context(|| format!("reading [agent].seed_diff {}", path.display())) + }) + .transpose() +} + +/// Run a composite domain: set up each component's checkout under one base workspace, build +/// the multi-workspace [`CompositeWorld`] + the combined gate, and drive the same loop. The components +/// co-locate under the base so the agent has one cwd / one sandbox upload tree spanning both repos. +fn run_composite(mut args: Args, manifest_path: PathBuf) -> Result<()> { + let m = manifest::CompositeManifest::load_frozen(&manifest_path)?; + let manifest_dir = manifest_path + .parent() + .filter(|p| !p.as_os_str().is_empty()) + .map(Path::to_path_buf) + .unwrap_or_else(|| PathBuf::from(".")); + let base = m.base_dir(&manifest_dir); + let components = m.resolve_components(&manifest_dir)?; + eprintln!( + "composite `{}`: {} components — {}", + m.composite.name, + components.len(), + components + .iter() + .map(|c| format!("{} ({})", c.name, c.domain_dir.display())) + .collect::>() + .join(", ") + ); + + // Check out each component into /, then make each its own git repo (the per-component + // overlay CompositeWorld commits). The combined external deployment setup is a follow-up. + for c in &components { + if !c.workspace.exists() { + let repo = &c.manifest.repo; + let src = repo + .url + .clone() + .or_else(|| repo.path.clone()) + .with_context(|| format!("component `{}` [repo] needs url or path", c.name))?; + crate::cli::workspace::clone_repo(&src, repo.git_ref.as_deref(), &c.workspace) + .with_context(|| format!("cloning component `{}`", c.name))?; + } + vcs::ensure_repo(&c.workspace) + .with_context(|| format!("ensuring component `{}` is a git repo", c.name))?; + } + + // The world's comparability key: one component entry per checkout, all + // pinned now that every workspace has a HEAD. + let identity = crate::identity::for_composite(&manifest_path, &base, &components, &m) + .context("computing run identity")?; + + let state = args + .state_dir + .clone() + .unwrap_or_else(|| manifest_dir.join("state")); + let skills = m.agent.toolbox_dir.as_ref().map(|d| manifest_dir.join(d)); + // The agent's cwd is the base (it sees every component checkout as a subdir). + let p = Paths::for_manifest(base, state, &manifest_dir, skills); + std::fs::create_dir_all(&p.state) + .with_context(|| format!("creating state dir {}", p.state.display()))?; + let harness = crate::cli::setup::pin_agent(&mut args, &m.agent)?; + crate::cli::workspace::install_toolbox( + &p, + &m.agent.toolbox_exclude, + harness.spec().skills_dir, + )?; + + // A composite has no single-repo [publish]; its forks are per component. + let bounds = crate::cli::setup::run_bounds( + &m.outputs, + &m.build, + Some(args.pr_repo.as_str()), + &std::collections::BTreeMap::new(), + ); + let frozen = crate::cli::setup::FrozenProjection { + broker_env: crate::cli::setup::broker_bounds_env(&bounds, &p.session_log)?, + disclosure: Some(crate::exposure::covered_from( + crate::exposure::composite_capabilities(&m.agent, &m.capabilities), + )), + bounds: Some(bounds), + }; + crate::cli::setup::apply_agent_cfg(&mut args, &m.agent, &m.secrets, &p.workspace, &frozen)?; + // The per-component fork map for publish-on-keep, manifest-owned via [[component]].pr_repo. + args.component_pr_repos = m.component_pr_repos(); + let (goal, template) = resolve_goal_template(&args, &m.agent, &manifest_dir)?; + let prior = publish::fetch_prior_results(&args.results_bucket, &goal).unwrap_or_default(); + let prep = Prepared { + run_id: publish::run_id(&goal), + prior, + goal, + template, + identity, + skip_baseline: m.judge.skip_baseline, + preflight: m.preflight.clone(), + preflight_modes: m + .measure + .as_ref() + .and_then(crate::manifest::MeasureCfg::build_modes) + .unwrap_or_default(), + seed_diff: read_seed_diff(&manifest_dir, m.agent.seed_diff.as_deref())?, + }; + + args.search = m.search.clone(); + args.workflow = m.workflow.clone(); + args.workflow_frozen_injects = Vec::new(); + args.workflow_toolbox_exclude = m.agent.toolbox_exclude.clone(); + let world = m.build_world(&manifest_dir)?; + let judge = m.build_judge(&manifest_dir)?; + drive_loop(args, p, prep, world, judge) +} + +/// Resolve the run's goal + method-prompt template. `--goal`/`--goal-file` override the manifest (the +/// forge trigger injects a per-issue goal); otherwise the manifest's inline `goal` / `goal_file`. +fn resolve_goal_template( + args: &Args, + agent: &manifest::AgentCfg, + manifest_dir: &Path, +) -> Result<(String, String), ScoredError> { + let goal = if let Some(g) = &args.goal { + g.clone() + } else if let Some(f) = &args.goal_file { + std::fs::read_to_string(f).map_err(FileError::at("reading --goal-file", f))? + } else { + match (&agent.goal, &agent.goal_file) { + (Some(g), _) => g.clone(), + (None, Some(f)) => { + let path = manifest_dir.join(f); + std::fs::read_to_string(&path).map_err(FileError::at("reading goal_file", &path))? + } + (None, None) => return Err(ScoredError::NoGoal), + } + }; + let template = match &agent.method_prompt { + Some(mp) => { + let path = manifest_dir.join(mp); + std::fs::read_to_string(&path).map_err(FileError::at("reading method_prompt", &path))? + } + None => "{{GOAL}}\n\nStatus: {{STATUS}}\n{{STEER}}".to_string(), + }; + Ok((goal, template)) +} + +/// The shared loop tail: install Ctrl+C, then pick the front-end (resume / jsonl / stream / +/// console) and drive [`run_loop`]. Single-domain and composite runs both end here. +fn drive_loop( + args: Args, + p: Paths, + prep: Prepared, + world: Arc, + judge: Arc, +) -> Result<()> { + install_ctrlc()?; + if args.control_port.is_some() && !args.resume && args.ui != Ui::Stream { + return Err(ScoredError::ControlPortNeedsStream.into()); + } + + // When the controller dispatched this loop pod, adopt its dispatch span as the run's trace + // parent so Tempo shows controller → run → turn in one tree; the openshell turn spans nest under + // this span because they're created on this same thread. `None` (a local run, or telemetry off) + // leaves the turn spans rooting themselves independently. Held across the whole loop, then + // dropped below so the span closes and the OTLP layer batches it before `flush`. + let run_span = crate::agent::engine::run_span(&p.workspace.to_string_lossy(), &prep.run_id); + // A signal would otherwise kill the process with this span still open and the batch unflushed, + // so every rolled loop pod loses its run span. Installed here, where the span exists, rather + // than behind a static. + crate::agent::engine::abort_on_signal(run_span.clone()); + + // The liveness beat runs for the whole loop, parented to the run span so its beats hang off the + // run in Tempo. Declared AFTER `run_span` so the guard's Drop runs first and the beat is joined + // before the span it holds goes away, on the error returns below as well as the success path. + let heartbeat = crate::control::heartbeat::period_from_env() + .map(|period| crate::control::heartbeat::start(period, run_span.clone())); + let beat = heartbeat + .as_ref() + .map(crate::control::heartbeat::BeatGuard::beat); + + let outcome = { + let _run_guard = run_span.as_ref().map(tracing::Span::enter); + if args.resume { + // Replay the parked log, then continue in append mode. A NoOp exits 0 + // WITHOUT re-running the finish path (replaying finish re-published the + // kept candidate each crash-loop lap); Refuse keeps exit code 2's meaning. + let recovered = classify_session(&p.session_log)?; + match plan_recovery(&recovered, args.iterations, args.max_cost) { + RecoveryPlan::NoOp { message } => { + eprintln!("resume: {message}"); + return Ok(()); + } + RecoveryPlan::Refuse { message } => { + return Err(ScoredError::ResumeRefused { message }.into()); + } + RecoveryPlan::Continue { + repark, + pending_regime, + } => { + let recovery = ResumeRecovery { + class: recovered.classification.class(), + iter: recovered.classification.iter(), + detail: recovered.classification.detail(), + repark, + pending_regime, + }; + let meta = report::reporter::RunMeta::from_args(&args); + let r = stream::SessionReporter::resume(&p, meta)?; + // Fold the prior run's admissions before the bridge is up, so no + // inbound command can land on a half-built index. + let ledger = open_admission_ledger(&p, forge::ndjson::Open::Fold)?; + let control = start_control_bridge(&args, &p, &ledger)?; + let (_reporter, outcome) = run_loop( + &args, + &p, + &prep, + r, + &world, + &judge, + LoopRuntime { + control: control.clone(), + resume: Some(recovered.resume), + recovery: Some(recovery), + ledger: Some(ledger), + heartbeat: beat.clone(), + }, + ); + outcome? + } + } + } else { + let meta = report::reporter::RunMeta::from_args(&args); + match args.ui { + Ui::Jsonl => { + let r = stream::SessionReporter::stdout(meta); + let (_reporter, outcome) = run_loop( + &args, + &p, + &prep, + r, + &world, + &judge, + LoopRuntime { + heartbeat: beat.clone(), + ..LoopRuntime::default() + }, + ); + outcome? + } + Ui::Stream => { + let r = stream::SessionReporter::stream(&p, meta)?; + // A fresh run must not inherit the last run's un-drained inputs. + let ledger = open_admission_ledger(&p, forge::ndjson::Open::Truncate)?; + let control = start_control_bridge(&args, &p, &ledger)?; + let (_reporter, outcome) = run_loop( + &args, + &p, + &prep, + r, + &world, + &judge, + LoopRuntime { + control: control.clone(), + ledger: Some(ledger), + heartbeat: beat.clone(), + ..LoopRuntime::default() + }, + ); + outcome? + } + _ => { + let r = console::ConsoleReporter; + let (_reporter, outcome) = run_loop( + &args, + &p, + &prep, + r, + &world, + &judge, + LoopRuntime { + heartbeat: beat.clone(), + ..LoopRuntime::default() + }, + ); + outcome? + } + } + } + }; + crate::report::ingest_client::deliver_run_evidence(&p); + // Explicit because this path ends in `process::exit`, which runs no destructors: the beat + // thread holds a clone of the run span, so a live beat would keep it open past the flush below. + drop(heartbeat); + // Close the run span (drop it, now that its guard is gone) so the OTLP layer batches it, THEN + // flush: the loop exits via process::exit, which skips EngineCtx::Drop. + drop(run_span); + crate::agent::engine::flush(); + std::process::exit(outcome.exit_code()); +} + +/// Ctrl+C stops cleanly at the next checkpoint. +fn install_ctrlc() -> Result<()> { + ctrlc::set_handler(|| { + STOP.store(true, Ordering::SeqCst); + crate::process::pid_registry::kill_all(); + eprintln!("\n[crucible] interrupt received — wrapping up the current step…"); + }) + .context("installing Ctrl+C handler") +} + +fn start_control_bridge( + args: &Args, + p: &Paths, + ledger: &std::sync::Arc, +) -> Result>> { + args.control_port + .map(|port| control::bridge::spawn_bridge(port, p.clone(), ledger.clone())) + .transpose() +} + +/// Every external input is recorded here before it takes effect, so this must exist +/// before anything can deliver one. +fn open_admission_ledger( + p: &Paths, + mode: forge::ndjson::Open, +) -> Result> { + crate::control::admission::AdmissionLedger::open(&p.admissions, mode).map(std::sync::Arc::new) +} diff --git a/crucible/src/runloop/selftest.rs b/crucible/src/cli/selftest.rs similarity index 93% rename from crucible/src/runloop/selftest.rs rename to crucible/src/cli/selftest.rs index 05d61fca..fedfe0fd 100644 --- a/crucible/src/runloop/selftest.rs +++ b/crucible/src/cli/selftest.rs @@ -7,6 +7,7 @@ use crate::manifest::SelftestCfg; use anyhow::{Context, Result}; use crucible::crucible::Direction; use crucible::crucible::{Judge, MeasureCtx, World}; +use crucible_contract::refine::{ControlEvidence, ReadingEvidence, SelftestEvidence}; use std::path::Path; #[derive(Debug, thiserror::Error)] @@ -142,9 +143,42 @@ fn stage(workspace: &Path, cmd: &str) -> Result<()> { Ok(()) } +impl From<&SelftestReport> for SelftestEvidence { + fn from(r: &SelftestReport) -> Self { + let direction = match r.direction { + crucible::crucible::Direction::Higher => "higher", + crucible::crucible::Direction::Lower => "lower", + } + .to_string(); + SelftestEvidence { + direction, + runs: r.runs, + good: control_evidence(&r.good), + bad: control_evidence(&r.bad), + } + } +} + +fn control_evidence(c: &ControlResult) -> ControlEvidence { + ControlEvidence { + cmd: c.cmd.clone(), + mean: c.mean_score, + all_valid: c.all_valid, + readings: c + .readings + .iter() + .map(|r| ReadingEvidence { + valid: r.valid, + score: r.score, + note: r.note.clone(), + }) + .collect(), + } +} + #[cfg(test)] mod tests { - use super::*; + use crate::cli::selftest::*; use crate::manifest::SelftestCfg; use crucible::command_world::GitWorld; use std::fs; diff --git a/crucible/src/control/bridge.rs b/crucible/src/control/bridge.rs new file mode 100644 index 00000000..607e6adf --- /dev/null +++ b/crucible/src/control/bridge.rs @@ -0,0 +1,1507 @@ +//! In-process remote control bridge for a stream-mode loop. +//! +//! The bridge is intentionally transport-agnostic: it listens on a plain TCP socket, +//! broadcasts new `state/session.jsonl` lines with a monotonic `seq`, and applies +//! inbound NDJSON commands against the live loop process. + +use crate::args::Paths; +use crate::control::admission::{AdmissionLedger, Admitted}; +use crate::process::STOP; +use anyhow::{Context, Result}; +use crucible_contract::LoopPhase; +use crucible_contract::admission::{ + AdmissionKey, AdmissionOutcome, AdmittedInput, MAX_KEY_LEN, SteerSource, +}; +use serde::{Deserialize, Serialize}; +use serde_json::{Value, json}; +use std::collections::VecDeque; +use std::fs::{File, OpenOptions}; +use std::io::{BufRead, BufReader, Seek, SeekFrom, Write}; +use std::net::{TcpListener, TcpStream}; +use std::path::Path; +use std::sync::atomic::{AtomicBool, AtomicU64, Ordering}; +use std::sync::{Arc, Mutex}; +use std::thread; +use std::time::{Duration, SystemTime, UNIX_EPOCH}; + +type Client = Arc>; + +/// How many recently-broadcast session lines the bridge retains in memory for replay. A `tail +/// from_seq=N` inside this window is served straight from the ring; anything older is replayed from +/// `session.jsonl` itself (see [`tail_with_file_replay`]), the seq IS the 1-based file line index, +/// so the file is the authoritative pre-ring history and a late joiner gets the whole run. +const HISTORY_CAP: usize = 4096; + +/// One session line stamped with its monotonic `seq`, kept in the bridge's replay ring. +#[derive(Clone)] +struct Stamped { + seq: u64, + line: String, +} + +/// The broadcast hub: a bounded replay ring plus the currently-subscribed client writers, under one +/// lock so a new subscriber's replay-then-live handoff is atomic against the tail thread's +/// broadcasts, every line is delivered exactly once, in order, with no gap at the boundary. +#[derive(Default)] +struct Hub { + inner: Mutex, +} + +#[derive(Default)] +struct HubInner { + history: VecDeque, + clients: Vec, +} + +impl Hub { + /// Stamp-and-broadcast one line: retain it in the replay ring (bounded) and write it to every + /// live subscriber, dropping any whose socket has closed. + fn publish(&self, seq: u64, line: String) { + let Ok(mut g) = self.inner.lock() else { + return; + }; + g.history.push_back(Stamped { + seq, + line: line.clone(), + }); + while g.history.len() > HISTORY_CAP { + g.history.pop_front(); + } + g.clients.retain(|c| write_line(c, &line)); + } + + /// The newest seq in the ring (0 before anything was published), the file-replay loop's + /// per-pass target. + fn newest_seq(&self) -> u64 { + self.inner + .lock() + .map(|g| g.history.back().map(|s| s.seq).unwrap_or(0)) + .unwrap_or(0) + } + + /// Attempt the atomic file→ring handoff for a client that has already received everything up to + /// `last` (from the file): if the ring still covers `last + 1` (or nothing was ever published), + /// replay the retained lines after `last` and subscribe, all under the one lock, so nothing + /// published concurrently is missed or doubled. Returns false when the ring's oldest has moved + /// past `last + 1` (the file grew faster than the replay): the caller reads more file first. + fn try_handoff(&self, writer: &Client, last: u64) -> bool { + let Ok(mut g) = self.inner.lock() else { + return true; // poisoned: behave like the plain subscribe arms elsewhere + }; + let covered = match g.history.front() { + None => true, + Some(oldest) => oldest.seq <= last + 1, + }; + if !covered { + return false; + } + for s in g.history.iter().filter(|s| s.seq > last) { + let _ = write_line(writer, &s.line); + } + g.clients.push(writer.clone()); + true + } + + /// Drop a client (its reader thread ended / socket closed) so broadcasts stop targeting it. + fn unsubscribe(&self, writer: &Client) { + let Ok(mut g) = self.inner.lock() else { + return; + }; + g.clients.retain(|c| !Arc::ptr_eq(c, writer)); + } +} + +#[derive(Default)] +pub(crate) struct ControlState { + paused: AtomicBool, + next_seq: AtomicU64, + live_max_cost: Mutex>, + status: Mutex, + /// Pending re-scope, drained by the loop at the next iteration head. The key travels + /// with the regime so the loop can settle the admission where it applies it. + rescope: Mutex>, + /// The regime an open approval would grant (attended path): the loop records it from the agent's + /// pending marker so an operator `approve` keystroke can turn it into a `rescope` without naming + /// the regime. `None` when there's no approval awaiting an operator. + pending_regime: Mutex>, + /// A rejected pending approval, drained by the park as the terminal "not granted" + /// outcome. + deny: Mutex>, +} + +#[derive(Clone, Debug, Serialize)] +pub(crate) struct StatusSnapshot { + pub phase: LoopPhase, + pub iter: u32, + #[serde(skip_serializing_if = "Option::is_none")] + pub best_score: Option, + pub spend: f64, + pub paused: bool, + #[serde(skip_serializing_if = "Option::is_none")] + pub max_cost: Option, +} + +impl Default for StatusSnapshot { + fn default() -> Self { + Self { + phase: LoopPhase::Starting, + iter: 0, + best_score: None, + spend: 0.0, + paused: false, + max_cost: None, + } + } +} + +impl ControlState { + pub(crate) fn is_paused(&self) -> bool { + self.paused.load(Ordering::SeqCst) + } + + pub(crate) fn live_max_cost(&self) -> Option { + self.live_max_cost.lock().ok().and_then(|g| *g) + } + + /// Drained by the loop at the iteration head, which re-baselines and settles it. + pub(crate) fn take_rescope(&self) -> Option<(AdmissionKey, String)> { + self.rescope.lock().ok()?.take() + } + + /// Non-consuming peek: is a re-scope pending? The park polls this so the drain + /// ([`take_rescope`](Self::take_rescope)) stays the single place that re-baselines. + pub(crate) fn has_rescope(&self) -> bool { + self.rescope.lock().is_ok_and(|g| g.is_some()) + } + + #[cfg(test)] + pub(crate) fn phase(&self) -> LoopPhase { + self.status + .lock() + .map(|s| s.phase) + .unwrap_or(LoopPhase::Starting) + } + + pub(crate) fn set_phase(&self, phase: LoopPhase) { + if let Ok(mut status) = self.status.lock() { + status.phase = phase; + } + } + + pub(crate) fn set_progress(&self, iter: u32, best_score: Option, spend: f64) { + let Ok(mut status) = self.status.lock() else { + return; + }; + status.iter = iter; + status.best_score = best_score; + status.spend = spend; + } + + pub(crate) fn set_spend(&self, spend: f64) { + if let Ok(mut status) = self.status.lock() { + status.spend = spend; + } + } + + pub(crate) fn pause(&self) { + self.paused.store(true, Ordering::SeqCst); + } + + fn resume(&self) { + self.paused.store(false, Ordering::SeqCst); + } + + pub(crate) fn set_live_max_cost(&self, usd: f64) { + if let Ok(mut cap) = self.live_max_cost.lock() { + *cap = Some(usd); + } + } + + /// Arm the re-scope the next iteration head drains. Returns the key of an undrained + /// re-scope this displaced, which the caller settles as superseded. + pub(crate) fn set_rescope(&self, key: AdmissionKey, regime: String) -> Option { + let Ok(mut slot) = self.rescope.lock() else { + return None; + }; + slot.replace((key, regime)).map(|(old, _)| old) + } + + /// Record the regime an open approval would grant, so an operator `approve` can resolve it. + /// The loop sets this when it reads the agent's pending-provisioning marker. + pub(crate) fn set_pending_regime(&self, regime: String) { + if let Ok(mut slot) = self.pending_regime.lock() { + *slot = Some(regime); + } + } + + /// Take the regime an open approval would grant. The caller admits the derived + /// re-scope BEFORE arming it, and hands the regime back if the record can't be + /// written. + pub(crate) fn take_pending_regime(&self) -> Option { + self.pending_regime.lock().ok()?.take() + } + + /// Record a denial, clearing any pending regime so a later `approve` can't resurrect + /// a denied ask. Returns the key of an undrained denial this displaced. + pub(crate) fn set_deny(&self, key: AdmissionKey, reason: String) -> Option { + if let Ok(mut pending) = self.pending_regime.lock() { + *pending = None; + } + let Ok(mut slot) = self.deny.lock() else { + return None; + }; + slot.replace((key, reason)).map(|(old, _)| old) + } + + /// Drain a denial if one arrived (the terminal "not granted" outcome the park waits on). + pub(crate) fn take_deny(&self) -> Option<(AdmissionKey, String)> { + self.deny.lock().ok()?.take() + } + + fn snapshot(&self) -> StatusSnapshot { + let mut status = self + .status + .lock() + .map(|s| s.clone()) + .unwrap_or_else(|_| StatusSnapshot::default()); + status.paused = self.is_paused(); + status.max_cost = self.live_max_cost(); + status + } + + /// Assign the next monotonic `seq` and stamp it onto the line, returning both (the seq feeds the + /// replay ring, the stamped line goes on the wire). `None` for a blank/torn line, the seq is + /// still consumed so the numbering matches the file's line count. + fn stamp_session_line(&self, line: &str) -> Option<(u64, String)> { + let seq = self.next_seq.fetch_add(1, Ordering::SeqCst) + 1; + stamp_session_line(line, seq).map(|stamped| (seq, stamped)) + } +} + +/// Start a detached TCP bridge thread. The listener binds to all interfaces so it works +/// inside a pod behind `kubectl port-forward`, while still being reachable as localhost +/// for local smoke tests. +pub(crate) fn spawn_bridge( + port: u16, + paths: Paths, + ledger: Arc, +) -> Result> { + let listener = TcpListener::bind(("0.0.0.0", port)) + .with_context(|| format!("binding control port {port}"))?; + let state = Arc::new(ControlState::default()); + let hub = Arc::new(Hub::default()); + spawn_session_tail(paths.session_log.clone(), state.clone(), hub.clone()); + + let accept_state = state.clone(); + thread::spawn(move || { + for stream in listener.incoming() { + match stream { + Ok(stream) => serve_client( + stream, + &paths, + accept_state.clone(), + hub.clone(), + ledger.clone(), + ), + Err(_) => thread::sleep(Duration::from_millis(100)), + } + } + }); + + Ok(state) +} + +fn spawn_session_tail(path: std::path::PathBuf, state: Arc, hub: Arc) { + thread::spawn(move || { + let mut pos: Option = None; + loop { + if let Ok(mut file) = File::open(&path) { + let len = file.metadata().map(|m| m.len()).unwrap_or(0); + // Start from offset 0 (not the live edge): every line the file already holds gets + // stamped, so seq == the 1-based file line index and file replay can serve history + // older than the ring. + let current = match pos { + Some(p) if len >= p => p, + Some(_) => 0, + None => 0, + }; + let _ = len; + pos = Some(read_new_lines(&mut file, current, &state, &hub)); + } + thread::sleep(Duration::from_millis(100)); + } + }); +} + +fn read_new_lines(file: &mut File, start: u64, state: &ControlState, hub: &Hub) -> u64 { + if file.seek(SeekFrom::Start(start)).is_err() { + return start; + } + let mut pos = start; + let mut reader = BufReader::new(file); + loop { + let mut line = String::new(); + let n = match reader.read_line(&mut line) { + Ok(0) => break, + Ok(n) => n as u64, + Err(_) => break, + }; + if !line.ends_with('\n') { + break; + } + pos += n; + if let Some((seq, stamped)) = state.stamp_session_line(&line) { + hub.publish(seq, stamped); + } + } + pos +} + +/// Write one NDJSON line to a client and flush, reporting whether the socket is still alive (a dead +/// one is dropped from the broadcast set by the caller). +fn write_line(client: &Client, line: &str) -> bool { + let Ok(mut stream) = client.lock() else { + return false; + }; + writeln!(stream, "{line}") + .and_then(|_| stream.flush()) + .is_ok() +} + +/// Serve a `tail from_seq` that may reach back past the in-memory ring: replay the missing range +/// from `session.jsonl` itself (seq == the 1-based file line index, unparseable lines consume their +/// seq silently, the same rule the live stamping applies), then hand off to the ring atomically +/// via [`Hub::try_handoff`]. Loops because the file can grow while a pass streams: each pass reads +/// up to the ring's newest at pass start, and the handoff only succeeds when the ring still covers +/// the next line, so every line is delivered exactly once, in order, with no gap at the boundary. +fn tail_with_file_replay(path: &Path, hub: &Hub, writer: &Client, from_seq: u64) { + let mut last = from_seq; + loop { + if hub.try_handoff(writer, last) { + return; + } + let target = hub.newest_seq(); + if target <= last { + // The ring is ahead of `last+1`'s coverage yet has nothing newer than `last`, only a + // transient between a publish and our read. Yield and retry. + thread::sleep(Duration::from_millis(10)); + continue; + } + last = replay_file_range(path, writer, last, target); + } +} + +/// Stream file lines with 1-based index in `(after, upto]` to the client, stamped exactly like the +/// live path. Returns the last index actually reached (== `upto` unless the file is unexpectedly +/// short, then the handoff loop retries from there). Only full (newline-terminated) lines within +/// `upto` exist by construction: the tail thread already stamped them. +fn replay_file_range(path: &Path, writer: &Client, after: u64, upto: u64) -> u64 { + let Ok(file) = File::open(path) else { + return upto; // vanished file: fall through to the ring, which is all that's left + }; + let reader = BufReader::new(file); + let mut idx: u64 = 0; + for line in reader.lines() { + let Ok(line) = line else { break }; + idx += 1; + if idx > upto { + break; + } + if idx <= after { + continue; + } + if let Some(stamped) = stamp_session_line(&line, idx) { + let _ = write_line(writer, &stamped); + } + } + idx.min(upto).max(after) +} + +fn serve_client( + stream: TcpStream, + paths: &Paths, + state: Arc, + hub: Arc, + ledger: Arc, +) { + let Ok(writer) = stream.try_clone() else { + return; + }; + let writer = Arc::new(Mutex::new(writer)); + + let paths = paths.clone(); + thread::spawn(move || { + let reader = BufReader::new(stream); + for line in reader.lines() { + let reply = match line { + // `tail` subscribes this socket to the live broadcast (replaying from `from_seq`), + // so it's handled here where the writer lives rather than in `apply_command`. + Ok(line) => match parse_request(&line) { + Ok(ControlRequest { + cmd: ControlCommand::Tail { from_seq }, + .. + }) => { + tail_with_file_replay(&paths.session_log, &hub, &writer, from_seq); + json!({"ok": true, "cmd": "tail", "from_seq": from_seq}) + } + Ok(req) => apply_command(req, &state, &ledger), + Err(e) => json!({"ok": false, "error": e}), + }, + Err(e) => json!({"ok": false, "error": e.to_string()}), + }; + write_reply(&writer, &reply); + } + // The reader loop ends when the client closes the socket: drop it from the broadcast set so + // publishes stop targeting a dead writer (a tail subscriber that just went away). + hub.unsubscribe(&writer); + }); +} + +fn write_reply(writer: &Client, reply: &Value) { + let Ok(mut stream) = writer.lock() else { + return; + }; + let _ = writeln!(stream, "{reply}"); + let _ = stream.flush(); +} + +/// Nothing mutates the run before its `Admitted` line is on disk; a redelivered command +/// converges; `stop`/`abort` fire even when the record can't be written. +fn apply_command(req: ControlRequest, state: &ControlState, ledger: &AdmissionLedger) -> Value { + let ControlRequest { id, cmd } = req; + match cmd { + ControlCommand::Tail { from_seq } => { + // Subscription happens at the socket layer ([`serve_client`]) where the writer lives; a + // direct caller that reaches here (never the live relay) only gets the ack. + json!({"ok": true, "cmd": "tail", "from_seq": from_seq}) + } + ControlCommand::Abort => stop_the_run(id, AdmittedInput::Abort, ledger), + ControlCommand::Stop => stop_the_run(id, AdmittedInput::Stop, ledger), + ControlCommand::Pause => admitted(ledger, id, AdmittedInput::Pause, |key| { + state.pause(); + let _ = ledger.settle(key, AdmissionOutcome::Applied, "paused"); + json!({"ok": true, "cmd": "pause"}) + }), + ControlCommand::Resume => admitted(ledger, id, AdmittedInput::Resume, |key| { + state.resume(); + let _ = ledger.settle(key, AdmissionOutcome::Applied, "resumed"); + json!({"ok": true, "cmd": "resume"}) + }), + // The ledger IS the steer queue: the bridge no longer writes STEER.md. + ControlCommand::Steer { text } => admitted( + ledger, + id, + AdmittedInput::Steer { + text, + from: SteerSource::Operator, + }, + |_| json!({"ok": true, "cmd": "steer"}), + ), + ControlCommand::SetBudget { usd } => { + admitted(ledger, id, AdmittedInput::SetBudget { usd }, |key| { + state.set_live_max_cost(usd); + let _ = ledger.settle(key, AdmissionOutcome::Applied, "live cap set"); + json!({"ok": true, "cmd": "set-budget", "usd": usd}) + }) + } + ControlCommand::Rescope { regime } => admitted( + ledger, + id, + AdmittedInput::Rescope { + regime: regime.clone(), + }, + |key| { + supersede(ledger, state.set_rescope(key.clone(), regime.clone()), key); + json!({"ok": true, "cmd": "rescope", "regime": regime}) + }, + ), + ControlCommand::Approve => admitted(ledger, id, AdmittedInput::Approve, |key| { + grant_approval(state, ledger, key) + }), + ControlCommand::Deny { reason } => admitted( + ledger, + id, + AdmittedInput::Deny { + reason: reason.clone(), + }, + |key| { + supersede(ledger, state.set_deny(key.clone(), reason.clone()), key); + json!({"ok": true, "cmd": "deny", "reason": reason}) + }, + ), + ControlCommand::Status => { + json!({"ok": true, "cmd": "status", "status": state.snapshot()}) + } + } +} + +/// The grant is recorded BEFORE the slot is armed, under a key derived from the ASK, so +/// approvals of the same ask converge and a resume can recognize a recorded grant. +fn grant_approval( + state: &ControlState, + ledger: &AdmissionLedger, + approve_key: &AdmissionKey, +) -> Value { + let Some(regime) = state.take_pending_regime() else { + let _ = ledger.settle( + approve_key, + AdmissionOutcome::Rejected, + "no approval was awaiting an operator", + ); + return json!({"ok": false, "cmd": "approve", "error": "no pending approval"}); + }; + let derived = AdmissionKey::rescope_from(&AdmissionKey::approve(®ime)); + let recorded = ledger.admit( + Some(derived), + AdmittedInput::Rescope { + regime: regime.clone(), + }, + ); + match recorded { + Ok(Admitted::Fresh(rescope_key)) => { + supersede( + ledger, + state.set_rescope(rescope_key.clone(), regime.clone()), + &rescope_key, + ); + let _ = ledger.settle( + approve_key, + AdmissionOutcome::Applied, + &format!("granted '{regime}'"), + ); + json!({"ok": true, "cmd": "approve", "regime": regime}) + } + // The grant is already on the record (a redelivered approve, or one sent after a + // resume re-registered the ask): converge on it instead of arming a second re-scope. + Ok(Admitted::Duplicate(..)) => { + let _ = ledger.settle( + approve_key, + AdmissionOutcome::Applied, + &format!("'{regime}' was already granted"), + ); + json!({"ok": true, "cmd": "approve", "regime": regime, "dup": true}) + } + Ok(Admitted::Conflict(key)) => { + state.set_pending_regime(regime); + let _ = ledger.settle(approve_key, AdmissionOutcome::Rejected, "grant conflict"); + json!({"ok": false, "cmd": "approve", + "error": format!("idempotency conflict on the derived grant {key}")}) + } + Err(e) => { + // Hand the ask back: an approval we could not record must stay approvable + // rather than vanish. + state.set_pending_regime(regime); + json!({"ok": false, "cmd": "approve", + "error": format!("admission not recorded: {e:#}")}) + } + } +} + +/// Admit, then apply. The effect runs only for a fresh admission; a conflict or ledger +/// write failure refuses the command rather than acting unrecorded. +fn admitted( + ledger: &AdmissionLedger, + id: Option, + input: AdmittedInput, + effect: impl FnOnce(&AdmissionKey) -> Value, +) -> Value { + let cmd = input.cmd(); + match ledger.admit(id, input) { + Ok(Admitted::Fresh(key)) => { + let mut reply = effect(&key); + with_field(&mut reply, "key", json!(key.as_str())); + reply + } + Ok(Admitted::Duplicate(key, outcome)) => { + let mut reply = json!({"ok": true, "cmd": cmd, "key": key.as_str(), "dup": true}); + if let Some(outcome) = outcome { + with_field(&mut reply, "outcome", json!(outcome.as_str())); + } + reply + } + Ok(Admitted::Conflict(key)) => json!({ + "ok": false, "cmd": cmd, "key": key.as_str(), + "error": "idempotency conflict: key already admitted with a different payload" + }), + Err(e) => json!({ + "ok": false, "cmd": cmd, "error": format!("admission not recorded: {e:#}") + }), + } +} + +/// Stop and abort are safety valves: they apply even when the ledger write fails, and say +/// so (`unrecorded`) rather than refusing to stop a running loop over a disk error. +fn stop_the_run(id: Option, input: AdmittedInput, ledger: &AdmissionLedger) -> Value { + let cmd = input.cmd(); + let abort = matches!(input, AdmittedInput::Abort); + let recorded = ledger.admit(id, input); + STOP.store(true, Ordering::SeqCst); + if abort { + crate::process::pid_registry::kill_all(); + } + match recorded { + Ok(Admitted::Fresh(key)) => { + let _ = ledger.settle(&key, AdmissionOutcome::Applied, "stop flag set"); + json!({"ok": true, "cmd": cmd, "key": key.as_str()}) + } + Ok(Admitted::Duplicate(key, _)) => { + json!({"ok": true, "cmd": cmd, "key": key.as_str(), "dup": true}) + } + Ok(Admitted::Conflict(_)) | Err(_) => { + json!({"ok": true, "cmd": cmd, "unrecorded": true}) + } + } +} + +/// Settle the admission a newly-armed one displaced as superseded. +fn supersede(ledger: &AdmissionLedger, displaced: Option, by: &AdmissionKey) { + if let Some(old) = displaced { + let _ = ledger.settle( + &old, + AdmissionOutcome::Superseded, + &format!("replaced by {by} before the loop drained it"), + ); + } +} + +fn with_field(reply: &mut Value, field: &str, value: Value) { + if let Some(obj) = reply.as_object_mut() { + obj.insert(field.to_string(), value); + } +} + +/// The legacy cross-process stop file external tooling may write. +/// Unknown fields are ignored so older writers that include debugging metadata still work. +#[derive(Deserialize, Default)] +struct StopFile { + #[serde(default)] + stop: bool, +} + +pub(crate) fn stop_file_says_stop(path: &Path) -> bool { + std::fs::read_to_string(path) + .ok() + .and_then(|s| serde_json::from_str::(&s).ok()) + .map(|c| c.stop) + .unwrap_or(false) +} + +#[derive(Debug, PartialEq)] +enum ControlCommand { + Abort, + Stop, + Pause, + Resume, + Steer { + text: String, + }, + SetBudget { + usd: f64, + }, + Rescope { + regime: String, + }, + Approve, + Deny { + reason: String, + }, + Status, + /// Subscribe to the live session broadcast, replaying retained lines with `seq > from_seq` + /// first (resume support for the read-only live relay). `from_seq = 0` = from the ring's oldest. + Tail { + from_seq: u64, + }, +} + +/// One inbound command plus its optional idempotency key. No `id` means a generated one +/// (every delivery is a fresh input); a natural key makes redelivery converge. +#[derive(Debug, PartialEq)] +struct ControlRequest { + id: Option, + cmd: ControlCommand, +} + +fn parse_request(line: &str) -> std::result::Result { + let value: Value = serde_json::from_str(line.trim()).map_err(|e| e.to_string())?; + if let Some(cmd) = value.as_str() { + // String-form commands ("stop") carry no id; the server generates one. + return command_from_name(cmd, &value).map(|cmd| ControlRequest { id: None, cmd }); + } + let obj = value + .as_object() + .ok_or_else(|| "command must be a JSON object or string".to_string())?; + let cmd = obj + .get("cmd") + .or_else(|| obj.get("command")) + .or_else(|| obj.get("kind")) + .and_then(Value::as_str) + .ok_or_else(|| "command object needs cmd".to_string())?; + let id = parse_id(obj.get("id"))?; + command_from_name(cmd, &value).map(|cmd| ControlRequest { id, cmd }) +} + +fn parse_id(raw: Option<&Value>) -> std::result::Result, String> { + let Some(raw) = raw.filter(|v| !v.is_null()) else { + return Ok(None); + }; + let id = raw + .as_str() + .ok_or_else(|| "id must be a string".to_string())? + .trim(); + if id.is_empty() { + return Err("id must not be empty".into()); + } + if id.len() > MAX_KEY_LEN { + return Err(format!("id must be at most {MAX_KEY_LEN} bytes")); + } + Ok(Some(AdmissionKey::new(id))) +} + +fn command_from_name(cmd: &str, value: &Value) -> std::result::Result { + match cmd { + "abort" => Ok(ControlCommand::Abort), + "stop" => Ok(ControlCommand::Stop), + "pause" => Ok(ControlCommand::Pause), + "resume" => Ok(ControlCommand::Resume), + "approve" => Ok(ControlCommand::Approve), + "deny" => { + // Reason is optional (an operator may just reject); default to a generic note. + let reason = value + .get("reason") + .and_then(Value::as_str) + .unwrap_or("rejected") + .trim() + .to_string(); + Ok(ControlCommand::Deny { reason }) + } + "status" => Ok(ControlCommand::Status), + "tail" => { + // The seq to resume after; absent/0 = replay from the ring's oldest retained line. + let from_seq = value.get("from_seq").and_then(Value::as_u64).unwrap_or(0); + Ok(ControlCommand::Tail { from_seq }) + } + "steer" => { + let text = value + .get("text") + .and_then(Value::as_str) + .ok_or_else(|| "steer needs text".to_string())?; + Ok(ControlCommand::Steer { + text: text.trim().to_string(), + }) + } + "set-budget" | "set_budget" => { + let usd = value + .get("usd") + .and_then(Value::as_f64) + .ok_or_else(|| "set-budget needs numeric usd".to_string())?; + if !usd.is_finite() || usd < 0.0 { + return Err("set-budget usd must be a finite non-negative number".into()); + } + Ok(ControlCommand::SetBudget { usd }) + } + "rescope" | "re-scope" => { + // A judge-changing grant approved out-of-band: re-baseline into this regime. + let regime = value + .get("regime") + .and_then(Value::as_str) + .unwrap_or("rescoped") + .trim() + .to_string(); + Ok(ControlCommand::Rescope { regime }) + } + other => Err(format!("unknown control command {other:?}")), + } +} + +/// Append one steer payload in the marker-wrapped shape [`crate::control::admission::drain_steer`] +/// reads. `pr_watch`'s reseed sink writes this file when there is no live control bridge; +/// the bridge itself admits `steer` straight into the ledger. +pub(crate) fn append_steer(path: &Path, text: &str) -> std::io::Result<()> { + if let Some(dir) = path.parent().filter(|p| !p.as_os_str().is_empty()) { + std::fs::create_dir_all(dir)?; + } + let payload = format!( + "\n{}\n", + now_secs(), + text.trim() + ); + OpenOptions::new() + .create(true) + .append(true) + .open(path)? + .write_all(payload.as_bytes()) +} + +fn now_secs() -> u64 { + SystemTime::now() + .duration_since(UNIX_EPOCH) + .map(|d| d.as_secs()) + .unwrap_or(0) +} + +fn stamp_session_line(line: &str, seq: u64) -> Option { + let mut value: Value = serde_json::from_str(line.trim()).ok()?; + let Value::Object(obj) = &mut value else { + return None; + }; + obj.insert("seq".into(), Value::from(seq)); + serde_json::to_string(&value).ok() +} + +#[cfg(test)] +mod tests { + use crate::control::bridge::*; + + /// Just the command half of a request, for the parser tests that predate `id`. + fn parse_command(line: &str) -> std::result::Result { + parse_request(line).map(|r| r.cmd) + } + + /// A real ledger under a fresh temp dir (never a stub: the admit/settle contract IS + /// what these tests are checking). + fn ledger(name: &str) -> (AdmissionLedger, std::path::PathBuf) { + let path = tempdir(name).join("admissions.jsonl"); + let ledger = AdmissionLedger::open(&path, forge::ndjson::Open::Truncate).expect("ledger"); + (ledger, path) + } + + fn ledger_lines(path: &Path) -> Vec { + std::fs::read_to_string(path) + .unwrap_or_default() + .lines() + .filter_map(|l| serde_json::from_str(l).ok()) + .collect() + } + + fn request(line: &str) -> ControlRequest { + parse_request(line).expect("parses") + } + + #[test] + fn parses_control_commands() { + assert_eq!( + parse_command(r#"{"cmd":"abort"}"#).unwrap(), + ControlCommand::Abort + ); + assert_eq!(parse_command(r#""stop""#).unwrap(), ControlCommand::Stop); + assert_eq!( + parse_command(r#"{"kind":"steer","text":" try cache-first "}"#).unwrap(), + ControlCommand::Steer { + text: "try cache-first".into() + } + ); + assert_eq!( + parse_command(r#"{"cmd":"set-budget","usd":2.5}"#).unwrap(), + ControlCommand::SetBudget { usd: 2.5 } + ); + assert_eq!( + parse_command(r#"{"cmd":"rescope","regime":"concurrency=48"}"#).unwrap(), + ControlCommand::Rescope { + regime: "concurrency=48".into() + } + ); + assert!(parse_command(r#"{"cmd":"set-budget","usd":-1}"#).is_err()); + assert!(parse_command(r#"{"cmd":"steer"}"#).is_err()); + } + + #[test] + fn parses_the_optional_idempotency_key() { + // Absent: the server generates one, which is exactly the old behavior. + assert_eq!(request(r#"{"cmd":"stop"}"#).id, None); + assert_eq!(request(r#""stop""#).id, None); + assert_eq!( + request(r#"{"cmd":"steer","text":"go","id":" pr-comment:o/r#7:1 "}"#).id, + Some(AdmissionKey::new("pr-comment:o/r#7:1")), + "trimmed" + ); + assert!(parse_request(r#"{"cmd":"stop","id":""}"#).is_err()); + assert!(parse_request(r#"{"cmd":"stop","id":7}"#).is_err()); + let long = "x".repeat(MAX_KEY_LEN + 1); + assert!(parse_request(&format!(r#"{{"cmd":"stop","id":"{long}"}}"#)).is_err()); + } + + #[test] + fn a_redelivered_steer_is_admitted_once() { + let (l, path) = ledger("dup-steer"); + let state = ControlState::default(); + let line = r#"{"cmd":"steer","text":"cache first","id":"pr-comment:o/r#7:1"}"#; + + let first = apply_command(request(line), &state, &l); + assert_eq!(first["ok"], true); + assert_eq!(first["key"], "pr-comment:o/r#7:1"); + assert!(first.get("dup").is_none()); + + let second = apply_command(request(line), &state, &l); + assert_eq!(second["ok"], true, "convergence is success, not an error"); + assert_eq!(second["dup"], true); + + let written = ledger_lines(&path); + assert_eq!(written.len(), 1, "one admission, one steer"); + assert_eq!(written[0]["input"], "steer"); + assert_eq!(l.peek_steers().len(), 1); + } + + #[test] + fn the_same_key_with_different_text_is_refused_and_writes_nothing() { + let (l, path) = ledger("conflict"); + let state = ControlState::default(); + apply_command( + request(r#"{"cmd":"steer","text":"one","id":"k"}"#), + &state, + &l, + ); + let reply = apply_command( + request(r#"{"cmd":"steer","text":"two","id":"k"}"#), + &state, + &l, + ); + assert_eq!(reply["ok"], false); + assert!( + reply["error"] + .as_str() + .is_some_and(|e| e.contains("idempotency conflict")) + ); + assert_eq!(ledger_lines(&path).len(), 1); + } + + #[test] + fn commands_without_an_id_keep_todays_reply_shapes() { + let (l, _path) = ledger("compat"); + let state = ControlState::default(); + let budget = apply_command(request(r#"{"cmd":"set-budget","usd":2.5}"#), &state, &l); + assert_eq!(budget["ok"], true); + assert_eq!(budget["cmd"], "set-budget"); + assert_eq!(budget["usd"], 2.5); + assert_eq!(state.live_max_cost(), Some(2.5)); + + let rescope = apply_command(request(r#"{"cmd":"rescope","regime":"c=48"}"#), &state, &l); + assert_eq!(rescope["ok"], true); + assert_eq!(rescope["regime"], "c=48"); + + let deny = apply_command(request(r#"{"cmd":"deny","reason":"nope"}"#), &state, &l); + assert_eq!(deny["ok"], true); + assert_eq!(deny["reason"], "nope"); + + let stray = apply_command(request(r#""approve""#), &state, &l); + assert_eq!(stray["ok"], false); + assert_eq!(stray["error"], "no pending approval"); + } + + #[test] + fn a_second_rescope_supersedes_the_undrained_one() { + let (l, path) = ledger("supersede"); + let state = ControlState::default(); + apply_command( + request(r#"{"cmd":"rescope","regime":"c=24","id":"r1"}"#), + &state, + &l, + ); + apply_command( + request(r#"{"cmd":"rescope","regime":"c=48","id":"r2"}"#), + &state, + &l, + ); + // The loop only ever sees the newest, as before — but the overwrite is now recorded. + assert_eq!( + state.take_rescope(), + Some((AdmissionKey::new("r2"), "c=48".to_string())) + ); + let settled: Vec = ledger_lines(&path) + .into_iter() + .filter(|v| v["kind"] == "settled") + .collect(); + assert_eq!(settled.len(), 1); + assert_eq!(settled[0]["key"], "r1"); + assert_eq!(settled[0]["outcome"], "superseded"); + } + + #[test] + fn approve_records_the_grant_under_a_key_derived_from_the_ask() { + let (l, path) = ledger("approve"); + let state = ControlState::default(); + state.set_pending_regime("model=Q;c=48".into()); + + let reply = apply_command(request(r#"{"cmd":"approve","id":"op-1"}"#), &state, &l); + assert_eq!(reply["ok"], true); + assert_eq!(reply["regime"], "model=Q;c=48"); + + let derived = AdmissionKey::rescope_from(&AdmissionKey::approve("model=Q;c=48")); + assert_eq!( + state.take_rescope(), + Some((derived.clone(), "model=Q;c=48".to_string())), + "the loop drains the grant under the derived key" + ); + let written = ledger_lines(&path); + // approve admitted, derived rescope admitted, approve settled applied. + assert_eq!(written.len(), 3); + assert_eq!(written[0]["input"], "approve"); + assert_eq!(written[1]["key"], derived.as_str()); + assert_eq!(written[1]["regime"], "model=Q;c=48"); + assert_eq!(written[2]["kind"], "settled"); + assert_eq!(written[2]["outcome"], "applied"); + + // A second approve for the same ask converges: no second grant on the record. + state.set_pending_regime("model=Q;c=48".into()); + let again = apply_command(request(r#"{"cmd":"approve","id":"op-2"}"#), &state, &l); + assert_eq!(again["ok"], true); + assert_eq!(again["dup"], true); + assert_eq!( + ledger_lines(&path) + .iter() + .filter(|v| v["kind"] == "admitted" && v["input"] == "rescope") + .count(), + 1, + "one grant per ask" + ); + } + + #[test] + fn a_stray_approve_is_recorded_as_rejected() { + let (l, path) = ledger("stray-approve"); + let state = ControlState::default(); + let reply = apply_command(request(r#"{"cmd":"approve"}"#), &state, &l); + assert_eq!(reply["ok"], false); + let written = ledger_lines(&path); + assert_eq!(written.len(), 2); + assert_eq!(written[1]["outcome"], "rejected"); + } + + #[test] + fn stop_applies_even_when_it_cannot_be_recorded() { + let (l, path) = ledger("unrecorded-stop"); + let state = ControlState::default(); + // Burn the key on a different payload so the stop's own admission is refused: a + // safety valve must fire anyway and say that it went unrecorded. + apply_command( + request(r#"{"cmd":"steer","text":"x","id":"k"}"#), + &state, + &l, + ); + let before = STOP.load(Ordering::SeqCst); + let reply = apply_command(request(r#"{"cmd":"stop","id":"k"}"#), &state, &l); + assert_eq!(reply["ok"], true, "a stop is never refused"); + assert_eq!(reply["unrecorded"], true); + assert!(STOP.load(Ordering::SeqCst), "the flag is set regardless"); + STOP.store(before, Ordering::SeqCst); + assert_eq!(ledger_lines(&path).len(), 1, "nothing was written for it"); + } + + #[test] + fn stamps_session_lines_with_monotonic_seq() { + let state = ControlState::default(); + let (seq_one, one) = state + .stamp_session_line(r#"{"v":1,"kind":"note","msg":"a"}"#) + .unwrap(); + let (seq_two, two) = state + .stamp_session_line(r#"{"v":1,"kind":"note","msg":"b"}"#) + .unwrap(); + assert_eq!(seq_one, 1); + assert_eq!(seq_two, 2); + let one: Value = serde_json::from_str(&one).unwrap(); + let two: Value = serde_json::from_str(&two).unwrap(); + assert_eq!(one["seq"], 1); + assert_eq!(two["seq"], 2); + assert_eq!(one["kind"], "note"); + assert_eq!(two["msg"], "b"); + } + + #[test] + fn set_budget_zero_is_a_live_unlimited_cap() { + let state = ControlState::default(); + state.set_live_max_cost(0.0); + assert_eq!(state.live_max_cost(), Some(0.0)); + } + + #[test] + fn approve_turns_a_pending_regime_into_a_rescope() { + let (l, _path) = ledger("approve-slot"); + let state = ControlState::default(); + // Nothing pending yet: approve grants nothing. + assert_eq!( + apply_command(request(r#""approve""#), &state, &l)["ok"], + false + ); + assert!(!state.has_rescope()); + + // The loop records the regime an approval would grant; an operator `approve` resolves it. + state.set_pending_regime("concurrency=48".into()); + assert_eq!( + apply_command(request(r#""approve""#), &state, &l)["regime"], + "concurrency=48" + ); + assert!( + state.has_rescope(), + "approve sets the re-scope the loop drains" + ); + assert_eq!( + state.take_rescope().map(|(_, regime)| regime).as_deref(), + Some("concurrency=48") + ); + // Consumed: a second approve has nothing to grant. + assert_eq!( + apply_command(request(r#""approve""#), &state, &l)["ok"], + false + ); + } + + #[test] + fn parses_approve_command() { + assert_eq!( + parse_command(r#"{"cmd":"approve"}"#).unwrap(), + ControlCommand::Approve + ); + assert_eq!( + parse_command(r#""approve""#).unwrap(), + ControlCommand::Approve + ); + } + + #[test] + fn parses_deny_command_with_optional_reason() { + assert_eq!( + parse_command(r#"{"cmd":"deny","reason":"over budget"}"#).unwrap(), + ControlCommand::Deny { + reason: "over budget".into() + } + ); + // Bare deny defaults its reason. + assert_eq!( + parse_command(r#""deny""#).unwrap(), + ControlCommand::Deny { + reason: "rejected".into() + } + ); + } + + #[test] + fn deny_drops_a_recorded_pending_regime_so_approve_cant_resurrect_it() { + let (l, _path) = ledger("deny"); + let state = ControlState::default(); + state.set_pending_regime("concurrency=48".into()); + apply_command( + request(r#"{"cmd":"deny","reason":"policy says no"}"#), + &state, + &l, + ); + assert_eq!( + state.take_deny().map(|(_, why)| why).as_deref(), + Some("policy says no") + ); + // The pending regime is gone, so a stray `approve` grants nothing. + assert_eq!( + apply_command(request(r#""approve""#), &state, &l)["ok"], + false + ); + assert!(!state.has_rescope()); + } + + #[test] + fn stop_file_accepts_old_metadata_and_minimal_shape() { + let path = + std::env::temp_dir().join(format!("crucible-stop-file-{}.json", std::process::id())); + std::fs::write(&path, r#"{"stop":true,"kill_agent":true,"seq":42}"#).unwrap(); + assert!(stop_file_says_stop(&path)); + + std::fs::write(&path, r#"{"stop":true}"#).unwrap(); + assert!(stop_file_says_stop(&path)); + let _ = std::fs::remove_file(&path); + } + + #[test] + fn ignores_blank_or_torn_session_lines() { + assert!(stamp_session_line("", 1).is_none()); + assert!(stamp_session_line(r#"{"v":1,"kind":"note""#, 1).is_none()); + } + + #[test] + fn parses_tail_command_with_and_without_from_seq() { + assert_eq!( + parse_command(r#"{"cmd":"tail","from_seq":42}"#).unwrap(), + ControlCommand::Tail { from_seq: 42 } + ); + // Absent from_seq defaults to 0 (replay from the ring's oldest). + assert_eq!( + parse_command(r#""tail""#).unwrap(), + ControlCommand::Tail { from_seq: 0 } + ); + assert_eq!( + parse_command(r#"{"cmd":"tail","from_seq":7}"#).unwrap(), + ControlCommand::Tail { from_seq: 7 } + ); + } + + /// Read one NDJSON line from a stream with a bounded timeout, returning its parsed `seq`. + fn read_seq(reader: &mut BufReader) -> i64 { + let mut line = String::new(); + reader.read_line(&mut line).expect("read line"); + let v: Value = serde_json::from_str(line.trim()).expect("json"); + v["seq"].as_i64().expect("seq") + } + + #[test] + fn hub_replays_from_seq_then_delivers_live_exactly_once_in_order() { + // A real socket pair stands in for a subscribed client; the Hub is the actual replay core. + let listener = TcpListener::bind("127.0.0.1:0").expect("bind"); + let addr = listener.local_addr().expect("addr"); + let hub = Arc::new(Hub::default()); + + // Three lines already broadcast (retained in the ring; no subscriber yet). + hub.publish(1, r#"{"seq":1,"m":"a"}"#.to_string()); + hub.publish(2, r#"{"seq":2,"m":"b"}"#.to_string()); + hub.publish(3, r#"{"seq":3,"m":"c"}"#.to_string()); + + let client = TcpStream::connect(addr).expect("connect"); + client + .set_read_timeout(Some(Duration::from_secs(3))) + .expect("timeout"); + let (server_side, _) = listener.accept().expect("accept"); + let writer = Arc::new(Mutex::new(server_side)); + + // Resume after seq 1 (ring covers 2): the handoff replays 2 and 3 and subscribes... + assert!(hub.try_handoff(&writer, 1), "ring covers last+1"); + // ...then a live line arrives after subscription. + hub.publish(4, r#"{"seq":4,"m":"d"}"#.to_string()); + + let mut reader = BufReader::new(client); + assert_eq!(read_seq(&mut reader), 2, "replayed, seq 1 skipped"); + assert_eq!(read_seq(&mut reader), 3, "replayed in order"); + assert_eq!(read_seq(&mut reader), 4, "live after the replay seam"); + } + + #[test] + fn bridge_tail_command_streams_session_lines_over_tcp() { + // A full bridge over a real socket, tailing a real session.jsonl. Prove the `tail` command + // subscribes and streams stamped lines end to end. + let root = + std::env::temp_dir().join(format!("crucible-bridge-tail-{}", std::process::id())); + let state_dir = root.join("state"); + std::fs::create_dir_all(&state_dir).expect("state dir"); + let session_log = state_dir.join("session.jsonl"); + // The bridge tails from the file's current end, so any pre-existing content is not replayed; + // start it empty and append after it's up (matching the real loop, which writes as it runs). + std::fs::write(&session_log, "").expect("seed empty session log"); + + let paths = Paths { + workspace: root.clone(), + skills: None, + steer: root.join("STEER.md"), + state: state_dir.clone(), + session_log: session_log.clone(), + control: state_dir.join("control.json"), + escalation: root.join("ESCALATION.json"), + provisioning: root.join("PROVISIONING_PENDING.json"), + admissions: state_dir.join("admissions.jsonl"), + }; + + let port = { + let l = TcpListener::bind("127.0.0.1:0").expect("free port"); + l.local_addr().expect("addr").port() + }; + let ledger = Arc::new( + AdmissionLedger::open(&paths.admissions, forge::ndjson::Open::Truncate) + .expect("ledger"), + ); + spawn_bridge(port, paths, ledger).expect("spawn bridge"); + // Let the tail thread reach the (empty) file's end before we start appending. + thread::sleep(Duration::from_millis(200)); + + let append = |m: &str| { + let mut f = OpenOptions::new() + .append(true) + .open(&session_log) + .expect("reopen session log"); + writeln!(f, r#"{{"kind":"note","m":"{m}"}}"#).expect("append"); + }; + + // Two lines land while the bridge is up (no subscriber yet): they get seq 1 and 2 into the + // replay ring. Give the 100ms poll room to see both. + append("one"); + append("two"); + thread::sleep(Duration::from_millis(300)); + + let stream = TcpStream::connect(("127.0.0.1", port)).expect("connect"); + stream + .set_read_timeout(Some(Duration::from_secs(3))) + .expect("timeout"); + let mut writer = stream.try_clone().expect("clone"); + // Resume after seq 1: expect line 2 replayed, then a newly appended line live. + writeln!(writer, r#"{{"cmd":"tail","from_seq":1}}"#).expect("send tail"); + writer.flush().expect("flush"); + + // The subscribe replays before the command loop writes the `tail` ack, so the two frame + // kinds (session lines carrying `seq`, the ack carrying `cmd`) can interleave, classify by + // content, exactly as the live relay does, rather than assuming an order. + let mut reader = BufReader::new(stream); + let mut got_ack = false; + let next_seq = |reader: &mut BufReader, got_ack: &mut bool| -> i64 { + loop { + let mut line = String::new(); + reader.read_line(&mut line).expect("read frame"); + let v: Value = serde_json::from_str(line.trim()).expect("json"); + if v.get("cmd").is_some() { + *got_ack = true; + continue; + } + return v["seq"].as_i64().expect("seq"); + } + }; + + // The replayed line 2 (seq 1 skipped by from_seq=1). + assert_eq!( + next_seq(&mut reader, &mut got_ack), + 2, + "replayed after from_seq" + ); + + // A third line appended now is broadcast live to the subscriber. + append("three"); + assert_eq!( + next_seq(&mut reader, &mut got_ack), + 3, + "live-streamed appended line" + ); + assert!(got_ack, "the tail command was acked"); + + let _ = std::fs::remove_dir_all(&root); + } + + fn tempdir(name: &str) -> std::path::PathBuf { + let dir = std::env::temp_dir().join(format!( + "crucible-control-{name}-{}-{}", + std::process::id(), + now_secs() + )); + std::fs::create_dir_all(&dir).expect("tempdir"); + dir + } + + fn socket_pair() -> (BufReader, Client) { + let listener = TcpListener::bind("127.0.0.1:0").expect("bind"); + let addr = listener.local_addr().expect("addr"); + let client = TcpStream::connect(addr).expect("connect"); + client + .set_read_timeout(Some(Duration::from_secs(5))) + .expect("timeout"); + let (server_side, _) = listener.accept().expect("accept"); + (BufReader::new(client), Arc::new(Mutex::new(server_side))) + } + + #[test] + fn try_handoff_declines_when_the_ring_aged_past_the_resume_point() { + let hub = Hub::default(); + // Ring holds 100..=102, a client that has only seen up to 42 has a gap (43..=99 aged out). + for seq in 100..=102u64 { + hub.publish(seq, format!(r#"{{"seq":{seq}}}"#)); + } + let (_reader, writer) = socket_pair(); + assert!( + !hub.try_handoff(&writer, 42), + "gap: must read the file first" + ); + assert!( + hub.try_handoff(&writer, 99), + "contiguous: 100 is the next line" + ); + } + + #[test] + fn file_replay_streams_history_older_than_the_ring_then_hands_off() { + // 60 session lines on disk; the ring only retains the newest 3 (publishes 58..=60), a + // tail from 0 must get 1..=60 exactly once, in order, then live lines. + let dir = tempdir("file-replay"); + let path = dir.join("session.jsonl"); + let mut body = String::new(); + for i in 1..=60u64 { + body.push_str(&format!("{{\"v\":1,\"kind\":\"note\",\"msg\":\"m{i}\"}}\n")); + } + std::fs::write(&path, body).expect("write"); + + let hub = Arc::new(Hub::default()); + for seq in 58..=60u64 { + hub.publish(seq, format!(r#"{{"seq":{seq},"live":true}}"#)); + } + // Force the gap: drop everything older than 58 out of the ring window. + { + let mut g = hub.inner.lock().unwrap(); + while g.history.len() > 3 { + g.history.pop_front(); + } + } + + let (mut reader, writer) = socket_pair(); + tail_with_file_replay(&path, &hub, &writer, 0); + hub.publish(61, r#"{"seq":61,"live":true}"#.to_string()); + + for expect in 1..=61i64 { + assert_eq!(read_seq(&mut reader), expect, "in order, exactly once"); + } + } + + #[test] + fn file_replay_skips_unparseable_lines_but_their_seq_is_consumed() { + let dir = tempdir("torn-line"); + let path = dir.join("session.jsonl"); + // Line 2 is torn garbage: its seq is consumed (numbering = file line index) but not sent. + std::fs::write( + &path, + "{\"v\":1,\"kind\":\"note\",\"msg\":\"a\"}\n{\"broken\n{\"v\":1,\"kind\":\"note\",\"msg\":\"c\"}\n", + ) + .expect("write"); + // Ring holds only seq 3 (oldest = 3 > 0+1), so the tail must take the file pass. + let hub = Hub::default(); + hub.publish(3, r#"{"seq":3,"m":"c"}"#.to_string()); + let (mut reader, writer) = socket_pair(); + tail_with_file_replay(&path, &hub, &writer, 0); + assert_eq!(read_seq(&mut reader), 1); + assert_eq!( + read_seq(&mut reader), + 3, + "seq 2 consumed by the torn line, never sent" + ); + } + + #[test] + fn file_replay_converges_while_the_file_keeps_growing() { + // The race the handoff loop exists for: lines keep arriving while the file pass streams. + // A writer thread appends + publishes 20 more lines while the replay runs; the client must + // still see 1..=70 exactly once in order. + let dir = tempdir("growing"); + let path = dir.join("session.jsonl"); + let mut body = String::new(); + for i in 1..=50u64 { + body.push_str(&format!("{{\"v\":1,\"kind\":\"note\",\"msg\":\"m{i}\"}}\n")); + } + std::fs::write(&path, body).expect("write"); + + let hub = Arc::new(Hub::default()); + hub.publish(50, r#"{"seq":50}"#.to_string()); + { + let mut g = hub.inner.lock().unwrap(); + while g.history.len() > 1 { + g.history.pop_front(); + } + } + + let grower = { + let hub = hub.clone(); + let path = path.clone(); + thread::spawn(move || { + for i in 51..=70u64 { + let line = format!("{{\"v\":1,\"kind\":\"note\",\"msg\":\"m{i}\"}}\n"); + let mut f = OpenOptions::new().append(true).open(&path).expect("append"); + f.write_all(line.as_bytes()).expect("grow"); + hub.publish(i, format!(r#"{{"seq":{i}}}"#)); + thread::sleep(Duration::from_millis(1)); + } + }) + }; + + let (mut reader, writer) = socket_pair(); + tail_with_file_replay(&path, &hub, &writer, 0); + grower.join().expect("grower"); + // Anything not yet delivered at handoff arrives live; read all 70 in order. + for expect in 1..=70i64 { + assert_eq!(read_seq(&mut reader), expect, "exactly once, in order"); + } + } +} diff --git a/crucible/src/control/escalation.rs b/crucible/src/control/escalation.rs index a413084a..258da461 100644 --- a/crucible/src/control/escalation.rs +++ b/crucible/src/control/escalation.rs @@ -9,7 +9,7 @@ //! The engine is deliberately incurious about *why*: `category`/`reason`/`evidence` are //! free-form and never branched on here. Routing the report to a human is the whole point, //! an escalation is a stop-for-review, not a success (it gets its own exit code, see -//! [`crate::report::Outcome`]). +//! [`crate::report::reporter::Outcome`]). use serde::{Deserialize, Serialize}; use std::path::Path; diff --git a/crucible/src/control/mod.rs b/crucible/src/control/mod.rs index 44a61777..f35e4951 100644 --- a/crucible/src/control/mod.rs +++ b/crucible/src/control/mod.rs @@ -1,1516 +1,20 @@ -//! In-process remote control bridge for a stream-mode loop. -//! -//! The bridge is intentionally transport-agnostic: it listens on a plain TCP socket, -//! broadcasts new `state/session.jsonl` lines with a monotonic `seq`, and applies -//! inbound NDJSON commands against the live loop process. +//! Steering a run from outside the process: the scored loop's control bridge, admission ledger +//! and signals, and the broker a sandboxed turn reaches. +#[cfg(feature = "autoresearch")] pub(crate) mod admission; +#[cfg(feature = "autoresearch")] +pub(crate) mod bridge; pub(crate) mod broker; +#[cfg(feature = "autoresearch")] pub(crate) mod distress; +#[cfg(feature = "autoresearch")] pub(crate) mod escalation; +#[cfg(feature = "autoresearch")] pub(crate) mod heartbeat; +#[cfg(feature = "autoresearch")] pub(crate) mod pr_watch; +#[cfg(feature = "autoresearch")] pub(crate) mod provisioning; +#[cfg(feature = "autoresearch")] pub(crate) mod recovery; - -use crate::args::Paths; -use crate::control::admission::{AdmissionLedger, Admitted}; -use crate::process::STOP; -use anyhow::{Context, Result}; -use crucible_contract::LoopPhase; -use crucible_contract::admission::{ - AdmissionKey, AdmissionOutcome, AdmittedInput, MAX_KEY_LEN, SteerSource, -}; -use serde::{Deserialize, Serialize}; -use serde_json::{Value, json}; -use std::collections::VecDeque; -use std::fs::{File, OpenOptions}; -use std::io::{BufRead, BufReader, Seek, SeekFrom, Write}; -use std::net::{TcpListener, TcpStream}; -use std::path::Path; -use std::sync::atomic::{AtomicBool, AtomicU64, Ordering}; -use std::sync::{Arc, Mutex}; -use std::thread; -use std::time::{Duration, SystemTime, UNIX_EPOCH}; - -type Client = Arc>; - -/// How many recently-broadcast session lines the bridge retains in memory for replay. A `tail -/// from_seq=N` inside this window is served straight from the ring; anything older is replayed from -/// `session.jsonl` itself (see [`tail_with_file_replay`]), the seq IS the 1-based file line index, -/// so the file is the authoritative pre-ring history and a late joiner gets the whole run. -const HISTORY_CAP: usize = 4096; - -/// One session line stamped with its monotonic `seq`, kept in the bridge's replay ring. -#[derive(Clone)] -struct Stamped { - seq: u64, - line: String, -} - -/// The broadcast hub: a bounded replay ring plus the currently-subscribed client writers, under one -/// lock so a new subscriber's replay-then-live handoff is atomic against the tail thread's -/// broadcasts, every line is delivered exactly once, in order, with no gap at the boundary. -#[derive(Default)] -struct Hub { - inner: Mutex, -} - -#[derive(Default)] -struct HubInner { - history: VecDeque, - clients: Vec, -} - -impl Hub { - /// Stamp-and-broadcast one line: retain it in the replay ring (bounded) and write it to every - /// live subscriber, dropping any whose socket has closed. - fn publish(&self, seq: u64, line: String) { - let Ok(mut g) = self.inner.lock() else { - return; - }; - g.history.push_back(Stamped { - seq, - line: line.clone(), - }); - while g.history.len() > HISTORY_CAP { - g.history.pop_front(); - } - g.clients.retain(|c| write_line(c, &line)); - } - - /// The newest seq in the ring (0 before anything was published), the file-replay loop's - /// per-pass target. - fn newest_seq(&self) -> u64 { - self.inner - .lock() - .map(|g| g.history.back().map(|s| s.seq).unwrap_or(0)) - .unwrap_or(0) - } - - /// Attempt the atomic file→ring handoff for a client that has already received everything up to - /// `last` (from the file): if the ring still covers `last + 1` (or nothing was ever published), - /// replay the retained lines after `last` and subscribe, all under the one lock, so nothing - /// published concurrently is missed or doubled. Returns false when the ring's oldest has moved - /// past `last + 1` (the file grew faster than the replay): the caller reads more file first. - fn try_handoff(&self, writer: &Client, last: u64) -> bool { - let Ok(mut g) = self.inner.lock() else { - return true; // poisoned: behave like the plain subscribe arms elsewhere - }; - let covered = match g.history.front() { - None => true, - Some(oldest) => oldest.seq <= last + 1, - }; - if !covered { - return false; - } - for s in g.history.iter().filter(|s| s.seq > last) { - let _ = write_line(writer, &s.line); - } - g.clients.push(writer.clone()); - true - } - - /// Drop a client (its reader thread ended / socket closed) so broadcasts stop targeting it. - fn unsubscribe(&self, writer: &Client) { - let Ok(mut g) = self.inner.lock() else { - return; - }; - g.clients.retain(|c| !Arc::ptr_eq(c, writer)); - } -} - -#[derive(Default)] -pub(crate) struct ControlState { - paused: AtomicBool, - next_seq: AtomicU64, - live_max_cost: Mutex>, - status: Mutex, - /// Pending re-scope, drained by the loop at the next iteration head. The key travels - /// with the regime so the loop can settle the admission where it applies it. - rescope: Mutex>, - /// The regime an open approval would grant (attended path): the loop records it from the agent's - /// pending marker so an operator `approve` keystroke can turn it into a `rescope` without naming - /// the regime. `None` when there's no approval awaiting an operator. - pending_regime: Mutex>, - /// A rejected pending approval, drained by the park as the terminal "not granted" - /// outcome. - deny: Mutex>, -} - -#[derive(Clone, Debug, Serialize)] -pub(crate) struct StatusSnapshot { - pub phase: LoopPhase, - pub iter: u32, - #[serde(skip_serializing_if = "Option::is_none")] - pub best_score: Option, - pub spend: f64, - pub paused: bool, - #[serde(skip_serializing_if = "Option::is_none")] - pub max_cost: Option, -} - -impl Default for StatusSnapshot { - fn default() -> Self { - Self { - phase: LoopPhase::Starting, - iter: 0, - best_score: None, - spend: 0.0, - paused: false, - max_cost: None, - } - } -} - -impl ControlState { - pub(crate) fn is_paused(&self) -> bool { - self.paused.load(Ordering::SeqCst) - } - - pub(crate) fn live_max_cost(&self) -> Option { - self.live_max_cost.lock().ok().and_then(|g| *g) - } - - /// Drained by the loop at the iteration head, which re-baselines and settles it. - pub(crate) fn take_rescope(&self) -> Option<(AdmissionKey, String)> { - self.rescope.lock().ok()?.take() - } - - /// Non-consuming peek: is a re-scope pending? The park polls this so the drain - /// ([`take_rescope`](Self::take_rescope)) stays the single place that re-baselines. - pub(crate) fn has_rescope(&self) -> bool { - self.rescope.lock().is_ok_and(|g| g.is_some()) - } - - #[cfg(test)] - pub(crate) fn phase(&self) -> LoopPhase { - self.status - .lock() - .map(|s| s.phase) - .unwrap_or(LoopPhase::Starting) - } - - pub(crate) fn set_phase(&self, phase: LoopPhase) { - if let Ok(mut status) = self.status.lock() { - status.phase = phase; - } - } - - pub(crate) fn set_progress(&self, iter: u32, best_score: Option, spend: f64) { - let Ok(mut status) = self.status.lock() else { - return; - }; - status.iter = iter; - status.best_score = best_score; - status.spend = spend; - } - - pub(crate) fn set_spend(&self, spend: f64) { - if let Ok(mut status) = self.status.lock() { - status.spend = spend; - } - } - - pub(crate) fn pause(&self) { - self.paused.store(true, Ordering::SeqCst); - } - - fn resume(&self) { - self.paused.store(false, Ordering::SeqCst); - } - - pub(crate) fn set_live_max_cost(&self, usd: f64) { - if let Ok(mut cap) = self.live_max_cost.lock() { - *cap = Some(usd); - } - } - - /// Arm the re-scope the next iteration head drains. Returns the key of an undrained - /// re-scope this displaced, which the caller settles as superseded. - pub(crate) fn set_rescope(&self, key: AdmissionKey, regime: String) -> Option { - let Ok(mut slot) = self.rescope.lock() else { - return None; - }; - slot.replace((key, regime)).map(|(old, _)| old) - } - - /// Record the regime an open approval would grant, so an operator `approve` can resolve it. - /// The loop sets this when it reads the agent's pending-provisioning marker. - pub(crate) fn set_pending_regime(&self, regime: String) { - if let Ok(mut slot) = self.pending_regime.lock() { - *slot = Some(regime); - } - } - - /// Take the regime an open approval would grant. The caller admits the derived - /// re-scope BEFORE arming it, and hands the regime back if the record can't be - /// written. - pub(crate) fn take_pending_regime(&self) -> Option { - self.pending_regime.lock().ok()?.take() - } - - /// Record a denial, clearing any pending regime so a later `approve` can't resurrect - /// a denied ask. Returns the key of an undrained denial this displaced. - pub(crate) fn set_deny(&self, key: AdmissionKey, reason: String) -> Option { - if let Ok(mut pending) = self.pending_regime.lock() { - *pending = None; - } - let Ok(mut slot) = self.deny.lock() else { - return None; - }; - slot.replace((key, reason)).map(|(old, _)| old) - } - - /// Drain a denial if one arrived (the terminal "not granted" outcome the park waits on). - pub(crate) fn take_deny(&self) -> Option<(AdmissionKey, String)> { - self.deny.lock().ok()?.take() - } - - fn snapshot(&self) -> StatusSnapshot { - let mut status = self - .status - .lock() - .map(|s| s.clone()) - .unwrap_or_else(|_| StatusSnapshot::default()); - status.paused = self.is_paused(); - status.max_cost = self.live_max_cost(); - status - } - - /// Assign the next monotonic `seq` and stamp it onto the line, returning both (the seq feeds the - /// replay ring, the stamped line goes on the wire). `None` for a blank/torn line, the seq is - /// still consumed so the numbering matches the file's line count. - fn stamp_session_line(&self, line: &str) -> Option<(u64, String)> { - let seq = self.next_seq.fetch_add(1, Ordering::SeqCst) + 1; - stamp_session_line(line, seq).map(|stamped| (seq, stamped)) - } -} - -/// Start a detached TCP bridge thread. The listener binds to all interfaces so it works -/// inside a pod behind `kubectl port-forward`, while still being reachable as localhost -/// for local smoke tests. -pub(crate) fn spawn_bridge( - port: u16, - paths: Paths, - ledger: Arc, -) -> Result> { - let listener = TcpListener::bind(("0.0.0.0", port)) - .with_context(|| format!("binding control port {port}"))?; - let state = Arc::new(ControlState::default()); - let hub = Arc::new(Hub::default()); - spawn_session_tail(paths.session_log.clone(), state.clone(), hub.clone()); - - let accept_state = state.clone(); - thread::spawn(move || { - for stream in listener.incoming() { - match stream { - Ok(stream) => serve_client( - stream, - &paths, - accept_state.clone(), - hub.clone(), - ledger.clone(), - ), - Err(_) => thread::sleep(Duration::from_millis(100)), - } - } - }); - - Ok(state) -} - -fn spawn_session_tail(path: std::path::PathBuf, state: Arc, hub: Arc) { - thread::spawn(move || { - let mut pos: Option = None; - loop { - if let Ok(mut file) = File::open(&path) { - let len = file.metadata().map(|m| m.len()).unwrap_or(0); - // Start from offset 0 (not the live edge): every line the file already holds gets - // stamped, so seq == the 1-based file line index and file replay can serve history - // older than the ring. - let current = match pos { - Some(p) if len >= p => p, - Some(_) => 0, - None => 0, - }; - let _ = len; - pos = Some(read_new_lines(&mut file, current, &state, &hub)); - } - thread::sleep(Duration::from_millis(100)); - } - }); -} - -fn read_new_lines(file: &mut File, start: u64, state: &ControlState, hub: &Hub) -> u64 { - if file.seek(SeekFrom::Start(start)).is_err() { - return start; - } - let mut pos = start; - let mut reader = BufReader::new(file); - loop { - let mut line = String::new(); - let n = match reader.read_line(&mut line) { - Ok(0) => break, - Ok(n) => n as u64, - Err(_) => break, - }; - if !line.ends_with('\n') { - break; - } - pos += n; - if let Some((seq, stamped)) = state.stamp_session_line(&line) { - hub.publish(seq, stamped); - } - } - pos -} - -/// Write one NDJSON line to a client and flush, reporting whether the socket is still alive (a dead -/// one is dropped from the broadcast set by the caller). -fn write_line(client: &Client, line: &str) -> bool { - let Ok(mut stream) = client.lock() else { - return false; - }; - writeln!(stream, "{line}") - .and_then(|_| stream.flush()) - .is_ok() -} - -/// Serve a `tail from_seq` that may reach back past the in-memory ring: replay the missing range -/// from `session.jsonl` itself (seq == the 1-based file line index, unparseable lines consume their -/// seq silently, the same rule the live stamping applies), then hand off to the ring atomically -/// via [`Hub::try_handoff`]. Loops because the file can grow while a pass streams: each pass reads -/// up to the ring's newest at pass start, and the handoff only succeeds when the ring still covers -/// the next line, so every line is delivered exactly once, in order, with no gap at the boundary. -fn tail_with_file_replay(path: &Path, hub: &Hub, writer: &Client, from_seq: u64) { - let mut last = from_seq; - loop { - if hub.try_handoff(writer, last) { - return; - } - let target = hub.newest_seq(); - if target <= last { - // The ring is ahead of `last+1`'s coverage yet has nothing newer than `last`, only a - // transient between a publish and our read. Yield and retry. - thread::sleep(Duration::from_millis(10)); - continue; - } - last = replay_file_range(path, writer, last, target); - } -} - -/// Stream file lines with 1-based index in `(after, upto]` to the client, stamped exactly like the -/// live path. Returns the last index actually reached (== `upto` unless the file is unexpectedly -/// short, then the handoff loop retries from there). Only full (newline-terminated) lines within -/// `upto` exist by construction: the tail thread already stamped them. -fn replay_file_range(path: &Path, writer: &Client, after: u64, upto: u64) -> u64 { - let Ok(file) = File::open(path) else { - return upto; // vanished file: fall through to the ring, which is all that's left - }; - let reader = BufReader::new(file); - let mut idx: u64 = 0; - for line in reader.lines() { - let Ok(line) = line else { break }; - idx += 1; - if idx > upto { - break; - } - if idx <= after { - continue; - } - if let Some(stamped) = stamp_session_line(&line, idx) { - let _ = write_line(writer, &stamped); - } - } - idx.min(upto).max(after) -} - -fn serve_client( - stream: TcpStream, - paths: &Paths, - state: Arc, - hub: Arc, - ledger: Arc, -) { - let Ok(writer) = stream.try_clone() else { - return; - }; - let writer = Arc::new(Mutex::new(writer)); - - let paths = paths.clone(); - thread::spawn(move || { - let reader = BufReader::new(stream); - for line in reader.lines() { - let reply = match line { - // `tail` subscribes this socket to the live broadcast (replaying from `from_seq`), - // so it's handled here where the writer lives rather than in `apply_command`. - Ok(line) => match parse_request(&line) { - Ok(ControlRequest { - cmd: ControlCommand::Tail { from_seq }, - .. - }) => { - tail_with_file_replay(&paths.session_log, &hub, &writer, from_seq); - json!({"ok": true, "cmd": "tail", "from_seq": from_seq}) - } - Ok(req) => apply_command(req, &state, &ledger), - Err(e) => json!({"ok": false, "error": e}), - }, - Err(e) => json!({"ok": false, "error": e.to_string()}), - }; - write_reply(&writer, &reply); - } - // The reader loop ends when the client closes the socket: drop it from the broadcast set so - // publishes stop targeting a dead writer (a tail subscriber that just went away). - hub.unsubscribe(&writer); - }); -} - -fn write_reply(writer: &Client, reply: &Value) { - let Ok(mut stream) = writer.lock() else { - return; - }; - let _ = writeln!(stream, "{reply}"); - let _ = stream.flush(); -} - -/// Nothing mutates the run before its `Admitted` line is on disk; a redelivered command -/// converges; `stop`/`abort` fire even when the record can't be written. -fn apply_command(req: ControlRequest, state: &ControlState, ledger: &AdmissionLedger) -> Value { - let ControlRequest { id, cmd } = req; - match cmd { - ControlCommand::Tail { from_seq } => { - // Subscription happens at the socket layer ([`serve_client`]) where the writer lives; a - // direct caller that reaches here (never the live relay) only gets the ack. - json!({"ok": true, "cmd": "tail", "from_seq": from_seq}) - } - ControlCommand::Abort => stop_the_run(id, AdmittedInput::Abort, ledger), - ControlCommand::Stop => stop_the_run(id, AdmittedInput::Stop, ledger), - ControlCommand::Pause => admitted(ledger, id, AdmittedInput::Pause, |key| { - state.pause(); - let _ = ledger.settle(key, AdmissionOutcome::Applied, "paused"); - json!({"ok": true, "cmd": "pause"}) - }), - ControlCommand::Resume => admitted(ledger, id, AdmittedInput::Resume, |key| { - state.resume(); - let _ = ledger.settle(key, AdmissionOutcome::Applied, "resumed"); - json!({"ok": true, "cmd": "resume"}) - }), - // The ledger IS the steer queue: the bridge no longer writes STEER.md. - ControlCommand::Steer { text } => admitted( - ledger, - id, - AdmittedInput::Steer { - text, - from: SteerSource::Operator, - }, - |_| json!({"ok": true, "cmd": "steer"}), - ), - ControlCommand::SetBudget { usd } => { - admitted(ledger, id, AdmittedInput::SetBudget { usd }, |key| { - state.set_live_max_cost(usd); - let _ = ledger.settle(key, AdmissionOutcome::Applied, "live cap set"); - json!({"ok": true, "cmd": "set-budget", "usd": usd}) - }) - } - ControlCommand::Rescope { regime } => admitted( - ledger, - id, - AdmittedInput::Rescope { - regime: regime.clone(), - }, - |key| { - supersede(ledger, state.set_rescope(key.clone(), regime.clone()), key); - json!({"ok": true, "cmd": "rescope", "regime": regime}) - }, - ), - ControlCommand::Approve => admitted(ledger, id, AdmittedInput::Approve, |key| { - grant_approval(state, ledger, key) - }), - ControlCommand::Deny { reason } => admitted( - ledger, - id, - AdmittedInput::Deny { - reason: reason.clone(), - }, - |key| { - supersede(ledger, state.set_deny(key.clone(), reason.clone()), key); - json!({"ok": true, "cmd": "deny", "reason": reason}) - }, - ), - ControlCommand::Status => { - json!({"ok": true, "cmd": "status", "status": state.snapshot()}) - } - } -} - -/// The grant is recorded BEFORE the slot is armed, under a key derived from the ASK, so -/// approvals of the same ask converge and a resume can recognize a recorded grant. -fn grant_approval( - state: &ControlState, - ledger: &AdmissionLedger, - approve_key: &AdmissionKey, -) -> Value { - let Some(regime) = state.take_pending_regime() else { - let _ = ledger.settle( - approve_key, - AdmissionOutcome::Rejected, - "no approval was awaiting an operator", - ); - return json!({"ok": false, "cmd": "approve", "error": "no pending approval"}); - }; - let derived = AdmissionKey::rescope_from(&AdmissionKey::approve(®ime)); - let recorded = ledger.admit( - Some(derived), - AdmittedInput::Rescope { - regime: regime.clone(), - }, - ); - match recorded { - Ok(Admitted::Fresh(rescope_key)) => { - supersede( - ledger, - state.set_rescope(rescope_key.clone(), regime.clone()), - &rescope_key, - ); - let _ = ledger.settle( - approve_key, - AdmissionOutcome::Applied, - &format!("granted '{regime}'"), - ); - json!({"ok": true, "cmd": "approve", "regime": regime}) - } - // The grant is already on the record (a redelivered approve, or one sent after a - // resume re-registered the ask): converge on it instead of arming a second re-scope. - Ok(Admitted::Duplicate(..)) => { - let _ = ledger.settle( - approve_key, - AdmissionOutcome::Applied, - &format!("'{regime}' was already granted"), - ); - json!({"ok": true, "cmd": "approve", "regime": regime, "dup": true}) - } - Ok(Admitted::Conflict(key)) => { - state.set_pending_regime(regime); - let _ = ledger.settle(approve_key, AdmissionOutcome::Rejected, "grant conflict"); - json!({"ok": false, "cmd": "approve", - "error": format!("idempotency conflict on the derived grant {key}")}) - } - Err(e) => { - // Hand the ask back: an approval we could not record must stay approvable - // rather than vanish. - state.set_pending_regime(regime); - json!({"ok": false, "cmd": "approve", - "error": format!("admission not recorded: {e:#}")}) - } - } -} - -/// Admit, then apply. The effect runs only for a fresh admission; a conflict or ledger -/// write failure refuses the command rather than acting unrecorded. -fn admitted( - ledger: &AdmissionLedger, - id: Option, - input: AdmittedInput, - effect: impl FnOnce(&AdmissionKey) -> Value, -) -> Value { - let cmd = input.cmd(); - match ledger.admit(id, input) { - Ok(Admitted::Fresh(key)) => { - let mut reply = effect(&key); - with_field(&mut reply, "key", json!(key.as_str())); - reply - } - Ok(Admitted::Duplicate(key, outcome)) => { - let mut reply = json!({"ok": true, "cmd": cmd, "key": key.as_str(), "dup": true}); - if let Some(outcome) = outcome { - with_field(&mut reply, "outcome", json!(outcome.as_str())); - } - reply - } - Ok(Admitted::Conflict(key)) => json!({ - "ok": false, "cmd": cmd, "key": key.as_str(), - "error": "idempotency conflict: key already admitted with a different payload" - }), - Err(e) => json!({ - "ok": false, "cmd": cmd, "error": format!("admission not recorded: {e:#}") - }), - } -} - -/// Stop and abort are safety valves: they apply even when the ledger write fails, and say -/// so (`unrecorded`) rather than refusing to stop a running loop over a disk error. -fn stop_the_run(id: Option, input: AdmittedInput, ledger: &AdmissionLedger) -> Value { - let cmd = input.cmd(); - let abort = matches!(input, AdmittedInput::Abort); - let recorded = ledger.admit(id, input); - STOP.store(true, Ordering::SeqCst); - if abort { - crate::process::pid_registry::kill_all(); - } - match recorded { - Ok(Admitted::Fresh(key)) => { - let _ = ledger.settle(&key, AdmissionOutcome::Applied, "stop flag set"); - json!({"ok": true, "cmd": cmd, "key": key.as_str()}) - } - Ok(Admitted::Duplicate(key, _)) => { - json!({"ok": true, "cmd": cmd, "key": key.as_str(), "dup": true}) - } - Ok(Admitted::Conflict(_)) | Err(_) => { - json!({"ok": true, "cmd": cmd, "unrecorded": true}) - } - } -} - -/// Settle the admission a newly-armed one displaced as superseded. -fn supersede(ledger: &AdmissionLedger, displaced: Option, by: &AdmissionKey) { - if let Some(old) = displaced { - let _ = ledger.settle( - &old, - AdmissionOutcome::Superseded, - &format!("replaced by {by} before the loop drained it"), - ); - } -} - -fn with_field(reply: &mut Value, field: &str, value: Value) { - if let Some(obj) = reply.as_object_mut() { - obj.insert(field.to_string(), value); - } -} - -/// The legacy cross-process stop file external tooling may write. -/// Unknown fields are ignored so older writers that include debugging metadata still work. -#[derive(Deserialize, Default)] -struct StopFile { - #[serde(default)] - stop: bool, -} - -pub(crate) fn stop_file_says_stop(path: &Path) -> bool { - std::fs::read_to_string(path) - .ok() - .and_then(|s| serde_json::from_str::(&s).ok()) - .map(|c| c.stop) - .unwrap_or(false) -} - -#[derive(Debug, PartialEq)] -enum ControlCommand { - Abort, - Stop, - Pause, - Resume, - Steer { - text: String, - }, - SetBudget { - usd: f64, - }, - Rescope { - regime: String, - }, - Approve, - Deny { - reason: String, - }, - Status, - /// Subscribe to the live session broadcast, replaying retained lines with `seq > from_seq` - /// first (resume support for the read-only live relay). `from_seq = 0` = from the ring's oldest. - Tail { - from_seq: u64, - }, -} - -/// One inbound command plus its optional idempotency key. No `id` means a generated one -/// (every delivery is a fresh input); a natural key makes redelivery converge. -#[derive(Debug, PartialEq)] -struct ControlRequest { - id: Option, - cmd: ControlCommand, -} - -fn parse_request(line: &str) -> std::result::Result { - let value: Value = serde_json::from_str(line.trim()).map_err(|e| e.to_string())?; - if let Some(cmd) = value.as_str() { - // String-form commands ("stop") carry no id; the server generates one. - return command_from_name(cmd, &value).map(|cmd| ControlRequest { id: None, cmd }); - } - let obj = value - .as_object() - .ok_or_else(|| "command must be a JSON object or string".to_string())?; - let cmd = obj - .get("cmd") - .or_else(|| obj.get("command")) - .or_else(|| obj.get("kind")) - .and_then(Value::as_str) - .ok_or_else(|| "command object needs cmd".to_string())?; - let id = parse_id(obj.get("id"))?; - command_from_name(cmd, &value).map(|cmd| ControlRequest { id, cmd }) -} - -fn parse_id(raw: Option<&Value>) -> std::result::Result, String> { - let Some(raw) = raw.filter(|v| !v.is_null()) else { - return Ok(None); - }; - let id = raw - .as_str() - .ok_or_else(|| "id must be a string".to_string())? - .trim(); - if id.is_empty() { - return Err("id must not be empty".into()); - } - if id.len() > MAX_KEY_LEN { - return Err(format!("id must be at most {MAX_KEY_LEN} bytes")); - } - Ok(Some(AdmissionKey::new(id))) -} - -fn command_from_name(cmd: &str, value: &Value) -> std::result::Result { - match cmd { - "abort" => Ok(ControlCommand::Abort), - "stop" => Ok(ControlCommand::Stop), - "pause" => Ok(ControlCommand::Pause), - "resume" => Ok(ControlCommand::Resume), - "approve" => Ok(ControlCommand::Approve), - "deny" => { - // Reason is optional (an operator may just reject); default to a generic note. - let reason = value - .get("reason") - .and_then(Value::as_str) - .unwrap_or("rejected") - .trim() - .to_string(); - Ok(ControlCommand::Deny { reason }) - } - "status" => Ok(ControlCommand::Status), - "tail" => { - // The seq to resume after; absent/0 = replay from the ring's oldest retained line. - let from_seq = value.get("from_seq").and_then(Value::as_u64).unwrap_or(0); - Ok(ControlCommand::Tail { from_seq }) - } - "steer" => { - let text = value - .get("text") - .and_then(Value::as_str) - .ok_or_else(|| "steer needs text".to_string())?; - Ok(ControlCommand::Steer { - text: text.trim().to_string(), - }) - } - "set-budget" | "set_budget" => { - let usd = value - .get("usd") - .and_then(Value::as_f64) - .ok_or_else(|| "set-budget needs numeric usd".to_string())?; - if !usd.is_finite() || usd < 0.0 { - return Err("set-budget usd must be a finite non-negative number".into()); - } - Ok(ControlCommand::SetBudget { usd }) - } - "rescope" | "re-scope" => { - // A judge-changing grant approved out-of-band: re-baseline into this regime. - let regime = value - .get("regime") - .and_then(Value::as_str) - .unwrap_or("rescoped") - .trim() - .to_string(); - Ok(ControlCommand::Rescope { regime }) - } - other => Err(format!("unknown control command {other:?}")), - } -} - -/// Append one steer payload in the marker-wrapped shape [`crate::control::admission::drain_steer`] -/// reads. `pr_watch`'s reseed sink writes this file when there is no live control bridge; -/// the bridge itself admits `steer` straight into the ledger. -pub(crate) fn append_steer(path: &Path, text: &str) -> std::io::Result<()> { - if let Some(dir) = path.parent().filter(|p| !p.as_os_str().is_empty()) { - std::fs::create_dir_all(dir)?; - } - let payload = format!( - "\n{}\n", - now_secs(), - text.trim() - ); - OpenOptions::new() - .create(true) - .append(true) - .open(path)? - .write_all(payload.as_bytes()) -} - -fn now_secs() -> u64 { - SystemTime::now() - .duration_since(UNIX_EPOCH) - .map(|d| d.as_secs()) - .unwrap_or(0) -} - -fn stamp_session_line(line: &str, seq: u64) -> Option { - let mut value: Value = serde_json::from_str(line.trim()).ok()?; - let Value::Object(obj) = &mut value else { - return None; - }; - obj.insert("seq".into(), Value::from(seq)); - serde_json::to_string(&value).ok() -} - -#[cfg(test)] -mod tests { - use super::*; - - /// Just the command half of a request, for the parser tests that predate `id`. - fn parse_command(line: &str) -> std::result::Result { - parse_request(line).map(|r| r.cmd) - } - - /// A real ledger under a fresh temp dir (never a stub: the admit/settle contract IS - /// what these tests are checking). - fn ledger(name: &str) -> (AdmissionLedger, std::path::PathBuf) { - let path = tempdir(name).join("admissions.jsonl"); - let ledger = AdmissionLedger::open(&path, forge::ndjson::Open::Truncate).expect("ledger"); - (ledger, path) - } - - fn ledger_lines(path: &Path) -> Vec { - std::fs::read_to_string(path) - .unwrap_or_default() - .lines() - .filter_map(|l| serde_json::from_str(l).ok()) - .collect() - } - - fn request(line: &str) -> ControlRequest { - parse_request(line).expect("parses") - } - - #[test] - fn parses_control_commands() { - assert_eq!( - parse_command(r#"{"cmd":"abort"}"#).unwrap(), - ControlCommand::Abort - ); - assert_eq!(parse_command(r#""stop""#).unwrap(), ControlCommand::Stop); - assert_eq!( - parse_command(r#"{"kind":"steer","text":" try cache-first "}"#).unwrap(), - ControlCommand::Steer { - text: "try cache-first".into() - } - ); - assert_eq!( - parse_command(r#"{"cmd":"set-budget","usd":2.5}"#).unwrap(), - ControlCommand::SetBudget { usd: 2.5 } - ); - assert_eq!( - parse_command(r#"{"cmd":"rescope","regime":"concurrency=48"}"#).unwrap(), - ControlCommand::Rescope { - regime: "concurrency=48".into() - } - ); - assert!(parse_command(r#"{"cmd":"set-budget","usd":-1}"#).is_err()); - assert!(parse_command(r#"{"cmd":"steer"}"#).is_err()); - } - - #[test] - fn parses_the_optional_idempotency_key() { - // Absent: the server generates one, which is exactly the old behavior. - assert_eq!(request(r#"{"cmd":"stop"}"#).id, None); - assert_eq!(request(r#""stop""#).id, None); - assert_eq!( - request(r#"{"cmd":"steer","text":"go","id":" pr-comment:o/r#7:1 "}"#).id, - Some(AdmissionKey::new("pr-comment:o/r#7:1")), - "trimmed" - ); - assert!(parse_request(r#"{"cmd":"stop","id":""}"#).is_err()); - assert!(parse_request(r#"{"cmd":"stop","id":7}"#).is_err()); - let long = "x".repeat(MAX_KEY_LEN + 1); - assert!(parse_request(&format!(r#"{{"cmd":"stop","id":"{long}"}}"#)).is_err()); - } - - #[test] - fn a_redelivered_steer_is_admitted_once() { - let (l, path) = ledger("dup-steer"); - let state = ControlState::default(); - let line = r#"{"cmd":"steer","text":"cache first","id":"pr-comment:o/r#7:1"}"#; - - let first = apply_command(request(line), &state, &l); - assert_eq!(first["ok"], true); - assert_eq!(first["key"], "pr-comment:o/r#7:1"); - assert!(first.get("dup").is_none()); - - let second = apply_command(request(line), &state, &l); - assert_eq!(second["ok"], true, "convergence is success, not an error"); - assert_eq!(second["dup"], true); - - let written = ledger_lines(&path); - assert_eq!(written.len(), 1, "one admission, one steer"); - assert_eq!(written[0]["input"], "steer"); - assert_eq!(l.peek_steers().len(), 1); - } - - #[test] - fn the_same_key_with_different_text_is_refused_and_writes_nothing() { - let (l, path) = ledger("conflict"); - let state = ControlState::default(); - apply_command( - request(r#"{"cmd":"steer","text":"one","id":"k"}"#), - &state, - &l, - ); - let reply = apply_command( - request(r#"{"cmd":"steer","text":"two","id":"k"}"#), - &state, - &l, - ); - assert_eq!(reply["ok"], false); - assert!( - reply["error"] - .as_str() - .is_some_and(|e| e.contains("idempotency conflict")) - ); - assert_eq!(ledger_lines(&path).len(), 1); - } - - #[test] - fn commands_without_an_id_keep_todays_reply_shapes() { - let (l, _path) = ledger("compat"); - let state = ControlState::default(); - let budget = apply_command(request(r#"{"cmd":"set-budget","usd":2.5}"#), &state, &l); - assert_eq!(budget["ok"], true); - assert_eq!(budget["cmd"], "set-budget"); - assert_eq!(budget["usd"], 2.5); - assert_eq!(state.live_max_cost(), Some(2.5)); - - let rescope = apply_command(request(r#"{"cmd":"rescope","regime":"c=48"}"#), &state, &l); - assert_eq!(rescope["ok"], true); - assert_eq!(rescope["regime"], "c=48"); - - let deny = apply_command(request(r#"{"cmd":"deny","reason":"nope"}"#), &state, &l); - assert_eq!(deny["ok"], true); - assert_eq!(deny["reason"], "nope"); - - let stray = apply_command(request(r#""approve""#), &state, &l); - assert_eq!(stray["ok"], false); - assert_eq!(stray["error"], "no pending approval"); - } - - #[test] - fn a_second_rescope_supersedes_the_undrained_one() { - let (l, path) = ledger("supersede"); - let state = ControlState::default(); - apply_command( - request(r#"{"cmd":"rescope","regime":"c=24","id":"r1"}"#), - &state, - &l, - ); - apply_command( - request(r#"{"cmd":"rescope","regime":"c=48","id":"r2"}"#), - &state, - &l, - ); - // The loop only ever sees the newest, as before — but the overwrite is now recorded. - assert_eq!( - state.take_rescope(), - Some((AdmissionKey::new("r2"), "c=48".to_string())) - ); - let settled: Vec = ledger_lines(&path) - .into_iter() - .filter(|v| v["kind"] == "settled") - .collect(); - assert_eq!(settled.len(), 1); - assert_eq!(settled[0]["key"], "r1"); - assert_eq!(settled[0]["outcome"], "superseded"); - } - - #[test] - fn approve_records_the_grant_under_a_key_derived_from_the_ask() { - let (l, path) = ledger("approve"); - let state = ControlState::default(); - state.set_pending_regime("model=Q;c=48".into()); - - let reply = apply_command(request(r#"{"cmd":"approve","id":"op-1"}"#), &state, &l); - assert_eq!(reply["ok"], true); - assert_eq!(reply["regime"], "model=Q;c=48"); - - let derived = AdmissionKey::rescope_from(&AdmissionKey::approve("model=Q;c=48")); - assert_eq!( - state.take_rescope(), - Some((derived.clone(), "model=Q;c=48".to_string())), - "the loop drains the grant under the derived key" - ); - let written = ledger_lines(&path); - // approve admitted, derived rescope admitted, approve settled applied. - assert_eq!(written.len(), 3); - assert_eq!(written[0]["input"], "approve"); - assert_eq!(written[1]["key"], derived.as_str()); - assert_eq!(written[1]["regime"], "model=Q;c=48"); - assert_eq!(written[2]["kind"], "settled"); - assert_eq!(written[2]["outcome"], "applied"); - - // A second approve for the same ask converges: no second grant on the record. - state.set_pending_regime("model=Q;c=48".into()); - let again = apply_command(request(r#"{"cmd":"approve","id":"op-2"}"#), &state, &l); - assert_eq!(again["ok"], true); - assert_eq!(again["dup"], true); - assert_eq!( - ledger_lines(&path) - .iter() - .filter(|v| v["kind"] == "admitted" && v["input"] == "rescope") - .count(), - 1, - "one grant per ask" - ); - } - - #[test] - fn a_stray_approve_is_recorded_as_rejected() { - let (l, path) = ledger("stray-approve"); - let state = ControlState::default(); - let reply = apply_command(request(r#"{"cmd":"approve"}"#), &state, &l); - assert_eq!(reply["ok"], false); - let written = ledger_lines(&path); - assert_eq!(written.len(), 2); - assert_eq!(written[1]["outcome"], "rejected"); - } - - #[test] - fn stop_applies_even_when_it_cannot_be_recorded() { - let (l, path) = ledger("unrecorded-stop"); - let state = ControlState::default(); - // Burn the key on a different payload so the stop's own admission is refused: a - // safety valve must fire anyway and say that it went unrecorded. - apply_command( - request(r#"{"cmd":"steer","text":"x","id":"k"}"#), - &state, - &l, - ); - let before = STOP.load(Ordering::SeqCst); - let reply = apply_command(request(r#"{"cmd":"stop","id":"k"}"#), &state, &l); - assert_eq!(reply["ok"], true, "a stop is never refused"); - assert_eq!(reply["unrecorded"], true); - assert!(STOP.load(Ordering::SeqCst), "the flag is set regardless"); - STOP.store(before, Ordering::SeqCst); - assert_eq!(ledger_lines(&path).len(), 1, "nothing was written for it"); - } - - #[test] - fn stamps_session_lines_with_monotonic_seq() { - let state = ControlState::default(); - let (seq_one, one) = state - .stamp_session_line(r#"{"v":1,"kind":"note","msg":"a"}"#) - .unwrap(); - let (seq_two, two) = state - .stamp_session_line(r#"{"v":1,"kind":"note","msg":"b"}"#) - .unwrap(); - assert_eq!(seq_one, 1); - assert_eq!(seq_two, 2); - let one: Value = serde_json::from_str(&one).unwrap(); - let two: Value = serde_json::from_str(&two).unwrap(); - assert_eq!(one["seq"], 1); - assert_eq!(two["seq"], 2); - assert_eq!(one["kind"], "note"); - assert_eq!(two["msg"], "b"); - } - - #[test] - fn set_budget_zero_is_a_live_unlimited_cap() { - let state = ControlState::default(); - state.set_live_max_cost(0.0); - assert_eq!(state.live_max_cost(), Some(0.0)); - } - - #[test] - fn approve_turns_a_pending_regime_into_a_rescope() { - let (l, _path) = ledger("approve-slot"); - let state = ControlState::default(); - // Nothing pending yet: approve grants nothing. - assert_eq!( - apply_command(request(r#""approve""#), &state, &l)["ok"], - false - ); - assert!(!state.has_rescope()); - - // The loop records the regime an approval would grant; an operator `approve` resolves it. - state.set_pending_regime("concurrency=48".into()); - assert_eq!( - apply_command(request(r#""approve""#), &state, &l)["regime"], - "concurrency=48" - ); - assert!( - state.has_rescope(), - "approve sets the re-scope the loop drains" - ); - assert_eq!( - state.take_rescope().map(|(_, regime)| regime).as_deref(), - Some("concurrency=48") - ); - // Consumed: a second approve has nothing to grant. - assert_eq!( - apply_command(request(r#""approve""#), &state, &l)["ok"], - false - ); - } - - #[test] - fn parses_approve_command() { - assert_eq!( - parse_command(r#"{"cmd":"approve"}"#).unwrap(), - ControlCommand::Approve - ); - assert_eq!( - parse_command(r#""approve""#).unwrap(), - ControlCommand::Approve - ); - } - - #[test] - fn parses_deny_command_with_optional_reason() { - assert_eq!( - parse_command(r#"{"cmd":"deny","reason":"over budget"}"#).unwrap(), - ControlCommand::Deny { - reason: "over budget".into() - } - ); - // Bare deny defaults its reason. - assert_eq!( - parse_command(r#""deny""#).unwrap(), - ControlCommand::Deny { - reason: "rejected".into() - } - ); - } - - #[test] - fn deny_drops_a_recorded_pending_regime_so_approve_cant_resurrect_it() { - let (l, _path) = ledger("deny"); - let state = ControlState::default(); - state.set_pending_regime("concurrency=48".into()); - apply_command( - request(r#"{"cmd":"deny","reason":"policy says no"}"#), - &state, - &l, - ); - assert_eq!( - state.take_deny().map(|(_, why)| why).as_deref(), - Some("policy says no") - ); - // The pending regime is gone, so a stray `approve` grants nothing. - assert_eq!( - apply_command(request(r#""approve""#), &state, &l)["ok"], - false - ); - assert!(!state.has_rescope()); - } - - #[test] - fn stop_file_accepts_old_metadata_and_minimal_shape() { - let path = - std::env::temp_dir().join(format!("crucible-stop-file-{}.json", std::process::id())); - std::fs::write(&path, r#"{"stop":true,"kill_agent":true,"seq":42}"#).unwrap(); - assert!(stop_file_says_stop(&path)); - - std::fs::write(&path, r#"{"stop":true}"#).unwrap(); - assert!(stop_file_says_stop(&path)); - let _ = std::fs::remove_file(&path); - } - - #[test] - fn ignores_blank_or_torn_session_lines() { - assert!(stamp_session_line("", 1).is_none()); - assert!(stamp_session_line(r#"{"v":1,"kind":"note""#, 1).is_none()); - } - - #[test] - fn parses_tail_command_with_and_without_from_seq() { - assert_eq!( - parse_command(r#"{"cmd":"tail","from_seq":42}"#).unwrap(), - ControlCommand::Tail { from_seq: 42 } - ); - // Absent from_seq defaults to 0 (replay from the ring's oldest). - assert_eq!( - parse_command(r#""tail""#).unwrap(), - ControlCommand::Tail { from_seq: 0 } - ); - assert_eq!( - parse_command(r#"{"cmd":"tail","from_seq":7}"#).unwrap(), - ControlCommand::Tail { from_seq: 7 } - ); - } - - /// Read one NDJSON line from a stream with a bounded timeout, returning its parsed `seq`. - fn read_seq(reader: &mut BufReader) -> i64 { - let mut line = String::new(); - reader.read_line(&mut line).expect("read line"); - let v: Value = serde_json::from_str(line.trim()).expect("json"); - v["seq"].as_i64().expect("seq") - } - - #[test] - fn hub_replays_from_seq_then_delivers_live_exactly_once_in_order() { - // A real socket pair stands in for a subscribed client; the Hub is the actual replay core. - let listener = TcpListener::bind("127.0.0.1:0").expect("bind"); - let addr = listener.local_addr().expect("addr"); - let hub = Arc::new(Hub::default()); - - // Three lines already broadcast (retained in the ring; no subscriber yet). - hub.publish(1, r#"{"seq":1,"m":"a"}"#.to_string()); - hub.publish(2, r#"{"seq":2,"m":"b"}"#.to_string()); - hub.publish(3, r#"{"seq":3,"m":"c"}"#.to_string()); - - let client = TcpStream::connect(addr).expect("connect"); - client - .set_read_timeout(Some(Duration::from_secs(3))) - .expect("timeout"); - let (server_side, _) = listener.accept().expect("accept"); - let writer = Arc::new(Mutex::new(server_side)); - - // Resume after seq 1 (ring covers 2): the handoff replays 2 and 3 and subscribes... - assert!(hub.try_handoff(&writer, 1), "ring covers last+1"); - // ...then a live line arrives after subscription. - hub.publish(4, r#"{"seq":4,"m":"d"}"#.to_string()); - - let mut reader = BufReader::new(client); - assert_eq!(read_seq(&mut reader), 2, "replayed, seq 1 skipped"); - assert_eq!(read_seq(&mut reader), 3, "replayed in order"); - assert_eq!(read_seq(&mut reader), 4, "live after the replay seam"); - } - - #[test] - fn bridge_tail_command_streams_session_lines_over_tcp() { - // A full bridge over a real socket, tailing a real session.jsonl. Prove the `tail` command - // subscribes and streams stamped lines end to end. - let root = - std::env::temp_dir().join(format!("crucible-bridge-tail-{}", std::process::id())); - let state_dir = root.join("state"); - std::fs::create_dir_all(&state_dir).expect("state dir"); - let session_log = state_dir.join("session.jsonl"); - // The bridge tails from the file's current end, so any pre-existing content is not replayed; - // start it empty and append after it's up (matching the real loop, which writes as it runs). - std::fs::write(&session_log, "").expect("seed empty session log"); - - let paths = Paths { - workspace: root.clone(), - skills: None, - steer: root.join("STEER.md"), - state: state_dir.clone(), - session_log: session_log.clone(), - control: state_dir.join("control.json"), - escalation: root.join("ESCALATION.json"), - provisioning: root.join("PROVISIONING_PENDING.json"), - admissions: state_dir.join("admissions.jsonl"), - }; - - let port = { - let l = TcpListener::bind("127.0.0.1:0").expect("free port"); - l.local_addr().expect("addr").port() - }; - let ledger = Arc::new( - AdmissionLedger::open(&paths.admissions, forge::ndjson::Open::Truncate) - .expect("ledger"), - ); - spawn_bridge(port, paths, ledger).expect("spawn bridge"); - // Let the tail thread reach the (empty) file's end before we start appending. - thread::sleep(Duration::from_millis(200)); - - let append = |m: &str| { - let mut f = OpenOptions::new() - .append(true) - .open(&session_log) - .expect("reopen session log"); - writeln!(f, r#"{{"kind":"note","m":"{m}"}}"#).expect("append"); - }; - - // Two lines land while the bridge is up (no subscriber yet): they get seq 1 and 2 into the - // replay ring. Give the 100ms poll room to see both. - append("one"); - append("two"); - thread::sleep(Duration::from_millis(300)); - - let stream = TcpStream::connect(("127.0.0.1", port)).expect("connect"); - stream - .set_read_timeout(Some(Duration::from_secs(3))) - .expect("timeout"); - let mut writer = stream.try_clone().expect("clone"); - // Resume after seq 1: expect line 2 replayed, then a newly appended line live. - writeln!(writer, r#"{{"cmd":"tail","from_seq":1}}"#).expect("send tail"); - writer.flush().expect("flush"); - - // The subscribe replays before the command loop writes the `tail` ack, so the two frame - // kinds (session lines carrying `seq`, the ack carrying `cmd`) can interleave, classify by - // content, exactly as the live relay does, rather than assuming an order. - let mut reader = BufReader::new(stream); - let mut got_ack = false; - let next_seq = |reader: &mut BufReader, got_ack: &mut bool| -> i64 { - loop { - let mut line = String::new(); - reader.read_line(&mut line).expect("read frame"); - let v: Value = serde_json::from_str(line.trim()).expect("json"); - if v.get("cmd").is_some() { - *got_ack = true; - continue; - } - return v["seq"].as_i64().expect("seq"); - } - }; - - // The replayed line 2 (seq 1 skipped by from_seq=1). - assert_eq!( - next_seq(&mut reader, &mut got_ack), - 2, - "replayed after from_seq" - ); - - // A third line appended now is broadcast live to the subscriber. - append("three"); - assert_eq!( - next_seq(&mut reader, &mut got_ack), - 3, - "live-streamed appended line" - ); - assert!(got_ack, "the tail command was acked"); - - let _ = std::fs::remove_dir_all(&root); - } - - fn tempdir(name: &str) -> std::path::PathBuf { - let dir = std::env::temp_dir().join(format!( - "crucible-control-{name}-{}-{}", - std::process::id(), - now_secs() - )); - std::fs::create_dir_all(&dir).expect("tempdir"); - dir - } - - fn socket_pair() -> (BufReader, Client) { - let listener = TcpListener::bind("127.0.0.1:0").expect("bind"); - let addr = listener.local_addr().expect("addr"); - let client = TcpStream::connect(addr).expect("connect"); - client - .set_read_timeout(Some(Duration::from_secs(5))) - .expect("timeout"); - let (server_side, _) = listener.accept().expect("accept"); - (BufReader::new(client), Arc::new(Mutex::new(server_side))) - } - - #[test] - fn try_handoff_declines_when_the_ring_aged_past_the_resume_point() { - let hub = Hub::default(); - // Ring holds 100..=102, a client that has only seen up to 42 has a gap (43..=99 aged out). - for seq in 100..=102u64 { - hub.publish(seq, format!(r#"{{"seq":{seq}}}"#)); - } - let (_reader, writer) = socket_pair(); - assert!( - !hub.try_handoff(&writer, 42), - "gap: must read the file first" - ); - assert!( - hub.try_handoff(&writer, 99), - "contiguous: 100 is the next line" - ); - } - - #[test] - fn file_replay_streams_history_older_than_the_ring_then_hands_off() { - // 60 session lines on disk; the ring only retains the newest 3 (publishes 58..=60), a - // tail from 0 must get 1..=60 exactly once, in order, then live lines. - let dir = tempdir("file-replay"); - let path = dir.join("session.jsonl"); - let mut body = String::new(); - for i in 1..=60u64 { - body.push_str(&format!("{{\"v\":1,\"kind\":\"note\",\"msg\":\"m{i}\"}}\n")); - } - std::fs::write(&path, body).expect("write"); - - let hub = Arc::new(Hub::default()); - for seq in 58..=60u64 { - hub.publish(seq, format!(r#"{{"seq":{seq},"live":true}}"#)); - } - // Force the gap: drop everything older than 58 out of the ring window. - { - let mut g = hub.inner.lock().unwrap(); - while g.history.len() > 3 { - g.history.pop_front(); - } - } - - let (mut reader, writer) = socket_pair(); - tail_with_file_replay(&path, &hub, &writer, 0); - hub.publish(61, r#"{"seq":61,"live":true}"#.to_string()); - - for expect in 1..=61i64 { - assert_eq!(read_seq(&mut reader), expect, "in order, exactly once"); - } - } - - #[test] - fn file_replay_skips_unparseable_lines_but_their_seq_is_consumed() { - let dir = tempdir("torn-line"); - let path = dir.join("session.jsonl"); - // Line 2 is torn garbage: its seq is consumed (numbering = file line index) but not sent. - std::fs::write( - &path, - "{\"v\":1,\"kind\":\"note\",\"msg\":\"a\"}\n{\"broken\n{\"v\":1,\"kind\":\"note\",\"msg\":\"c\"}\n", - ) - .expect("write"); - // Ring holds only seq 3 (oldest = 3 > 0+1), so the tail must take the file pass. - let hub = Hub::default(); - hub.publish(3, r#"{"seq":3,"m":"c"}"#.to_string()); - let (mut reader, writer) = socket_pair(); - tail_with_file_replay(&path, &hub, &writer, 0); - assert_eq!(read_seq(&mut reader), 1); - assert_eq!( - read_seq(&mut reader), - 3, - "seq 2 consumed by the torn line, never sent" - ); - } - - #[test] - fn file_replay_converges_while_the_file_keeps_growing() { - // The race the handoff loop exists for: lines keep arriving while the file pass streams. - // A writer thread appends + publishes 20 more lines while the replay runs; the client must - // still see 1..=70 exactly once in order. - let dir = tempdir("growing"); - let path = dir.join("session.jsonl"); - let mut body = String::new(); - for i in 1..=50u64 { - body.push_str(&format!("{{\"v\":1,\"kind\":\"note\",\"msg\":\"m{i}\"}}\n")); - } - std::fs::write(&path, body).expect("write"); - - let hub = Arc::new(Hub::default()); - hub.publish(50, r#"{"seq":50}"#.to_string()); - { - let mut g = hub.inner.lock().unwrap(); - while g.history.len() > 1 { - g.history.pop_front(); - } - } - - let grower = { - let hub = hub.clone(); - let path = path.clone(); - thread::spawn(move || { - for i in 51..=70u64 { - let line = format!("{{\"v\":1,\"kind\":\"note\",\"msg\":\"m{i}\"}}\n"); - let mut f = OpenOptions::new().append(true).open(&path).expect("append"); - f.write_all(line.as_bytes()).expect("grow"); - hub.publish(i, format!(r#"{{"seq":{i}}}"#)); - thread::sleep(Duration::from_millis(1)); - } - }) - }; - - let (mut reader, writer) = socket_pair(); - tail_with_file_replay(&path, &hub, &writer, 0); - grower.join().expect("grower"); - // Anything not yet delivered at handoff arrives live; read all 70 in order. - for expect in 1..=70i64 { - assert_eq!(read_seq(&mut reader), expect, "exactly once, in order"); - } - } -} diff --git a/crucible/src/control/pr_watch.rs b/crucible/src/control/pr_watch.rs index 3b18d029..15cd6935 100644 --- a/crucible/src/control/pr_watch.rs +++ b/crucible/src/control/pr_watch.rs @@ -261,7 +261,7 @@ pub enum Sink { /// A live run's control-bridge address (host:port); delivered over TCP as a `steer` command. Steer(String), /// A file (typically the next run's `STEER.md`) appended directly, in the same - /// `` shape `control::append_steer` writes, no run needs to be up. + /// `` shape `control::bridge::append_steer` writes, no run needs to be up. Reseed(PathBuf), } @@ -283,7 +283,7 @@ impl Sink { send_steer(addr, text, key).context("sending steer over the control bridge") } Sink::Reseed(path) => { - control::append_steer(path, text).context("appending to reseed file") + control::bridge::append_steer(path, text).context("appending to reseed file") } } } @@ -591,7 +591,7 @@ mod tests { #[test] fn reseed_sink_appends_the_same_shape_the_loop_reads() { - // No live run: the reseed sink writes straight to a file (`control::append_steer`'s exact + // No live run: the reseed sink writes straight to a file (`control::bridge::append_steer`'s exact // marker-wrapped shape), which the next run's first steer drain reads. let path = std::env::temp_dir().join(format!( "crucible-pr-watch-reseed-{}-{}.md", diff --git a/crucible/src/main.rs b/crucible/src/main.rs index 1c640df2..734627f0 100644 --- a/crucible/src/main.rs +++ b/crucible/src/main.rs @@ -14,11 +14,14 @@ //! [`args::Paths`], [`args::Prepared`]); [`cli`] parses the command line and dispatches it; //! [`runloop`] holds the single orchestration loop; [`agent`] runs one turn; [`control`] steers a //! running loop from outside; [`report`] is how the loop talks to a human or a log; [`scope`] -//! is the scoping pipeline. The loop talks only to a [`report::Reporter`], so one loop drives +//! is the scoping pipeline. The loop talks only to a [`report::reporter::Reporter`], so one loop drives //! multiple front-ends: [`report::console::ConsoleReporter`] for headless runs and the NDJSON //! [`report::stream::SessionReporter`] for stdout/session-log runs. The choice is just `--ui` //! (default: auto by TTY). //! +//! The loop, [`scope`], and the loop-only parts of [`control`] and [`report`] build only with the +//! `autoresearch` feature; without it the binary is the playbook workflow engine. +//! //! Operator ergonomics: //! //! - Ctrl+C never just dies: it stops cleanly after the current step and prints a summary. Headless offers a steer/quit prompt. @@ -28,10 +31,14 @@ mod agent; mod args; mod cli; mod control; +#[cfg(feature = "autoresearch")] mod identity; +mod object_store; mod process; mod report; +#[cfg(feature = "autoresearch")] mod runloop; +#[cfg(feature = "autoresearch")] mod scope; #[cfg(test)] mod testing; @@ -61,6 +68,7 @@ mod plan { pub mod cli; pub mod events; pub mod harness; + pub mod template; } use anyhow::Result; diff --git a/crucible/src/object_store.rs b/crucible/src/object_store.rs new file mode 100644 index 00000000..91c67190 --- /dev/null +++ b/crucible/src/object_store.rs @@ -0,0 +1,134 @@ +//! Published objects addressed by URI: `s3://bucket[/key]` or an absolute `file:///path`. + +use anyhow::{Context, Result}; + +/// A URI this module cannot address. +#[derive(Debug, thiserror::Error, PartialEq)] +pub enum ObjectUriError { + #[error("results bucket must be an s3:// or file:// URI, got `{uri}`")] + NotAnS3Uri { uri: String }, + #[error("file:// results root must be absolute: `{uri}`")] + FileRootRelative { uri: String }, + #[error("results bucket URI has no bucket: `{uri}`")] + NoBucket { uri: String }, + #[error("fetch object URI has no key: `{uri}`")] + NoKey { uri: String }, +} + +/// A publish destination: S3 (`s3://bucket[/prefix]`) or a mounted filesystem +/// (`file:///abs/path`, e.g. an artifacts PVC on a cluster with no S3 reach). Both write the +/// exact same key layout, so reporting tools walk either. +pub(crate) enum Backend { + // The S3 half re-parses the URI where it's used (the async block owns bucket/base), so the + // variant carries nothing. + S3, + File { root: std::path::PathBuf }, +} + +pub(crate) fn backend(uri: &str) -> Result { + match uri.strip_prefix("file://") { + Some(path) if path.starts_with('/') => Ok(Backend::File { + root: std::path::PathBuf::from(path), + }), + Some(_) => Err(ObjectUriError::FileRootRelative { + uri: uri.to_owned(), + }), + None => parse_s3_uri(uri).map(|_| Backend::S3), + } +} + +/// `s3://bucket[/prefix]` → (bucket, prefix). Prefix is trimmed of slashes and may +/// be empty. +pub(crate) fn parse_s3_uri(uri: &str) -> Result<(String, String), ObjectUriError> { + let rest = uri + .strip_prefix("s3://") + .ok_or_else(|| ObjectUriError::NotAnS3Uri { + uri: uri.to_owned(), + })?; + let (bucket, prefix) = rest.split_once('/').unwrap_or((rest, "")); + if bucket.is_empty() { + return Err(ObjectUriError::NoBucket { + uri: uri.to_owned(), + }); + } + Ok((bucket.to_string(), prefix.trim_matches('/').to_string())) +} + +/// Download one published object at an exact `s3://bucket/key` URI to a local file, the general +/// GetObject the controller's artifact proxy shells (`crucible fetch`), keeping every S3 client out +/// of `crucible-controller` (that crate has no aws-sdk and never learns the bucket layout). Nothing +/// is appended to the URI: the caller passes the exact key it wants. Reuses the same IRSA creds the +/// publisher uses (GetObject is in the role's policy). +pub fn fetch_object(uri: &str, dest: &std::path::Path) -> Result<()> { + if let Backend::File { root } = backend(uri)? { + // The file URI IS the object path; a plain copy is the whole fetch. + std::fs::copy(&root, dest) + .with_context(|| format!("copying {} to {}", root.display(), dest.display()))?; + return Ok(()); + } + let (bucket, key) = parse_s3_uri(uri)?; + if key.is_empty() { + return Err(ObjectUriError::NoKey { + uri: uri.to_owned(), + } + .into()); + } + crate::agent::engine::handle()?.block_on(async { + let conf = aws_config::load_defaults(aws_config::BehaviorVersion::latest()).await; + let client = aws_sdk_s3::Client::new(&conf); + let out = client + .get_object() + .bucket(&bucket) + .key(&key) + .send() + .await + .with_context(|| format!("GetObject s3://{bucket}/{key}"))?; + let data = out + .body + .collect() + .await + .context("read object body")? + .into_bytes(); + std::fs::write(dest, &data) + .with_context(|| format!("writing {} ({} bytes)", dest.display(), data.len()))?; + Ok::<(), anyhow::Error>(()) + }) +} + +#[cfg(test)] +mod tests { + use crate::object_store::*; + + #[test] + fn parse_s3_uri_splits_bucket_and_prefix() { + assert_eq!( + parse_s3_uri("s3://my-bucket/autoresearch").unwrap(), + ("my-bucket".into(), "autoresearch".into()) + ); + assert_eq!( + parse_s3_uri("s3://my-bucket").unwrap(), + ("my-bucket".into(), String::new()) + ); + assert_eq!( + parse_s3_uri("s3://my-bucket/a/b/").unwrap(), + ("my-bucket".into(), "a/b".into()) + ); + assert!(parse_s3_uri("https://nope").is_err()); + assert!(parse_s3_uri("s3:///just-prefix").is_err()); + } + + #[test] + fn a_file_uri_fetches_by_copy_and_a_relative_one_is_refused() { + let dir = tempfile::tempdir().unwrap(); + let src = dir.path().join("summary.txt"); + std::fs::write(&src, "investigation").unwrap(); + let dest = dir.path().join("fetched.txt"); + fetch_object(&format!("file://{}", src.display()), &dest).unwrap(); + assert_eq!(std::fs::read_to_string(&dest).unwrap(), "investigation"); + assert!(fetch_object("file://relative/path", &dest).is_err()); + assert!( + fetch_object("s3://bucket", &dest).is_err(), + "an s3 URI with no key" + ); + } +} diff --git a/crucible/src/plan/cli.rs b/crucible/src/plan/cli.rs index 76575069..2a1d5d89 100644 --- a/crucible/src/plan/cli.rs +++ b/crucible/src/plan/cli.rs @@ -477,6 +477,7 @@ config: }) } +#[cfg(any(feature = "autoresearch", test))] /// Render a validated graph to PNG. pub fn render_png_to( plan: &ValidPlan, @@ -622,7 +623,7 @@ pub fn run( .validate() .context("building one-pass playbook")? } else { - crate::runloop::graph::iteration_template(Some(workflow), &caps)? + crate::plan::template::iteration_template(Some(workflow), &caps)? } } }; diff --git a/crucible/src/plan/harness.rs b/crucible/src/plan/harness.rs index 0f93e263..6d6e9512 100644 --- a/crucible/src/plan/harness.rs +++ b/crucible/src/plan/harness.rs @@ -1030,7 +1030,7 @@ mod tests { let mut manifest = crate::manifest::Manifest::load(&dir.join("crucible.toml")).unwrap(); manifest.resolve_workflow(dir).unwrap(); let workflow = manifest.workflow.as_ref().unwrap(); - let plan = crate::runloop::graph::iteration_template( + let plan = crate::plan::template::iteration_template( Some(workflow), &crate::plan::workflow::WorkflowCaps::for_lane(workflow.workflow_type), ) @@ -1609,7 +1609,7 @@ workflow(type = "playbook", tasks = [draft, shape, polish, audit_a, audit_b, rou ); assert!(manifest.is_task(), "a playbook carries no judge"); - let plan = crate::runloop::graph::iteration_template( + let plan = crate::plan::template::iteration_template( Some(workflow), &crate::plan::workflow::WorkflowCaps::for_lane(workflow.workflow_type) .with_persistent_sessions(), @@ -1751,7 +1751,7 @@ workflow(type = "playbook", tasks = [discover, audit, roundup]) let mut manifest = crate::manifest::Manifest::load(&dir.join("crucible.toml")).unwrap(); manifest.resolve_workflow(&dir).unwrap(); let workflow = manifest.workflow.as_ref().expect("workflow"); - let plan = crate::runloop::graph::iteration_template( + let plan = crate::plan::template::iteration_template( Some(workflow), &crate::plan::workflow::WorkflowCaps::for_lane(workflow.workflow_type), ) @@ -1827,7 +1827,7 @@ workflow(type = "playbook", tasks = [discover, audit, roundup]) .unwrap(); let mut narrow = crate::manifest::Manifest::load(&dir.join("crucible.toml")).unwrap(); narrow.resolve_workflow(&dir).unwrap(); - let plan = crate::runloop::graph::iteration_template( + let plan = crate::plan::template::iteration_template( Some(narrow.workflow.as_ref().unwrap()), &crate::plan::workflow::WorkflowCaps::for_lane( crate::plan::workflow::WorkflowType::Playbook, @@ -1921,7 +1921,7 @@ workflow(type = "playbook", tasks = [good, bad, after]) let mut manifest = crate::manifest::Manifest::load(&dir.join("crucible.toml")).unwrap(); manifest.resolve_workflow(&dir).unwrap(); let workflow = manifest.workflow.as_ref().unwrap(); - let plan = crate::runloop::graph::iteration_template( + let plan = crate::plan::template::iteration_template( Some(workflow), &crate::plan::workflow::WorkflowCaps::for_lane(workflow.workflow_type), ) @@ -2789,7 +2789,7 @@ workflow(type = "playbook", tasks = [discover, audit, roundup]) let mut manifest = crate::manifest::Manifest::load(&dir.join("crucible.toml")).unwrap(); manifest.resolve_workflow(dir).unwrap(); let workflow = manifest.workflow.as_ref().expect("workflow"); - crate::runloop::graph::iteration_template( + crate::plan::template::iteration_template( Some(workflow), &crate::plan::workflow::WorkflowCaps::for_lane(workflow.workflow_type), ) @@ -3646,7 +3646,7 @@ workflow(type = "playbook", tasks = [author, repro]) let mut manifest = crate::manifest::Manifest::load(&dir.join("crucible.toml")).unwrap(); manifest.resolve_workflow(&dir).unwrap(); let workflow = manifest.workflow.as_ref().unwrap(); - let plan = crate::runloop::graph::iteration_template( + let plan = crate::plan::template::iteration_template( Some(workflow), &crate::plan::workflow::WorkflowCaps::for_lane(workflow.workflow_type) .with_persistent_sessions(), diff --git a/crucible/src/plan/template.rs b/crucible/src/plan/template.rs new file mode 100644 index 00000000..ca3e7f04 --- /dev/null +++ b/crucible/src/plan/template.rs @@ -0,0 +1,189 @@ +//! The per-iteration plan an autoresearch workflow runs: the authored graph, or the default +//! propose, apply, measure, decide chain with any legacy splice tasks between propose and apply. + +use anyhow::{Context, Result}; + +use crate::plan::ir::{ + EngineOp, Join, Plan, PlanBudget, Stage, Task, TaskKind, TaskName, ValidPlan, +}; +use crate::plan::workflow::{WorkflowCaps, WorkflowCfg, WorkflowType}; + +/// Build and admit the default or authored iteration graph. +pub(crate) fn iteration_template( + workflow: Option<&WorkflowCfg>, + caps: &WorkflowCaps, +) -> Result { + if let Some(workflow) = workflow.filter(|workflow| !workflow.is_legacy_splice()) { + workflow + .admit(caps) + .context("admitting authored workflow into the autoresearch loop")?; + return Plan { + version: 1, + reason: None, + budget: PlanBudget { usd: f64::MAX }, + tasks: workflow.iteration_tasks(), + } + .validate() + .context("building authored iteration workflow"); + } + + let engine = + |name: &str, op: EngineOp, source: Option, deps: Vec| -> Task { + Task { + name: name.into(), + task: TaskKind::Engine { + op, + source, + tiebreak: None, + }, + depends_on: deps, + session: None, + needs: "any".to_string(), + required: true, + isolation: None, + join: Join::default(), + stage: Stage::Iteration, + emits: Vec::new(), + emits_files: Vec::new(), + over: None, + max_fanout: None, + when: None, + revise: None, + } + }; + let mut tasks = vec![engine("propose", EngineOp::Propose, None, vec![])]; + + // Legacy splice tasks run between `propose` and `apply`; `apply` waits on every sink. + // Epilogue tasks never splice: they run once post-loop, not per iteration. + let mut apply_deps = vec![TaskName("propose".to_string())]; + if let Some(w) = workflow.filter(|w| !w.tasks.is_empty()) { + for mut t in w.iteration_tasks() { + if t.depends_on.is_empty() { + t.depends_on = vec![TaskName("propose".to_string())]; + } + tasks.push(t); + } + let sinks = w.sinks(); + if !sinks.is_empty() { + apply_deps = sinks; + } + } + + tasks.push(engine("apply", EngineOp::Apply, None, apply_deps)); + tasks.push(engine( + "measure", + EngineOp::Measure, + None, + vec![TaskName("apply".to_string())], + )); + tasks.push(engine( + "decide", + EngineOp::Decide, + Some(TaskName("measure".to_string())), + vec![TaskName("measure".to_string())], + )); + let workflow = WorkflowCfg { + workflow_type: WorkflowType::Autoresearch, + result: Some("decide".into()), + tasks, + file: None, + resolved_from: None, + }; + workflow + .admit(caps) + .context("admitting the default autoresearch workflow")?; + Plan { + version: 1, + reason: None, + budget: PlanBudget { usd: f64::MAX }, + tasks: workflow.tasks, + } + .validate() + .context("building the iteration template") +} + +#[cfg(test)] +mod tests { + use crate::plan::template::*; + + #[test] + fn pack_tasks_splice_between_the_turn_and_the_gate() { + let w: WorkflowCfg = toml::from_str( + "[[task]]\nname = \"review\"\nkind = \"command\"\ncommand = \"true\"\n [[task]]\nname = \"lint\"\nkind = \"command\"\ncommand = \"true\"\ndepends_on = [\"review\"]\n", + ) + .unwrap(); + w.validate().unwrap(); + let plan = iteration_template(Some(&w), &WorkflowCaps::autoresearch_engine()).unwrap(); + let names: Vec<&str> = plan.tasks_topo().map(|t| t.name.0.as_str()).collect(); + assert_eq!( + names, + ["propose", "review", "lint", "apply", "measure", "decide"] + ); + let dep = |n: &str| { + plan.get(&n.into()) + .unwrap() + .depends_on + .iter() + .map(|d| d.0.clone()) + .collect::>() + }; + assert_eq!( + dep("review"), + ["propose"], + "an unattached task hangs off propose" + ); + assert_eq!( + dep("apply"), + ["lint"], + "apply waits on the sink, not on propose" + ); + assert_eq!(dep("measure"), ["apply"]); + assert_eq!(dep("decide"), ["measure"]); + } + + #[test] + fn template_is_the_canonical_chain() { + let plan = iteration_template(None, &WorkflowCaps::autoresearch_engine()).unwrap(); + let names: Vec<&str> = plan.tasks_topo().map(|t| t.name.0.as_str()).collect(); + assert_eq!(names, ["propose", "apply", "measure", "decide"]); + assert!(plan.tasks_topo().all(|t| t.required)); + let kinds: Vec<&str> = plan.tasks_topo().map(|t| t.task.label()).collect(); + assert_eq!( + kinds, + [ + "engine_propose", + "engine_apply", + "engine_measure", + "engine_decide" + ] + ); + } + + #[test] + fn authored_autoresearch_uses_semantics_instead_of_reserved_names() { + let workflow: WorkflowCfg = toml::from_str( + "type = \"autoresearch\"\nresult = \"keep-if-better\"\n\ + [[task]]\nname = \"invent\"\nkind = \"engine\"\nop = \"propose\"\n\ + [[task]]\nname = \"review\"\nkind = \"command\"\ncommand = \"true\"\ndepends_on = [\"invent\"]\n\ + [[task]]\nname = \"deploy-preview\"\nkind = \"engine\"\nop = \"apply\"\ndepends_on = [\"review\"]\n\ + [[task]]\nname = \"benchmark-a\"\nkind = \"engine\"\nop = \"measure\"\ndepends_on = [\"deploy-preview\"]\n\ + [[task]]\nname = \"explain-score\"\nkind = \"command\"\ncommand = \"true\"\ndepends_on = [\"benchmark-a\"]\n\ + [[task]]\nname = \"keep-if-better\"\nkind = \"engine\"\nop = \"decide\"\nsource = \"benchmark-a\"\ndepends_on = [\"benchmark-a\", \"explain-score\"]\n", + ) + .unwrap(); + let plan = + iteration_template(Some(&workflow), &WorkflowCaps::autoresearch_engine()).unwrap(); + let names: Vec<&str> = plan.tasks_topo().map(|task| task.name.0.as_str()).collect(); + assert_eq!( + names, + [ + "invent", + "review", + "deploy-preview", + "benchmark-a", + "explain-score", + "keep-if-better" + ] + ); + } +} diff --git a/crucible/src/process.rs b/crucible/src/process.rs index 81467fbc..11d58504 100644 --- a/crucible/src/process.rs +++ b/crucible/src/process.rs @@ -4,6 +4,7 @@ use std::sync::atomic::AtomicBool; pub(crate) static STOP: AtomicBool = AtomicBool::new(false); +#[cfg(feature = "autoresearch")] /// Send SIGTERM to one PID (no-op if zero/negative), via libc rather than the `kill` binary. pub(crate) fn kill_pid(pid: i32) { if pid > 0 { @@ -30,6 +31,7 @@ pub(crate) mod pid_registry { } } + #[cfg(feature = "autoresearch")] pub fn kill_all() { if let Ok(v) = PIDS.lock() { for &pid in v.iter() { diff --git a/crucible/src/report/console.rs b/crucible/src/report/console.rs index 3e4a91a7..88b0ac68 100644 --- a/crucible/src/report/console.rs +++ b/crucible/src/report/console.rs @@ -9,8 +9,8 @@ use crate::agent; use crate::agent::event::{AgentEvent, RawStream}; use crate::args::{Args, Paths}; use crate::process::STOP; +use crate::report::reporter::{AgentTurn, Reporter, Stop, TurnBudget}; use crate::report::session::Row; -use crate::report::{AgentTurn, Reporter, Stop, TurnBudget}; use crucible_contract::LoopPhase; use std::io::{IsTerminal, Write}; use std::sync::atomic::Ordering; @@ -47,7 +47,7 @@ impl Reporter for ConsoleReporter { if !row.evidence.is_empty() { println!( " evidence: {}", - crate::report::evidence_line(&row.evidence) + crate::report::reporter::evidence_line(&row.evidence) ); } if solved { @@ -226,7 +226,7 @@ fn print_rows(rows: &[Row]) { } else { format!( " [evidence: {}]", - crate::report::evidence_line(&r.evidence) + crate::report::reporter::evidence_line(&r.evidence) ) }; println!( diff --git a/crucible/src/report/mod.rs b/crucible/src/report/mod.rs index 1501da0b..5ca8d9dd 100644 --- a/crucible/src/report/mod.rs +++ b/crucible/src/report/mod.rs @@ -1,276 +1,19 @@ //! The boundary between the orchestration loop and how it talks to a human. //! -//! The loop (`crate::run_loop`) is written once and calls a [`Reporter`]; the +//! The loop (`crate::run_loop`) is written once and calls a [`reporter::Reporter`]; the //! implementations are [`crate::report::console::ConsoleReporter`] (headless: plain lines, //! for CI / in-cluster pods / pipes) and [`crate::report::stream::SessionReporter`] (NDJSON session //! events). All drive the identical keep/discard logic, so headless is a first-class //! mode, not a degraded fallback. +#[cfg(feature = "autoresearch")] pub(crate) mod console; pub(crate) mod flow_dd; pub(crate) mod ingest_client; +#[cfg(feature = "autoresearch")] +pub(crate) mod reporter; +#[cfg(feature = "autoresearch")] pub(crate) mod result_mode; pub(crate) mod session; +#[cfg(feature = "autoresearch")] pub(crate) mod stream; - -use crate::args::{Args, Paths}; -use crate::report::session::Row; -use crucible_contract::LoopPhase; -use std::sync::Arc; -use std::time::{Duration, Instant}; - -/// Cumulative-budget context for one agent turn. Cost is only authoritative at turn -/// end, so mid-turn the reporters price the streamed token samples (see -/// [`crate::agent::event::provisional_cost`]) and emit provisional budget updates; the -/// loop's turn-end budget call reconciles them. Without this a run could spend its -/// whole cap inside one turn before any guard fires. -#[derive(Clone)] -pub struct TurnBudget { - /// Run spend before this turn started (authoritative). - pub spent_before: f64, - /// When the run started, so provisional updates carry the run-relative clock. - pub started: Instant, - /// Effective cost cap resolved by the loop (live control override or the CLI - /// arg); 0 means uncapped. - pub max_cost: f64, - /// Live liveness telemetry for provisional, mid-turn spend. The authoritative - /// total still comes from the completed turn and overwrites this estimate. - pub(crate) heartbeat: Option>, -} - -impl TurnBudget { - /// Whether a provisional spend crosses the cap (same threshold as the loop's - /// between-iteration guard). - pub fn over_cap(&self, spent: f64) -> bool { - self.max_cost > 0.0 && spent >= self.max_cost - } - - /// Publish a provisional cumulative total while the agent is still running. - pub fn record_provisional(&self, iter: u32, spent: f64) { - if let Some(heartbeat) = &self.heartbeat { - heartbeat.record(iter, spent); - } - } -} - -/// One-line evidence rendering for the human-readable outputs (console rows, RESULTS.md, -/// PR bodies): `refcheck ✓ calc-diff ✓ tensor-pipe SKIPPED (reason)`. -pub(crate) fn evidence_line(evidence: &[crate::report::session::EvidenceEntry]) -> String { - evidence - .iter() - .map(ToString::to_string) - .collect::>() - .join(" ") -} - -/// Run-wide context every front-end needs to render the start banner. Built once -/// from [`Args`] so the NDJSON reporters don't each re-derive it. -#[derive(Clone)] -pub struct RunMeta { - pub namespace: String, - pub model: String, - pub iters_total: u32, - pub max_cost: f64, - pub max_secs: u64, -} - -impl RunMeta { - pub fn from_args(args: &Args) -> Self { - Self { - namespace: args.namespace.clone(), - model: args.model().to_string(), - iters_total: args.iterations, - max_cost: args.max_cost, - max_secs: args.max_time().map(|d| d.as_secs()).unwrap_or(0), - } - } -} - -/// What the run achieved, for the process exit code. -#[derive(Debug, Clone, Copy, Default)] -pub struct Outcome { - pub improved: bool, - pub solved: bool, - /// The agent declared the harness inadequate (harness-inadequate escalation): the run stopped for - /// human review, distinct from both success and a plain no-improvement. - pub escalated: bool, -} - -impl Outcome { - /// CI exit code. Escalation gets its own code (2) so the outer pod / CI can route a - /// "needs human" stop separately from success (0) and no-improvement (1). - pub fn exit_code(&self) -> i32 { - if self.escalated { - 2 - } else if self.improved || self.solved { - 0 - } else { - 1 - } - } -} - -/// What the operator decided at an interrupt checkpoint. -pub enum Stop { - Continue, - Quit, -} - -/// The outcome of one agent turn the loop needs beyond the streamed events: the -/// cost to budget on, and whether the CLI reported the turn as an error so the loop -/// can discard a no-op instead of measuring an unchanged workspace as a success. -#[derive(Debug, Clone, Default)] -pub struct AgentTurn { - /// The turn's cost (USD). - pub cost: f64, - /// The `result` event's `is_error` flag, a failed turn (e.g. a credential-less - /// "Not logged in" no-op) even when its subtype is "success". - pub is_error: bool, - /// The failure text the CLI reported (the synthetic message), for the discard note. - pub error: Option, -} - -/// Everything the loop needs from its front-end. The loop never prints directly. -pub trait Reporter { - /// The run is starting with this goal and objective label. - fn start(&mut self, goal: &str, objective: &str); - /// The loop moved to `phase` at iteration `iter` (0 before the first). - fn phase(&mut self, phase: LoopPhase, iter: u32); - /// A free-form progress note (setup steps, steer injection, ...). - fn note(&mut self, msg: &str); - /// A decided row (baseline / keep / discard); `solved` flags a winning fix. - fn row(&mut self, row: &Row, solved: bool); - /// Run the agent subprocess for `it`, streaming its output to the front-end. - /// Returns the turn's cost and its error verdict ([`AgentTurn`]) so the loop can - /// budget on it and discard a failed no-op turn. `budget` carries the run spend - /// so far and the cost cap: reporters stream provisional budget updates from - /// mid-turn token samples and may stop a local agent once the provisional spend - /// crosses the cap. - #[allow(clippy::too_many_arguments)] - fn run_agent( - &mut self, - args: &Args, - p: &Paths, - it: u32, - prompt: &str, - resume_prompt: Option<&str>, - session: Option<&str>, - budget: TurnBudget, - ) -> AgentTurn; - /// Cumulative budget after a turn; lets the front-end draw a gauge / warn. - fn budget(&mut self, _spent: f64, _elapsed: Duration) {} - /// Interrupt checkpoint: decide whether to stop. - fn check_interrupt(&mut self, p: &Paths, rows: &[Row]) -> Stop; - /// A (re-)scope boundary (a re-scope boundary): a fresh baseline + `fingerprint` begin a new comparable - /// segment (`regime` labels the new evaluation regime). The default renders a note; the - /// session reporter overrides it to emit a structured [`crate::report::session::SessionEvent::Segment`]. - fn segment(&mut self, fingerprint: &str, baseline_score: f64, regime: &str) { - self.note(&format!( - "segment [{regime}] fingerprint={fingerprint} baseline={baseline_score}" - )); - } - /// The run's comparability key, reported once at run start (and again on - /// `--resume`, the freshly recomputed value). The default renders a note; the session - /// reporter overrides it to emit a structured [`crate::report::session::SessionEvent::Identity`]. - fn identity(&mut self, identity: &crate::identity::RunIdentity) { - self.note(&format!( - "run identity {} ({} component(s), measure_cmd={})", - identity.digest, - identity.components.len(), - identity.measure_cmd - )); - } - /// The agent escalated (declared the harness inadequate), halting the run for human review. - /// The default renders it as a note; the session reporter overrides this to emit a structured - /// [`crate::report::session::SessionEvent::Escalation`]. - fn escalation(&mut self, esc: &crate::control::escalation::Escalation) { - let evidence = if esc.evidence.trim().is_empty() { - String::new() - } else { - format!(" — evidence: {}", esc.evidence.trim()) - }; - self.note(&format!( - "ESCALATION [{}]: {}{evidence}", - esc.category, esc.reason - )); - } - /// An additive work-graph wire line (`PlanAdmitted` / `TaskResult`) from a loop iteration's plan - /// iteration. Default no-op: only the session reporter persists them; the console - /// front-end has no plan rendering, and the legacy sequencing path never emits one. - fn plan_event(&mut self, _ev: &crate::report::session::SessionEvent) {} - /// How this resume classified the previous shutdown; the session reporter emits a - /// structured [`crate::report::session::SessionEvent::Recovery`]. - fn recovery(&mut self, class: crate::report::session::RecoveryClass, iter: u32, detail: &str) { - self.note(&format!("recovery: {class} (iter {iter}): {detail}")); - } - /// Opens the approval bracket a resume's classifier reads: a dangling wait means the - /// run died with the approval open. - fn approval_wait( - &mut self, - handle: &str, - trace_id: &str, - mode: crate::control::provisioning::WaitMode, - ) { - self.note(&format!( - "approval wait [{}] {handle} ({trace_id})", - mode.as_str() - )); - } - /// The wait reached a terminal outcome (`granted`/`denied`/`timeout`), closing the - /// bracket. Deliberately NOT called on a stop-while-parked: a stop doesn't resolve - /// the ask, and the still-open bracket makes a resume re-park on it. - fn approval_resolved(&mut self, outcome: &str, reason: &str) { - self.note(&format!("approval {outcome}: {reason}")); - } - /// The draft PR(s) publish-on-keep opened, reported once after publish so the durable session - /// log carries them (the controller's pull-ingest folds them onto the kept candidates' `pr_url`). - /// The default is a no-op, the console already prints each URL via `note`; only the session - /// reporter persists a structured [`crate::report::session::SessionEvent::PrLinks`] line. Best-effort: - /// an append failure must never fail the run. - fn pr_links(&mut self, _links: &[crate::report::session::PrLinkWire]) {} - /// The run finished; render the final summary. - fn summary(&mut self, rows: &[Row], objective: &str, best_score: f64); - /// The loop is exiting, called exactly once as the very last thing `run_loop` does before - /// returning (every exit path, including an early-return error). `outcome` is one of - /// `finished`/`solved`/`budget`/`stopped`/`escalated`/`error`; `reason` is a short - /// human-readable detail. The default renders a note; the session reporter overrides it to - /// emit a structured [`crate::report::session::SessionEvent::Shutdown`] as the log's final line, so a - /// consumer tailing it can tell "the run ended" from "the stream just went quiet". - fn shutdown(&mut self, outcome: &str, reason: &str) { - self.note(&format!("run ended: {outcome} ({reason})")); - } -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn over_cap_matches_the_loop_guard_threshold() { - let b = TurnBudget { - spent_before: 0.0, - started: Instant::now(), - max_cost: 0.0, - heartbeat: None, - }; - assert!(!b.over_cap(1e9), "0 means uncapped"); - let b = TurnBudget { max_cost: 5.0, ..b }; - assert!(!b.over_cap(4.99)); - assert!(b.over_cap(5.0), ">= like over_budget"); - } - - #[test] - fn provisional_spend_reaches_the_heartbeat() { - let heartbeat = Arc::new(crate::control::heartbeat::Heartbeat::new()); - let budget = TurnBudget { - spent_before: 1.0, - started: Instant::now(), - max_cost: 5.0, - heartbeat: Some(Arc::clone(&heartbeat)), - }; - - budget.record_provisional(2, 1.2345); - - assert!((heartbeat.spent_usd() - 1.235).abs() < 1e-9); - } -} diff --git a/crucible/src/report/reporter.rs b/crucible/src/report/reporter.rs new file mode 100644 index 00000000..0267d610 --- /dev/null +++ b/crucible/src/report/reporter.rs @@ -0,0 +1,263 @@ +//! The [`Reporter`] a scored loop talks to, and the turn, run and outcome context it passes. + +use crate::args::{Args, Paths}; +use crate::report::session::Row; +use crucible_contract::LoopPhase; +use std::sync::Arc; +use std::time::{Duration, Instant}; + +/// Cumulative-budget context for one agent turn. Cost is only authoritative at turn +/// end, so mid-turn the reporters price the streamed token samples (see +/// [`crate::agent::event::provisional_cost`]) and emit provisional budget updates; the +/// loop's turn-end budget call reconciles them. Without this a run could spend its +/// whole cap inside one turn before any guard fires. +#[derive(Clone)] +pub struct TurnBudget { + /// Run spend before this turn started (authoritative). + pub spent_before: f64, + /// When the run started, so provisional updates carry the run-relative clock. + pub started: Instant, + /// Effective cost cap resolved by the loop (live control override or the CLI + /// arg); 0 means uncapped. + pub max_cost: f64, + /// Live liveness telemetry for provisional, mid-turn spend. The authoritative + /// total still comes from the completed turn and overwrites this estimate. + pub(crate) heartbeat: Option>, +} + +impl TurnBudget { + /// Whether a provisional spend crosses the cap (same threshold as the loop's + /// between-iteration guard). + pub fn over_cap(&self, spent: f64) -> bool { + self.max_cost > 0.0 && spent >= self.max_cost + } + + /// Publish a provisional cumulative total while the agent is still running. + pub fn record_provisional(&self, iter: u32, spent: f64) { + if let Some(heartbeat) = &self.heartbeat { + heartbeat.record(iter, spent); + } + } +} + +/// One-line evidence rendering for the human-readable outputs (console rows, RESULTS.md, +/// PR bodies): `refcheck ✓ calc-diff ✓ tensor-pipe SKIPPED (reason)`. +pub(crate) fn evidence_line(evidence: &[crate::report::session::EvidenceEntry]) -> String { + evidence + .iter() + .map(ToString::to_string) + .collect::>() + .join(" ") +} + +/// Run-wide context every front-end needs to render the start banner. Built once +/// from [`Args`] so the NDJSON reporters don't each re-derive it. +#[derive(Clone)] +pub struct RunMeta { + pub namespace: String, + pub model: String, + pub iters_total: u32, + pub max_cost: f64, + pub max_secs: u64, +} + +impl RunMeta { + pub fn from_args(args: &Args) -> Self { + Self { + namespace: args.namespace.clone(), + model: args.model().to_string(), + iters_total: args.iterations, + max_cost: args.max_cost, + max_secs: args.max_time().map(|d| d.as_secs()).unwrap_or(0), + } + } +} + +/// What the run achieved, for the process exit code. +#[derive(Debug, Clone, Copy, Default)] +pub struct Outcome { + pub improved: bool, + pub solved: bool, + /// The agent declared the harness inadequate (harness-inadequate escalation): the run stopped for + /// human review, distinct from both success and a plain no-improvement. + pub escalated: bool, +} + +impl Outcome { + /// CI exit code. Escalation gets its own code (2) so the outer pod / CI can route a + /// "needs human" stop separately from success (0) and no-improvement (1). + pub fn exit_code(&self) -> i32 { + if self.escalated { + 2 + } else if self.improved || self.solved { + 0 + } else { + 1 + } + } +} + +/// What the operator decided at an interrupt checkpoint. +pub enum Stop { + Continue, + Quit, +} + +/// The outcome of one agent turn the loop needs beyond the streamed events: the +/// cost to budget on, and whether the CLI reported the turn as an error so the loop +/// can discard a no-op instead of measuring an unchanged workspace as a success. +#[derive(Debug, Clone, Default)] +pub struct AgentTurn { + /// The turn's cost (USD). + pub cost: f64, + /// The `result` event's `is_error` flag, a failed turn (e.g. a credential-less + /// "Not logged in" no-op) even when its subtype is "success". + pub is_error: bool, + /// The failure text the CLI reported (the synthetic message), for the discard note. + pub error: Option, +} + +/// Everything the loop needs from its front-end. The loop never prints directly. +pub trait Reporter { + /// The run is starting with this goal and objective label. + fn start(&mut self, goal: &str, objective: &str); + /// The loop moved to `phase` at iteration `iter` (0 before the first). + fn phase(&mut self, phase: LoopPhase, iter: u32); + /// A free-form progress note (setup steps, steer injection, ...). + fn note(&mut self, msg: &str); + /// A decided row (baseline / keep / discard); `solved` flags a winning fix. + fn row(&mut self, row: &Row, solved: bool); + /// Run the agent subprocess for `it`, streaming its output to the front-end. + /// Returns the turn's cost and its error verdict ([`AgentTurn`]) so the loop can + /// budget on it and discard a failed no-op turn. `budget` carries the run spend + /// so far and the cost cap: reporters stream provisional budget updates from + /// mid-turn token samples and may stop a local agent once the provisional spend + /// crosses the cap. + #[allow(clippy::too_many_arguments)] + fn run_agent( + &mut self, + args: &Args, + p: &Paths, + it: u32, + prompt: &str, + resume_prompt: Option<&str>, + session: Option<&str>, + budget: TurnBudget, + ) -> AgentTurn; + /// Cumulative budget after a turn; lets the front-end draw a gauge / warn. + fn budget(&mut self, _spent: f64, _elapsed: Duration) {} + /// Interrupt checkpoint: decide whether to stop. + fn check_interrupt(&mut self, p: &Paths, rows: &[Row]) -> Stop; + /// A (re-)scope boundary (a re-scope boundary): a fresh baseline + `fingerprint` begin a new comparable + /// segment (`regime` labels the new evaluation regime). The default renders a note; the + /// session reporter overrides it to emit a structured [`crate::report::session::SessionEvent::Segment`]. + fn segment(&mut self, fingerprint: &str, baseline_score: f64, regime: &str) { + self.note(&format!( + "segment [{regime}] fingerprint={fingerprint} baseline={baseline_score}" + )); + } + /// The run's comparability key, reported once at run start (and again on + /// `--resume`, the freshly recomputed value). The default renders a note; the session + /// reporter overrides it to emit a structured [`crate::report::session::SessionEvent::Identity`]. + fn identity(&mut self, identity: &crate::identity::RunIdentity) { + self.note(&format!( + "run identity {} ({} component(s), measure_cmd={})", + identity.digest, + identity.components.len(), + identity.measure_cmd + )); + } + /// The agent escalated (declared the harness inadequate), halting the run for human review. + /// The default renders it as a note; the session reporter overrides this to emit a structured + /// [`crate::report::session::SessionEvent::Escalation`]. + fn escalation(&mut self, esc: &crate::control::escalation::Escalation) { + let evidence = if esc.evidence.trim().is_empty() { + String::new() + } else { + format!(" — evidence: {}", esc.evidence.trim()) + }; + self.note(&format!( + "ESCALATION [{}]: {}{evidence}", + esc.category, esc.reason + )); + } + /// An additive work-graph wire line (`PlanAdmitted` / `TaskResult`) from a loop iteration's plan + /// iteration. Default no-op: only the session reporter persists them; the console + /// front-end has no plan rendering, and the legacy sequencing path never emits one. + fn plan_event(&mut self, _ev: &crate::report::session::SessionEvent) {} + /// How this resume classified the previous shutdown; the session reporter emits a + /// structured [`crate::report::session::SessionEvent::Recovery`]. + fn recovery(&mut self, class: crate::report::session::RecoveryClass, iter: u32, detail: &str) { + self.note(&format!("recovery: {class} (iter {iter}): {detail}")); + } + /// Opens the approval bracket a resume's classifier reads: a dangling wait means the + /// run died with the approval open. + fn approval_wait( + &mut self, + handle: &str, + trace_id: &str, + mode: crate::control::provisioning::WaitMode, + ) { + self.note(&format!( + "approval wait [{}] {handle} ({trace_id})", + mode.as_str() + )); + } + /// The wait reached a terminal outcome (`granted`/`denied`/`timeout`), closing the + /// bracket. Deliberately NOT called on a stop-while-parked: a stop doesn't resolve + /// the ask, and the still-open bracket makes a resume re-park on it. + fn approval_resolved(&mut self, outcome: &str, reason: &str) { + self.note(&format!("approval {outcome}: {reason}")); + } + /// The draft PR(s) publish-on-keep opened, reported once after publish so the durable session + /// log carries them (the controller's pull-ingest folds them onto the kept candidates' `pr_url`). + /// The default is a no-op, the console already prints each URL via `note`; only the session + /// reporter persists a structured [`crate::report::session::SessionEvent::PrLinks`] line. Best-effort: + /// an append failure must never fail the run. + fn pr_links(&mut self, _links: &[crate::report::session::PrLinkWire]) {} + /// The run finished; render the final summary. + fn summary(&mut self, rows: &[Row], objective: &str, best_score: f64); + /// The loop is exiting, called exactly once as the very last thing `run_loop` does before + /// returning (every exit path, including an early-return error). `outcome` is one of + /// `finished`/`solved`/`budget`/`stopped`/`escalated`/`error`; `reason` is a short + /// human-readable detail. The default renders a note; the session reporter overrides it to + /// emit a structured [`crate::report::session::SessionEvent::Shutdown`] as the log's final line, so a + /// consumer tailing it can tell "the run ended" from "the stream just went quiet". + fn shutdown(&mut self, outcome: &str, reason: &str) { + self.note(&format!("run ended: {outcome} ({reason})")); + } +} + +#[cfg(test)] +mod tests { + use crate::report::reporter::*; + + #[test] + fn over_cap_matches_the_loop_guard_threshold() { + let b = TurnBudget { + spent_before: 0.0, + started: Instant::now(), + max_cost: 0.0, + heartbeat: None, + }; + assert!(!b.over_cap(1e9), "0 means uncapped"); + let b = TurnBudget { max_cost: 5.0, ..b }; + assert!(!b.over_cap(4.99)); + assert!(b.over_cap(5.0), ">= like over_budget"); + } + + #[test] + fn provisional_spend_reaches_the_heartbeat() { + let heartbeat = Arc::new(crate::control::heartbeat::Heartbeat::new()); + let budget = TurnBudget { + spent_before: 1.0, + started: Instant::now(), + max_cost: 5.0, + heartbeat: Some(Arc::clone(&heartbeat)), + }; + + budget.record_provisional(2, 1.2345); + + assert!((heartbeat.spent_usd() - 1.235).abs() < 1e-9); + } +} diff --git a/crucible/src/report/session.rs b/crucible/src/report/session.rs index 8e4dc4bd..11362704 100644 --- a/crucible/src/report/session.rs +++ b/crucible/src/report/session.rs @@ -64,12 +64,14 @@ impl From<&Row> for RowWire { } } +#[cfg(feature = "autoresearch")] /// Bridges [`RowWire`] (the wire mirror) back to [`Row`] (in-process state). A trait rather than /// an inherent method since `RowWire` is defined in `crucible-contract`. pub trait IntoRow { fn into_row(self) -> Row; } +#[cfg(feature = "autoresearch")] impl IntoRow for RowWire { fn into_row(self) -> Row { Row { @@ -90,9 +92,9 @@ impl IntoRow for RowWire { } } -#[cfg(test)] +#[cfg(all(test, feature = "autoresearch"))] mod tests { - use super::*; + use crate::report::session::*; #[test] fn row_bridge_round_trips() { diff --git a/crucible/src/report/stream.rs b/crucible/src/report/stream.rs index cae111b7..8c834722 100644 --- a/crucible/src/report/stream.rs +++ b/crucible/src/report/stream.rs @@ -12,8 +12,8 @@ use crate::agent; use crate::agent::event::AgentEvent; use crate::args::{Args, Paths}; use crate::process::STOP; +use crate::report::reporter::{AgentTurn, Reporter, RunMeta, Stop, TurnBudget}; use crate::report::session::{self, Row, RowWire, SessionEvent}; -use crate::report::{AgentTurn, Reporter, RunMeta, Stop, TurnBudget}; use anyhow::{Context, Result}; use crucible_contract::LoopPhase; use std::fs::{File, OpenOptions}; @@ -256,7 +256,7 @@ impl Reporter for SessionReporter { return Stop::Quit; } if let Some(control) = &self.control - && crate::control::stop_file_says_stop(control) + && crate::control::bridge::stop_file_says_stop(control) { crate::process::pid_registry::kill_all(); return Stop::Quit; diff --git a/crucible/src/runloop/driver.rs b/crucible/src/runloop/driver.rs index 2b5c0190..2542fd70 100644 --- a/crucible/src/runloop/driver.rs +++ b/crucible/src/runloop/driver.rs @@ -10,9 +10,9 @@ use crate::args::{Args, Paths, Prepared}; use crate::control; use crate::control::provisioning; use crate::process::STOP; +use crate::report::reporter::{Outcome, Reporter, Stop}; use crate::report::session; use crate::report::session::Row; -use crate::report::{Outcome, Reporter, Stop}; use crate::runloop::publish; use anyhow::{Context, Result}; use crucible::crucible::{Judge, World}; @@ -40,7 +40,7 @@ struct BaselineInvalid { /// one argument so the core loop boundary stays small as front-ends evolve. #[derive(Default)] pub(crate) struct LoopRuntime { - pub control: Option>, + pub control: Option>, pub resume: Option, /// How this resume classified the previous shutdown; present only with `resume`. pub recovery: Option, @@ -1232,7 +1232,7 @@ fn restore_kept_best( } } -fn wait_if_paused(control: Option<&control::ControlState>, r: &mut R) { +fn wait_if_paused(control: Option<&control::bridge::ControlState>, r: &mut R) { let Some(control) = control else { return; }; @@ -1257,7 +1257,7 @@ fn beat_position(heartbeat: Option<&crate::control::heartbeat::Heartbeat>, iter: } fn update_control_progress( - control: Option<&control::ControlState>, + control: Option<&control::bridge::ControlState>, iter: u32, best_score: f64, spend: f64, @@ -1269,7 +1269,7 @@ fn update_control_progress( fn over_budget( args: &Args, - control: Option<&control::ControlState>, + control: Option<&control::bridge::ControlState>, spent: f64, started: Instant, parked_total: Duration, @@ -1315,7 +1315,7 @@ enum ParkOutcome { /// broker drives the approval+capture and sends the terminal `rescope`/`deny` over the control /// bridge. With no control bridge nothing could deliver an outcome, so we note and proceed. fn park_for_approval( - control: Option<&control::ControlState>, + control: Option<&control::bridge::ControlState>, ledger: Option<&crate::control::admission::AdmissionLedger>, r: &mut R, parked_total: &mut Duration, @@ -1595,7 +1595,7 @@ fn fingerprint(goal: &str, objective: &str, regime: &str) -> String { /// stops and un-granted approves are closed out. fn replay_admissions( ledger: &crate::control::admission::AdmissionLedger, - control: Option<&control::ControlState>, + control: Option<&control::bridge::ControlState>, replay: crate::control::admission::ResumeReplay, r: &mut R, ) { @@ -1683,7 +1683,7 @@ fn write_results(p: &Paths, goal: &str, prior: &str, rows: &[Row]) -> Result<()> detail.push(' '); } detail.push_str("evidence: "); - detail.push_str(&crate::report::evidence_line(&r.evidence)); + detail.push_str(&crate::report::reporter::evidence_line(&r.evidence)); } s.push_str(&format!( "| {} | {} | {} | {} |\n", @@ -1700,8 +1700,8 @@ fn write_results(p: &Paths, goal: &str, prior: &str, rows: &[Row]) -> Result<()> mod tests { use super::*; use crate::control::recovery::{ResumeFold, resume_finished}; + use crate::report::reporter::{AgentTurn, Stop, TurnBudget}; use crate::report::session::Row; - use crate::report::{AgentTurn, Stop, TurnBudget}; use crate::runloop::step::{TurnVerdict, drain_turn_markers}; use crucible_contract::LoopPhase; use crucible_contract::admission::AdmissionKey; @@ -2273,7 +2273,7 @@ mod tests { // The broker would deliver a rescope over the control bridge when the human approves; here // a thread plays that role. The park must wake, accrue the idle time, and leave the rescope // for the iteration-head drain to consume (the single re-baseline site). - let control = std::sync::Arc::new(control::ControlState::default()); + let control = std::sync::Arc::new(control::bridge::ControlState::default()); let deliver = control.clone(); let h = std::thread::spawn(move || { std::thread::sleep(Duration::from_millis(60)); @@ -2315,7 +2315,7 @@ mod tests { fn park_returns_denied_on_a_deny_signal() { // The broker (or an operator) rejects the ask; the park must wake with a Denied outcome so // the caller can escalate (block had no fallback). - let control = std::sync::Arc::new(control::ControlState::default()); + let control = std::sync::Arc::new(control::bridge::ControlState::default()); let deliver = control.clone(); let h = std::thread::spawn(move || { std::thread::sleep(Duration::from_millis(60)); @@ -2342,7 +2342,7 @@ mod tests { #[test] fn park_times_out_into_denied() { // No signal ever arrives; a short --max-park bounds the wait and resolves to Denied. - let control = std::sync::Arc::new(control::ControlState::default()); + let control = std::sync::Arc::new(control::bridge::ControlState::default()); let mut r = NoteCapture::default(); let mut parked = Duration::ZERO; let outcome = park_for_approval( @@ -3895,7 +3895,7 @@ mod tests { solved: false, fail_baseline: false, }); - let control = Arc::new(control::ControlState::default()); + let control = Arc::new(control::bridge::ControlState::default()); let r = RecordingReporter::default(); let recovery = crate::control::recovery::ResumeRecovery { class: crate::report::session::RecoveryClass::DiedBetweenIterations, @@ -4070,7 +4070,7 @@ mod tests { .admit(Some(AdmissionKey::new("s1")), AdmittedInput::Stop) .expect("stop"); - let control = Arc::new(control::ControlState::default()); + let control = Arc::new(control::bridge::ControlState::default()); let mut r = NoteCapture::default(); replay_admissions(&ledger, Some(&control), ledger.replay_for_resume(), &mut r); @@ -4237,7 +4237,7 @@ mod tests { &[r#"echo '{"pass":true}'"#], Some(r#"echo '{"score":88.0,"note":"seeded"}'"#), )); - let control = std::sync::Arc::new(control::ControlState::default()); + let control = std::sync::Arc::new(control::bridge::ControlState::default()); control.set_rescope(AdmissionKey::new("g1"), "concurrency=48".into()); let world = world_of(FakeWorld); let judge = judge_of(FakeJudge { diff --git a/crucible/src/runloop/graph.rs b/crucible/src/runloop/graph.rs index 5790ad0b..1ca1455b 100644 --- a/crucible/src/runloop/graph.rs +++ b/crucible/src/runloop/graph.rs @@ -25,11 +25,11 @@ use crate::plan::exec::{ use crate::plan::ir::{ EngineOp, Isolation, Join, Plan, PlanBudget, Stage, Task, TaskKind, TaskName, ValidPlan, }; -use crate::plan::workflow::{WorkflowCaps, WorkflowCfg, WorkflowType}; +use crate::plan::workflow::{WorkflowCaps, WorkflowCfg}; use crate::process::STOP; +use crate::report::reporter::{Reporter, TurnBudget}; use crate::report::session::Row; use crate::report::session::{EvidenceDisposition, EvidenceEntry}; -use crate::report::{Reporter, TurnBudget}; use crate::runloop::step::{Decided, IterStep, Measured, TurnVerdict}; use crucible::crucible::Direction; use crucible::crucible::{Judge, MeasureCtx, Reading, World}; @@ -88,7 +88,7 @@ pub(crate) fn run_iteration( if agent::supports_persistent_sessions(&runner.args) { caps = caps.with_persistent_sessions(); } - let plan = iteration_template(cx.workflow, &caps)?; + let plan = crate::plan::template::iteration_template(cx.workflow, &caps)?; let result_task = cx .workflow .filter(|workflow| !workflow.is_legacy_splice()) @@ -183,100 +183,6 @@ pub(crate) fn run_iteration( Ok((step, outcome.spent_usd)) } -/// Build and admit the default or authored iteration graph. -pub(crate) fn iteration_template( - workflow: Option<&WorkflowCfg>, - caps: &WorkflowCaps, -) -> Result { - if let Some(workflow) = workflow.filter(|workflow| !workflow.is_legacy_splice()) { - workflow - .admit(caps) - .context("admitting authored workflow into the autoresearch loop")?; - return Plan { - version: 1, - reason: None, - budget: PlanBudget { usd: f64::MAX }, - tasks: workflow.iteration_tasks(), - } - .validate() - .context("building authored iteration workflow"); - } - - let engine = - |name: &str, op: EngineOp, source: Option, deps: Vec| -> Task { - Task { - name: name.into(), - task: TaskKind::Engine { - op, - source, - tiebreak: None, - }, - depends_on: deps, - session: None, - needs: "any".to_string(), - required: true, - isolation: None, - join: Join::default(), - stage: Stage::Iteration, - emits: Vec::new(), - emits_files: Vec::new(), - over: None, - max_fanout: None, - when: None, - revise: None, - } - }; - let mut tasks = vec![engine("propose", EngineOp::Propose, None, vec![])]; - - // Legacy splice tasks run between `propose` and `apply`; `apply` waits on every sink. - // Epilogue tasks never splice: they run once post-loop, not per iteration. - let mut apply_deps = vec![TaskName("propose".to_string())]; - if let Some(w) = workflow.filter(|w| !w.tasks.is_empty()) { - for mut t in w.iteration_tasks() { - if t.depends_on.is_empty() { - t.depends_on = vec![TaskName("propose".to_string())]; - } - tasks.push(t); - } - let sinks = w.sinks(); - if !sinks.is_empty() { - apply_deps = sinks; - } - } - - tasks.push(engine("apply", EngineOp::Apply, None, apply_deps)); - tasks.push(engine( - "measure", - EngineOp::Measure, - None, - vec![TaskName("apply".to_string())], - )); - tasks.push(engine( - "decide", - EngineOp::Decide, - Some(TaskName("measure".to_string())), - vec![TaskName("measure".to_string())], - )); - let workflow = WorkflowCfg { - workflow_type: WorkflowType::Autoresearch, - result: Some("decide".into()), - tasks, - file: None, - resolved_from: None, - }; - workflow - .admit(caps) - .context("admitting the default autoresearch workflow")?; - Plan { - version: 1, - reason: None, - budget: PlanBudget { usd: f64::MAX }, - tasks: workflow.tasks, - } - .validate() - .context("building the iteration template") -} - /// Build the workflow's run-scoped epilogue subgraph; `None` when it declares none. pub(crate) fn epilogue_template(workflow: &WorkflowCfg) -> Result> { let tasks = workflow.epilogue_tasks(); @@ -482,7 +388,7 @@ pub(crate) struct LoopTaskRunner { p: Paths, world: Arc, judge: Arc, - control: Option>, + control: Option>, heartbeat: Option>, pub(crate) started: Instant, pub(crate) r: R, @@ -524,7 +430,7 @@ impl LoopTaskRunner { r: R, world: Arc, judge: Arc, - control: Option>, + control: Option>, heartbeat: Option>, ) -> Self { Self { @@ -1523,41 +1429,6 @@ mod tests { ); } - #[test] - fn pack_tasks_splice_between_the_turn_and_the_gate() { - let w: WorkflowCfg = toml::from_str( - "[[task]]\nname = \"review\"\nkind = \"command\"\ncommand = \"true\"\n [[task]]\nname = \"lint\"\nkind = \"command\"\ncommand = \"true\"\ndepends_on = [\"review\"]\n", - ) - .unwrap(); - w.validate().unwrap(); - let plan = iteration_template(Some(&w), &WorkflowCaps::autoresearch_engine()).unwrap(); - let names: Vec<&str> = plan.tasks_topo().map(|t| t.name.0.as_str()).collect(); - assert_eq!( - names, - ["propose", "review", "lint", "apply", "measure", "decide"] - ); - let dep = |n: &str| { - plan.get(&n.into()) - .unwrap() - .depends_on - .iter() - .map(|d| d.0.clone()) - .collect::>() - }; - assert_eq!( - dep("review"), - ["propose"], - "an unattached task hangs off propose" - ); - assert_eq!( - dep("apply"), - ["lint"], - "apply waits on the sink, not on propose" - ); - assert_eq!(dep("measure"), ["apply"]); - assert_eq!(dep("decide"), ["measure"]); - } - /// Epilogue tasks stay out of the per-iteration plan entirely (legacy splice and /// fully-authored form) and land in their own post-loop template. #[test] @@ -1568,7 +1439,11 @@ mod tests { ) .unwrap(); w.validate().unwrap(); - let plan = iteration_template(Some(&w), &WorkflowCaps::autoresearch_engine()).unwrap(); + let plan = crate::plan::template::iteration_template( + Some(&w), + &WorkflowCaps::autoresearch_engine(), + ) + .unwrap(); let names: Vec<&str> = plan.tasks_topo().map(|t| t.name.0.as_str()).collect(); assert_eq!(names, ["propose", "review", "apply", "measure", "decide"]); assert_eq!( @@ -1590,8 +1465,11 @@ mod tests { [[task]]\nname = \"racecheck\"\nkind = \"command\"\ncommand = \"true\"\nstage = \"epilogue\"\n", ) .unwrap(); - let plan = - iteration_template(Some(&authored), &WorkflowCaps::autoresearch_engine()).unwrap(); + let plan = crate::plan::template::iteration_template( + Some(&authored), + &WorkflowCaps::autoresearch_engine(), + ) + .unwrap(); assert!( plan.get(&"racecheck".into()).is_none(), "authored iteration plan must not carry the epilogue task" @@ -1647,52 +1525,6 @@ mod tests { assert_eq!(trace.shutdown, "finished", "a rejection is not a run error"); } - #[test] - fn template_is_the_canonical_chain() { - let plan = iteration_template(None, &WorkflowCaps::autoresearch_engine()).unwrap(); - let names: Vec<&str> = plan.tasks_topo().map(|t| t.name.0.as_str()).collect(); - assert_eq!(names, ["propose", "apply", "measure", "decide"]); - assert!(plan.tasks_topo().all(|t| t.required)); - let kinds: Vec<&str> = plan.tasks_topo().map(|t| t.task.label()).collect(); - assert_eq!( - kinds, - [ - "engine_propose", - "engine_apply", - "engine_measure", - "engine_decide" - ] - ); - } - - #[test] - fn authored_autoresearch_uses_semantics_instead_of_reserved_names() { - let workflow: WorkflowCfg = toml::from_str( - "type = \"autoresearch\"\nresult = \"keep-if-better\"\n\ - [[task]]\nname = \"invent\"\nkind = \"engine\"\nop = \"propose\"\n\ - [[task]]\nname = \"review\"\nkind = \"command\"\ncommand = \"true\"\ndepends_on = [\"invent\"]\n\ - [[task]]\nname = \"deploy-preview\"\nkind = \"engine\"\nop = \"apply\"\ndepends_on = [\"review\"]\n\ - [[task]]\nname = \"benchmark-a\"\nkind = \"engine\"\nop = \"measure\"\ndepends_on = [\"deploy-preview\"]\n\ - [[task]]\nname = \"explain-score\"\nkind = \"command\"\ncommand = \"true\"\ndepends_on = [\"benchmark-a\"]\n\ - [[task]]\nname = \"keep-if-better\"\nkind = \"engine\"\nop = \"decide\"\nsource = \"benchmark-a\"\ndepends_on = [\"benchmark-a\", \"explain-score\"]\n", - ) - .unwrap(); - let plan = - iteration_template(Some(&workflow), &WorkflowCaps::autoresearch_engine()).unwrap(); - let names: Vec<&str> = plan.tasks_topo().map(|task| task.name.0.as_str()).collect(); - assert_eq!( - names, - [ - "invent", - "review", - "deploy-preview", - "benchmark-a", - "explain-score", - "keep-if-better" - ] - ); - } - #[test] fn authored_measurement_subgraph_drives_real_decisions() { let workflow = r#" @@ -2066,7 +1898,8 @@ mod tests { }; let world = m.build_world(workspace.clone()); let judge = m.build_judge(workspace.clone(), Vec::new()).unwrap(); - let r = stream::SessionReporter::stream(&p, report::RunMeta::from_args(&args)).unwrap(); + let r = stream::SessionReporter::stream(&p, report::reporter::RunMeta::from_args(&args)) + .unwrap(); let (_r, outcome) = run_loop(&args, &p, &prep, r, &world, &judge, LoopRuntime::default()); let outcome = outcome.unwrap(); if workflow.is_none() { diff --git a/crucible/src/runloop/machine.rs b/crucible/src/runloop/machine.rs index 9c13e8e8..aa9b32e3 100644 --- a/crucible/src/runloop/machine.rs +++ b/crucible/src/runloop/machine.rs @@ -6,8 +6,8 @@ //! which event moves it, and how a run ends. A transition the table does not list is a bug in //! the driver, reported as [`IllegalTransition`] rather than silently taken. -use crate::control::ControlState; -use crate::report::Reporter; +use crate::control::bridge::ControlState; +use crate::report::reporter::Reporter; use crucible::diagram::{self, Cluster, Cursor, Digraph, Edge, IllegalTransition, Node, NodeKind}; use crucible_contract::LoopPhase; use std::sync::Arc; @@ -442,16 +442,16 @@ mod tests { _: &str, _: Option<&str>, _: Option<&str>, - _: crate::report::TurnBudget, - ) -> crate::report::AgentTurn { - crate::report::AgentTurn::default() + _: crate::report::reporter::TurnBudget, + ) -> crate::report::reporter::AgentTurn { + crate::report::reporter::AgentTurn::default() } fn check_interrupt( &mut self, _: &crate::args::Paths, _: &[crate::report::session::Row], - ) -> crate::report::Stop { - crate::report::Stop::Continue + ) -> crate::report::reporter::Stop { + crate::report::reporter::Stop::Continue } fn summary(&mut self, _: &[crate::report::session::Row], _: &str, _: f64) {} } diff --git a/crucible/src/runloop/mod.rs b/crucible/src/runloop/mod.rs index 905c89e3..b407efc5 100644 --- a/crucible/src/runloop/mod.rs +++ b/crucible/src/runloop/mod.rs @@ -5,5 +5,4 @@ pub(crate) mod graph; pub(crate) mod machine; pub(crate) mod preflight; pub(crate) mod publish; -pub(crate) mod selftest; pub(crate) mod step; diff --git a/crucible/src/runloop/preflight.rs b/crucible/src/runloop/preflight.rs index 9d7fbb8a..7251860f 100644 --- a/crucible/src/runloop/preflight.rs +++ b/crucible/src/runloop/preflight.rs @@ -16,7 +16,7 @@ use serde_json::Value; use crate::manifest::{MODE_PLACEHOLDER, PreflightCfg}; use crate::process::STOP; -use crate::report::Reporter; +use crate::report::reporter::Reporter; /// The `{digest}` placeholder, filled from the most recent `digest` a rung emitted. const DIGEST_PLACEHOLDER: &str = "{digest}"; @@ -263,8 +263,8 @@ fn stderr_tail(stderr: &str) -> String { mod tests { use super::*; use crate::args::{Args, Paths}; + use crate::report::reporter::{AgentTurn, Stop, TurnBudget}; use crate::report::session::Row; - use crate::report::{AgentTurn, Stop, TurnBudget}; /// Collects notes; every other `Reporter` call is inert. Preflight only ever notes. #[derive(Default)] diff --git a/crucible/src/runloop/publish.rs b/crucible/src/runloop/publish.rs index aa7b46db..8958db64 100644 --- a/crucible/src/runloop/publish.rs +++ b/crucible/src/runloop/publish.rs @@ -17,8 +17,9 @@ use crate::args::{Args, Paths}; use crate::flow::model::{finite, goal_line}; +use crate::object_store::{Backend, ObjectUriError, backend, parse_s3_uri}; use crate::outputs::OutputTally; -use crate::report::Reporter; +use crate::report::reporter::Reporter; use crate::report::session::Row; use anyhow::{Context, Result}; use crucible_contract::outputs::{BoundViolation, OutputKind}; @@ -409,28 +410,6 @@ fn keys(base: &str, goal_slug: &str, run_id: &str) -> Keys { } } -/// A publish destination: S3 (`s3://bucket[/prefix]`) or a mounted filesystem -/// (`file:///abs/path`, e.g. an artifacts PVC on a cluster with no S3 reach). Both write the -/// exact same key layout, so reporting tools walk either. -enum Backend { - // The S3 half re-parses the URI where it's used (the async block owns bucket/base), so the - // variant carries nothing. - S3, - File { root: std::path::PathBuf }, -} - -fn backend(uri: &str) -> Result { - match uri.strip_prefix("file://") { - Some(path) if path.starts_with('/') => Ok(Backend::File { - root: std::path::PathBuf::from(path), - }), - Some(_) => Err(PublishError::FileRootRelative { - uri: uri.to_owned(), - }), - None => parse_s3_uri(uri).map(|_| Backend::S3), - } -} - /// Write the run record to whichever backend the results URI names. fn record(rec: &Record<'_>, prs: &[PrLink]) -> Result { match backend(&rec.args.results_bucket)? { @@ -633,47 +612,6 @@ fn s3_record(rec: &Record<'_>, prs: &[PrLink]) -> Result { Ok(record_uri) } -/// Download one published object at an exact `s3://bucket/key` URI to a local file, the general -/// GetObject the controller's artifact proxy shells (`crucible fetch`), keeping every S3 client out -/// of `crucible-controller` (that crate has no aws-sdk and never learns the bucket layout). Nothing -/// is appended to the URI: the caller passes the exact key it wants. Reuses the same IRSA creds the -/// publisher uses (GetObject is in the role's policy). -pub fn fetch_object(uri: &str, dest: &std::path::Path) -> Result<()> { - if let Backend::File { root } = backend(uri)? { - // The file URI IS the object path; a plain copy is the whole fetch. - std::fs::copy(&root, dest) - .with_context(|| format!("copying {} to {}", root.display(), dest.display()))?; - return Ok(()); - } - let (bucket, key) = parse_s3_uri(uri)?; - if key.is_empty() { - return Err(PublishError::NoKey { - uri: uri.to_owned(), - } - .into()); - } - crate::agent::engine::handle()?.block_on(async { - let conf = aws_config::load_defaults(aws_config::BehaviorVersion::latest()).await; - let client = aws_sdk_s3::Client::new(&conf); - let out = client - .get_object() - .bucket(&bucket) - .key(&key) - .send() - .await - .with_context(|| format!("GetObject s3://{bucket}/{key}"))?; - let data = out - .body - .collect() - .await - .context("read object body")? - .into_bytes(); - std::fs::write(dest, &data) - .with_context(|| format!("writing {} ({} bytes)", dest.display(), data.len()))?; - Ok::<(), anyhow::Error>(()) - }) -} - /// Cross-run memory: fetch the PREVIOUS run's tried-ideas ledger for this goal from S3, so a fresh /// run inherits what's already been tried instead of re-walking dead ends. Reads the per-goal /// `latest.json` pointer, then that run's `RESULTS.md`, and returns its candidate rows (baseline @@ -769,14 +707,8 @@ async fn put( /// publishing refuses on its own terms; everything else is plumbing and stays `anyhow`. #[derive(Debug, thiserror::Error, PartialEq)] pub enum PublishError { - #[error("results bucket must be an s3:// or file:// URI, got `{uri}`")] - NotAnS3Uri { uri: String }, - #[error("file:// results root must be absolute: `{uri}`")] - FileRootRelative { uri: String }, - #[error("results bucket URI has no bucket: `{uri}`")] - NoBucket { uri: String }, - #[error("fetch object URI has no key: `{uri}`")] - NoKey { uri: String }, + #[error(transparent)] + Uri(#[from] ObjectUriError), #[error("gh pr edit failed: {stderr}")] PrEditFailed { stderr: String }, #[error("gh pr create failed: {stderr}")] @@ -790,23 +722,6 @@ pub enum PublishError { }, } -/// `s3://bucket[/prefix]` → (bucket, prefix). Prefix is trimmed of slashes and may -/// be empty. -fn parse_s3_uri(uri: &str) -> Result<(String, String), PublishError> { - let rest = uri - .strip_prefix("s3://") - .ok_or_else(|| PublishError::NotAnS3Uri { - uri: uri.to_owned(), - })?; - let (bucket, prefix) = rest.split_once('/').unwrap_or((rest, "")); - if bucket.is_empty() { - return Err(PublishError::NoBucket { - uri: uri.to_owned(), - }); - } - Ok((bucket.to_string(), prefix.trim_matches('/').to_string())) -} - // --- git PR channel -------------------------------------------------------- /// The mediation point a refused draft PR is recorded against on the session log. @@ -1458,7 +1373,7 @@ fn kept_section(row: &Row) -> String { if !row.evidence.is_empty() { s.push_str(&format!( "\nDeclared checks: {}\n", - crate::report::evidence_line(&row.evidence) + crate::report::reporter::evidence_line(&row.evidence) )); } s.push('\n'); @@ -1903,7 +1818,7 @@ mod tests { // fetch_object: the file URI is the object path. let dest = base.join("fetched.txt"); - fetch_object( + crate::object_store::fetch_object( &format!( "file://{}", run.join("artifacts/codegen-out/summary.txt").display() @@ -2560,24 +2475,6 @@ mod tests { assert_eq!(rows.lines().count(), 2); } - #[test] - fn parse_s3_uri_splits_bucket_and_prefix() { - assert_eq!( - parse_s3_uri("s3://my-bucket/autoresearch").unwrap(), - ("my-bucket".into(), "autoresearch".into()) - ); - assert_eq!( - parse_s3_uri("s3://my-bucket").unwrap(), - ("my-bucket".into(), String::new()) - ); - assert_eq!( - parse_s3_uri("s3://my-bucket/a/b/").unwrap(), - ("my-bucket".into(), "a/b".into()) - ); - assert!(parse_s3_uri("https://nope").is_err()); - assert!(parse_s3_uri("s3:///just-prefix").is_err()); - } - /// A run bounded to `count` draft PRs against `repo`, and nothing else. fn draft_pr_bounds(count: u32, repo: &str) -> crate::outputs::RunBounds { use crucible_contract::outputs::{ diff --git a/crucible/src/runloop/step.rs b/crucible/src/runloop/step.rs index 82efa61a..b4c874c4 100644 --- a/crucible/src/runloop/step.rs +++ b/crucible/src/runloop/step.rs @@ -5,8 +5,8 @@ use crate::args::{Args, Paths}; use crate::control; use crate::control::escalation; use crate::control::provisioning; +use crate::report::reporter::{AgentTurn, Reporter, Stop}; use crate::report::session::Row; -use crate::report::{AgentTurn, Reporter, Stop}; use anyhow::Result; use crucible::crucible::{Judge, World}; @@ -114,7 +114,7 @@ fn is_transport_turn_error(why: &str) -> bool { pub(crate) fn drain_turn_markers( r: &mut R, p: &Paths, - control: Option<&control::ControlState>, + control: Option<&control::bridge::ControlState>, it: u32, turn: &AgentTurn, rows: &[Row], @@ -275,9 +275,9 @@ pub(crate) fn decide_row( /// True when a cost/time cap is set and reached; notes it on `r`. `parked_total` is idle time /// spent waiting on a human approval, excluded from the wall-clock the time cap measures. /// The effective cost cap: a live control override wins over the CLI arg. -pub(crate) fn live_max_cost(args: &Args, control: Option<&control::ControlState>) -> f64 { +pub(crate) fn live_max_cost(args: &Args, control: Option<&control::bridge::ControlState>) -> f64 { control - .and_then(control::ControlState::live_max_cost) + .and_then(control::bridge::ControlState::live_max_cost) .unwrap_or(args.max_cost) } diff --git a/crucible/src/scope/pipeline.rs b/crucible/src/scope/pipeline.rs index 6b163895..94bd5ccc 100644 --- a/crucible/src/scope/pipeline.rs +++ b/crucible/src/scope/pipeline.rs @@ -1058,7 +1058,7 @@ fn render_workflow_preview(manifest_path: &Path, pack: &Path) -> Result<(u32, u3 } .validate()? } - workflow => crate::runloop::graph::iteration_template(workflow, &workflow_caps)?, + workflow => crate::plan::template::iteration_template(workflow, &workflow_caps)?, }; // Preview authored capabilities; execution still admits against the real substrate. let caps = plan diff --git a/crucible/src/scope/refine.rs b/crucible/src/scope/refine.rs index c50570d1..f489b287 100644 --- a/crucible/src/scope/refine.rs +++ b/crucible/src/scope/refine.rs @@ -9,11 +9,7 @@ //! orchestration and this stays trivially unit-testable. The records themselves are //! [`crucible_contract::refine`], which a controller depends on directly to read a frozen trail. -use crate::runloop::selftest::SelftestReport; -use crucible_contract::refine::{ - Attack, ControlEvidence, FailureEvidence, ReadingEvidence, RoundOutcome, RoundRecord, - SelftestEvidence, -}; +use crucible_contract::refine::{Attack, FailureEvidence, RoundOutcome, RoundRecord}; use serde::{Deserialize, Serialize}; /// The engine-embedded refine prompt: seeded from `scope-propose.md`'s contract sections, focused @@ -110,39 +106,6 @@ pub fn render_adversary_prompt( .replace("{{OUT_DIR}}", &out_dir.display().to_string()) .replace("{{TRAIL}}", &trail) } -impl From<&SelftestReport> for SelftestEvidence { - fn from(r: &SelftestReport) -> Self { - let direction = match r.direction { - crucible::crucible::Direction::Higher => "higher", - crucible::crucible::Direction::Lower => "lower", - } - .to_string(); - SelftestEvidence { - direction, - runs: r.runs, - good: control_evidence(&r.good), - bad: control_evidence(&r.bad), - } - } -} - -fn control_evidence(c: &crate::runloop::selftest::ControlResult) -> ControlEvidence { - ControlEvidence { - cmd: c.cmd.clone(), - mean: c.mean_score, - all_valid: c.all_valid, - readings: c - .readings - .iter() - .map(|r| ReadingEvidence { - valid: r.valid, - score: r.score, - note: r.note.clone(), - }) - .collect(), - } -} - /// Render the refine prompt for `round`: the goal, the pack's on-disk location (the agent edits in /// place), the concrete failure evidence from the prior round, the round number, and the /// confirmed tier so a refine turn doesn't quietly slide a T1 harness back toward a diff --git a/docs/getting-started.md b/docs/getting-started.md index 1684af59..541ef381 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -27,7 +27,7 @@ build from source: ```bash git clone https://github.com/neuralmagic/crucible.git cd crucible -cargo build --release -p crucible +cargo build --release -p crucible --features autoresearch install -m 755 target/release/crucible ~/.local/bin/ # or anywhere on PATH ``` @@ -53,7 +53,7 @@ memory) with no cluster and no model. ```bash crucible --manifest examples/counter/crucible.toml --iterations 6 -# or, from a source checkout: cargo run -p crucible -- --manifest examples/counter/crucible.toml --iterations 6 +# or, from a source checkout: cargo run -p crucible --features autoresearch -- --manifest examples/counter/crucible.toml --iterations 6 ``` The manifest is the whole story (`examples/counter/crucible.toml`): diff --git a/justfile b/justfile index 91ae3567..f51f07e1 100644 --- a/justfile +++ b/justfile @@ -32,9 +32,9 @@ install-tools: for f in tools/*.nu; do [ -e "$f" ] && ln -sf "$PWD/$f" "{{cargo_bin}}/$(basename "$f" .nu)"; done @echo "linked tools -> {{cargo_bin}}" -# Build the whole Rust workspace. +# Build the whole Rust workspace, the scored loop included. build-loop: - cargo build --release + cargo build --release --features crucible/autoresearch # Score the agent-stream decoder (examples/selfhost's gate): ns/line over the synthetic corpus. bench-stream: @@ -42,7 +42,7 @@ bench-stream: # Lint + test the Rust workspace. lint: - cargo fmt --check && cargo clippy --workspace --all-targets && cargo test --workspace + cargo fmt --check && cargo clippy --workspace --all-targets --all-features && cargo clippy -p crucible --all-targets && cargo test --workspace --all-features && cargo test -p crucible # Module dependency graph of one crate (crucible by default; `--root crucible-controller/src` # for the controller): cycles, fan-in/out, duplicate item names. `just modgraph --check` fails diff --git a/scripts/state-docs.sh b/scripts/state-docs.sh index c435e487..f74d9092 100755 --- a/scripts/state-docs.sh +++ b/scripts/state-docs.sh @@ -9,7 +9,7 @@ set -euo pipefail root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" -crucible=(cargo run --quiet --manifest-path "$root/Cargo.toml" -p crucible --) +crucible=(cargo run --quiet --manifest-path "$root/Cargo.toml" -p crucible --features autoresearch --) # page-or-dot path, then the command that produces it. outputs=(