From 022ad5e2710dc2919b1cc04b4428a786217a4cb6 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Thu, 17 Sep 2026 00:19:45 +0000 Subject: [PATCH 01/32] feat(record): give a verb-written family a declaration, so a module can read it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `batten record named ` wrote a branch-keyed store and no policy module could read one. `recorder_records` builds the set of families it projects into `input.tree.records` as the declared `[[recorder]]` rows unioned with `record::VERB_WRITTEN`, and a caller-named family is in neither — nor can it join either. `VERB_WRITTEN` is fixed because the engine owns both halves of those three stores, and a consumer's family cannot enter it without this crate knowing a consumer's name (rule 1). A `[[recorder]]` requires `tool`, because that table selects on a mediated tool call, and a family a `mise` task writes answers to no tool call at all. So the store was written, the row registered, the module reading `input.tree.records[""]` — and the key absent, every rule beneath it undefined, the gate green. Measured over `branch-age`: a record naming a 36-day branch against a two-day threshold, `batten check` exit 0. CLOUD-1707's dead gate, one surface over, and the third time this campaign has met an absent input that is byte-identical to a real negative. `[[record]]` is the missing half. Config rather than a column on the rule that reads it, on the reasoning `rules.rs` already states at that call site: the fact is what THIS repository's producers accumulated, so a per-rule declaration would be a second home for one answer. DECLARED RATHER THAN SWEPT, and that is the design rather than a detail. The projection could read whatever files sit in the store directory; it must not, because then a leftover file from a retired producer answers as a live measurement and nothing names what SHOULD be there. The declaration is also what keeps could-not-look readable: an absent record under a declared family is "the producer did not run", where the same absence with nothing declared is not a reading at all. `writer` is never executed. Section 5 keeps the spawn outside `check`, so it is a pointer — but it makes a declared family always name who fills it, and a store with no producer is a row that could only ever answer could-not-look. AUTHORITY-ONLY, and the direction is why: declaring a family arms a gate, so the edit worth refusing is the REMOVAL that disarms one. `resolve` reads the table from the committed authority alone and the key is absent from `OverrideConfig`, which makes that removal unwritable in the layer an author controls at PR time. `trust.rs` records it as not-policy-bearing by unreachability, `board`'s own shape. THREE CASES, over the compiled binary, because two of them would pass on a fix that is not one. A declared family with a record decides; a declared family with no record says nothing — a projection reading the DECLARATION rather than the store would report a finding there; an undeclared family stays unprojected whatever its store holds — the sweep, as an exit code. Both mutations sit on the chain they mutate: dropping the declared names restores the defect, and replacing them with a hardcoded name is the sweep, which only the third case sees. Measured after: the same record, the same rule, `branch watch stale` and `branch name duplicate` on `claude/ancient`, exit 2. Refs: CLOUD-1810 Admits: 53414e5287cdb5b4c82bda91f7a939f3b1e2168a9f33ea61b8c47de603c94b15 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: batten.toml Admits-anchor: call:605e5d4e96fbf80abb307344ccaf7a98327291d4 Admits-epoch: 8bdb07bbd131b8390f0955979b243cfc4d4c549a644c801c00800ce5767f7ec7 Admits-author: alec@wenzowski.com Admits-prev: e6b87d87b6ee1d840fa55f42244ec486f88db6e13db9ea803ddbda5a31bcb4f5 Admits-answer-lost: The projection this change adds carries nothing: `policy/branch-age.rego` is registered and still decides nothing, because `input.tree.records["branch-age"]` needs the family DECLARED before `recorder_records` will read its store. Without the row the engine change is unexercised and the nine CLOUD-1707-group ports stay blocked on a fix that is already written. Admits-answer-precondition: The `[[record]]` table is the declaration surface CLOUD-1810 adds, and a family is declared in `batten.toml` and nowhere else — `resolve` reads the table from the committed authority alone, deliberately, so that no local layer can remove a declaration and disarm the rule reading that store. There is no second file the row can live in. It lands in the same diff as the engine change that reads it, where a reviewer sees both halves together. Admits-answer-rejected-route: config read first — I read `[[recorder]]`'s rows and the `board`/`epoch` authority-only precedent to settle both the table's shape and where it resolves from, and that reading is what produced the row rather than what avoided it: a declaration is an addition and cannot be satisfied by reading. patch run first is a commit-message route and does not apply to a config addition. Admits: ff70d7405fd339ebe4ddd020a29e65f4cd18ecdc8dc3731c955230ddcf454500 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: mise.toml Admits-anchor: call:605e5d4e96fbf80abb307344ccaf7a98327291d4 Admits-epoch: ba0d29a9bb89a9544da54e66e06e4e190c8cbba7ee9caac53c1c88f116b2065e Admits-author: alec@wenzowski.com Admits-prev: c7f928e56d30b0a479909e63ac35beb33a7d551917beffe77e27ceb429308214 Admits-answer-lost: `mutate census` refuses with `crates/batten/src/rules.rs uncovered`, so the branch cannot land — and the thing it refuses is the two mutations that prove CLOUD-1810's fix discriminates. Without the entry the choice is to drop the mutation rows, which would ship the projection covered by nothing stronger than "its suite is green", the exact insufficiency CLOUD-418 measured four times. Admits-answer-precondition: `$MUTANT_GATES` is a single quoted string on one line of `mise.toml` and exists on no other surface — `mutate census` reads it to decide which gates are mutation-enforced, and a file carrying a `#MUTANT` row whose name is absent from the list is reported uncovered. `crates/batten/src/rules.rs` became a mutation subject in this delta, so `engine-rules` has to join the list in the same commit as the rows that make it one. The addition is one token a reviewer reads beside the rows it covers. Admits-answer-rejected-route: config read first — I read the list and the `engine-land`/`engine-hook`/`engine-policy` entries to establish both that the convention is `engine-` and that the list is the only surface, and that reading is what produced the token rather than what avoided it: a name absent from a list cannot be added by reading it. patch run first is a commit-message route and does not apply to a manifest edit. --- batten.toml | 25 ++++ crates/batten/src/config.rs | 33 +++++ crates/batten/src/hook.rs | 2 + crates/batten/src/lib.rs | 5 + crates/batten/src/policy.rs | 12 ++ crates/batten/src/record.rs | 97 +++++++++++++ crates/batten/src/resolve.rs | 14 ++ crates/batten/src/rules.rs | 30 +++- crates/batten/src/trust.rs | 14 ++ crates/batten/src/verdict.rs | 20 +++ crates/batten/tests/it/admission_narrowing.rs | 2 + crates/batten/tests/it/agentic_record.rs | 1 + crates/batten/tests/it/bats_invocation.rs | 1 + crates/batten/tests/it/cfg_gated_test.rs | 1 + crates/batten/tests/it/ci_cache_declared.rs | 1 + crates/batten/tests/it/ci_hygiene.rs | 1 + crates/batten/tests/it/ci_parity.rs | 2 + crates/batten/tests/it/ci_suite_lane.rs | 1 + crates/batten/tests/it/claim_order.rs | 1 + crates/batten/tests/it/config_fault_class.rs | 9 ++ crates/batten/tests/it/document_read_count.rs | 1 + crates/batten/tests/it/filed_here.rs | 1 + crates/batten/tests/it/fixture_forks.rs | 1 + crates/batten/tests/it/frontmatter_gates.rs | 1 + crates/batten/tests/it/hk_fix_selection.rs | 1 + crates/batten/tests/it/identity_churn.rs | 1 + crates/batten/tests/it/landing_roster.rs | 1 + crates/batten/tests/it/mise_preset.rs | 1 + .../batten/tests/it/mutation_declared_case.rs | 1 + crates/batten/tests/it/nextest_slow.rs | 1 + crates/batten/tests/it/obligations_bound.rs | 1 + crates/batten/tests/it/plan_complete.rs | 1 + crates/batten/tests/it/policy_test_suite.rs | 1 + crates/batten/tests/it/policy_tree.rs | 1 + crates/batten/tests/it/prebuilt_lint.rs | 1 + crates/batten/tests/it/prose_only.rs | 2 + crates/batten/tests/it/record_families.rs | 134 ++++++++++++++++++ .../tests/it/release_provision_parity.rs | 1 + crates/batten/tests/it/remedy_authorship.rs | 1 + crates/batten/tests/it/review_dispatched.rs | 3 + crates/batten/tests/it/rule_cost_census.rs | 1 + crates/batten/tests/it/shell_retirement.rs | 1 + crates/batten/tests/it/spawn_widening.rs | 1 + crates/batten/tests/it/suite_subjects.rs | 1 + crates/batten/tests/it/test_targets.rs | 1 + crates/batten/tests/it/verdict_registry.rs | 2 + crates/batten/tests/policy_modules.rs | 2 + hk.pkl | 1 + mise.toml | 2 +- schema/batten.schema.json | 26 ++++ 50 files changed, 462 insertions(+), 4 deletions(-) diff --git a/batten.toml b/batten.toml index 0b0921171..dff02ec09 100644 --- a/batten.toml +++ b/batten.toml @@ -2109,6 +2109,31 @@ regex = '(?i)(^|[^0-9A-Za-z-])(clos(e|es|ed)|fix(|es|ed)|resolv(e|es|ed))[[:blan id = "whole-number" regex = '^[0-9]+$' +# THE VERB-WRITTEN RECORD FAMILIES THIS REPOSITORY FILLS (CLOUD-1810). +# +# The sibling of `[[recorder]]` across one split: a recorder row is filled from a +# mediated tool call, and one of these is filled by a producer calling `batten +# record named `. Both project into `input.tree.records`; until this table +# existed only the first could, so a store a `mise` task wrote was invisible to +# every module and the row reading it reported clean over a record that said +# otherwise. +# +# DECLARED RATHER THAN SWEPT. The engine could have projected whatever files sit +# in the store directory and does not, because then a leftover file from a retired +# producer answers as a live measurement and nothing names what SHOULD be there. +# The declaration is also what keeps could-not-look readable: an absent record +# under a declared family is "the producer did not run", where the same absence +# with nothing declared is not a reading at all. +# +# `writer` IS NEVER EXECUTED, and that is not a gap. Section 5 keeps the spawn +# outside `check`, so this is a pointer — the job `[[verdict.route]]`'s `target` +# already does. What it buys is that a declared family always says who fills it, +# because a store with no producer is a row that can only ever read +# could-not-look. +[[record]] +record = "branch-age" +writer = "mise run branch-age-record" + [[pattern]] id = "closed-issue-status" regex = '^(done|canceled|duplicate)$' diff --git a/crates/batten/src/config.rs b/crates/batten/src/config.rs index 9652720df..edd2ad56b 100644 --- a/crates/batten/src/config.rs +++ b/crates/batten/src/config.rs @@ -357,6 +357,27 @@ pub struct Config { /// [`Config::mints`]' — same false claim, same reason.) #[serde(default, rename = "recorder", skip_serializing_if = "Vec::is_empty")] pub recorders: Vec, + /// The verb-written record families this repository's producers fill + /// (CLOUD-1810). + /// + /// The sibling of [`Self::recorders`] on the other side of one split: a + /// `[[recorder]]` row is filled from a mediated tool call, and one of these + /// is filled by a producer calling `batten record named `. Both + /// project into `input.tree.records`, and until this table existed only the + /// first could — so a store a `mise` task wrote was invisible to every + /// module, and the row reading it reported clean over a record that said + /// otherwise. + /// + /// **Declared rather than swept**, which is what keeps could-not-look + /// readable: an absent record under a declared family is "the producer did + /// not run", where the same absence with nothing declared is not a reading at + /// all. [`crate::record::Declared`] carries the whole argument. + /// + /// Consumer-owned, like the two tables above it: which measurements a + /// repository records, and what fills each, are facts about that repository + /// and never about the engine (non-negotiable rule 1). + #[serde(default, rename = "record", skip_serializing_if = "Vec::is_empty")] + pub records: Vec, /// The programs a `[[recorder]]` may run, by id. /// /// Named rather than inline so one program has one spelling, which is @@ -1970,6 +1991,12 @@ fn validate_tables(config: &Config, text: &str, source: &str, grammar: Grammar) .collect(), ), )?; + // The verb-written families, beside the recorder table because the two are + // halves of one question: which record names a module may read (CLOUD-1810). + under( + Native::RecordTableRefused, + crate::record::validate(&config.records), + )?; validate_sections(config) } @@ -3414,6 +3441,7 @@ impl Config { credential: None, forge: None, unresolvable: Vec::new(), + records: Vec::new(), version: SUPPORTED_VERSION, deferrals: Vec::new(), host: None, @@ -4000,6 +4028,11 @@ mod tests { "crate::recorder::validate(", Native::RecorderTableRefused, ), + ( + "records", + "crate::record::validate(", + Native::RecordTableRefused, + ), ]; /// The one CLASSED refusal that is not a `Config` table. diff --git a/crates/batten/src/hook.rs b/crates/batten/src/hook.rs index 1a00d699b..568be3228 100644 --- a/crates/batten/src/hook.rs +++ b/crates/batten/src/hook.rs @@ -3417,6 +3417,7 @@ impl Policy { // The hot path does not re-check an authoring property (CLOUD-1638). words: None, recorders: &resolved.recorders, + records: &resolved.records, }, crate::policy::ModuleChecks::SkipOnHotPath, reference, @@ -12253,6 +12254,7 @@ mod tests { verdicts: &fixture_verdicts, words: None, recorders: &[], + records: &[], }, crate::policy::ModuleChecks::Run, None, diff --git a/crates/batten/src/lib.rs b/crates/batten/src/lib.rs index 787979e2e..617eb62c6 100644 --- a/crates/batten/src/lib.rs +++ b/crates/batten/src/lib.rs @@ -816,6 +816,7 @@ fn run_baseline( verdicts: &config.verdicts, words: (!config.vocabulary.is_empty()).then_some(&config.vocabulary), recorders: &config.recorders, + records: &config.records, }, &root, rules::RunOptions { @@ -6361,6 +6362,7 @@ fn admission_anchor( verdicts: &config.verdicts, words: (!config.vocabulary.is_empty()).then_some(&config.vocabulary), recorders: &config.recorders, + records: &config.records, }, root, rules::RunOptions { @@ -6647,6 +6649,7 @@ fn run_policy_test(json: bool, overrides: &Overrides, out: &mut dyn Write) -> Re verdicts: &config.verdicts, words: (!config.vocabulary.is_empty()).then_some(&config.vocabulary), recorders: &config.recorders, + records: &config.records, }, policy::ModuleChecks::Run, overrides.config_from.as_deref(), @@ -15884,6 +15887,7 @@ fn filed_here_pointers( verdicts: &config.verdicts, words: (!config.vocabulary.is_empty()).then_some(&config.vocabulary), recorders: &config.recorders, + records: &config.records, }; // `run_static_over` WITH AN INSTANT, because the four-argument wrapper hands // `now: None` to `minted_facts`, which reads it as epoch 0 — so every receipt @@ -18991,6 +18995,7 @@ fn run_rules( verdicts: &config.verdicts, words: (!config.vocabulary.is_empty()).then_some(&config.vocabulary), recorders: &config.recorders, + records: &config.records, }; let (selected, checks) = select_rules(&config.rules, only)?; let scope = scope.resolve(&root)?; diff --git a/crates/batten/src/policy.rs b/crates/batten/src/policy.rs index ac805f27f..5ee5ff687 100644 --- a/crates/batten/src/policy.rs +++ b/crates/batten/src/policy.rs @@ -659,6 +659,14 @@ pub struct Vocabulary<'a> { /// has the recorders too. The alternative was a fifth positional on four /// public entry points, which is the shape this parameter exists to prevent. pub recorders: &'a [crate::recorder::Declared], + /// The `[[record]]` table (CLOUD-1810). + /// + /// Here for the same reason `recorders` is, and it is the same question: a + /// module reads a record projected from a DECLARATION, so a caller holding + /// the recorders needs the verb-written families too or half the store is + /// invisible to it. Carrying it on this struct is what keeps the arity of the + /// four public entry points from growing again. + pub records: &'a [crate::record::Declared], } impl Vocabulary<'_> { @@ -672,6 +680,7 @@ impl Vocabulary<'_> { verdicts: &[], words: None, recorders: &[], + records: &[], }; } @@ -682,6 +691,7 @@ impl<'a> From<&'a crate::config::Config> for Vocabulary<'a> { verdicts: &config.verdicts, words: (!config.vocabulary.is_empty()).then_some(&config.vocabulary), recorders: &config.recorders, + records: &config.records, } } } @@ -698,6 +708,7 @@ impl<'a> From<&'a crate::resolve::Resolved> for Vocabulary<'a> { verdicts: &resolved.verdicts, words: (!resolved.vocabulary.is_empty()).then_some(&resolved.vocabulary), recorders: &resolved.recorders, + records: &resolved.records, } } } @@ -767,6 +778,7 @@ pub fn load( verdicts, words, recorders: _, + records: _, } = vocabulary; // The table is validated at PARSE, beside `verbs` and `redirects` and for // their reason (`config.rs`'s `VALIDATED_AT_LOAD` census asserts the call diff --git a/crates/batten/src/record.rs b/crates/batten/src/record.rs index fe90878f7..72490d458 100644 --- a/crates/batten/src/record.rs +++ b/crates/batten/src/record.rs @@ -322,6 +322,103 @@ fn claim_of(git_dir: &Path, branch: &str) -> Option { /// not express. pub const VERB_WRITTEN: &[&str] = &["claim", "plan", crate::land::LAP_RECORD]; +/// One family a producer writes through [`run_named`], declared by the consumer +/// (CLOUD-1810). +/// +/// # The gap this closes, and why neither existing surface could +/// +/// [`run_named`] writes a branch-keyed store, and until this existed nothing +/// could read one. [`crate::rules`]' projection builds the set of families it +/// hands a module as the declared [`crate::recorder::Declared`] rows unioned with +/// [`VERB_WRITTEN`], and a caller-named family is in neither: +/// +/// - [`VERB_WRITTEN`] is a fixed list because the ENGINE owns both halves of +/// those three stores. A consumer's family cannot join it without this crate +/// knowing a consumer's name, which non-negotiable rule 1 forbids outright. +/// - A `[[recorder]]` cannot express one either: [`crate::recorder::Declared`] +/// requires `tool`, because that table selects on a mediated tool call. A +/// family a `mise` task writes answers to no tool call at all. +/// +/// So the store was written, the row was registered, the module read +/// `input.tree.records[""]` — and the key was absent, every rule beneath +/// it undefined, and the gate green. Measured over `branch-age`: a record naming +/// a 36-day branch against a two-day threshold, `batten check` exit `0`. That is +/// CLOUD-1707's dead gate one surface over. +/// +/// # Declared rather than swept, which is the whole design +/// +/// The projection could have read whatever files happen to sit in the store +/// directory. It must not, for the reason [`crate::rules`] already gives about a +/// sibling fact: a family set cannot become an ambient sweep of whatever records +/// happen to be on disk, because then a leftover file from a retired producer +/// answers as a live measurement and nothing names what SHOULD be there. +/// +/// A declaration is also what makes could-not-look readable. An absent record +/// under a DECLARED family is "the producer did not run"; the same absence under +/// no declaration is not a reading at all, and collapsing the two is the error in +/// the fact model this whole store exists to avoid. +/// +/// # Config rather than a column on the rule that reads it +/// +/// [`crate::rules`] settles this at its own call site: the fact is what THIS +/// repository's producers accumulated, so a per-rule declaration would be a +/// second place for the same answer to live. Two rules reading one family is +/// ordinary; two rules disagreeing about what writes it is not expressible. +#[derive( + Debug, Clone, PartialEq, Eq, serde::Deserialize, serde::Serialize, schemars::JsonSchema, +)] +#[serde(deny_unknown_fields, rename_all = "kebab-case")] +pub struct Declared { + /// The family name, which is also its file name under the store. + /// + /// Held to [`safe_component`]'s grammar at validation rather than at write + /// time alone, so a family that could never be written is refused while its + /// author is watching instead of on the first producer run. + pub record: String, + /// What writes it, as a runnable command. + /// + /// **Never executed, and that is not a gap.** House style §5 keeps the spawn + /// outside `check`, so this is a pointer — the job `[[verdict.route]]`'s + /// `target` already does. What it buys is that a declared family always says + /// who fills it: a store with no producer is a row that can only ever read + /// could-not-look, and the moment to catch that is at config load rather than + /// after a green run nobody questions. + pub writer: String, +} + +/// Prove every declared family well formed (CLOUD-253's obligation). +/// +/// # Errors +/// +/// A [`UsageError`] (→ exit `1`) for a family whose name is not a single path +/// component, for an empty `writer`, and for two rows naming one family — the +/// last because a second row is a second answer to "who writes this", which is +/// the one question the table exists to settle. +pub fn validate(declared: &[Declared]) -> Result<()> { + let mut seen: std::collections::BTreeSet<&str> = std::collections::BTreeSet::new(); + for family in declared { + // The same grammar the writer enforces, checked here so the refusal lands + // at load. A name that escapes its store is why `safe_component` exists; + // reaching it only from `run_named` would let a config sit green until a + // producer ran. + safe_component("record", &family.record)?; + if family.writer.trim().is_empty() { + return Err(UsageError::raise(format!( + "record `{}`: `writer` names what fills this store and is empty, so the \ + family could only ever read could-not-look", + family.record + ))); + } + if !seen.insert(family.record.as_str()) { + return Err(UsageError::raise(format!( + "record `{}` is declared twice; one family has one writer", + family.record + ))); + } + } + Ok(()) +} + /// The statuses a plan entry may carry. /// /// The vocabulary four harnesses already converged on, which is what makes a diff --git a/crates/batten/src/resolve.rs b/crates/batten/src/resolve.rs index 5234c151f..15d174187 100644 --- a/crates/batten/src/resolve.rs +++ b/crates/batten/src/resolve.rs @@ -602,6 +602,15 @@ pub struct Resolved { /// rather than a rule someone has to remember. #[serde(rename = "recorder")] pub recorders: Vec, + /// The verb-written record families (CLOUD-1810), from the **committed + /// authority alone**. + /// + /// A local layer cannot add one, and that is the safe direction rather than a + /// restriction: declaring a family only ever makes a gate LIVE, so nothing is + /// lost by refusing an override here — while a layer that could REMOVE one + /// would disarm every rule reading that store, which is the weakening §8 + /// admits no route for. `board`'s structural guarantee, for `epoch`'s reason. + pub records: Vec, /// The programs a recorder may run, authority-only for the same reason and /// separately, because the indirection is the sharper half: repointing an id /// here changes what every column reading it records while the recorder rows @@ -1709,6 +1718,7 @@ fn assemble( // own note for why the local layer may not reach this one. mints: repo.mints.clone(), recorders: repo.recorders.clone(), + records: repo.records.clone(), programs: repo.programs.clone(), markers: repo.markers.clone(), exec: repo.exec, @@ -1792,6 +1802,10 @@ fn attribution( // weakening dressed as a setting, and §8 admits only raises. ("ready", authority_set(repo.ready.is_some())), ("board", authority_set(repo.board.is_some())), + // AUTHORITY-ONLY, and the direction is why: a declared family arms a gate, + // so the dangerous edit is the REMOVAL a local layer must not be able to + // make (CLOUD-1810). + ("records", authority_set(!repo.records.is_empty())), // AUTHORITY-ONLY for the same reason, one table over: every row RAISES a // path's threshold, so a local layer that could add one would be the // weakening §8 refuses. diff --git a/crates/batten/src/rules.rs b/crates/batten/src/rules.rs index 61f212d46..366d62be2 100644 --- a/crates/batten/src/rules.rs +++ b/crates/batten/src/rules.rs @@ -7011,9 +7011,13 @@ fn run( // when some unrelated recorder happened to be declared would make a gate's // liveness depend on a table it has nothing to do with. let records = match (crate::git::git_dir(root), crate::git::current_branch(root)) { - (Ok(git_dir), Ok(Some(branch))) => { - recorder_records(&git_dir, &branch, recorders, crate::record::VERB_WRITTEN) - } + (Ok(git_dir), Ok(Some(branch))) => recorder_records( + &git_dir, + &branch, + recorders, + vocabulary.records, + crate::record::VERB_WRITTEN, + ), _ => BTreeMap::new(), }; // CLOUD-1126, read UNCONDITIONALLY for `VERB_WRITTEN`'s reason: the engine @@ -7424,10 +7428,29 @@ fn recorder_blocked(git_dir: &std::path::Path, branch: &str) -> BTreeMap BTreeMap> { let mut found: BTreeMap> = BTreeMap::new(); @@ -7439,6 +7462,7 @@ fn recorder_records( let names = recorders .iter() .map(|recorder| recorder.record.as_str()) + .chain(families.iter().map(|family| family.record.as_str())) .chain(verb_written.iter().copied()); // THE SAME PARTITION THE WRITER USED (CLOUD-1300). A branch name outlives the // branch it described, so reading by name alone let this attempt read the diff --git a/crates/batten/src/trust.rs b/crates/batten/src/trust.rs index 30b804d0c..aba71394a 100644 --- a/crates/batten/src/trust.rs +++ b/crates/batten/src/trust.rs @@ -1280,6 +1280,20 @@ pub const CENSUS: &[FieldCoverage] = &[ WeakeningKind::RecorderChanged, ]), }, + FieldCoverage { + field: "records", + coverage: Coverage::NotPolicyBearing( + "the verb-written record families this repository's producers fill (CLOUD-1810). \ + It IS read by gates — a module reading `input.tree.records.` decides \ + nothing at all until the family is declared — so the reason is not that it lacks \ + policy weight. It is that an override cannot speak to it: the key is absent from \ + `OverrideConfig` and `resolve` reads the table from the committed authority \ + alone, `board`'s structural guarantee for `epoch`'s reason. The direction is \ + what makes that sufficient rather than merely convenient — DECLARING a family \ + only ever arms a gate, and the REMOVAL that would disarm one is unwritable in \ + the layer an author controls at PR time", + ), + }, FieldCoverage { field: "programs", coverage: Coverage::Compared(&[WeakeningKind::ProgramChanged]), diff --git a/crates/batten/src/verdict.rs b/crates/batten/src/verdict.rs index 0b01d1a25..b50e8028c 100644 --- a/crates/batten/src/verdict.rs +++ b/crates/batten/src/verdict.rs @@ -1282,6 +1282,12 @@ pub enum Native { MintTableRefused, /// The `[[recorder]]` table would not load. RecorderTableRefused, + /// The `[[record]]` table would not load (CLOUD-1810). + /// + /// Its own class rather than the recorder's, on the reason the environment + /// pair one screen up already states: the two tables carry different rows and + /// a refusal has to name which one to edit. + RecordTableRefused, /// The `[[provision]]` table would not load. ProvisionTableRefused, /// The `[[startup]]` table would not load. @@ -1376,6 +1382,7 @@ impl Native { Native::FactTableRefused, Native::MintTableRefused, Native::RecorderTableRefused, + Native::RecordTableRefused, Native::ProvisionTableRefused, Native::StartupTableRefused, Native::PlanReadStale, @@ -1410,6 +1417,7 @@ impl Native { Native::FactTableRefused, Native::MintTableRefused, Native::RecorderTableRefused, + Native::RecordTableRefused, Native::ProvisionTableRefused, Native::StartupTableRefused, ]; @@ -1459,6 +1467,7 @@ impl Native { Native::FactTableRefused => "fact declare refused", Native::MintTableRefused => "mint declare refused", Native::RecorderTableRefused => "recorder declare refused", + Native::RecordTableRefused => "record declare refused", Native::ProvisionTableRefused => "provision declare refused", Native::StartupTableRefused => "startup declare refused", } @@ -2116,6 +2125,16 @@ the ids are known to be well formed themselves.", routes: &[read("config read first", "batten.toml")], applicability: Applicability::Advice, }, + VendoredVerdict { + id: "record declare refused", + gloss: "the record table would not load", + class: "`[[record]]` declares a family a producer fills with `batten record named`, so \ +that a module may read its store at all. A row whose name could never be written, or whose \ +writer is unnamed, is a family that can only ever answer could-not-look -- and the moment to \ +say so is at load, not after a green run nobody questioned.", + routes: &[read("config read first", "batten.toml")], + applicability: Applicability::Advice, + }, VendoredVerdict { id: "provision declare refused", gloss: "the provision table would not load", @@ -2526,6 +2545,7 @@ mod tests { | Native::FactTableRefused | Native::MintTableRefused | Native::RecorderTableRefused + | Native::RecordTableRefused | Native::ProvisionTableRefused | Native::StartupTableRefused => native.id(), }; diff --git a/crates/batten/tests/it/admission_narrowing.rs b/crates/batten/tests/it/admission_narrowing.rs index 78eedc677..1ac9e1bed 100644 --- a/crates/batten/tests/it/admission_narrowing.rs +++ b/crates/batten/tests/it/admission_narrowing.rs @@ -105,6 +105,7 @@ fn load_fixture(root: &Path, rows: &[Rule]) -> Vec { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, policy::ModuleChecks::RunOverSelection, None, @@ -206,6 +207,7 @@ fn the_committed_bundles_publish_no_engine_side_rule_name() { verdicts: &config.verdicts, words: None, recorders: &config.recorders, + records: &[], }, policy::ModuleChecks::RunOverSelection, None, diff --git a/crates/batten/tests/it/agentic_record.rs b/crates/batten/tests/it/agentic_record.rs index 5a47d37b8..49b63ea89 100644 --- a/crates/batten/tests/it/agentic_record.rs +++ b/crates/batten/tests/it/agentic_record.rs @@ -436,6 +436,7 @@ fn replayed_findings( verdicts, words: None, recorders: &[], + records: &[], }, dir, ) diff --git a/crates/batten/tests/it/bats_invocation.rs b/crates/batten/tests/it/bats_invocation.rs index d4a7ec37f..8cffc41a6 100644 --- a/crates/batten/tests/it/bats_invocation.rs +++ b/crates/batten/tests/it/bats_invocation.rs @@ -134,6 +134,7 @@ fn findings_declared_by(root: &Path, vocabulary_root: &Path) -> Vec<(String, Opt verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/cfg_gated_test.rs b/crates/batten/tests/it/cfg_gated_test.rs index 176e270eb..86118e647 100644 --- a/crates/batten/tests/it/cfg_gated_test.rs +++ b/crates/batten/tests/it/cfg_gated_test.rs @@ -121,6 +121,7 @@ fn scan(root: &Path) -> rules::Scan { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/ci_cache_declared.rs b/crates/batten/tests/it/ci_cache_declared.rs index 9d6fd70d4..b192725d2 100644 --- a/crates/batten/tests/it/ci_cache_declared.rs +++ b/crates/batten/tests/it/ci_cache_declared.rs @@ -119,6 +119,7 @@ fn findings_declared_by(root: &Path, vocabulary_root: &Path) -> Vec<(String, Opt verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/ci_hygiene.rs b/crates/batten/tests/it/ci_hygiene.rs index 209c550c8..819730f0e 100644 --- a/crates/batten/tests/it/ci_hygiene.rs +++ b/crates/batten/tests/it/ci_hygiene.rs @@ -102,6 +102,7 @@ fn findings(root: &Path) -> Vec<(String, Option)> { verdicts: &[], words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/ci_parity.rs b/crates/batten/tests/it/ci_parity.rs index a3e99859d..ff8ce491a 100644 --- a/crates/batten/tests/it/ci_parity.rs +++ b/crates/batten/tests/it/ci_parity.rs @@ -242,6 +242,7 @@ fn findings_declared_by(root: &Path, vocabulary_root: &Path) -> Vec<(String, Opt verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) @@ -275,6 +276,7 @@ fn verdicts_raised(root: &Path) -> Vec { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/ci_suite_lane.rs b/crates/batten/tests/it/ci_suite_lane.rs index 99b368676..4aad64ce2 100644 --- a/crates/batten/tests/it/ci_suite_lane.rs +++ b/crates/batten/tests/it/ci_suite_lane.rs @@ -103,6 +103,7 @@ fn findings_declared_by(root: &Path, vocabulary_root: &Path) -> Vec<(String, Opt verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/claim_order.rs b/crates/batten/tests/it/claim_order.rs index 531d8d0b3..5f951bd8f 100644 --- a/crates/batten/tests/it/claim_order.rs +++ b/crates/batten/tests/it/claim_order.rs @@ -121,6 +121,7 @@ fn findings_declared_by(root: &Path, vocabulary_root: &Path) -> Vec { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/config_fault_class.rs b/crates/batten/tests/it/config_fault_class.rs index 2b6f257cc..10b150603 100644 --- a/crates/batten/tests/it/config_fault_class.rs +++ b/crates/batten/tests/it/config_fault_class.rs @@ -130,6 +130,15 @@ const FAULTS: &[(&str, &str, &str)] = &[ [[mint]]\nname = \"twice\"\ntool = \"Bash\"\nkey = \"branch\"\n\ mode = \"replace\"\nbody = \"b\"\n", ), + ( + // Two rows naming one family: a second row is a second answer to "who + // writes this", which is the one question the table exists to settle. + "record declare refused", + "record", + "version = 1\n\ + [[record]]\nrecord = \"twice\"\nwriter = \"mise run measure\"\n\ + [[record]]\nrecord = \"twice\"\nwriter = \"mise run measure\"\n", + ), ( "recorder declare refused", "recorder", diff --git a/crates/batten/tests/it/document_read_count.rs b/crates/batten/tests/it/document_read_count.rs index 13d738d0d..f334f25fb 100644 --- a/crates/batten/tests/it/document_read_count.rs +++ b/crates/batten/tests/it/document_read_count.rs @@ -59,6 +59,7 @@ fn fixtures(root: &Path) -> batten::policy::Vocabulary<'static> { verdicts: table, words: None, recorders: &[], + records: &[], } } diff --git a/crates/batten/tests/it/filed_here.rs b/crates/batten/tests/it/filed_here.rs index 345fe9e2a..55ca02968 100644 --- a/crates/batten/tests/it/filed_here.rs +++ b/crates/batten/tests/it/filed_here.rs @@ -250,6 +250,7 @@ fn scan(root: &Path) -> rules::Scan { verdicts: &verdicts, words: None, recorders: &declared, + records: &[], }, root, ) diff --git a/crates/batten/tests/it/fixture_forks.rs b/crates/batten/tests/it/fixture_forks.rs index 0b47842a9..18aeec5a5 100644 --- a/crates/batten/tests/it/fixture_forks.rs +++ b/crates/batten/tests/it/fixture_forks.rs @@ -122,6 +122,7 @@ fn scan(root: &Path) -> rules::Scan { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/frontmatter_gates.rs b/crates/batten/tests/it/frontmatter_gates.rs index 9951f12b2..0460dd20b 100644 --- a/crates/batten/tests/it/frontmatter_gates.rs +++ b/crates/batten/tests/it/frontmatter_gates.rs @@ -95,6 +95,7 @@ fn findings_for(root: &Path, row: Rule, vocabulary_root: &Path) -> Vec { // the exemption the field documents rather than a gap in the setup. words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/hk_fix_selection.rs b/crates/batten/tests/it/hk_fix_selection.rs index 973458608..a9031e263 100644 --- a/crates/batten/tests/it/hk_fix_selection.rs +++ b/crates/batten/tests/it/hk_fix_selection.rs @@ -114,6 +114,7 @@ fn findings_declared_by(root: &Path, vocabulary_root: &Path) -> Vec<(String, Opt verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/identity_churn.rs b/crates/batten/tests/it/identity_churn.rs index f4d37b0bf..994c3209d 100644 --- a/crates/batten/tests/it/identity_churn.rs +++ b/crates/batten/tests/it/identity_churn.rs @@ -85,6 +85,7 @@ impl Scan { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/landing_roster.rs b/crates/batten/tests/it/landing_roster.rs index f800ba0d3..d4e9b6c29 100644 --- a/crates/batten/tests/it/landing_roster.rs +++ b/crates/batten/tests/it/landing_roster.rs @@ -115,6 +115,7 @@ fn scan(root: &Path) -> rules::Scan { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/mise_preset.rs b/crates/batten/tests/it/mise_preset.rs index 5892011b6..f4df9d562 100644 --- a/crates/batten/tests/it/mise_preset.rs +++ b/crates/batten/tests/it/mise_preset.rs @@ -94,6 +94,7 @@ fn findings(root: &Path) -> Vec<(String, Option)> { verdicts: &[], words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/mutation_declared_case.rs b/crates/batten/tests/it/mutation_declared_case.rs index 61eab29f2..fab6ff948 100644 --- a/crates/batten/tests/it/mutation_declared_case.rs +++ b/crates/batten/tests/it/mutation_declared_case.rs @@ -89,6 +89,7 @@ fn verdicts(root: &Path) -> Vec { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/nextest_slow.rs b/crates/batten/tests/it/nextest_slow.rs index 4a604d5b0..2c37a0d31 100644 --- a/crates/batten/tests/it/nextest_slow.rs +++ b/crates/batten/tests/it/nextest_slow.rs @@ -103,6 +103,7 @@ fn scan(root: &Path) -> rules::Scan { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/obligations_bound.rs b/crates/batten/tests/it/obligations_bound.rs index 1874e3a34..16c8af3aa 100644 --- a/crates/batten/tests/it/obligations_bound.rs +++ b/crates/batten/tests/it/obligations_bound.rs @@ -159,6 +159,7 @@ fn verdicts(root: &Path) -> Vec { verdicts: &verdicts, words: None, recorders: &declared, + records: &[], }, root, ) diff --git a/crates/batten/tests/it/plan_complete.rs b/crates/batten/tests/it/plan_complete.rs index 720d5af17..2c87bc4a2 100644 --- a/crates/batten/tests/it/plan_complete.rs +++ b/crates/batten/tests/it/plan_complete.rs @@ -132,6 +132,7 @@ fn scan(root: &Path) -> rules::Scan { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/policy_test_suite.rs b/crates/batten/tests/it/policy_test_suite.rs index f0c80f9aa..37ef0072a 100644 --- a/crates/batten/tests/it/policy_test_suite.rs +++ b/crates/batten/tests/it/policy_test_suite.rs @@ -66,6 +66,7 @@ fn fixtures(root: &Path) -> batten::policy::Vocabulary<'static> { verdicts: table, words: None, recorders: &[], + records: &[], } } diff --git a/crates/batten/tests/it/policy_tree.rs b/crates/batten/tests/it/policy_tree.rs index 887aa9808..61e40de76 100644 --- a/crates/batten/tests/it/policy_tree.rs +++ b/crates/batten/tests/it/policy_tree.rs @@ -49,6 +49,7 @@ fn fixtures(root: &Path) -> batten::policy::Vocabulary<'static> { verdicts: table, words: None, recorders: &[], + records: &[], } } diff --git a/crates/batten/tests/it/prebuilt_lint.rs b/crates/batten/tests/it/prebuilt_lint.rs index ab1e28143..6a2c73161 100644 --- a/crates/batten/tests/it/prebuilt_lint.rs +++ b/crates/batten/tests/it/prebuilt_lint.rs @@ -78,6 +78,7 @@ fn findings(root: &Path) -> Vec { verdicts: &[], words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/prose_only.rs b/crates/batten/tests/it/prose_only.rs index 4312f7c23..631120405 100644 --- a/crates/batten/tests/it/prose_only.rs +++ b/crates/batten/tests/it/prose_only.rs @@ -116,6 +116,7 @@ fn findings(root: &Path) -> Vec { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) @@ -485,6 +486,7 @@ fn the_finding_carries_a_count_and_never_a_path() { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, &root, ) diff --git a/crates/batten/tests/it/record_families.rs b/crates/batten/tests/it/record_families.rs index b0dc4818d..ec62f64f4 100644 --- a/crates/batten/tests/it/record_families.rs +++ b/crates/batten/tests/it/record_families.rs @@ -209,3 +209,137 @@ fn a_family_that_would_escape_its_store_is_refused() { let refused = run_with_stdin(&dir, &["record", "keyed", "../escape", "k"], "v\n"); assert_eq!(refused.status.code(), Some(1)); } + +// --------------------------------------------------------------------------- +// The projection: a declared family reaches a module (CLOUD-1810). +// --------------------------------------------------------------------------- + +/// A module that decides from one record and nothing else. +/// +/// Deliberately the thinnest thing that can tell the three states apart: it +/// reports when the record holds a `hit` line, and is undefined when the family +/// is not projected at all. A richer module would let a case pass for a reason +/// that has nothing to do with the projection. +const READS_A_FAMILY: &str = r#"# METADATA +# description: reads one verb-written record family. +# schemas: +# - input: schema["policy-input.schema"] +package batten.reads_a_family + +import rego.v1 + +rules contains "record read other" + +recorded := input.tree.records["measured"] + +violation contains { + "rule": "record read other", + "verdict": "record read other", +} if { + some line in recorded + line == "hit" +} +"#; + +/// Config registering that module, with `declares` deciding whether the family +/// is declared at all. +fn family_config(declares: bool) -> String { + let table = if declares { + "\n[[record]]\nrecord = \"measured\"\nwriter = \"mise run measure\"\n" + } else { + "" + }; + format!( + r#"version = 1 +scope = ["**"] + +[[verdict]] +id = "record read other" +gloss = "the record this rule reads says so" +class = "A test fixture's class." + +[[verdict.route]] +id = "record read first" +kind = "document" +target = "the record this rule reads" + +[[rule]] +id = "record read other" +kind = "policy" +scope = "tree" +module = "policy/reads-a-family.rego" +severity = "deny" +{table}"# + ) +} + +/// The tree both cases below run over, differing only in the declaration. +fn family_repo(name: &str, declares: bool) -> std::path::PathBuf { + let dir = repo(name); + write(&dir, "batten.toml", &family_config(declares)); + write(&dir, "policy/reads-a-family.rego", READS_A_FAMILY); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-qm", "register the module"]); + dir +} + +#[test] +fn a_declared_family_reaches_the_module_that_reads_it() { + // THE DEFECT THIS ROW EXISTS FOR (CLOUD-1810). Before the declaration table, + // `record named` wrote a store no module could read: the key never reached + // `input.tree.records`, every rule beneath it was undefined, and the row + // reported clean over a record that said otherwise. Measured over + // `branch-age`: a 36-day branch against a two-day threshold, exit 0. + let dir = family_repo("projected", true); + let written = run_with_stdin( + &dir, + &["record", "named", "measured"], + "hit +", + ); + assert!(written.status.success(), "the setup write lands"); + + let decided = run(&dir, &["check"]); + assert_eq!( + decided.status.code(), + Some(2), + "a declared family's record must reach the module and decide\n{}", + String::from_utf8_lossy(&decided.stderr) + ); +} + +#[test] +fn an_absent_record_under_a_declared_family_says_nothing() { + // THE OTHER HALF OF THE PAIR, and the half a weaker tier skips. Asserting + // only the case above passes on a projection that reads the DECLARATION + // rather than the store — it would report the finding with nothing written. + // Absent is could-not-look, never a pass and never a violation. + let dir = family_repo("absent", true); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "an absent record is could-not-look, so the module says nothing\n{}", + String::from_utf8_lossy(&quiet.stderr) + ); +} + +#[test] +fn an_undeclared_family_is_not_projected_whatever_the_store_holds() { + // DECLARED RATHER THAN SWEPT, as an exit code. If the projection read the + // store directory instead of the table, this record would decide — and a + // leftover file from a retired producer would answer as a live measurement + // with nothing naming what should be there. + let dir = family_repo("undeclared", false); + let written = run_with_stdin(&dir, &["record", "named", "measured"], "hit\n"); + assert!(written.status.success(), "the setup write lands"); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "an undeclared family is not a family, whatever sits in its store\n{}", + String::from_utf8_lossy(&quiet.stderr) + ); +} diff --git a/crates/batten/tests/it/release_provision_parity.rs b/crates/batten/tests/it/release_provision_parity.rs index e3c4e109c..c006ddd90 100644 --- a/crates/batten/tests/it/release_provision_parity.rs +++ b/crates/batten/tests/it/release_provision_parity.rs @@ -128,6 +128,7 @@ fn findings_declared_by(root: &Path, vocabulary_root: &Path) -> Vec { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/remedy_authorship.rs b/crates/batten/tests/it/remedy_authorship.rs index 4bc660a5f..c4ca0fb9d 100644 --- a/crates/batten/tests/it/remedy_authorship.rs +++ b/crates/batten/tests/it/remedy_authorship.rs @@ -95,6 +95,7 @@ fn scan(root: &Path, rule: Rule) -> rules::Scan { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/review_dispatched.rs b/crates/batten/tests/it/review_dispatched.rs index 4d64840e6..ed419bccb 100644 --- a/crates/batten/tests/it/review_dispatched.rs +++ b/crates/batten/tests/it/review_dispatched.rs @@ -164,6 +164,7 @@ fn verdicts_for(root: &Path, declared: bool) -> Vec { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) @@ -224,6 +225,7 @@ fn the_mediated_surface_resolves_no_effect_fact_and_withholds_the_rule() { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }; let rows = [row(&root, true)]; @@ -489,6 +491,7 @@ fn verdicts_with(root: &Path, extra: &serde_json::Value) -> Vec { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/rule_cost_census.rs b/crates/batten/tests/it/rule_cost_census.rs index b399b209a..83a478610 100644 --- a/crates/batten/tests/it/rule_cost_census.rs +++ b/crates/batten/tests/it/rule_cost_census.rs @@ -41,6 +41,7 @@ fn vocabulary() -> batten::policy::Vocabulary<'static> { verdicts: &[], words: None, recorders: &[], + records: &[], } } diff --git a/crates/batten/tests/it/shell_retirement.rs b/crates/batten/tests/it/shell_retirement.rs index a8b0f33be..cce43b9d3 100644 --- a/crates/batten/tests/it/shell_retirement.rs +++ b/crates/batten/tests/it/shell_retirement.rs @@ -131,6 +131,7 @@ pub(crate) fn scan(root: &Path) -> rules::Scan { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/spawn_widening.rs b/crates/batten/tests/it/spawn_widening.rs index 8cf4d1712..961d712d9 100644 --- a/crates/batten/tests/it/spawn_widening.rs +++ b/crates/batten/tests/it/spawn_widening.rs @@ -129,6 +129,7 @@ fn verdicts(root: &Path) -> Vec { verdicts: &declared, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/suite_subjects.rs b/crates/batten/tests/it/suite_subjects.rs index 004f3b826..dad3dbe22 100644 --- a/crates/batten/tests/it/suite_subjects.rs +++ b/crates/batten/tests/it/suite_subjects.rs @@ -85,6 +85,7 @@ fn findings_declared_by(root: &Path, vocabulary_root: &Path) -> Vec<(String, Opt verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/test_targets.rs b/crates/batten/tests/it/test_targets.rs index 6308416b3..d1c524242 100644 --- a/crates/batten/tests/it/test_targets.rs +++ b/crates/batten/tests/it/test_targets.rs @@ -103,6 +103,7 @@ fn scan(root: &Path) -> rules::Scan { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, root, ) diff --git a/crates/batten/tests/it/verdict_registry.rs b/crates/batten/tests/it/verdict_registry.rs index d300f9534..a7281be52 100644 --- a/crates/batten/tests/it/verdict_registry.rs +++ b/crates/batten/tests/it/verdict_registry.rs @@ -68,6 +68,7 @@ fn load( verdicts, words: None, recorders: &[], + records: &[], }, policy::ModuleChecks::Run, None, @@ -456,6 +457,7 @@ fn route_findings(name: &str, authority: &str, manifest: &str) -> Vec { verdicts: &verdicts, words: None, recorders: &[], + records: &[], }, &root, ) diff --git a/crates/batten/tests/policy_modules.rs b/crates/batten/tests/policy_modules.rs index 1adf19891..9714a06f8 100644 --- a/crates/batten/tests/policy_modules.rs +++ b/crates/batten/tests/policy_modules.rs @@ -99,6 +99,7 @@ fn fixtures(root: &Path) -> policy::Vocabulary<'static> { verdicts: table, words: None, recorders: &[], + records: &[], } } @@ -112,6 +113,7 @@ fn fixtures_with( verdicts: fixtures(root).verdicts, words: None, recorders: &[], + records: &[], } } diff --git a/hk.pkl b/hk.pkl index 56a350f82..ec853d42c 100644 --- a/hk.pkl +++ b/hk.pkl @@ -471,6 +471,7 @@ local gate = new Mapping { "crates/batten/src/pattern.rs", "crates/batten/src/provision.rs", "crates/batten/src/prune.rs", + "crates/batten/src/record.rs", "crates/batten/src/recorder.rs", "crates/batten/src/redirect.rs", "crates/batten/src/refusal.rs", diff --git a/mise.toml b/mise.toml index bd09d20ab..74f85e14c 100644 --- a/mise.toml +++ b/mise.toml @@ -617,7 +617,7 @@ CI_FANIN_WORKFLOW = ".github/workflows/ci.yml" BATS_TEST_TIMEOUT = "300" REGORUS_OPA_COMPLIANCE = "1.2.0" REGORUS_OPA_COMPLIANCE_FOR = "0.11" -MUTANT_GATES = "mise,attestation-check,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age-check,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure-check,evaluator-io-check,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence-assert,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link-check,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict-assert,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus-check,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook" +MUTANT_GATES = "mise,attestation-check,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age-check,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure-check,evaluator-io-check,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence-assert,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link-check,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict-assert,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus-check,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" # --- GitHub reachability behind an egress proxy (Claude Code web sandbox etc.) --- # mise resolves every tool's release through GitHub's *API* host, api.github.com. diff --git a/schema/batten.schema.json b/schema/batten.schema.json index 1b4863969..1fe5d37de 100644 --- a/schema/batten.schema.json +++ b/schema/batten.schema.json @@ -388,6 +388,13 @@ } ] }, + "record": { + "description": "The verb-written record families this repository's producers fill\n(CLOUD-1810).\n\nThe sibling of [`Self::recorders`] on the other side of one split: a\n`[[recorder]]` row is filled from a mediated tool call, and one of these\nis filled by a producer calling `batten record named `. Both\nproject into `input.tree.records`, and until this table existed only the\nfirst could — so a store a `mise` task wrote was invisible to every\nmodule, and the row reading it reported clean over a record that said\notherwise.\n\n**Declared rather than swept**, which is what keeps could-not-look\nreadable: an absent record under a declared family is \"the producer did\nnot run\", where the same absence with nothing declared is not a reading at\nall. [`crate::record::Declared`] carries the whole argument.\n\nConsumer-owned, like the two tables above it: which measurements a\nrepository records, and what fills each, are facts about that repository\nand never about the engine (non-negotiable rule 1).", + "type": "array", + "items": { + "$ref": "#/$defs/Declared4" + } + }, "recorder": { "description": "Records written from the tool result that earned them (CLOUD-1051).\n\nThe third selector on the post-tool event, and the one that can carry a\nvalue another gate decided. A `[[mint]]` renders a template over the\npayload; a `[[recorder]]` may additionally run a declared program and\nrecord its verdict, which is what a board write's refinement column IS.\n\nConsumer-owned for the same reason `[[mint]]` is, and more so: the column\nnames, the verdict tokens and the programs are all a tracker's vocabulary,\nand every one of them is declared here rather than chosen by the engine.\n(The scan this paragraph used to report having run is gone with\n[`Config::mints`]' — same false claim, same reason.)", "type": "array", @@ -1418,6 +1425,25 @@ "columns" ] }, + "Declared4": { + "description": "One family a producer writes through [`run_named`], declared by the consumer\n(CLOUD-1810).\n\n# The gap this closes, and why neither existing surface could\n\n[`run_named`] writes a branch-keyed store, and until this existed nothing\ncould read one. [`crate::rules`]' projection builds the set of families it\nhands a module as the declared [`crate::recorder::Declared`] rows unioned with\n[`VERB_WRITTEN`], and a caller-named family is in neither:\n\n- [`VERB_WRITTEN`] is a fixed list because the ENGINE owns both halves of\n those three stores. A consumer's family cannot join it without this crate\n knowing a consumer's name, which non-negotiable rule 1 forbids outright.\n- A `[[recorder]]` cannot express one either: [`crate::recorder::Declared`]\n requires `tool`, because that table selects on a mediated tool call. A\n family a `mise` task writes answers to no tool call at all.\n\nSo the store was written, the row was registered, the module read\n`input.tree.records[\"\"]` — and the key was absent, every rule beneath\nit undefined, and the gate green. Measured over `branch-age`: a record naming\na 36-day branch against a two-day threshold, `batten check` exit `0`. That is\nCLOUD-1707's dead gate one surface over.\n\n# Declared rather than swept, which is the whole design\n\nThe projection could have read whatever files happen to sit in the store\ndirectory. It must not, for the reason [`crate::rules`] already gives about a\nsibling fact: a family set cannot become an ambient sweep of whatever records\nhappen to be on disk, because then a leftover file from a retired producer\nanswers as a live measurement and nothing names what SHOULD be there.\n\nA declaration is also what makes could-not-look readable. An absent record\nunder a DECLARED family is \"the producer did not run\"; the same absence under\nno declaration is not a reading at all, and collapsing the two is the error in\nthe fact model this whole store exists to avoid.\n\n# Config rather than a column on the rule that reads it\n\n[`crate::rules`] settles this at its own call site: the fact is what THIS\nrepository's producers accumulated, so a per-rule declaration would be a\nsecond place for the same answer to live. Two rules reading one family is\nordinary; two rules disagreeing about what writes it is not expressible.", + "type": "object", + "properties": { + "record": { + "description": "The family name, which is also its file name under the store.\n\nHeld to [`safe_component`]'s grammar at validation rather than at write\ntime alone, so a family that could never be written is refused while its\nauthor is watching instead of on the first producer run.", + "type": "string" + }, + "writer": { + "description": "What writes it, as a runnable command.\n\n**Never executed, and that is not a gap.** House style §5 keeps the spawn\noutside `check`, so this is a pointer — the job `[[verdict.route]]`'s\n`target` already does. What it buys is that a declared family always says\nwho fills it: a store with no producer is a row that can only ever read\ncould-not-look, and the moment to catch that is at config load rather than\nafter a green run nobody questions.", + "type": "string" + } + }, + "additionalProperties": false, + "required": [ + "record", + "writer" + ] + }, "DeclaredVerdict": { "description": "One declared refusal class.", "type": "object", From 4da311c5dbdcd8e02f7540af4b6b90041fabf06f Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Thu, 17 Sep 2026 00:44:17 +0000 Subject: [PATCH 02/32] refactor(gate)!: retire branch-age-check onto the module and the producer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The first of CLOUD-1707's nine, and the one that proves the group is unblocked: `policy/branch-age.rego` has been registered since the forge producer landed and decided nothing until CLOUD-1810 gave a `record named` family a declaration. It decides now, so the program goes. THE SPLIT IS FORCED, NOT CHOSEN. `policy/branch-age.rego` takes every DECISION the program made — a tip past the threshold, a name heading more than one merged pull request while still on the remote, a present record naming no branch at all. `[tasks.branch-age-record]` keeps the two things that are not decisions: the `gh` reads and the civil-calendar subtraction. Section 5 makes `check` `read` and structurally incapable of spawning, and `Fact::Instant` projects `null` to every module, which `clippy.toml` and `crates/batten/tests/clock_ban.rs` hold the engine to. CLOUD-1559's reading rule from the other side: carry the decisions, not the steps. THE TIER IS NOT THE MODULE'S OWN SUITE, and the difference is this port's whole lesson. The module carries eleven `test_` rules and every one fabricates its input with `with input as` — so all eleven were green through a whole session in which the row decided nothing on any real checkout. `crates/batten/tests/it/branch_age.rs` runs the real module over a record the real producer's verb wrote, read through the engine's own projection: the one case no `with input as` can reach. THE LEDGER ACCOUNTS FOR ALL THIRTEEN of the dying suite's cases. Seven carried. Three `changed` onto `mise.toml`, because the trunk filter and both forge reads moved to the producer — and the could-not-look pair records a real behaviour change rather than a relocation: a failed read writes nothing and nothing is silence here, where the shell exited 2. A module refusing there would refuse every checkout with no credential. Two `changed` onto the module: there is no custom threshold to honour, and an unparseable line is a torn store because the producer refuses a malformed one while its author is watching. One `withdrawn`: "a nonsense today is exit 2" asserted a property of an arithmetic step that does not exist on a surface which reads no clock. One case title is qualified `suite::case`. `perf-compare.bats` carried one spelled identically, and `bats count dropped` reported the ambiguity against `perf_compare.rs:56` — a file this delta never touched. The ratchet conserves cases by TITLE across the whole tree. The module's `#MUTANT-EXEMPT` placeholder gives way to the three mutations it drafted, `$MUTANT_GATES` trades `branch-age-check` for `branch-age`, and `branch-hygiene.yml` runs the producer then the engine. 85 programs to 84. Refs: CLOUD-1717 --- .github/workflows/branch-hygiene.yml | 11 +- bench/gates/RESULTS.md | 1 - bench/suites/RESULTS.md | 1 - crates/batten/tests/it/branch_age.rs | 237 +++++++++++++++++++++++++++ crates/batten/tests/it/main.rs | 1 + mise-tasks/branch-age-check.sh | 169 ------------------- mise.toml | 2 +- policy/branch-age.rego | 5 +- tests/branch-age-check.bats | 145 ---------------- 9 files changed, 252 insertions(+), 320 deletions(-) create mode 100644 crates/batten/tests/it/branch_age.rs delete mode 100755 mise-tasks/branch-age-check.sh delete mode 100644 tests/branch-age-check.bats diff --git a/.github/workflows/branch-hygiene.yml b/.github/workflows/branch-hygiene.yml index 42fefb75f..0a0fe3386 100644 --- a/.github/workflows/branch-hygiene.yml +++ b/.github/workflows/branch-hygiene.yml @@ -9,7 +9,8 @@ name: branch-hygiene # Failure here is informational: it means branches are accumulating, not that # anything is broken. `land` deletes the head of a PR it merges, so a growing # list means branches abandoned without landing, or names reused across stories. -# Run `mise run branch-age-check` locally and delete what has landed. +# Run `mise run branch-age-record` locally, then `batten check --rule 'branch +# watch loose'`, and delete what has landed. on: schedule: @@ -53,7 +54,13 @@ jobs: # the reason ci.yml states: the whole toolchain is ~10s a job that # this one has no use for. install_args: aqua:cli/cli aqua:jqlang/jq - - run: mise run branch-age-check + # THE SPLIT THE PORT FORCED (CLOUD-1717). The producer makes the two + # `gh` reads and the calendar subtraction; the engine adjudicates what + # it recorded. Neither half can hold the other: section 5 makes `check` + # `read` and incapable of spawning, and the engine calls no clock on any + # evaluation path. + - run: mise run branch-age-record + - run: mise run batten -- check --rule 'branch watch loose' # The landing lease is a ref, so its hygiene belongs in the ref sweep # rather than in a cron of its own — that is the whole reason it lands # here: an added step on a job that already spins costs no runner. diff --git a/bench/gates/RESULTS.md b/bench/gates/RESULTS.md index 1931f1115..e440222fa 100644 --- a/bench/gates/RESULTS.md +++ b/bench/gates/RESULTS.md @@ -68,7 +68,6 @@ commit, and a named ref resolves a declared one. | `awk-regex-check.sh` | git | `tracked` | | `batten-glob-check.sh` | git | `location` | | `board-sweep.sh` | forge | `location`, `log` | -| `branch-age-check.sh` | forge | — | | `cap-drift.sh` | tree | — | | `checks-green.sh` | forge | `head` | | `ci-drift.sh` | forge | `location` | diff --git a/bench/suites/RESULTS.md b/bench/suites/RESULTS.md index a845c7846..2fa87f133 100644 --- a/bench/suites/RESULTS.md +++ b/bench/suites/RESULTS.md @@ -82,7 +82,6 @@ to it pays. | 0.3 | 0.2% | `tests/sonar-gate.bats` | | 0.3 | 0.2% | `tests/no-doctests.bats` | | 0.3 | 0.2% | `tests/report-only-check.bats` | -| 0.3 | 0.2% | `tests/branch-age-check.bats` | | 0.3 | 0.2% | `tests/license-table-check.bats` | | 0.3 | 0.2% | `tests/board-payloads.bats` | | 0.3 | 0.2% | `tests/nonverdict-assert.bats` | diff --git a/crates/batten/tests/it/branch_age.rs b/crates/batten/tests/it/branch_age.rs new file mode 100644 index 000000000..3fc70171f --- /dev/null +++ b/crates/batten/tests/it/branch_age.rs @@ -0,0 +1,237 @@ +//! `branch watch loose` over the compiled binary (CLOUD-349, CLOUD-1717). +//! +//! # Why this tier exists and the module's own `test_` rules do not suffice +//! +//! `policy/branch-age.rego` carries eleven load-time cases, and every one of +//! them fabricates its input with `with input as`. That is the shape +//! `rules/policy-modules.md` warns about: a case can assert over a fact the +//! engine is unable to produce, and the module stays green while the row decides +//! nothing on any real checkout. This module spent a whole session in exactly +//! that state — registered, green, and reading a key `recorder_records` never +//! projected (CLOUD-1810) — so the case that matters here is the one no +//! `with input as` can reach: a record WRITTEN by the producer, read through the +//! engine's own projection, deciding. +//! +//! # RETIREMENT LEDGER, PER PATH — what `shell retire partial` reads +//! +//! The program's successor is `policy/branch-age.rego` for every DECISION it +//! made, and `[tasks.branch-age-record]` for the two steps that are not +//! decisions: the `gh` reads and the civil-calendar subtraction. That split is +//! forced rather than chosen — house style §5 makes `check` `read` and +//! structurally incapable of spawning, and `Fact::Instant` projects `null` to +//! every module, which `clippy.toml`'s `disallowed-methods` and +//! `crates/batten/tests/clock_ban.rs` hold the engine to. CLOUD-1559's reading +//! rule says the same thing from the other side: carry the decisions, not the +//! steps. +//! +//! Three cases below the file arms are NOT carried, and each says why in its own +//! row rather than being dropped quietly. +//! +// carried: mise-tasks/branch-age-check.sh policy/branch-age.rego kind:mechanism crates/batten/tests/it/branch_age.rs +// carried: tests/branch-age-check.bats policy/branch-age.rego kind:mechanism crates/batten/tests/it/branch_age.rs +// carried: "a remote carrying only fresh branches passes" policy/branch-age.rego kind:mechanism +// carried: "a branch past the threshold is refused, and named with its age" policy/branch-age.rego kind:mechanism +// The title is qualified because `perf-compare.bats` carried one spelled +// identically, and two arms claiming one title is what `bats count dropped` +// reported at `perf_compare.rs:56`. The `suite::case` form is the corpus's +// own, used in `connector_verbs.rs` for the same collision. +// carried: "branch-age-check.bats::the threshold is a boundary, not a suggestion" policy/branch-age.rego kind:mechanism +// carried: "a name heading more than one merged PR is refused, and counted" policy/branch-age.rego kind:mechanism +// carried: "a reused name whose branch is already gone is not counted" policy/branch-age.rego kind:mechanism +// carried: "a clean remote reaches green, which is the state the gate must be able to reach" policy/branch-age.rego kind:mechanism +// carried: "a remote reporting no branches at all is exit 2, not a pass" policy/branch-age.rego kind:mechanism +// changed: "the trunk is never counted, however old or however many PRs it heads" mise.toml the exclusion is a STEP rather than a decision, so it moved to the producer with the `gh` call that needs it: `[tasks.branch-age-record]` filters the trunk before recording, and the module never sees a trunk line to count. A case here would assert over input the producer cannot emit +// changed: "an unreadable refs reading is exit 2, not a pass" mise.toml the reading is the producer's and so is its failure: `[tasks.branch-age-record]` writes NOTHING when it cannot reach the forge, which is the could-not-look the module then reads as an absent record. The decision half is `an_absent_record_says_nothing_rather_than_passing`; what changed is that could-not-look is silence here rather than exit 2, because a module refusing there would refuse every checkout with no credential +// changed: "an unreadable PR reading is exit 2, not a pass" mise.toml the same split as the refs reading directly above, over the second of the producer's two forge calls: a failed read writes nothing, and nothing is could-not-look +// changed: "a custom threshold is honoured in both the verdict and the message" policy/branch-age.rego there is no custom threshold to honour: the figure is the practice's own "couple of days" and lives in the module, where moving it costs a diff a reviewer reads. A config knob invites raising it until nothing fires, which is `repetition-without-progress`'s reasoning one row over +// changed: "an unparseable tip date is reported rather than silently skipped" policy/branch-age.rego the producer refuses a malformed line at WRITE time, while its author is watching, so an unparseable line at read time is a torn store rather than a branch with a bad date. `a_line_this_reader_cannot_parse_is_skipped` pins the reader's half, and a surviving good line in that case is what stops it passing for want of any readable ref +// withdrawn: "a nonsense today is exit 2, not an arithmetic answer" there is no `today` for a caller to make nonsense of. The engine calls no clock on any evaluation path, so the module never subtracts dates at all — it compares a number the producer already computed against a threshold. The case asserted a property of an arithmetic step that no longer exists on this surface + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use common::{git_in, init_repo, run, run_with_stdin, scratch, write}; + +/// A repository registering the real module against a declared family. +/// +/// The REAL module and the REAL row, never a fixture copy: the whole point of +/// this tier is that the module decides over the engine's own projection, and a +/// stand-in would be one more `with input as` wearing a different costume. +fn repo(name: &str) -> std::path::PathBuf { + let dir = scratch(&format!("branch-age-{name}")); + let module = std::fs::read_to_string("../../policy/branch-age.rego") + .expect("the module this tier exists for"); + write(&dir, "policy/branch-age.rego", &module); + write( + &dir, + "batten.toml", + r#"version = 1 +scope = ["**"] + +[[pattern]] +id = "whole-number" +regex = '^[0-9]+$' + +[[verdict]] +id = "branch watch stale" +gloss = "a remote branch has outlived the story it was cut for" +class = "A branch is either finished, in which case delete it, or it is not short-lived." + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run branch-age-record" + +[[verdict]] +id = "branch name duplicate" +gloss = "one branch name heads more than one merged pull request and is still on the remote" +class = "A short-lived branch sleepwalking into a long-lived one." + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run branch-age-record" + +[[verdict]] +id = "branch list empty" +gloss = "the producer looked and the remote reported no branches at all" +class = "A remote with a trunk cannot report no branches; the listing failed while exiting zero." + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run branch-age-record" + +[[rule]] +id = "branch watch loose" +kind = "policy" +scope = "tree" +module = "policy/branch-age.rego" +severity = "deny" + +[[record]] +record = "branch-age" +writer = "mise run branch-age-record" +"#, + ); + init_repo(&dir); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-qm", "register the module"]); + dir +} + +/// Write the producer's record, as `mise run branch-age-record` would. +fn record(dir: &std::path::Path, lines: &str) { + let written = run_with_stdin(dir, &["record", "named", "branch-age"], lines); + assert!( + written.status.success(), + "the setup write lands: {}", + String::from_utf8_lossy(&written.stderr) + ); +} + +#[test] +fn a_recorded_branch_past_the_threshold_is_reported_through_the_engines_own_projection() { + // THE CASE NO `with input as` CAN REACH, and the one that was false for a + // whole session: the module was registered, its own suite green, and + // `recorder_records` projected no `record named` family at all — so this + // exact tree exited 0 over a 36-day branch (CLOUD-1810). + let dir = repo("stale"); + record(&dir, "ref\tclaude/ancient\t36\nref\tclaude/fresh\t1\n"); + + let decided = run(&dir, &["check"]); + assert_eq!( + decided.status.code(), + Some(2), + "a branch past the threshold decides\n{}", + String::from_utf8_lossy(&decided.stderr) + ); + // BOTH STREAMS, because which one a finding lands on is the output contract's + // business and not this case's: what is asserted is that the pointer reaches + // the reader at all. + let said = format!( + "{}{}", + String::from_utf8_lossy(&decided.stdout), + String::from_utf8_lossy(&decided.stderr) + ); + assert!( + said.contains("claude/ancient"), + "the finding names the branch to delete\n{said}" + ); + assert!( + !said.contains("claude/fresh"), + "and says nothing about one inside the threshold\n{said}" + ); +} + +#[test] +fn an_absent_record_says_nothing_rather_than_passing() { + // COULD NOT LOOK IS NOT A PASS, and on this surface it is also not a + // refusal. The producer writes nothing when it cannot reach the forge, so a + // module that refused here would refuse every checkout with no credential — + // and one that PASSED would be the silent green the whole port exists to + // remove. The discrimination is only visible with the record absent. + let dir = repo("absent"); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "an absent record is could-not-look\n{}", + String::from_utf8_lossy(&quiet.stderr) + ); +} + +#[test] +fn a_present_record_naming_no_branch_is_refused_rather_than_read_as_clean() { + // PRESENT-AND-EMPTY IS THE THIRD STATE, and collapsing it into the second is + // where the whole gate evaporates: a remote reporting no branches "cannot be + // true of a repository with a trunk", so the honest reading is that the + // listing failed while exiting zero. + let dir = repo("empty"); + record(&dir, "merged\tclaude/gone\n"); + + let refused = run(&dir, &["check"]); + assert_eq!( + refused.status.code(), + Some(2), + "a record that looked and found no branch at all is refused\n{}", + String::from_utf8_lossy(&refused.stderr) + ); +} + +#[test] +fn a_reused_name_still_on_the_remote_is_reported_and_one_already_deleted_is_not() { + // THE SURVIVOR CONJUNCT, as an exit code rather than as an argument. Merged + // pull requests are immutable, so a name that headed two of them heads two + // forever; without intersecting against what the remote still carries this + // would be an alarm no action could clear, which is the shape that gets a + // gate switched off. + let live = repo("reused-live"); + record( + &live, + "ref\tclaude/reused\t1\nmerged\tclaude/reused\nmerged\tclaude/reused\n", + ); + let reported = run(&live, &["check"]); + assert_eq!( + reported.status.code(), + Some(2), + "a reused name still on the remote is reported\n{}", + String::from_utf8_lossy(&reported.stderr) + ); + + let gone = repo("reused-gone"); + record( + &gone, + "ref\tclaude/other\t1\nmerged\tclaude/deleted\nmerged\tclaude/deleted\n", + ); + let quiet = run(&gone, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "and the same history with the branch deleted is clean — the remedy worked\n{}", + String::from_utf8_lossy(&quiet.stderr) + ); +} diff --git a/crates/batten/tests/it/main.rs b/crates/batten/tests/it/main.rs index 1190a6ef8..0f9405de8 100644 --- a/crates/batten/tests/it/main.rs +++ b/crates/batten/tests/it/main.rs @@ -61,6 +61,7 @@ mod board_receipts; mod board_record; mod board_state_claim; mod bot_lane; +mod branch_age; mod bundle; mod bypass_precondition; mod bypass_scrub; diff --git a/mise-tasks/branch-age-check.sh b/mise-tasks/branch-age-check.sh deleted file mode 100755 index e0c998ccb..000000000 --- a/mise-tasks/branch-age-check.sh +++ /dev/null @@ -1,169 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Gate: no remote branch outlives its story — none older than BRANCH_MAX_AGE_DAYS, and no name heading more than one merged PR" -# -# CLOUD-349. Trunk-based development says a review branch "can (and should) be -# deleted after the code review is complete and be very short-lived", and names -# the hazard this measures: a short-lived feature branch "sleepwalking into a -# long-lived feature branch". Its own words on the tooling — "You cannot with -# tools today, but it would be cool if you could have a ticking clock or count -# down on those branches at creation to enforce its 'temporary' intention." -# -# This is that clock, after the fact. Branch lifetime was the one quantity in -# this repository's whole trunk-based apparatus with no number attached: build -# time has a p95 and a budget grammar, the landing loop has LAND_MAX_LAPS, the -# instruction set has a token budget, and a branch had nothing. Measured on -# 2026-08-11, before `land` learned to delete: 23 remote branches, ten of them -# `release-plz-*` from five days earlier. -# -# TWO PROPERTIES, because staleness is the smaller half: -# -# stale a branch whose tip is older than BRANCH_MAX_AGE_DAYS (2, which is -# the source's "a couple of days"). The ordinary leftover. -# reused a branch name that is the head of more than one MERGED pull -# request AND still exists on the remote. The second conjunct is -# what keeps this a gate rather than a permanent alarm: merged PRs -# are immutable, so an unintersected count could never be cleared by -# any action. This is the one the per-PR lifetime metric cannot see: -# `claude/phase-3-sequential-landing-h26kx0` headed eight consecutive -# PRs, each landing inside an hour, while the branch itself lived for -# hours across unrelated stories. Reuse is also the stale-tracking-ref -# deadlock CLOUD-345 records. -# -# NOT IN THE hk GATE, deliberately, and this is the same split `lock-complete` -# and `lock-currency` made: the answer here is a property of the WORLD (the -# remote's refs), not of the commit being judged. On the landing path it would -# fail whichever PR happened to be in flight over somebody else's leftover -# branch — a finding that PR did not cause and cannot fix. So it runs on a -# clock (.github/workflows/branch-hygiene.yml) and by hand. -# -# Both readings are injectable so the suite runs offline and with no `gh`: -# BRANCH_AGE_REFS file of `` lines -# BRANCH_AGE_PRS file of merged pull requests' head branch names, one a line -# BRANCH_AGE_TODAY today as YYYY-MM-DD -# -# Exit 0 clean / 1 over threshold / 2 could-not-look — never a silent pass, per -# the rule that a gate which cannot run must exit non-zero and leave no receipt. -# A gate listed in $MUTANT_GATES with no row here fails `mise run mutant`. -#MUTANT stale-branch-passes|s/^\texit 1$/\texit 0/|past the threshold is refused - -set -uo pipefail - -max_age="${BRANCH_MAX_AGE_DAYS:-2}" -trunk="${BRANCH_AGE_TRUNK:-main}" - -cannot_look() { - echo "::error:: branch-age-check: $1" >&2 - exit 2 -} - -today="${BRANCH_AGE_TODAY:-$(date -u +%Y-%m-%d)}" -case "$today" in -[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]) ;; -*) cannot_look "cannot read today's date ('$today')" ;; -esac - -# --- the two readings --------------------------------------------------------- -# -# One GraphQL call for the refs rather than a REST branch list plus one commit -# lookup per branch: the tip date is not on the REST branch object, and 23 -# follow-up calls to learn it is the shape `mem:github-rest-etiquette` warns -# about. -if [[ -n "${BRANCH_AGE_REFS:-}" ]]; then - refs=$(cat "$BRANCH_AGE_REFS" 2>/dev/null) || - cannot_look "cannot read BRANCH_AGE_REFS ($BRANCH_AGE_REFS)" -else - slug=$(gh repo view --json owner,name --jq '.owner.login + " " + .name' 2>/dev/null) || - cannot_look "cannot resolve the repository — is \`gh\` authenticated?" - # shellcheck disable=SC2016 # $owner/$name are GraphQL variables, not shell - refs=$(gh api graphql \ - -f query='query($owner:String!,$name:String!){ - repository(owner:$owner,name:$name){ - refs(refPrefix:"refs/heads/",first:100){ - nodes{ name target{ ... on Commit { committedDate } } } - } - } - }' \ - -F owner="${slug% *}" -F name="${slug#* }" \ - --jq '.data.repository.refs.nodes[] | [.name, .target.committedDate] | @tsv' 2>/dev/null) || - cannot_look "cannot list the remote's branches" -fi -[[ -n "$refs" ]] || cannot_look "the remote reports no branches at all, which cannot be true of a repository with a trunk" - -if [[ -n "${BRANCH_AGE_PRS:-}" ]]; then - prs=$(cat "$BRANCH_AGE_PRS" 2>/dev/null) || - cannot_look "cannot read BRANCH_AGE_PRS ($BRANCH_AGE_PRS)" -else - prs=$(gh pr list --state merged --limit 200 --json headRefName --jq '.[].headRefName' 2>/dev/null) || - cannot_look "cannot list merged pull requests" -fi - -# --- the arithmetic ----------------------------------------------------------- -# -# Days-since-epoch by the civil-calendar formula, in integer arithmetic only. -# `mktime` would be shorter and is a gawk extension mawk does not carry — the -# same portability trap `awk-regex-check` exists for, and the reason CLOUD-282 -# lists two GNU-only gates as macOS blockers. This runs identically on both. -findings=$(printf '%s\n' "$refs" | awk -F'\t' -v today="$today" -v trunk="$trunk" -v max="$max_age" ' - function days(y, m, d, era, yoe, doy, doe) { - if (m <= 2) y -= 1 - era = int((y >= 0 ? y : y - 399) / 400) - yoe = y - era * 400 - doy = int((153 * (m + (m > 2 ? -3 : 9)) + 2) / 5) + d - 1 - doe = yoe * 365 + int(yoe / 4) - int(yoe / 100) + doy - return era * 146097 + doe - 719468 - } - function daynum(iso, p, s) { - # Shape first, then split. A field count alone is not a validation: - # `not-a-date` splits into three parts on the hyphen just as - # `2026-08-11` does, and the arithmetic then answers 740236 days - # instead of admitting it cannot read the value. - s = substr(iso, 1, 10) - if (s !~ /^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]$/) return "NaN" - split(s, p, "-") - return days(p[1] + 0, p[2] + 0, p[3] + 0) - } - BEGIN { now = daynum(today) } - $1 == "" { next } - $1 == trunk { next } - { - d = daynum($2) - if (d == "NaN") { print "unreadable\t" $1 "\t" $2; next } - age = now - d - if (age > max) print "stale\t" $1 "\t" age - } -') - -# Reuse counts only while the name STILL EXISTS on the remote, and that -# intersection is the whole difference between a gate and a permanent alarm. -# Merged PRs are immutable: a name that headed three of them and was then -# deleted stays in the PR list forever, so an unintersected count can never be -# cleared by any action anyone can take. It would have made this gate red on its -# first scheduled run and red on every run after it — and a gate that cannot -# reach green stops being read, then gets switched off (the desensitising shape -# CLOUD-245 names). Intersected, the count means "a live branch you can delete", -# which is a thing a person can finish. -live=$(printf '%s\n' "$refs" | cut -f1 | sed '/^$/d' | sort -u) -reused=$(printf '%s\n' "$prs" | sed '/^$/d' | sort | uniq -c | - awk -v trunk="$trunk" '$1 > 1 && $2 != trunk { print $2 "\t" $1 }' | - while IFS=$'\t' read -r name count; do - # A here-string, not a pipe: `grep -q` exits at the first match and a - # still-writing producer would take SIGPIPE, which pipefail promotes - # into a failure ON A MATCH (`pipefail-grep-check`). - grep -qxF "$name" <<<"$live" && printf 'reused\t%s\t%s\n' "$name" "$count" - done) - -# --- the verdict -------------------------------------------------------------- -# -# Pointer-only per non-negotiable rule 4: a branch name, a rule id and a count. -# Never a commit message, never a diff — a branch name is already the most a -# reader needs to act. -all=$(printf '%s\n%s\n' "$findings" "$reused" | sed '/^$/d') -if [[ -n "$all" ]]; then - printf '%s\n' "$all" >&2 - stale_n=$(printf '%s\n' "$all" | grep -c '^stale' || true) - reused_n=$(printf '%s\n' "$all" | grep -c '^reused' || true) - echo "::error:: branch-age-check: $stale_n branch(es) older than ${max_age}d, $reused_n live branch(es) reused across merged PRs. Delete what has landed; a branch that outlives its story is a long-lived branch." >&2 - exit 1 -fi - -echo "branch-age-check: no branch older than ${max_age}d and no live branch reused across merged PRs" diff --git a/mise.toml b/mise.toml index 74f85e14c..8bc962593 100644 --- a/mise.toml +++ b/mise.toml @@ -617,7 +617,7 @@ CI_FANIN_WORKFLOW = ".github/workflows/ci.yml" BATS_TEST_TIMEOUT = "300" REGORUS_OPA_COMPLIANCE = "1.2.0" REGORUS_OPA_COMPLIANCE_FOR = "0.11" -MUTANT_GATES = "mise,attestation-check,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age-check,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure-check,evaluator-io-check,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence-assert,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link-check,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict-assert,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus-check,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" +MUTANT_GATES = "mise,attestation-check,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure-check,evaluator-io-check,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence-assert,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link-check,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict-assert,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus-check,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" # --- GitHub reachability behind an egress proxy (Claude Code web sandbox etc.) --- # mise resolves every tool's release through GitHub's *API* host, api.github.com. diff --git a/policy/branch-age.rego b/policy/branch-age.rego index ea710caa9..d3d8adf37 100644 --- a/policy/branch-age.rego +++ b/policy/branch-age.rego @@ -39,7 +39,10 @@ # producer looked and the remote reported no branches, which the retired program # refused outright as impossible of a repository with a trunk. That refusal is # kept, because a silent pass there is the shape where the whole gate evaporates. -#MUTANT-EXEMPT CLOUD-1717|the compiled-binary tier this module's mutations would redden, `crates/batten/tests/it/branch_age.rs`, is not written yet: the module does not decide, because `recorder_records` projects no `record named` family and `input.tree.records["branch-age"]` never reaches it. The four mutations are drafted in this file's history and go back with the tier, in the same delta that retires `mise-tasks/branch-age-check.sh` — the program stays until then, so nothing is uncovered that was covered before. +#MUTANT-SUITE crates/batten/tests/it/branch_age.rs +#MUTANT age-unread|s@^\tage > threshold$@\tfalse@|a_recorded_branch_past_the_threshold_is_reported_through_the_engines_own_projection +#MUTANT reuse-needs-no-survivor|s@^\tcount(\[name | some name in on_remote; name == reused\]) > 0$@\ttrue@|a_reused_name_still_on_the_remote_is_reported_and_one_already_deleted_is_not +#MUTANT empty-remote-passes|s@^\tcount(refs) == 0$@\tfalse@|a_present_record_naming_no_branch_is_refused_rather_than_read_as_clean # METADATA # description: | diff --git a/tests/branch-age-check.bats b/tests/branch-age-check.bats deleted file mode 100644 index b2ea4d201..000000000 --- a/tests/branch-age-check.bats +++ /dev/null @@ -1,145 +0,0 @@ -#!/usr/bin/env bats -# subject: mise-tasks/branch-age-check.sh -# The ticking clock on a branch's life (CLOUD-349). -# -# Both readings are injected, so every case runs offline and with no `gh`: the -# gate's job is the arithmetic and the verdict, and a suite that needed a remote -# would be testing the remote. -# -# The discipline is the one the other gates hold: a clean tree passes, each -# finding fires on its own and is NAMED, the trunk is never counted, and an -# unreadable reading is exit 2 rather than a pass. That last one is the whole -# reason this file exists — a gate that cannot look and says nothing reports -# hygiene it never measured. - -setup() { - GATE="$BATS_TEST_DIRNAME/../mise-tasks/branch-age-check.sh" - REFS="$BATS_TEST_TMPDIR/refs.tsv" - PRS="$BATS_TEST_TMPDIR/prs.txt" - export BRANCH_AGE_REFS="$REFS" BRANCH_AGE_PRS="$PRS" BRANCH_AGE_TODAY=2026-08-11 - printf 'main\t2026-08-11T10:00:00Z\n' >"$REFS" - : >"$PRS" -} - -# ``, the shape the GraphQL reading emits. -ref() { printf '%s\t%s\n' "$1" "$2" >>"$REFS"; } - -@test "a remote carrying only fresh branches passes" { - ref feat 2026-08-10T09:00:00Z - printf 'feat\n' >"$PRS" - run "$GATE" - [ "$status" -eq 0 ] - [[ "$output" == *"no branch older than 2d"* ]] -} - -@test "a branch past the threshold is refused, and named with its age" { - # The ordinary leftover: ten `release-plz-*` branches sat five days stale - # when this was measured. - ref stale-one 2026-08-01T09:00:00Z - run "$GATE" - [ "$status" -eq 1 ] - [[ "$output" == *"stale stale-one 10"* ]] - [[ "$output" == *"1 branch(es) older than 2d"* ]] -} - -@test "the threshold is a boundary, not a suggestion" { - # Exactly BRANCH_MAX_AGE_DAYS old is still short-lived — the source says "a - # couple of days", and a gate that fired AT two days would refuse the branch - # it is meant to permit. - ref two-days 2026-08-09T09:00:00Z - ref three-days 2026-08-08T09:00:00Z - run "$GATE" - [ "$status" -eq 1 ] - [[ "$output" != *"two-days"* ]] - [[ "$output" == *"stale three-days 3"* ]] -} - -@test "a name heading more than one merged PR is refused, and counted" { - # The half the per-PR lifetime metric cannot see: eight consecutive PRs on - # one branch name all landed within the hour, while the branch itself - # outlived every one of their stories. - ref busy 2026-08-11T09:00:00Z - printf 'busy\nbusy\nbusy\nother\n' >"$PRS" - run "$GATE" - [ "$status" -eq 1 ] - [[ "$output" == *"reused busy 3"* ]] - [[ "$output" == *"1 live branch(es) reused across merged PRs"* ]] - [[ "$output" != *"other"* ]] -} - -@test "a reused name whose branch is already gone is not counted" { - # THE PROPERTY THAT KEEPS THIS A GATE. Merged PRs are immutable, so a name - # that headed three of them and was then deleted stays in the PR list - # forever. Counting it would make the gate red on its first scheduled run - # and red on every run after it, with no action anyone could take to clear - # it — and a gate that cannot reach green stops being read, then gets - # switched off. Only a name that still exists is a name someone can delete. - printf 'ghost\nghost\nghost\n' >"$PRS" - run "$GATE" - [ "$status" -eq 0 ] - [[ "$output" != *"ghost"* ]] -} - -@test "a clean remote reaches green, which is the state the gate must be able to reach" { - # Asserted directly rather than inferred from the cases above: the whole - # point of the intersection is that deleting the offending branches is - # sufficient. If nothing is stale and nothing live is reused, green. - ref fresh 2026-08-10T09:00:00Z - printf 'gone\ngone\nfresh\n' >"$PRS" - run "$GATE" - [ "$status" -eq 0 ] -} - -@test "the trunk is never counted, however old or however many PRs it heads" { - # `main` is the one long-lived branch by design. Counting it would make the - # gate permanently red and therefore permanently ignored. - printf 'main\t2020-01-01T00:00:00Z\n' >"$REFS" - printf 'main\nmain\n' >"$PRS" - run "$GATE" - [ "$status" -eq 0 ] -} - -@test "a custom threshold is honoured in both the verdict and the message" { - ref four-days 2026-08-07T09:00:00Z - BRANCH_MAX_AGE_DAYS=7 run "$GATE" - [ "$status" -eq 0 ] - [[ "$output" == *"older than 7d"* ]] -} - -@test "an unreadable refs reading is exit 2, not a pass" { - # The failure this gate exists to not have: a hygiene check that could not - # look reports hygiene it never measured. - BRANCH_AGE_REFS="$BATS_TEST_TMPDIR/nope" run "$GATE" - [ "$status" -eq 2 ] - [[ "$output" == *"cannot read BRANCH_AGE_REFS"* ]] -} - -@test "an unreadable PR reading is exit 2, not a pass" { - BRANCH_AGE_PRS="$BATS_TEST_TMPDIR/nope" run "$GATE" - [ "$status" -eq 2 ] - [[ "$output" == *"cannot read BRANCH_AGE_PRS"* ]] -} - -@test "a remote reporting no branches at all is exit 2, not a pass" { - # A repository with a trunk has at least one branch, so an empty answer is - # a broken reading rather than a spotless remote. - : >"$REFS" - run "$GATE" - [ "$status" -eq 2 ] - [[ "$output" == *"no branches at all"* ]] -} - -@test "an unparseable tip date is reported rather than silently skipped" { - # A date the arithmetic cannot read is not a fresh branch. Counting it as - # one is how a gate goes quietly blind to whatever changes the format. - ref weird "not-a-date" - run "$GATE" - [ "$status" -eq 1 ] - [[ "$output" == *"unreadable weird"* ]] -} - -@test "a nonsense today is exit 2, not an arithmetic answer" { - BRANCH_AGE_TODAY=yesterday run "$GATE" - [ "$status" -eq 2 ] - [[ "$output" == *"cannot read today's date"* ]] -} From f81f7a8230bb302c9e935d4dabac0aaa1c8c3b03 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Thu, 17 Sep 2026 01:13:12 +0000 Subject: [PATCH 03/32] fix(forge): a skipped check is not a grading, so a draft records nothing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `verify` refused this branch with `forge check red` and `head grade twice` over a commit the forge had never judged. PR #962 is a draft, this repository's contract says CI does not run on a draft, and all twelve of its check-runs therefore conclude `skipped`. The producer recorded that as a verdict and `forge-verdict-required` read `final skipped` as the forge having looked and refused. THE GUARD WAS WRITTEN FOR THE OTHER NON-VERDICT. `map(select(.conclusion != null))` excludes a PENDING run, which is what it was reasoned about — and `skipped` is non-null, so it sailed through as graded. Two states are not a verdict and the filter knew about one. The cost is structural rather than occasional: every pull request starts as a draft here, so the first `verify` after opening one writes a false red and `land` can never take its first lap. Measured 2026-09-17 on #962 — twelve checks skipped, `.git/batten-forge/488ac399…` spelling `final skipped`, `verify` refusing both forge rows, the branch unable to move. With the guard narrowed the producer writes nothing and the record stays absent, which is could-not-look. This is CLOUD-1707's own defect one layer in. That row existed because a `deny` gate had no producer and so decided nothing; this is its producer turning "CI has not run" into "CI refused" — the same collapse of could-not-look into a verdict, in the opposite direction. Filtered at the parse rather than at the fan-in test alone: a skipped job in a real run rendered no verdict either, so recording ` skipped` would say that job refused. Refs: CLOUD-1707 Admits: bb0e61c9cd57e1f69c063baa9c8eeabb0a9b3b707eda60123d558cb7e1525078 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: mise.toml Admits-anchor: call:488ac399c4402bded559276685e244bfe6c077a5 Admits-epoch: ba0d29a9bb89a9544da54e66e06e4e190c8cbba7ee9caac53c1c88f116b2065e Admits-author: alec@wenzowski.com Admits-prev: ff70d7405fd339ebe4ddd020a29e65f4cd18ecdc8dc3731c955230ddcf454500 Admits-answer-lost: Every pull request starts as a draft by this repository's own contract, so the first `verify` after opening one writes a false red and `land` can never take a lap. Measured now on PR #962: all twelve checks `skipped`, `.git/batten-forge/488ac399…` records `final skipped`, `verify` refuses `forge check red` plus `head grade twice`, and the branch cannot land. This is a could-not-look recorded as a verdict — the exact defect CLOUD-1707 existed to remove, reintroduced by its own producer's guard. Admits-answer-precondition: The producer's fan-in guard is a `jq` filter inside `[tasks.record-verdicts]`, and a mise task body lives on exactly one surface. The defect is in that filter: `map(select(.conclusion != null))` admits `skipped`, so a draft PR — where CI does not run and every check-run concludes `skipped` — is read as GRADED and the record spells `final skipped`, which `forge-verdict-required` reads as a refusal. There is no other file the filter can be corrected in. Admits-answer-rejected-route: config read first — I read the guard, the record it wrote and the live check-runs for the head, and that reading is what identified the wrong conjunct rather than what avoided the edit: a filter admitting a value it should exclude cannot be corrected by reading it. patch run first is a commit-message route and does not apply to a task-body fix. --- mise.toml | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/mise.toml b/mise.toml index 8bc962593..0aaaf80f9 100644 --- a/mise.toml +++ b/mise.toml @@ -2081,8 +2081,26 @@ if [ -n "$forge_sha" ] && [ -n "$forge_fanin" ]; then # last would win by listing order rather than by recency. `checks-green` draws # the same distinction with `started_at` and `id`; this is that reduction, # spelled once, at the producer. + # A `skipped` CONCLUSION IS NOT A GRADING, and admitting one is how this + # producer reintroduced the defect it exists to remove (CLOUD-1717). The + # filter read `.conclusion != null`, which was written for the PENDING case + # — and `skipped` is non-null. Every pull request starts as a draft by this + # repository's own contract, CI does not run on one, and a draft's twelve + # check-runs all conclude `skipped`. So the fan-in read as GRADED, the record + # spelled `final skipped`, and `forge-verdict-required` read that as the + # forge having looked and refused. + # + # Measured 2026-09-17 on PR #962: twelve checks skipped, the record written, + # `verify` refusing `forge check red` and `head grade twice`, and the branch + # unable to take its first lap — from the state the contract says every PR + # begins in. A could-not-look recorded as a verdict, which is precisely what + # CLOUD-1707 was about, one layer in. + # + # Filtered here rather than at the fan-in test alone, because a skipped job in + # a real run rendered no verdict either: recording ` skipped` would say + # that job refused. if forge_runs=$(gh api "repos/$forge_repo/commits/$forge_sha/check-runs?per_page=100" \ - --jq '.check_runs | map(select(.conclusion != null)) | group_by(.name) | map(max_by([(.started_at // ""), (.id // 0)])) | .[] | "\(.name)\t\(.conclusion)"' 2>/dev/null); then + --jq '.check_runs | map(select(.conclusion != null and .conclusion != "skipped")) | group_by(.name) | map(max_by([(.started_at // ""), (.id // 0)])) | .[] | "\(.name)\t\(.conclusion)"' 2>/dev/null); then # The fan-in decides whether ANY record is written; the name filter then # decides what the record may spell. forge_graded=$(printf '%s\n' "$forge_runs" | From 0804a3473e1ea731aeb1e4e0d997dcb9bd198d59 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Thu, 17 Sep 2026 01:51:50 +0000 Subject: [PATCH 04/32] refactor(policy): extract source_key, and move the record table's validator MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both functions CLOUD-1810 added a line to sat exactly at the hundred-line cap, so both went one over. Compressing my own additions onto single lines cleared it locally and did not survive `rustfmt`, which split them back — so this is the structural fix rather than a second attempt at squeezing. `policy::load` gives up `source_key`, which reads better as a named function than as an inline `.or().or().ok_or_else()` chain: it answers one question, "which source does this policy row name", and the reason it refuses at all is the reason it is worth naming — `validate` already refuses a row naming none of the three, and this is the LOCATED restatement so a caller reaching `load` directly cannot get a silent skip. `validate_tables` gives up the record table's validator to `validate_sections`. `config.rs`'s own census scans both bodies, so the call site is still found and the table is still classified — which is the property that matters, because a table nothing validates is a refusal that cannot fire (CLOUD-253). The placement is forced by the cap rather than chosen, and the comment there says so. Neither cap is raised and no expectation is added. A function at its limit is a signal to extract, which is what the limit is for. Refs: CLOUD-1810 --- crates/batten/src/config.rs | 17 ++++++++++------ crates/batten/src/policy.rs | 40 +++++++++++++++++++++++-------------- 2 files changed, 36 insertions(+), 21 deletions(-) diff --git a/crates/batten/src/config.rs b/crates/batten/src/config.rs index edd2ad56b..1e681c53b 100644 --- a/crates/batten/src/config.rs +++ b/crates/batten/src/config.rs @@ -1991,12 +1991,6 @@ fn validate_tables(config: &Config, text: &str, source: &str, grammar: Grammar) .collect(), ), )?; - // The verb-written families, beside the recorder table because the two are - // halves of one question: which record names a module may read (CLOUD-1810). - under( - Native::RecordTableRefused, - crate::record::validate(&config.records), - )?; validate_sections(config) } @@ -2018,6 +2012,17 @@ fn validate_tables(config: &Config, text: &str, source: &str, grammar: Grammar) /// /// As [`validate_tables`]. fn validate_sections(config: &Config) -> Result<()> { + // The verb-written record families (CLOUD-1810). HERE rather than beside the + // recorder table one function up, and the placement is forced rather than + // chosen: `validate_tables` sits exactly at its hundred-line cap, so the call + // had to land in the other half of the loader. `config.rs`'s own census scans + // both bodies, so the call site is still found and the table is still + // classified — which is the property that matters, since a table nothing + // validates is a refusal that cannot fire (CLOUD-253). + under( + Native::RecordTableRefused, + crate::record::validate(&config.records), + )?; // `[budget]` is a table rather than a list, so the census below (which scans // `Vec` fields) does not reach it — but the failure it guards against is // the same one: a table that parses and gates nothing. A `[budget]` header diff --git a/crates/batten/src/policy.rs b/crates/batten/src/policy.rs index 5ee5ff687..6361daa4f 100644 --- a/crates/batten/src/policy.rs +++ b/crates/batten/src/policy.rs @@ -750,6 +750,30 @@ pub enum ModuleChecks { SkipOnHotPath, } +/// Which source a policy row names: its `module`, `bundle` or `preset`. +/// +/// Extracted from [`load`] rather than inlined, and the reason is the same one +/// that keeps it a refusal at all: `validate` already refuses a policy row naming +/// none of the three and one naming more than one, so this is the LOCATED +/// restatement — a caller reaching `load` directly cannot get a silent skip +/// instead of a refusal. +/// +/// # Errors +/// +/// A [`UsageError`] (exit `1`) for a policy row naming none of the three. +fn source_key(rule: &Rule) -> Result<&str> { + rule.module + .as_deref() + .or(rule.bundle.as_deref()) + .or(rule.preset.as_deref()) + .ok_or_else(|| { + UsageError::raise(format!( + "rule `{}` is a policy row naming neither `module`, `bundle` nor `preset`", + rule.id + )) + }) +} + /// Load, compile and smoke-test every module the rule set registers. /// /// Boundary I/O, called once per process from the config resolution path — never @@ -810,21 +834,7 @@ pub fn load( // difference between a pointer and a complaint. let mut ids: BTreeMap = BTreeMap::new(); for rule in rules.iter().filter(|r| r.kind == RuleKind::Policy) { - // `validate` already refuses a policy row naming none of the three - // sources, and one naming more than one; this is the located - // restatement, so a caller reaching `load` directly cannot get a silent - // skip instead of a refusal. - let source_key = rule - .module - .as_deref() - .or(rule.bundle.as_deref()) - .or(rule.preset.as_deref()) - .ok_or_else(|| { - UsageError::raise(format!( - "rule `{}` is a policy row naming neither `module`, `bundle` nor `preset`", - rule.id - )) - })?; + let source_key = source_key(rule)?; // Two rows naming one source AT ONE SCOPE is dead config: the second // enablement decides nothing the first did not, and "which one denied // me" is not a question a reviewer should have to answer. From 2d929ee2278ebc3bedb65084614b2b93655f0479 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Thu, 17 Sep 2026 04:42:21 +0000 Subject: [PATCH 05/32] refactor(gate)!: retire attestation-check and timeout-drift onto the engine MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The second and third of CLOUD-1707's nine, both taking the split `branch-age-check` established: the module gets every decision, the producer keeps every step section 5 forbids inside `check`. ATTESTATION — ONE DISTINCTION IS THE WHOLE GATE. `gh attestation verify` exits 1 both when an artifact has no provenance and when the platform never offered any, and those are opposite facts: the first is a release to fix, the second a plan feature this private repository does not have. The control that separates them is the attestations endpoint's own status code, so `[tasks.attestation-record]` probes with an all-zeros digest and records the posture beside each archive's verdict. A `posture 404` record fires nothing — `release-artifacts.yml` publishes unattested BY DESIGN until the repo is public (CLOUD-585) — while still being a READING, which is why posture is recorded rather than inferred from an empty archive list. The verifier, the download and the unpack are processes, and `check` is `read` and cannot start one. So the spawn stayed out and only the adjudication moved. THE `deny` ROW SURVIVES, REPOINTED. `release check unread` ran `attestation-check --precondition`; it now runs `[tasks.attestation-record] --precondition` — the same one fact, decided from the same local reading, at the same severity. Deleting the row would have removed a gate and owed a weakening; this changes only which program carries it. Its glob follows the check to `mise.toml`, which is both provably covered by hk's list and where the body now lives, keeping the trigger and the mechanism one object. TIMEOUT-DRIFT — THE POSTURE IS PRESERVED BY SEVERITY. The retired program reported and never blocked: it failed only its own scheduled run, filed no issue, posted no comment, because a budget that no longer matches reality means nothing is broken and no branch is at fault. `severity = "warn"` is that on the engine's contract, and a case asserts it — the same record that fires under `--fail-on-warning` leaves a plain `check` green. Its measurement cannot move in at all: a p95 over run durations is arithmetic on instants, and the engine calls no clock on any evaluation path. The producer subtracts the instants, pools matrix legs into one distribution, and records a row per job; the classification into loose/tight/grandfathered/unmeasurable is the module's. BOTH EXIT CONTRACTS CHANGED, and nine ledger arms ride on it. The shell ran `0/1/2` with `2` as could-not-look; the engine runs `0/1/2/3` where `2` is a FINDING. Carrying the shell's spelling over would have made every could-not-look a violation — so each of those arms is now the producer refusing at write time, loudly, while its author is watching, and writing nothing. An absent record is the module's silence. The retired `timeout-drift` gives the reason that carries: reporting a healthy budget as drifted on a network blip is what gets a scheduled gate switched off. Two gates caught real defects in this delta rather than formalities. The ledger's first draft used case titles I had written rather than the dying suites' own, and `bats count dropped` named all five. And mise runs a task body under `sh` unless it declares otherwise, so both producers' `set -euo pipefail` died on `Illegal option -o pipefail` — which the repointed `release check unread` row reported, correctly, as a check that could not run. 84 programs to 82. Refs: CLOUD-1717 Admits: 3527f63c46b336594562519ea8fce553d6d4b1ac890afad8a4443baa051812b5 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: batten.toml Admits-anchor: call:384d6a955f75d33fd4af6d408a89e8be603c3f20 Admits-epoch: e8acee7f2cdd47739fc37d4cfaf4cad3e2a25fa71c089d7b7fa878bcb05a7bc5 Admits-author: alec@wenzowski.com Admits-prev: 53414e5287cdb5b4c82bda91f7a939f3b1e2168a9f33ea61b8c47de603c94b15 Admits-answer-lost: Both ported modules sit in the tree deciding nothing — unregistered, so `batten check` never loads them, and their families unprojected, so the records their producers write are invisible. The two retirements they exist for cannot proceed, and `mise-tasks/attestation-check.sh` and `mise-tasks/timeout-drift.sh` stay in the corpus. Admits-answer-precondition: A policy module is registered by a `[[rule]]` row and by nothing else; the verdict strings it emits must exist as `[[verdict]]` rows or the engine refuses the module at load; and a verb-written record family reaches no module until a `[[record]]` row declares it. `policy/attestation.rego` and `policy/timeout-drift.rego` are new modules with new verdicts over new families, so eleven rows have to be written into batten.toml directly. There is no other surface any of them lives on, and they land in one diff beside the modules they register. Admits-answer-rejected-route: config read first — I read the `branch watch loose` registration this campaign already landed and copied its shape for both, and that reading is what produced the rows rather than what avoided them: registration is an addition and cannot be satisfied by reading. patch run first is a commit-message route and does not apply to a config addition. Admits: a1cbd3a670ac38e341ff70fa991ce1d8e59ab0a4a00b5a4b372683920cec67a2 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: mise.toml Admits-anchor: call:384d6a955f75d33fd4af6d408a89e8be603c3f20 Admits-epoch: 7197ce2c3bafb81890a303c42f62505d47b9c00e2bab6094c6bf864ca35963ee Admits-author: alec@wenzowski.com Admits-prev: bb0e61c9cd57e1f69c063baa9c8eeabb0a9b3b707eda60123d558cb7e1525078 Admits-answer-lost: `policy/attestation.rego` and `policy/timeout-drift.rego` decide over stores nothing writes, which is exactly the could-not-look-reads-as-pass shape this campaign exists to remove. Both retirements stall and `mise-tasks/attestation-check.sh` and `mise-tasks/timeout-drift.sh` stay in the corpus. Admits-answer-precondition: A mise task body lives on exactly one surface. `[tasks.attestation-record]` and `[tasks.timeout-drift-record]` are the producer halves of two ports: they make the forge reads, the download, the unpack, the `gh attestation verify` spawn and the p95 arithmetic over instants — every one of which section 5 keeps outside `check`, and the arithmetic additionally because `Fact::Instant` projects `null` to every module. There is no other file a task body can be written in, and each lands beside the module it feeds. Admits-answer-rejected-route: config read first — I read `[tasks.branch-age-record]`, the producer this campaign already landed, and matched its shape for both; the reading is what made them correct rather than what made them unnecessary, because a producer that does not exist cannot be read into existence. patch run first is a commit-message route and does not apply to a task addition. Weakens: rule-predicate-changed rule[release check unread].glob Weakens: rule-predicate-changed rule[release check unread].check --- batten.toml | 175 ++++++++++++++++- bench/gates/RESULTS.md | 1 - bench/suites/RESULTS.md | 2 - crates/batten/tests/it/attestation.rs | 249 ++++++++++++++++++++++++ crates/batten/tests/it/main.rs | 2 + crates/batten/tests/it/timeout_drift.rs | 236 ++++++++++++++++++++++ mise-tasks/attestation-check.sh | 220 --------------------- mise-tasks/timeout-drift.sh | 191 ------------------ mise.toml | 201 ++++++++++++++++++- policy/attestation.rego | 193 ++++++++++++++++++ policy/timeout-drift.rego | 237 ++++++++++++++++++++++ tests/attestation-check.bats | 225 --------------------- tests/timeout-drift.bats | 175 ----------------- 13 files changed, 1287 insertions(+), 820 deletions(-) create mode 100644 crates/batten/tests/it/attestation.rs create mode 100644 crates/batten/tests/it/timeout_drift.rs delete mode 100755 mise-tasks/attestation-check.sh delete mode 100755 mise-tasks/timeout-drift.sh create mode 100644 policy/attestation.rego create mode 100644 policy/timeout-drift.rego delete mode 100644 tests/attestation-check.bats delete mode 100644 tests/timeout-drift.bats diff --git a/batten.toml b/batten.toml index dff02ec09..ef14d8452 100644 --- a/batten.toml +++ b/batten.toml @@ -2134,6 +2134,22 @@ regex = '^[0-9]+$' record = "branch-age" writer = "mise run branch-age-record" +# CLOUD-583's probe and verifier, ported off `attestation-check.sh` under +# CLOUD-1717. The producer makes the posture probe, downloads the release's +# archives, unpacks each one and runs `gh attestation verify` over the BINARY +# inside — every step of which section 5 keeps outside `check`. +[[record]] +record = "attestation" +writer = "mise run attestation-record" + +# CLOUD-266's other half, ported off `timeout-drift.sh` under CLOUD-1717. The +# producer reads successful runs from the Actions API, subtracts their instants +# and computes a p95 per job; the engine calls no clock, so the arithmetic cannot +# move in and only the classification does. +[[record]] +record = "timeout-drift" +writer = "mise run timeout-drift-record" + [[pattern]] id = "closed-issue-status" regex = '^(done|canceled|duplicate)$' @@ -5537,8 +5553,8 @@ no_fix_reason = "the missing fields do not exist in a cargo lockfile, so no comm # Release provenance, adopted as `gh attestation verify` (CLOUD-583, CLOUD-279 # verdict 1 — `slsa-verifier` is redundant against the same absent provenance and -# `cosign` has no signing identity here). `mise-tasks/attestation-check.sh` carries -# the design; what belongs here is why this is ONE row and which half it runs. +# `cosign` has no signing identity here). `policy/attestation.rego` carries the +# design; what belongs here is why this is ONE row and which half it runs. # # THE OFFLINE HALF, and that is the whole of the wiring decision. Whether a # published release is attested changes with no diff, so it is a property of the @@ -5569,11 +5585,23 @@ no_fix_reason = "the missing fields do not exist in a cargo lockfile, so no comm # had anything to do with attestation. CLOUD-614 carries the general defect; naming # the file the row exists to run is what makes the trigger and the mechanism the # same object rather than a coincidence. +# REPOINTED, NOT REMOVED (CLOUD-1717). `mise-tasks/attestation-check.sh` is +# retired, and its precondition mode survives as `[tasks.attestation-record] +# --precondition`: the same one fact, decided from the same local reading, still +# at `deny` on every gate invocation. Retiring the row instead would have removed +# a gate and owed a weakening; this changes only which program carries it. +# +# THE GLOB FOLLOWS THE CHECK, which is this row's own discipline one paragraph +# down: it must name a path `batten-glob-check` can prove covered, and it must not +# match inside a fixture tree, because a `mise run` check resolves only where a +# mise.toml is findable. `mise.toml` is both — it is in hk's `batten-check` list, +# and it is where the task body now lives, so the trigger and the mechanism stay +# the same object. [[rule]] id = "release check unread" kind = "command" -glob = "mise-tasks/attestation-check.sh" -check = "mise run attestation-check --precondition" +glob = "mise.toml" +check = "mise run attestation-record --precondition" severity = "deny" scope = "tree" no_fix_reason = "install the pinned `gh` (`mise install aqua:cli/cli`) or supply GH_TOKEN; neither is a change to this tree's content" @@ -5607,7 +5635,7 @@ no_fix_reason = "install the pinned `gh` (`mise install aqua:cli/cli`) or supply # # `deny` is safe because the check is offline and local: it reads one tracked # file, makes no network call, and needs no credential — so it cannot block on -# ambient environment the way an earlier `attestation-check` did inside fixture +# ambient environment the way an earlier attestation precondition did inside fixture # repositories. The only way it reds is the workflow actually losing its wiring. [[rule]] id = "release wire missing" @@ -6240,6 +6268,34 @@ scope = "tree" module = "policy/branch-age.rego" severity = "warn" +# CLOUD-583, ported off `mise-tasks/attestation-check.sh` under CLOUD-1717. +# +# `deny` BECAUSE THE GAP IS NOT A VERDICT. The module fires nothing while the +# platform offers no attestation — a 404 posture judges no artifact — so the only +# way this reds is a release whose binaries the verifier actually refused where +# attestation IS available. That is a release to fix, which is what `deny` means. +[[rule]] +id = "release grade unsafe" +kind = "policy" +scope = "tree" +module = "policy/attestation.rego" +severity = "deny" + +# CLOUD-266's world half, ported off `mise-tasks/timeout-drift.sh` under +# CLOUD-1717. +# +# `warn` IS THE PORTED POSTURE, not a hedge. The retired program reported and +# never blocked: it failed only its own scheduled run, filed no issue and posted +# no comment, because a budget that no longer matches reality means nothing is +# broken and no branch is at fault. A warn finding is reported without failing the +# run, which is that posture on the engine's contract. +[[rule]] +id = "bound grade other" +kind = "policy" +scope = "tree" +module = "policy/timeout-drift.rego" +severity = "warn" + [[rule]] id = "plan cover partial" kind = "policy" @@ -13014,6 +13070,115 @@ id = "patch run first" kind = "command" target = "git commit -F " +# CLOUD-583's three readings, ported off `attestation-check.sh` under CLOUD-1717. +[[verdict]] +id = "release ship unsafe" +gloss = "a release archive's binary carries no verifiable provenance" +class = """ +`gh attestation verify` refused the executable inside a published archive while \ +the platform DOES offer attestation for this repository. That is a release to \ +fix rather than a gap to report, and the two are told apart by the attestations \ +endpoint's own status code: 200 with an empty array where the feature exists, \ +404 on the resource where it does not. The subject is the BINARY and not the \ +archive, because `release-artifacts.yml` attests the executable deliberately so \ +that repackaging cannot launder the claim. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run attestation-record" + +[[verdict]] +id = "release carry missing" +gloss = "a release archive carries no executable to verify" +class = """ +The archive unpacked and held no `batten` binary, so there was nothing for the \ +verifier to judge. A packaging problem rather than a provenance one, and its own \ +class for that reason: collapsing it into the unverified finding would send a \ +reader after a signing identity when the dist matrix dropped a file. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run attestation-record" + +[[verdict]] +id = "release list empty" +gloss = "the producer looked at a tag and found no archive on it" +class = """ +A green verdict over a release carrying nothing would be about nothing. \ +Present-and-empty and absent are different readings and must not collapse: an \ +absent record is the producer unable to look, where this is the producer having \ +looked and found a tag with no archives. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run attestation-record" + +# CLOUD-266's four readings, ported off `timeout-drift.sh` under CLOUD-1717. +[[verdict]] +id = "bound pin loose" +gloss = "a job's declared timeout sits well above what its measurement justifies" +class = """ +A budget is a ceiling rather than a target, so headroom is correct and a slack \ +allowance keeps this off a job that merely got a little faster. Past that \ +allowance the number has gone slack, which is the direction a report that only \ +complained about tightness would let rot upward forever. Nothing is broken and no \ +branch is at fault: re-derive the number and commit the new comment. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run timeout-drift-record" + +[[verdict]] +id = "bound pin wrong" +gloss = "a job's measurement has outgrown its declared timeout" +class = """ +The p95 of recent successful runs, times the repo-wide headroom multiplier, is \ +already above the committed budget. Raise it before it starts failing healthy \ +runs — this is the direction that turns into a red job nobody caused. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run timeout-drift-record" + +[[verdict]] +id = "bound pin stale" +gloss = "a dated debt entry now has a usable sample and can become a measured budget" +class = """ +The prompt, never the conversion. A bot re-baselining the number it is supposed \ +to defend is the move section 4 forbids outright, so this reports that the debt \ +is now convertible and a deliberate commit does the converting. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run timeout-drift-record" + +[[verdict]] +id = "bound measure partial" +gloss = "too few successful runs to characterise a job, so no budget is proposed" +class = """ +Below the declared minimum a job is uncharacterised rather than fast, and saying \ +so is itself the useful signal. Most jobs here run weekly or on release, so a \ +naive percentile over a short window would compute a confident value from two \ +samples and propose tightening a release job on it. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run timeout-drift-record" + # CLOUD-349's two readings, ported off `branch-age-check.sh` under CLOUD-1717. [[verdict]] id = "branch watch stale" diff --git a/bench/gates/RESULTS.md b/bench/gates/RESULTS.md index e440222fa..44f6f3ab5 100644 --- a/bench/gates/RESULTS.md +++ b/bench/gates/RESULTS.md @@ -64,7 +64,6 @@ commit, and a named ref resolves a declared one. | task | bucket | git variants | | ------------------------------ | ------ | ---------------------------- | -| `attestation-check.sh` | git | `location`, `remote` | | `awk-regex-check.sh` | git | `tracked` | | `batten-glob-check.sh` | git | `location` | | `board-sweep.sh` | forge | `location`, `log` | diff --git a/bench/suites/RESULTS.md b/bench/suites/RESULTS.md index 2fa87f133..512df6527 100644 --- a/bench/suites/RESULTS.md +++ b/bench/suites/RESULTS.md @@ -62,8 +62,6 @@ to it pays. | 0.7 | 0.4% | `tests/doctor.bats` | | 0.6 | 0.4% | `tests/commit-attribution.bats` | | 0.6 | 0.4% | `tests/done-pr-check.bats` | -| 0.6 | 0.4% | `tests/attestation-check.bats` | -| 0.6 | 0.4% | `tests/timeout-drift.bats` | | 0.6 | 0.4% | `tests/evaluator-closure-check.bats` | | 0.6 | 0.4% | `tests/sbom-binary.bats` | | 0.6 | 0.4% | `tests/mcp-timeout-budget.bats` | diff --git a/crates/batten/tests/it/attestation.rs b/crates/batten/tests/it/attestation.rs new file mode 100644 index 000000000..d9fad2057 --- /dev/null +++ b/crates/batten/tests/it/attestation.rs @@ -0,0 +1,249 @@ +//! `release grade unsafe` over the compiled binary (CLOUD-583, CLOUD-1717). +//! +//! # Why this tier and not the module's own `test_` rules +//! +//! Every case in `policy/attestation.rego` fabricates its input with +//! `with input as`, which cannot see a fact the engine never projects. That is +//! not hypothetical here: `policy/branch-age.rego` spent a whole session +//! registered, its own suite green, and deciding nothing, because +//! `recorder_records` projected no `record named` family at all (CLOUD-1810). +//! These cases run the real module over a record the real verb wrote. +//! +//! # RETIREMENT LEDGER, PER PATH — what `shell retire partial` reads +//! +//! The program's successor is `policy/attestation.rego` for every DECISION it +//! made, and `[tasks.attestation-record]` for everything that is not one: the +//! posture probe, the release download, the unpack, and `gh attestation verify` +//! itself. House style §5 makes `check` `read` and structurally incapable of +//! spawning a process, so that half could not move whatever the ledger said — +//! CLOUD-1559's rule, carry the decisions rather than the steps. +//! +//! THE EXIT CONTRACT CHANGED, AND FIVE ARMS RIDE ON IT. The shell ran `0` pass / +//! `1` an artifact failed / `2` could-not-look. The engine runs `0/1/2/3` where +//! `2` is a FINDING, so carrying the shell's `2` over would have turned every +//! could-not-look into a violation. Each of those arms is now the producer +//! refusing at write time — loudly, while its author is watching — and writing +//! nothing, which leaves the record absent and the module silent. +//! +// carried: mise-tasks/attestation-check.sh policy/attestation.rego kind:mechanism crates/batten/tests/it/attestation.rs +// carried: tests/attestation-check.bats policy/attestation.rego kind:mechanism crates/batten/tests/it/attestation.rs +// carried: "THE GAP IS NOT A VERDICT: a 404 endpoint reports the platform gap and exits 0" policy/attestation.rego kind:mechanism +// carried: "with the platform available and provenance present, the run passes" policy/attestation.rego kind:mechanism +// carried: "with the platform available and provenance absent, the run fails" policy/attestation.rego kind:mechanism +// carried: "THE SUBJECT IS THE BINARY, NOT THE ARCHIVE" policy/attestation.rego kind:mechanism +// carried: "a release carrying no archive is exit 2, not a green verdict about nothing" policy/attestation.rego kind:mechanism +// carried: "output is pointer-only — no attestation body reaches the log" policy/attestation.rego kind:mechanism +// changed: "the gap names the repository it asked about, derived from the remote" mise.toml the slug is the producer's to derive and the producer's to name: it reads the origin remote, and a module naming a repository would be a consumer identifier inside a decision surface. The gap is still reported — `posture 404` is recorded and readable — but the repository it asked about is named where it was asked +// changed: "a download that fails is exit 2 — could not look is not a verdict" mise.toml the download is a step, so its failure is the producer's: it refuses and records nothing, and an absent record is the module's silence. On the engine's contract exit 2 is a FINDING, so the shell's spelling would have made could-not-look a violation +// changed: "a status that is neither 200 nor 404 is exit 2, naming the code" mise.toml the same split: the producer reads the status, so an unreadable posture refuses there and records nothing. Naming the code stays in the producer's own message, where the reader who can act on it is looking +// changed: "a missing credential IS exit 2 in the world half — a 404 could not be told from a denial" mise.toml the credential is what the producer needs to make the probe at all, so its absence refuses before anything is recorded +// changed: "no github.com remote is exit 2 in the world half, and irrelevant to the precondition" mise.toml the remote is how the producer derives the repository to ask about, so its absence refuses there +// changed: "the precondition holds when the verifier resolves" mise.toml the precondition survives as `[tasks.attestation-record] --precondition`, which the `release check unread` row still runs at `deny` — the gate is preserved rather than removed, and only the program carrying it changed +// changed: "THE SEVERITY SPLIT: the precondition holds while the platform gap is open" mise.toml the same split, on the same row: the precondition is local and offline, and the platform gap is the world half the record carries +// changed: "the precondition makes no network call" mise.toml asserted of the producer's precondition mode now; the property is what keeps the `deny` row safe to run on every gate invocation +// changed: "an absent verifier is exit 2 in precondition mode" mise.toml the producer's precondition refuses when `gh` does not resolve, which is the one fact that mode ever asserted +// changed: "a missing credential does NOT fail the precondition — cannot-look is not a deny" mise.toml kept in the producer's precondition mode: a credential is could-not-look, and the landing path must not block on ambient environment + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use common::{git_in, init_repo, run, run_with_stdin, scratch, write}; + +/// A repository registering the real module against a declared family. +fn repo(name: &str) -> std::path::PathBuf { + let dir = scratch(&format!("attestation-{name}")); + let module = std::fs::read_to_string("../../policy/attestation.rego") + .expect("the module this tier exists for"); + write(&dir, "policy/attestation.rego", &module); + write( + &dir, + "batten.toml", + r#"version = 1 +scope = ["**"] + +[[verdict]] +id = "release ship unsafe" +gloss = "a release archive's binary carries no verifiable provenance" +class = "The verifier refused the executable where attestation IS available." + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run attestation-record" + +[[verdict]] +id = "release carry missing" +gloss = "a release archive carries no executable to verify" +class = "A packaging problem rather than a provenance one." + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run attestation-record" + +[[verdict]] +id = "release list empty" +gloss = "the producer looked at a tag and found no archive on it" +class = "A green verdict over a release carrying nothing would be about nothing." + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run attestation-record" + +[[rule]] +id = "release grade unsafe" +kind = "policy" +scope = "tree" +module = "policy/attestation.rego" +severity = "deny" + +[[record]] +record = "attestation" +writer = "mise run attestation-record" +"#, + ); + init_repo(&dir); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-qm", "register the module"]); + dir +} + +/// Write the producer's record, as `mise run attestation-record` would. +fn record(dir: &std::path::Path, lines: &str) { + let written = run_with_stdin(dir, &["record", "named", "attestation"], lines); + assert!( + written.status.success(), + "the setup write lands: {}", + String::from_utf8_lossy(&written.stderr) + ); +} + +/// Both streams: which one carries a finding is the output contract's business, +/// and what these cases assert is that the pointer reaches the reader. +fn said(out: &std::process::Output) -> String { + format!( + "{}{}", + String::from_utf8_lossy(&out.stdout), + String::from_utf8_lossy(&out.stderr) + ) +} + +#[test] +fn an_unverified_archive_is_reported_over_the_engines_projection() { + let dir = repo("unverified"); + record( + &dir, + "posture\t200\narchive\tbatten-x86_64.tar.gz\tunverified\n", + ); + + let decided = run(&dir, &["check"]); + assert_eq!( + decided.status.code(), + Some(2), + "an archive the verifier refused decides\n{}", + said(&decided) + ); + assert!( + said(&decided).contains("batten-x86_64.tar.gz"), + "and the finding names the archive\n{}", + said(&decided) + ); +} + +#[test] +fn a_verified_archive_is_clean() { + let dir = repo("verified"); + record( + &dir, + "posture\t200\narchive\tbatten-x86_64.tar.gz\tverified\n", + ); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "provenance present is the state the gate must be able to reach\n{}", + said(&quiet) + ); +} + +#[test] +fn a_platform_gap_judges_nothing_even_with_archives_recorded() { + // THE GAP IS NOT A VERDICT (CLOUD-585). `gh attestation verify` exits 1 both + // when an artifact has no provenance and when the platform never offered + // any, and those are opposite facts. With the endpoint answering 404 the + // verifier refuses everything, so judging here would red every release for a + // reason no branch causes. + let dir = repo("gap"); + record( + &dir, + "posture\t404\narchive\tbatten-x86_64.tar.gz\tunverified\n", + ); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "a platform gap judges no artifact\n{}", + said(&quiet) + ); +} + +#[test] +fn an_absent_record_says_nothing_rather_than_passing() { + // Every could-not-look arm of the retired program — no credential, no + // remote, an unreadable status, a failed download — is now the producer + // refusing and writing nothing. This is what that absence must read as. + let dir = repo("absent"); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "an absent record is could-not-look\n{}", + said(&quiet) + ); +} + +#[test] +fn a_tag_carrying_no_archive_is_refused_rather_than_read_as_clean() { + // "A green verdict would be about nothing." Present-and-empty and absent are + // different readings: the case above is nobody having looked, and this is the + // producer having looked and found a tag with no archives. + let dir = repo("empty"); + record(&dir, "posture\t200\n"); + + let refused = run(&dir, &["check"]); + assert_eq!( + refused.status.code(), + Some(2), + "a tag with no archive is refused\n{}", + said(&refused) + ); +} + +#[test] +fn the_report_names_the_archive_and_carries_no_attestation_body() { + // POINTER, NEVER PAYLOAD (rule 4). The retired program discarded the + // verifier's own output deliberately — it names the attesting workflow and + // signer, which is not this gate's to republish. + let dir = repo("pointer"); + record( + &dir, + "posture\t200\narchive\tbatten-aarch64.tar.gz\tno-binary\n", + ); + + let refused = run(&dir, &["check"]); + let text = said(&refused); + assert!( + text.contains("batten-aarch64.tar.gz"), + "the asset name is the pointer\n{text}" + ); + assert!( + !text.contains("sha256:"), + "and no digest or bundle travels with it\n{text}" + ); +} diff --git a/crates/batten/tests/it/main.rs b/crates/batten/tests/it/main.rs index 0f9405de8..c0d2a10f5 100644 --- a/crates/batten/tests/it/main.rs +++ b/crates/batten/tests/it/main.rs @@ -52,6 +52,7 @@ mod agentic_record; mod ambient_authority; mod ask_disposition; mod assertion_gates; +mod attestation; mod attribution; mod attribution_provenance; mod authority_replay; @@ -280,6 +281,7 @@ mod task_prose; mod task_receipt; mod task_registry; mod test_targets; +mod timeout_drift; mod todo_promotion; mod tool_selector; mod tool_verdict_facts; diff --git a/crates/batten/tests/it/timeout_drift.rs b/crates/batten/tests/it/timeout_drift.rs new file mode 100644 index 000000000..05e82364b --- /dev/null +++ b/crates/batten/tests/it/timeout_drift.rs @@ -0,0 +1,236 @@ +//! `bound grade other` over the compiled binary (CLOUD-266, CLOUD-1717). +//! +//! # Why this tier and not the module's own `test_` rules +//! +//! Every case in `policy/timeout-drift.rego` fabricates its input with +//! `with input as`, which cannot see a fact the engine never projects — the state +//! `policy/branch-age.rego` sat in for a whole session while its own suite stayed +//! green (CLOUD-1810). These run the real module over a record the real verb +//! wrote, and one of them asserts the thing no load-time case can: that a `warn` +//! row REPORTS without failing the run. +//! +//! # RETIREMENT LEDGER, PER PATH — what `shell retire partial` reads +//! +//! The program's successor is `policy/timeout-drift.rego` for the +//! classification, and `[tasks.timeout-drift-record]` for the measurement. That +//! split is forced: the samples are durations of successful runs read from the +//! Actions API, and a p95 over them is arithmetic on instants — §5 makes `check` +//! `read` and incapable of spawning, and `Fact::Instant` projects `null` to every +//! module, which `clippy.toml` and `crates/batten/tests/clock_ban.rs` hold the +//! engine to. +//! +//! THE POSTURE IS PRESERVED BY SEVERITY. The retired program reported and never +//! blocked — it failed only its own scheduled run, filed no issue, posted no +//! comment — because a budget that no longer matches reality means nothing is +//! broken and no branch is at fault. `severity = "warn"` is that on the engine's +//! contract, and `a_drifted_budget_reports_without_failing_the_run` is what holds +//! it there. +//! +// carried: mise-tasks/timeout-drift.sh policy/timeout-drift.rego kind:mechanism crates/batten/tests/it/timeout_drift.rs +// carried: tests/timeout-drift.bats policy/timeout-drift.rego kind:mechanism crates/batten/tests/it/timeout_drift.rs +// carried: "a measured budget matching its measurement reports clean" policy/timeout-drift.rego kind:mechanism +// carried: "a budget the measurement has outgrown reports drift-tight, naming both numbers" policy/timeout-drift.rego kind:mechanism +// carried: "a budget gone slack because the job got faster reports drift-loose — the ratchet" policy/timeout-drift.rego kind:mechanism +// carried: "a small slack is not drift — a budget is a ceiling, not a target" policy/timeout-drift.rego kind:mechanism +// carried: "a job with fewer than the minimum samples reports unmeasurable, never a number" policy/timeout-drift.rego kind:mechanism +// carried: "a grandfathered entry with a usable sample is prompted for conversion" policy/timeout-drift.rego kind:mechanism +// carried: "a grandfathered entry with too small a sample is unmeasurable, not a conversion prompt" policy/timeout-drift.rego kind:mechanism +// changed: "matrix legs pool into one distribution — one timeout bounds them all" mise.toml the pooling is a STEP: the API reports a matrix leg as `dist ()`, and matching the job key or the key followed by " (" is how the producer gathers one distribution before computing its p95. The module receives one row per job with the p95 already over the pooled samples, so there is nothing left here to pool +// changed: "a failed API query is exit 2, never a drift verdict" mise.toml the query is the producer's and so is its failure: it refuses at write time and records nothing, and an absent record is the module's silence. On the engine's contract exit 2 is a FINDING, so the shell's spelling would have made could-not-look a violation. The reason the retired program gives is the one that carries — reporting a healthy budget as drifted on a network blip is what gets a scheduled gate switched off +// changed: "an absent gh is exit 2, never a pass" mise.toml the producer needs `gh` to read the Actions API at all, so its absence refuses there and records nothing +// changed: "a missing workflow directory is exit 2, never a pass" mise.toml the declared budgets are read from the workflows by the producer, so an unreadable directory refuses before anything is recorded + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use common::{git_in, init_repo, run, run_with_stdin, scratch, write}; + +fn repo(name: &str) -> std::path::PathBuf { + let dir = scratch(&format!("timeout-drift-{name}")); + let module = std::fs::read_to_string("../../policy/timeout-drift.rego") + .expect("the module this tier exists for"); + write(&dir, "policy/timeout-drift.rego", &module); + write( + &dir, + "batten.toml", + r#"version = 1 +scope = ["**"] + +[[pattern]] +id = "whole-number" +regex = '^[0-9]+$' + +[[verdict]] +id = "bound pin loose" +gloss = "a job's declared timeout sits well above what its measurement justifies" +class = "A budget is a ceiling rather than a target, and past the slack it has gone slack." + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run timeout-drift-record" + +[[verdict]] +id = "bound pin wrong" +gloss = "a job's measurement has outgrown its declared timeout" +class = "Raise it before it starts failing healthy runs." + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run timeout-drift-record" + +[[verdict]] +id = "bound pin stale" +gloss = "a dated debt entry now has a usable sample" +class = "The prompt, never the conversion." + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run timeout-drift-record" + +[[verdict]] +id = "bound measure partial" +gloss = "too few successful runs to characterise a job" +class = "Below the minimum a job is uncharacterised rather than fast." + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run timeout-drift-record" + +[[rule]] +id = "bound grade other" +kind = "policy" +scope = "tree" +module = "policy/timeout-drift.rego" +severity = "warn" + +[[record]] +record = "timeout-drift" +writer = "mise run timeout-drift-record" +"#, + ); + init_repo(&dir); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-qm", "register the module"]); + dir +} + +fn record(dir: &std::path::Path, lines: &str) { + let written = run_with_stdin(dir, &["record", "named", "timeout-drift"], lines); + assert!( + written.status.success(), + "the setup write lands: {}", + String::from_utf8_lossy(&written.stderr) + ); +} + +fn said(out: &std::process::Output) -> String { + format!( + "{}{}", + String::from_utf8_lossy(&out.stdout), + String::from_utf8_lossy(&out.stderr) + ) +} + +#[test] +fn a_slack_budget_is_reported_as_loose_over_the_engines_projection() { + // `justified(120s) == 6m`, so a declared 12 is past the five-minute slack. + let dir = repo("loose"); + record(&dir, "job\tci.yml\tbats\t12\t120\t25\tmeasured\n"); + + let reported = run(&dir, &["check", "--fail-on-warning"]); + assert_eq!( + reported.status.code(), + Some(2), + "a slack budget is reported\n{}", + said(&reported) + ); + assert!( + said(&reported).contains("bats"), + "and the report names the job\n{}", + said(&reported) + ); +} + +#[test] +fn a_drifted_budget_reports_without_failing_the_run() { + // THE PORTED POSTURE, and the case no load-time rule can make. The retired + // program reported and never blocked; `warn` is that on this contract, so the + // same record that fires above must leave an ordinary `check` green. + let dir = repo("warn"); + record(&dir, "job\tci.yml\tbats\t12\t120\t25\tmeasured\n"); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "a report must not fail the run — nothing is broken and no branch is at fault\n{}", + said(&quiet) + ); +} + +#[test] +fn a_budget_matching_its_measurement_is_clean() { + let dir = repo("clean"); + record(&dir, "job\tci.yml\tbats\t6\t120\t25\tmeasured\n"); + + let quiet = run(&dir, &["check", "--fail-on-warning"]); + assert_eq!( + quiet.status.code(), + Some(0), + "a correct budget is the state the report must be able to reach\n{}", + said(&quiet) + ); +} + +#[test] +fn a_budget_the_measurement_has_outgrown_is_reported_as_tight() { + let dir = repo("tight"); + record(&dir, "job\tci.yml\tbats\t5\t120\t25\tmeasured\n"); + + let reported = run(&dir, &["check", "--fail-on-warning"]); + assert_eq!( + reported.status.code(), + Some(2), + "a budget below what the measurement justifies is reported\n{}", + said(&reported) + ); +} + +#[test] +fn a_job_with_too_few_samples_is_unmeasurable_rather_than_fast() { + // A NAIVE PERCENTILE OVER TWO SAMPLES would propose tightening a release job + // on it. Ten of the fourteen jobs here run weekly or on release, so this arm + // is the common case rather than the defensive one. + let dir = repo("unmeasurable"); + record(&dir, "job\tci.yml\tbats\t30\t120\t2\tmeasured\n"); + + let reported = run(&dir, &["check", "--fail-on-warning"]); + assert_eq!( + reported.status.code(), + Some(2), + "an uncharacterised job is reported as such\n{}", + said(&reported) + ); +} + +#[test] +fn an_absent_record_says_nothing_rather_than_reporting_drift() { + // Every could-not-look arm of the retired program is now the producer + // refusing and writing nothing. Reporting a healthy budget as drifted on a + // network blip is the failure mode that gets a scheduled gate switched off. + let dir = repo("absent"); + + let quiet = run(&dir, &["check", "--fail-on-warning"]); + assert_eq!( + quiet.status.code(), + Some(0), + "an absent record is could-not-look\n{}", + said(&quiet) + ); +} diff --git a/mise-tasks/attestation-check.sh b/mise-tasks/attestation-check.sh deleted file mode 100755 index f0b2cc9d3..000000000 --- a/mise-tasks/attestation-check.sh +++ /dev/null @@ -1,220 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Gate: a release's binaries carry build provenance — or the platform gap is reported, never mistaken for an unverified artifact" -# -# CLOUD-583, adopting `gh attestation verify` (CLOUD-279 verdict 1) rather than -# `slsa-verifier` or `cosign`: the first is redundant against the same absent -# provenance, the second has no signing identity here. -# -# THE WHOLE DESIGN IS THE PRECONDITION. `gh attestation verify` exits 1 both when -# an artifact has no provenance and when the platform never offered any, and those -# are opposite facts: the first is a release that should be fixed, the second is a -# plan feature this private repo does not have. Measured 2026-08-14 on -# `batten-v0.0.74-x86_64-unknown-linux-gnu.tar.gz`, reproducing CLOUD-583's v0.0.52 -# reading exactly — `Error: HTTP 404: Not Found`, exit 1. -# -# The control that separates them is the endpoint's own status code: where the -# feature IS available, an unknown digest answers **200** with an empty array; -# **404 on the resource** is the feature being absent for the repository. -# Measured here against an all-zeros digest: 404. So this reports the gap and -# exits 0 rather than calling anything unverified — a gate that cannot tell the -# two apart is worse than no gate, because it reds every release for a reason no -# branch causes. -# -# TWO MODES, and the split is the CLOUD-410 one (a gate on the landing path -# answers a question about THIS COMMIT; whether a published release is attested -# today is a question about the WORLD, and it changes with no diff): -# -# --precondition OFFLINE, and narrow on purpose: the verifier resolves. This -# is what the `release-attestation-precondition` row in -# batten.toml runs on every gate invocation, so the landing -# path makes no network call at all — see its own note below -# for why the credential is NOT part of it. -# [] The world question, on the `release-assets-check` model: -# probe the platform, then verify the release's binaries. -# Defaults to the latest release. -# -# IT VERIFIES THE BINARY, NOT THE ARCHIVE, and that is a correction to the issue's -# own wording rather than a detail. `release-artifacts.yml` attests -# `steps.dist.outputs.binary` — deliberately, so repackaging cannot launder the -# claim — so the digest provenance binds to is the executable's. Verifying the -# `.tar.gz` would compute a digest nothing ever attested and report a failure that -# means nothing, which is the same conflation this gate exists to prevent, one -# layer down. -# -# Exit 0 pass or reported gap / 1 an artifact failed to verify / 2 could-not-look, -# matching the other `*-check` programs. Pointer-only (rule 4): asset names, -# counts and status codes, never an attestation, a bundle, or a digest's contents. -# A gate listed in $MUTANT_GATES with no row here fails `mise run mutant`. -#MUTANT provenance-absent-passes|s/^\texit 1$/\texit 0/|provenance absent, the run fails - -set -euo pipefail - -cd "${ATTESTATION_CHECK_ROOT:-$(git rev-parse --show-toplevel)}" - -# The verifier, named through an override rather than taken off PATH alone — the -# `BATTEN_BIN` idiom, and load-bearing for the suite, which must be able to drive -# a `gh` whose answers it chooses without shadowing the real shim. -GH_BIN="${ATTESTATION_GH:-gh}" - -# A digest no artifact has, which is what makes the probe a question about the -# REPOSITORY rather than about any file: where attestation is available the answer -# is 200 with an empty array, and where it is not the resource itself is 404. -readonly ZERO_DIGEST=0000000000000000000000000000000000000000000000000000000000000000 - -usage() { - cat >&2 <<-EOF - usage: mise run attestation-check [] - mise run attestation-check --precondition - - release tag to verify (defaults to the latest release) - --precondition offline: the verifier resolves - EOF -} - -# The repository, read from the remote rather than written down: which repo this -# is belongs to the consumer, never to a task (non-negotiable rule 1, and the -# same derivation `[tasks.scorecard]` makes). -repo_slug() { - local remote slug - remote=$(git remote get-url origin) || return 1 - slug=${remote#*github.com[:/]} - slug=${slug%.git} - [[ -n "$slug" ]] || return 1 - printf '%s' "$slug" -} - -# --- the offline half --------------------------------------------------------- -# -# ONE FACT, and the narrowness is the design. It asserts that the verifier -# resolves — nothing about the repository, the credential, or the platform — -# because this is what the `deny` row runs on every gate invocation, and a gate -# that blocks on ambient environment is a gate that blocks everything the moment -# an environment differs. Measured: an earlier version also required GH_TOKEN and -# a github.com remote, and it reported a violation inside -# `tests/prebuilt-lint.bats`' fixture repositories, which carry neither. A -# credential is "cannot look", and cannot-look is reported by the world half -# below rather than enforced here (the landing loop's fail-open rule). -precondition() { - if ! command -v "$GH_BIN" >/dev/null 2>&1; then - echo "::error:: attestation-check: no verifier at '$GH_BIN' (\$ATTESTATION_GH overrides), so no attestation could ever be checked. Run: mise install aqua:cli/cli" >&2 - return 2 - fi - echo "attestation-check: precondition holds — the verifier resolves" - return 0 -} - -if [[ "${1:-}" = "--precondition" ]]; then - precondition - exit $? -fi -case "${1:-}" in --h | --help) - usage - exit 0 - ;; -esac - -# --- the world half ----------------------------------------------------------- - -# The world half needs everything the precondition deliberately does not: a -# credential to read the endpoint with, and a repository to read it for. Each -# absent is "could not look" (exit 2), never a verdict about an artifact. -if ! precondition >/dev/null; then - exit 2 -fi -if [[ -z "${GH_TOKEN:-${GITHUB_TOKEN:-}}" ]]; then - echo "::error:: attestation-check: no GH_TOKEN/GITHUB_TOKEN, so the attestations endpoint cannot be read and a 404 could not be told from a denial." >&2 - exit 2 -fi -if ! slug=$(repo_slug); then - echo "::error:: attestation-check: no github.com origin remote, so there is no repository to ask about." >&2 - exit 2 -fi - -# The status code IS the object this decides over — a protocol answer, not a -# report to interpret (rule 3). `-i` prints the status line; nothing else in the -# response is read. -status=$("$GH_BIN" api "repos/$slug/attestations/sha256:$ZERO_DIGEST" -i 2>/dev/null | - awk 'NR==1 { for (i = 1; i <= NF; i++) if ($i ~ /^[0-9][0-9][0-9]$/) { print $i; exit } }' || true) - -case "$status" in -404) - # THE REPORTED GAP, and the reason this exits 0: nothing here is a claim about - # an artifact. `release-artifacts.yml` already runs its attestation step - # `continue-on-error: true` for the same fact, so a release is published - # unattested by design until the repository is public (CLOUD-585). - echo "attestation-check: $slug:0 attestation-unavailable — the platform offers no attestation for this repository (endpoint 404), so no release artifact is judged" - exit 0 - ;; -200) ;; -"") - echo "::error:: attestation-check: the attestations endpoint returned no readable status, so the platform's posture is unknown and nothing was judged." >&2 - exit 2 - ;; -*) - echo "::error:: attestation-check: the attestations endpoint answered $status — neither 200 (available) nor 404 (absent), so the platform's posture is unknown and nothing was judged." >&2 - exit 2 - ;; -esac - -tag="${1:-}" -if [[ -z "$tag" ]]; then - if ! tag=$("$GH_BIN" release view --json tagName --jq '.tagName' 2>/dev/null) || [[ -z "$tag" ]]; then - echo "::error:: attestation-check: no tag given and no latest release to read, so there is nothing to verify." >&2 - exit 2 - fi -fi - -scratch=$(mktemp -d) -trap 'rm -rf "$scratch"' EXIT - -# Archives only: a release also carries a schema, an SBOM and a checksum manifest, -# and none of those is what the dist matrix attests. -if ! "$GH_BIN" release download "$tag" --dir "$scratch" --pattern '*.tar.gz' --pattern '*.zip' >/dev/null 2>&1; then - echo "::error:: attestation-check: could not download $tag's archives, so their provenance is unverified." >&2 - exit 2 -fi - -violations=0 -checked=0 -report() { # pointer-only (rule 4): the asset name and the rule id, never a bundle - echo "$1 $2" >&2 - violations=$((violations + 1)) -} - -for archive in "$scratch"/*.tar.gz "$scratch"/*.zip; do - [[ -f "$archive" ]] || continue - name=${archive##*/} - # The attested subject is the BINARY inside, so the archive is opened and the - # executable handed to the verifier — see the header. - binary_dir="$scratch/x-$name" - mkdir -p "$binary_dir" - case "$name" in - *.zip) unzip -q -o "$archive" -d "$binary_dir" || true ;; - *) tar -xzf "$archive" -C "$binary_dir" || true ;; - esac - binary=$(find "$binary_dir" -type f \( -name batten -o -name batten.exe \) | head -n1) - if [[ -z "$binary" ]]; then - report "$name:0" "attestation-no-binary (the archive carries no batten executable to verify)" - continue - fi - checked=$((checked + 1)) - # THE VERDICT, and the only thing that is one: the verifier's exit status. Its - # own output is discarded rather than read (CLOUD-93), and it names the - # attesting workflow and signer, which is not this gate's to republish. - if ! "$GH_BIN" attestation verify "$binary" --repo "$slug" >/dev/null 2>&1; then - report "$name:0" "attestation-unverified" - fi -done - -if [[ "$checked" -eq 0 ]] && [[ "$violations" -eq 0 ]]; then - echo "::error:: attestation-check: $tag carries no archive to verify, so a green verdict would be about nothing." >&2 - exit 2 -fi - -if [[ "$violations" -ne 0 ]]; then - echo "::error:: attestation-check: $violations of $checked archive(s) in $tag carry no verifiable provenance. The platform DOES offer attestation here, so this is a release to fix, not a gap to report." >&2 - exit 1 -fi - -echo "attestation-check: $checked archive(s) in $tag carry verifiable provenance for $slug" diff --git a/mise-tasks/timeout-drift.sh b/mise-tasks/timeout-drift.sh deleted file mode 100755 index f4588780a..000000000 --- a/mise-tasks/timeout-drift.sh +++ /dev/null @@ -1,191 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Report (never gate) whether each job's committed timeout budget still matches measured reality — a question about the world, so it runs on a clock" -# -# CLOUD-266, the other half. `timeout-check` asks a question about the commit: -# is every timeout justified? This asks a question about the world: is the -# justification still TRUE? A property of the commit belongs in the gate, a -# property of the world belongs on a clock — the `lock-complete` / -# `lock-currency` split `.claude/rules/toolchain.md` sets. -# -# It reports and never blocks the landing path. On drift it fails its own -# scheduled workflow run and does nothing else: no issue filed, no comment -# posted, exactly the posture `lock-currency.yml` states in its header. Here a -# failure means a budget no longer matches reality, not that anything is broken. -# -# DRIFT IS REPORTED IN BOTH DIRECTIONS, and the loose direction is the point. A -# budget that has gone slack because the job got faster is the ratchet this issue -# exists for; a report that only complained about tightness would let every -# number rot upward forever. So: -# -# drift-loose the declared minutes are well above what the measurement now -# justifies — the number to tighten -# drift-tight the measurement has outgrown the declared minutes — the number -# to raise, before it starts failing healthy runs -# grandfathered a dated debt entry that now HAS a usable sample, so it can be -# converted to a measured budget. This is the prompt; converting -# it is a deliberate commit, never a bot re-baselining the number -# it is supposed to defend (§4: nothing derived). -# unmeasurable fewer than MIN_SAMPLES successful runs in the window -# -# A SMALL SAMPLE MUST REPORT `unmeasurable`, NEVER A NUMBER. Ten of the fourteen -# jobs run weekly or on release, so a naive p95 over a two-week window would -# compute a confident value from two samples and propose tightening a release job -# on it. Below the stated minimum the job is reported as unmeasurable, which is -# itself the useful signal about a job nobody can characterise. -# -# Matrix legs pool. `release-artifacts.yml`'s `dist` declares one -# `timeout-minutes` covering seven targets, and the API reports each leg under -# `dist ()`. One budget bounds them all, so every leg's duration feeds one -# distribution — matched on the job key, or the key followed by ` (`. -# -# Exit 0 clean, 1 drift, 2 could-not-look. `2` for an absent `gh`, a failed query -# or an unreadable workflow: reporting a healthy budget as drifted on a network -# blip is the failure mode that gets a scheduled gate switched off. -set -euo pipefail - -# The one repo-wide headroom multiplier, the same constant `timeout-check` gates. -BUDGET_MULTIPLIER="${BATTEN_BUDGET_MULTIPLIER:-3}" -# Below this many successful runs a job is unmeasurable, not fast. -MIN_SAMPLES="${BATTEN_BUDGET_MIN_SAMPLES:-5}" -# How many recent successful runs per workflow to read. -WINDOW="${BATTEN_BUDGET_WINDOW:-25}" -# How far a declared value may sit from what the measurement justifies before it -# is called loose. A budget is a ceiling, not a target, so some slack is correct. -SLACK_MINUTES="${BATTEN_BUDGET_SLACK:-5}" - -if ! command -v gh >/dev/null 2>&1; then - echo "::error:: gh is not on PATH — cannot read the Actions API" >&2 - exit 2 -fi - -workflows=".github/workflows" -[[ "$#" -eq 0 ]] || workflows="$1" -if [[ ! -d "$workflows" ]]; then - echo "::error:: $workflows is not a directory" >&2 - exit 2 -fi - -drift=0 -report() { - echo "$1" >&2 - drift=$((drift + 1)) -} - -# Seconds between two RFC 3339 instants. GNU `date` on the runner; the bats suite -# drives the same path through its stub. -seconds_between() { - local from to - from=$(date -d "$1" +%s 2>/dev/null) || return 1 - to=$(date -d "$2" +%s 2>/dev/null) || return 1 - echo $((to - from)) -} - -# The p95 of a newline-separated list of integers: the ceil(0.95 * n)-th smallest. -p95_of() { - local sorted count index - sorted=$(sort -n <<<"$1") - count=$(grep -c . <<<"$sorted") - index=$((((95 * count) + 99) / 100)) - [[ "$index" -ge 1 ]] || index=1 - sed -n "${index}p" <<<"$sorted" -} - -# `ceil(p95 * multiplier / 60)`, the same arithmetic `timeout-check` gates. -budget_minutes() { - echo $(((($1 * $2) + 59) / 60)) -} - -# The declared budget per job, read from the workflow itself so this task and the -# gate can never disagree about what is committed. -jobs_of() { - awk ' - function flush() { - if (job != "") printf "%s\t%s\t%s\n", job, declared, budget - } - /^jobs:/ { injobs = 1; next } - injobs && /^[a-z]/ { injobs = 0 } - injobs && /^ [A-Za-z0-9_-]+:[[:space:]]*$/ { - flush() - job = $1; sub(":", "", job); declared = ""; budget = "" - } - injobs && job != "" && /^ timeout-minutes:[[:space:]]*[0-9]+/ { - value = $0 - sub(/^[[:space:]]*timeout-minutes:[[:space:]]*/, "", value) - sub(/[^0-9].*$/, "", value) - declared = value - budget = "" - if (index($0, "#") > 0) budget = substr($0, index($0, "#")) - } - END { flush() } - ' "$1" -} - -checked=0 -for wf in "$workflows"/*.yml; do - [[ -e "$wf" ]] || continue - file="$(basename "$wf")" - - # Successful runs only: a cancelled or failed run's duration says nothing - # about what the job costs when it works. - if ! runs=$(gh api "repos/{owner}/{repo}/actions/workflows/$file/runs?status=success&per_page=$WINDOW" \ - --jq '.workflow_runs[].id' 2>/dev/null); then - echo "::error:: cannot read runs for $file" >&2 - exit 2 - fi - - # One duration per successful job, keyed by the API's job name. - durations="" - while IFS= read -r run; do - [[ -n "$run" ]] || continue - if ! legs=$(gh api "repos/{owner}/{repo}/actions/runs/$run/jobs" \ - --jq '.jobs[] | select(.conclusion == "success") | [.name, .started_at, .completed_at] | @tsv' 2>/dev/null); then - echo "::error:: cannot read jobs for run $run" >&2 - exit 2 - fi - while IFS=$'\t' read -r name started completed; do - [[ -n "$name" ]] || continue - elapsed=$(seconds_between "$started" "$completed") || continue - [[ "$elapsed" -ge 0 ]] || continue - durations+="$name $elapsed"$'\n' - done <<<"$legs" - done <<<"$runs" - - while IFS=$'\t' read -r job declared budget; do - [[ -n "$job" ]] || continue - [[ -n "$declared" ]] || continue - checked=$((checked + 1)) - - # The job key, or the key followed by " (" — which is how a matrix leg - # reports. Every leg feeds one distribution, because one timeout bounds - # them all. - samples=$(awk -F'\t' -v job="$job" \ - '$1 == job || index($1, job " (") == 1 { print $2 }' <<<"$durations") - count=$(grep -c . <<<"$samples" || true) - - if [[ "$count" -lt "$MIN_SAMPLES" ]]; then - report "$file $job $declared unmeasurable ($count/$MIN_SAMPLES samples)" - continue - fi - - p95=$(p95_of "$samples") - justified=$(budget_minutes "$p95" "$BUDGET_MULTIPLIER") - - if [[ $budget =~ grandfathered ]]; then - report "$file $job $declared grandfathered (p95=${p95}s x$BUDGET_MULTIPLIER is ${justified}m — convert it)" - continue - fi - - if [[ "$declared" -lt "$justified" ]]; then - report "$file $job $declared drift-tight (p95=${p95}s x$BUDGET_MULTIPLIER needs ${justified}m)" - elif [[ "$declared" -gt $((justified + SLACK_MINUTES)) ]]; then - report "$file $job $declared drift-loose (p95=${p95}s x$BUDGET_MULTIPLIER justifies ${justified}m)" - fi - done <<<"$(jobs_of "$wf")" -done - -if [[ "$drift" -ne 0 ]]; then - echo "::error:: timeout-drift: $drift budget(s) no longer match reality. This is a report, not a gate — nothing is broken and no branch is at fault. Re-derive the ones named above and commit the new comment; a bot must not re-baseline the number it is supposed to defend." >&2 - exit 1 -fi - -echo "timeout-drift: $checked job(s), every budget still matches its measurement" diff --git a/mise.toml b/mise.toml index 0aaaf80f9..adbac2f05 100644 --- a/mise.toml +++ b/mise.toml @@ -617,7 +617,7 @@ CI_FANIN_WORKFLOW = ".github/workflows/ci.yml" BATS_TEST_TIMEOUT = "300" REGORUS_OPA_COMPLIANCE = "1.2.0" REGORUS_OPA_COMPLIANCE_FOR = "0.11" -MUTANT_GATES = "mise,attestation-check,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure-check,evaluator-io-check,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence-assert,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link-check,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict-assert,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus-check,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" +MUTANT_GATES = "mise,attestation,timeout-drift,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure-check,evaluator-io-check,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence-assert,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link-check,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict-assert,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus-check,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" # --- GitHub reachability behind an egress proxy (Claude Code web sandbox etc.) --- # mise resolves every tool's release through GitHub's *API* host, api.github.com. @@ -1866,6 +1866,205 @@ counts=$(jq -r --argjson authored "$authored" ' # undefined and it says nothing; a record that IS written and names no branch is # the different, refused state. Collapsing those two is the silent pass this # campaign keeps finding. +# CLOUD-583's probe and verifier, ported off `attestation-check.sh` under +# CLOUD-1717. +# +# THE SPAWN STAYS OUT HERE AND ONLY THE ADJUDICATION MOVED IN (house-style §5). +# `policy/attestation.rego` decides what an unverified archive is; this fetches +# what it decides over, and every step of the fetching is a process `check` may +# not start: the endpoint probe, the release download, the unpack, and +# `gh attestation verify` itself. +# +# EVERY COULD-NOT-LOOK WRITES NOTHING, which is the half that makes the module's +# silence honest. No credential, no github.com remote, a posture that is neither +# 200 nor 404, an unreadable status line, a tag that will not resolve, a download +# that fails — each refuses here, loudly, while its author is watching, and leaves +# the record absent. The retired program spelled these as exit 2; on the engine's +# contract 2 means a finding, so carrying that over would have made could-not-look +# a violation. +# +# THE ZERO DIGEST IS THE PROBE, and it is a question about the REPOSITORY rather +# than about any file: where attestation is available an unknown digest answers +# 200 with an empty array, and where it is not the resource itself answers 404. +[tasks.attestation-record] +description = "Record the platform's attestation posture and each release archive's verdict, for `release grade unsafe`" +shell = "bash -c" +run = ''' +set -euo pipefail + +gh_bin="${ATTESTATION_GH:-gh}" +zero=0000000000000000000000000000000000000000000000000000000000000000 + +# THE OFFLINE HALF, and the narrowness is the design. It asserts that the +# verifier resolves and NOTHING about the repository, the credential or the +# platform, because this is what the `release check unread` row runs at `deny` on +# every gate invocation — and a gate that blocks on ambient environment blocks +# everything the moment an environment differs. Measured on the retired program: +# an earlier version also required GH_TOKEN and a github.com remote, and it +# reported a violation inside `tests/prebuilt-lint.bats`' fixture repositories, +# which carry neither. A credential is could-not-look, and could-not-look is the +# world half's to report rather than this one's to enforce. +if ! command -v "$gh_bin" >/dev/null 2>&1; then + echo "::error:: attestation-record: no verifier at '$gh_bin', so no attestation could ever be checked. Run: mise install aqua:cli/cli" >&2 + exit 2 +fi +if [ "${1:-}" = "--precondition" ]; then + echo "attestation-record: precondition holds — the verifier resolves" + exit 0 +fi +if [ -z "${GH_TOKEN:-${GITHUB_TOKEN:-}}" ]; then + echo "::error:: attestation-record: no GH_TOKEN/GITHUB_TOKEN, so a 404 could not be told from a denial. Nothing recorded." >&2 + exit 2 +fi + +remote=$(git remote get-url origin) +slug=${remote#*github.com[:/]} +slug=${slug%.git} +if [ -z "$slug" ]; then + echo "::error:: attestation-record: no github.com origin remote, so there is no repository to ask about." >&2 + exit 2 +fi + +status=$("$gh_bin" api "repos/$slug/attestations/sha256:$zero" -i 2>/dev/null | + awk 'NR==1 { for (i = 1; i <= NF; i++) if ($i ~ /^[0-9][0-9][0-9]$/) { print $i; exit } }' || true) +case "$status" in +404) printf 'posture\t404\n' | cargo run --quiet -p batten -- record named attestation; exit 0 ;; +200) ;; +*) + echo "::error:: attestation-record: the attestations endpoint answered '${status:-nothing readable}' — neither 200 nor 404, so the platform's posture is unknown and nothing was recorded." >&2 + exit 2 + ;; +esac + +tag="${1:-}" +if [ -z "$tag" ]; then + if ! tag=$("$gh_bin" release view --json tagName --jq '.tagName' 2>/dev/null) || [ -z "$tag" ]; then + echo "::error:: attestation-record: no tag given and no latest release to read, so there is nothing to verify." >&2 + exit 2 + fi +fi + +scratch=$(mktemp -d) +trap 'rm -rf "$scratch"' EXIT +if ! "$gh_bin" release download "$tag" --dir "$scratch" --pattern '*.tar.gz' --pattern '*.zip' >/dev/null 2>&1; then + echo "::error:: attestation-record: could not download $tag's archives, so their provenance is unread. Nothing recorded." >&2 + exit 2 +fi + +{ + printf 'posture\t200\n' + for archive in "$scratch"/*.tar.gz "$scratch"/*.zip; do + [ -f "$archive" ] || continue + name=${archive##*/} + into="$scratch/x-$name" + mkdir -p "$into" + case "$name" in + *.zip) unzip -q -o "$archive" -d "$into" || true ;; + *) tar -xzf "$archive" -C "$into" || true ;; + esac + binary=$(find "$into" -type f \( -name batten -o -name batten.exe \) | head -n1) + if [ -z "$binary" ]; then + printf 'archive\t%s\tno-binary\n' "$name" + elif "$gh_bin" attestation verify "$binary" --repo "$slug" >/dev/null 2>&1; then + printf 'archive\t%s\tverified\n' "$name" + else + printf 'archive\t%s\tunverified\n' "$name" + fi + done +} | cargo run --quiet -p batten -- record named attestation +''' + +# CLOUD-266's measurement half, ported off `timeout-drift.sh` under CLOUD-1717. +# +# THE ARITHMETIC CANNOT MOVE IN. A p95 over run durations is arithmetic over +# instants, and the engine calls no clock on any evaluation path — `clippy.toml`'s +# `disallowed-methods` and `crates/batten/tests/clock_ban.rs` hold it there. So the +# durations are subtracted here and the classification is +# `policy/timeout-drift.rego`'s: CLOUD-1559's rule, carry the decisions. +# +# MATRIX LEGS POOL. One `timeout-minutes` covers seven `dist` targets and the API +# reports each leg as `dist ()`, so every leg feeds one distribution — +# matched on the job key, or the key followed by " (". +[tasks.timeout-drift-record] +description = "Record each job's declared budget against its measured p95, for `bound grade other`" +shell = "bash -c" +run = ''' +set -euo pipefail + +window="${BATTEN_BUDGET_WINDOW:-25}" +workflows=".github/workflows" + +if ! command -v gh >/dev/null 2>&1; then + echo "::error:: timeout-drift-record: gh is not on PATH — cannot read the Actions API. Nothing recorded." >&2 + exit 2 +fi +if [ ! -d "$workflows" ]; then + echo "::error:: timeout-drift-record: $workflows is not a directory, so no budget could be read. Nothing recorded." >&2 + exit 2 +fi + +emit() { + for wf in "$workflows"/*.yml; do + [ -e "$wf" ] || continue + file="$(basename "$wf")" + if ! runs=$(gh api "repos/{owner}/{repo}/actions/workflows/$file/runs?status=success&per_page=$window" --jq '.workflow_runs[].id' 2>/dev/null); then + echo "::error:: timeout-drift-record: cannot read runs for $file. Nothing recorded." >&2 + return 2 + fi + durations="" + for run in $runs; do + [ -n "$run" ] || continue + if ! legs=$(gh api "repos/{owner}/{repo}/actions/runs/$run/jobs" --jq '.jobs[] | select(.conclusion == "success") | [.name, .started_at, .completed_at] | @tsv' 2>/dev/null); then + echo "::error:: timeout-drift-record: cannot read jobs for run $run. Nothing recorded." >&2 + return 2 + fi + durations="$durations$(printf '%s\n' "$legs" | awk -F'\t' ' + NF == 3 { + "date -d \"" $2 "\" +%s" | getline from + "date -d \"" $3 "\" +%s" | getline to + if (from != "" && to != "" && to >= from) printf "%s\t%d\n", $1, to - from + }') +" + done + awk -v file="$file" -v durations="$durations" ' + function flush( samples, n, i, sorted, idx, p95) { + if (job == "" || declared == "") return + n = 0 + split(durations, rows, "\n") + for (i in rows) { + split(rows[i], f, "\t") + if (f[1] == job || index(f[1], job " (") == 1) samples[++n] = f[2] + 0 + } + p95 = 0 + if (n > 0) { + asort(samples) + idx = int(((95 * n) + 99) / 100) + if (idx < 1) idx = 1 + p95 = samples[idx] + } + printf "job\t%s\t%s\t%s\t%d\t%d\t%s\n", file, job, declared, p95, n, basis + } + /^jobs:/ { injobs = 1; next } + injobs && /^[a-z]/ { injobs = 0 } + injobs && /^ [A-Za-z0-9_-]+:[[:space:]]*$/ { + flush() + job = $1; sub(":", "", job); declared = ""; basis = "measured" + } + injobs && job != "" && /^ timeout-minutes:[[:space:]]*[0-9]+/ { + value = $0 + sub(/^[[:space:]]*timeout-minutes:[[:space:]]*/, "", value) + sub(/[^0-9].*$/, "", value) + declared = value + basis = ($0 ~ /grandfathered/) ? "grandfathered" : "measured" + } + END { flush() } + ' "$wf" + done +} + +emit | cargo run --quiet -p batten -- record named timeout-drift +''' + [tasks.branch-age-record] description = "Effect: record every remote branch's age in days and which names head more than one merged PR (CLOUD-349)" shell = "bash -c" diff --git a/policy/attestation.rego b/policy/attestation.rego new file mode 100644 index 000000000..9d9cd820a --- /dev/null +++ b/policy/attestation.rego @@ -0,0 +1,193 @@ +# A release's binaries carry build provenance (CLOUD-583, ported under CLOUD-1717). +# +# THE WHOLE DESIGN IS ONE DISTINCTION, and the retired program's header names it: +# `gh attestation verify` exits 1 both when an artifact has no provenance and when +# the platform never offered any, and those are opposite facts. The first is a +# release to fix; the second is a plan feature this private repository does not +# have. A gate that cannot tell them apart is worse than no gate, because it reds +# every release for a reason no branch causes. +# +# The control that separates them is the endpoint's own status code, and it stays +# the producer's reading: where attestation IS available an unknown digest answers +# 200 with an empty array, and 404 on the resource is the feature being absent. +# `[tasks.attestation-record]` probes with an all-zeros digest and records which. +# +# WHY THE SPAWN STAYS OUTSIDE, and it is not a convention. The verifier is a +# process — `gh attestation verify` over a binary unpacked from a downloaded +# archive — and house style §5 makes `check` `read` and structurally incapable of +# spawning one. So what moves in here is not the verification but the ADJUDICATION +# of what the verifier said, which is CLOUD-1559's reading rule: carry the +# decisions, not the steps. The download, the unpack and the verify are steps. +# +# A 404 POSTURE FIRES NOTHING, deliberately, and that is the ported behaviour +# rather than a gap. The retired program exited 0 and reported the gap, because +# nothing there is a claim about an artifact — `release-artifacts.yml` runs its +# attestation step `continue-on-error: true` for the same fact, so a release is +# published unattested BY DESIGN until the repository is public (CLOUD-585). The +# record being present with `posture 404` is what keeps that readable: it says the +# producer looked and the platform offers none, where an absent record says nobody +# looked at all. +# +# COULD-NOT-LOOK IS AN ABSENT RECORD, and every one of the retired program's exit-2 +# arms is now the producer refusing at write time — no credential, no remote, a +# failed download, an unreadable status, a posture that is neither 200 nor 404. The +# producer refuses while its author is watching and writes nothing; this module +# then says nothing, which is the honest reading and not a pass. Carrying the +# shell's exit 2 in here would have made could-not-look a VIOLATION on the engine's +# contract, where 2 means a finding. +#MUTANT-SUITE crates/batten/tests/it/attestation.rs +#MUTANT unverified-passes|s@^\tentry.verdict == "unverified"$@\tfalse@|an_unverified_archive_is_reported_over_the_engines_projection +#MUTANT gap-judged-as-unverified|s@^\tposture == "200"$@\ttrue@|a_platform_gap_judges_nothing_even_with_archives_recorded +#MUTANT empty-tag-passes|s@^\tcount(archives) == 0$@\tfalse@|a_tag_carrying_no_archive_is_refused_rather_than_read_as_clean + +# METADATA +# description: | +# Bound to the TREE surface: this row is `scope = "tree"`, so it reads +# `input.tree` and never the mediated call. +# THIS BLOCK IS YAML AND MUST STAY THE LAST COMMENT BLOCK BEFORE `package`. +# schemas: +# - input: schema["policy-input.schema"] +package batten.attestation + +import rego.v1 + +rules contains "release ship unsafe" + +rules contains "release carry missing" + +rules contains "release list empty" + +# The producer's lines, or nothing. `recorded` being undefined is the +# could-not-look the header describes, and every rule below inherits it. +recorded := input.tree.records.attestation + +# `posture ` — the one line that decides whether anything below is judged +# at all. One line, because the producer makes one probe. +posture := columns[1] if { + some raw in recorded + columns := split(raw, "\t") + count(columns) == 2 + columns[0] == "posture" +} + +# `archive ` — one per archive the producer downloaded, carrying +# what the verifier said about the BINARY inside it. +# +# THE SUBJECT IS THE BINARY, NOT THE ARCHIVE, and that is the retired program's own +# correction to its issue's wording. `release-artifacts.yml` attests the binary +# deliberately, so that repackaging cannot launder the claim — verifying a +# `.tar.gz` would compute a digest nothing ever attested. +archives contains {"name": columns[1], "verdict": columns[2]} if { + some raw in recorded + columns := split(raw, "\t") + count(columns) == 3 + columns[0] == "archive" +} + +# An archive whose binary the verifier refused. +# +# GATED ON THE POSTURE, which is the distinction this module exists for: with the +# platform absent the verifier refuses everything, so firing here would report +# every release as unverifiable for a reason no branch causes. +violation contains { + "rule": "release ship unsafe", + "verdict": "release ship unsafe", + "subjects": [{"artifact": entry.name}], +} if { + posture == "200" + some entry in archives + entry.verdict == "unverified" +} + +# An archive carrying no executable to verify at all. +# +# A DIFFERENT FINDING FROM THE ONE ABOVE rather than a variant of it: an archive +# whose binary failed verification is a provenance problem, and one carrying no +# binary is a packaging problem. Collapsing them would send a reader after a +# signing identity when the dist matrix dropped a file. +violation contains { + "rule": "release carry missing", + "verdict": "release carry missing", + "subjects": [{"artifact": entry.name}], +} if { + posture == "200" + some entry in archives + entry.verdict == "no-binary" +} + +# A TAG THE PRODUCER LOOKED AT AND FOUND NO ARCHIVE ON. The retired program said +# why this is a refusal rather than a pass: "a green verdict would be about +# nothing". Present-and-empty and absent must not collapse, which is the same +# three-valued reading `posture` carries one rule up. +violation contains { + "rule": "release list empty", + "verdict": "release list empty", +} if { + posture == "200" + count(archives) == 0 +} + +# --- cases --------------------------------------------------------------- + +tree(lines) := {"tree": {"records": {"attestation": lines}}} + +available(lines) := tree(array.concat(["posture\t200"], lines)) + +test_an_unverified_archive_is_refused if { + some v in violation with input as available(["archive\tbatten-x86_64.tar.gz\tunverified"]) + v.verdict == "release ship unsafe" +} + +test_a_verified_archive_is_clean if { + count(violation) == 0 with input as available(["archive\tbatten-x86_64.tar.gz\tverified"]) +} + +# POINTER, NEVER PAYLOAD: the asset name, never a bundle, a digest, or a byte of +# the verifier's report — the retired program's rule 4 boundary, kept. +test_the_finding_names_the_archive_and_nothing_else if { + some v in violation with input as available(["archive\tbatten-x86_64.tar.gz\tunverified"]) + v.subjects == [{"artifact": "batten-x86_64.tar.gz"}] +} + +test_an_archive_with_no_binary_is_its_own_finding if { + some v in violation with input as available(["archive\tbatten-x86_64.tar.gz\tno-binary"]) + v.verdict == "release carry missing" +} + +# THE GAP IS NOT A VERDICT (CLOUD-585). With the platform offering no attestation +# the verifier refuses every artifact, so judging here would red every release for +# a reason no branch causes. +test_a_platform_gap_judges_nothing if { + count(violation) == 0 with input as tree([ + "posture\t404", + "archive\tbatten-x86_64.tar.gz\tunverified", + ]) +} + +# AND THE GAP IS STILL A READING. This case is why `posture` is recorded at all +# rather than inferred from an empty archive list: a present 404 record says the +# producer looked, where the case below says nobody did. +test_a_gap_record_is_present_and_readable if { + posture == "404" with input as tree(["posture\t404"]) +} + +test_no_record_at_all_says_nothing if { + count(violation) == 0 with input as {"tree": {"records": {}}} +} + +test_a_tag_carrying_no_archive_is_refused if { + some v in violation with input as tree(["posture\t200"]) + v.verdict == "release list empty" +} + +# A LINE THIS READER CANNOT PARSE IS SKIPPED, the posture every other record reader +# here takes: the producer refuses a malformed line at write time, so an +# unparseable line at read time is a torn store. The surviving good line is part of +# the case — without it the record holds no archive and `release list empty` fires, +# which would let this pass for a reason that has nothing to do with skipping. +test_a_line_this_reader_cannot_parse_is_skipped if { + count(violation) == 0 with input as available([ + "archive\tbatten-x86_64.tar.gz\tverified", + "nonsense", + ]) +} diff --git a/policy/timeout-drift.rego b/policy/timeout-drift.rego new file mode 100644 index 000000000..377d711a2 --- /dev/null +++ b/policy/timeout-drift.rego @@ -0,0 +1,237 @@ +# A job's committed timeout budget still matches measured reality (CLOUD-266, +# ported under CLOUD-1717). +# +# REPORT, NEVER GATE, and the severity is the whole of that. `timeout-check` asks +# a question about the COMMIT — is every timeout justified — and belongs on the +# landing path. This asks whether the justification is still TRUE, which is a +# question about the world and changes with no diff. The retired program said so +# by failing only its own scheduled run: no issue filed, no comment posted, +# nothing blocked. `severity = "warn"` is that posture expressed on the engine's +# contract — a warn finding is reported without failing the run. +# +# DRIFT IS REPORTED IN BOTH DIRECTIONS, and the loose direction is the point. A +# budget gone slack because the job got faster is the ratchet the row exists for; +# a report that only complained about tightness would let every number rot upward +# forever. +# +# WHY THE MEASUREMENT STAYS OUTSIDE. The samples are durations of successful runs +# read from the Actions API, and the p95 over them is arithmetic over instants — +# two things this surface cannot do. §5 makes `check` `read` and incapable of +# spawning, and `Fact::Instant` projects `null` to every module, which +# `clippy.toml`'s `disallowed-methods` and `crates/batten/tests/clock_ban.rs` hold +# the engine to. So `[tasks.timeout-drift-record]` reads the runs, subtracts the +# instants, computes the p95 and records a row per job; the classification — which +# is the decision — is here. CLOUD-1559: carry the decisions, not the steps. +# +# A SMALL SAMPLE REPORTS `unmeasurable`, NEVER A NUMBER, and that arm is load +# bearing rather than defensive. Ten of the fourteen jobs run weekly or on +# release, so a naive p95 over a two-week window would compute a confident value +# from two samples and propose tightening a release job on it. Below the declared +# minimum the job is reported as uncharacterised, which is itself the useful +# signal. +# +# COULD-NOT-LOOK IS AN ABSENT RECORD. Every exit-2 arm of the retired program — no +# `gh`, a query that would not answer, an unreadable workflow directory — is now +# the producer refusing at write time and writing nothing. Reporting a healthy +# budget as drifted on a network blip is the failure mode that gets a scheduled +# gate switched off, and an absent record cannot do it. +#MUTANT-SUITE crates/batten/tests/it/timeout_drift.rs +#MUTANT loose-budget-passes|s@^\tentry.declared > (entry.justified + slack)$@\tfalse@|a_slack_budget_is_reported_as_loose_over_the_engines_projection +#MUTANT tight-budget-passes|s@^\tentry.declared < entry.justified$@\tfalse@|a_budget_the_measurement_has_outgrown_is_reported_as_tight +#MUTANT small-sample-yields-a-number|s@^\tentry.samples < minimum$@\tfalse@|a_job_with_too_few_samples_is_unmeasurable_rather_than_fast + +# METADATA +# description: | +# Bound to the TREE surface: this row is `scope = "tree"`, so it reads +# `input.tree` and never the mediated call. +# THIS BLOCK IS YAML AND MUST STAY THE LAST COMMENT BLOCK BEFORE `package`. +# schemas: +# - input: schema["policy-input.schema"] +package batten.timeout_drift + +import rego.v1 + +rules contains "bound pin loose" + +rules contains "bound pin wrong" + +rules contains "bound pin stale" + +rules contains "bound measure partial" + +# The repo-wide headroom multiplier `timeout-check` gates, and the two thresholds +# the retired program carried as knobs. +# +# IN THE MODULE RATHER THAN IN CONFIG, on `repetition-without-progress`'s +# reasoning: these are the practice's own figures, and a config knob invites +# raising them until nothing is ever reported. Moving one costs a diff a reviewer +# reads. The retired program took them from the environment, which is how a +# scheduled report ends up measured against numbers nobody chose — its own +# `fail_input` arm existed because a typo'd knob read as a working setting. +multiplier := 3 + +minimum := 5 + +slack := 5 + +# The producer's lines, or nothing. +recorded := input.tree.records["timeout-drift"] + +# `job ` — one per +# job the producer characterised. `basis` is `measured` or `grandfathered`: a dated +# debt entry that now HAS a usable sample is a different report from a number that +# has drifted, because the remedy is to convert it rather than to move it. +jobs contains entry if { + some raw in recorded + columns := split(raw, "\t") + count(columns) == 7 + columns[0] == "job" + regex.match(data.batten.patterns["whole-number"], columns[3]) + regex.match(data.batten.patterns["whole-number"], columns[4]) + regex.match(data.batten.patterns["whole-number"], columns[5]) + entry := { + "file": columns[1], + "name": columns[2], + "declared": to_number(columns[3]), + "justified": justified(to_number(columns[4])), + "samples": to_number(columns[5]), + "basis": columns[6], + } +} + +# `ceil(p95 * multiplier / 60)`, the same arithmetic `timeout-check` gates. +# +# `ceil` RATHER THAN THE SHELL'S `(x + 59) / 60`, and the difference is not +# cosmetic: that idiom is integer division, and Rego's `/` is float — carrying it +# over verbatim would make `justified(120)` 6.98 rather than 6, so every +# comparison below would read against a number no budget can equal. The shell's +# arithmetic is a STEP; what carries is the decision it computed. +justified(p95) := ceil((p95 * multiplier) / 60) + +# A JOB NOBODY CAN CHARACTERISE, reported first because the arms below would +# otherwise compute a confident classification from two samples. +violation contains { + "rule": "bound measure partial", + "verdict": "bound measure partial", + "subjects": [{"artifact": entry.name}, {"count": entry.samples}], +} if { + some entry in jobs + entry.samples < minimum +} + +# THE DEBT ENTRY THAT CAN NOW BE CONVERTED. This is the prompt, never the +# conversion: a bot re-baselining the number it is supposed to defend is the one +# move §4 forbids outright. +violation contains { + "rule": "bound pin stale", + "verdict": "bound pin stale", + "subjects": [{"artifact": entry.name}, {"count": entry.justified}], +} if { + some entry in jobs + entry.samples >= minimum + entry.basis == "grandfathered" +} + +# THE NUMBER THE MEASUREMENT HAS OUTGROWN — raise it before it starts failing +# healthy runs. +violation contains { + "rule": "bound pin wrong", + "verdict": "bound pin wrong", + "subjects": [{"artifact": entry.name}, {"count": entry.justified}], +} if { + some entry in jobs + entry.samples >= minimum + entry.basis == "measured" + entry.declared < entry.justified +} + +# THE NUMBER THAT HAS GONE SLACK. A budget is a ceiling rather than a target, so +# some headroom is correct and `slack` is what keeps this off every job that +# merely got a little faster. +violation contains { + "rule": "bound pin loose", + "verdict": "bound pin loose", + "subjects": [{"artifact": entry.name}, {"count": entry.justified}], +} if { + some entry in jobs + entry.samples >= minimum + entry.basis == "measured" + entry.declared > entry.justified + slack +} + +# --- cases --------------------------------------------------------------- + +tree(lines) := {"tree": {"records": {"timeout-drift": lines}}} + +# `justified(120) == 6`, so a declared 6 is exactly right, 5 is tight, and 12 is +# loose once the five-minute slack is spent. +job(declared, p95, samples, basis) := tree([sprintf( + "job\tci.yml\tbats\t%d\t%d\t%d\t%s", + [declared, p95, samples, basis], +)]) + +test_a_budget_matching_its_measurement_is_clean if { + count(violation) == 0 with input as job(6, 120, 25, "measured") +} + +test_a_slack_budget_is_reported_as_loose if { + some v in violation with input as job(12, 120, 25, "measured") + v.verdict == "bound pin loose" +} + +# THE SLACK IS A BOUNDARY, not a suggestion: at exactly `justified + slack` the +# budget is still correct, because a ceiling is allowed headroom. +test_a_budget_inside_the_slack_is_not_loose if { + count(violation) == 0 with input as job(11, 120, 25, "measured") +} + +test_a_budget_the_measurement_has_outgrown_is_reported_as_tight if { + some v in violation with input as job(5, 120, 25, "measured") + v.verdict == "bound pin wrong" +} + +# POINTER, NEVER PAYLOAD: the job's name and the minutes the measurement +# justifies, which is what the remedy needs and nothing more. +test_the_report_carries_a_name_and_a_count if { + some v in violation with input as job(12, 120, 25, "measured") + v.subjects == [{"artifact": "bats"}, {"count": 6}] +} + +test_a_job_with_too_few_samples_is_unmeasurable_rather_than_fast if { + some v in violation with input as job(30, 120, 2, "measured") + v.verdict == "bound measure partial" +} + +# AND IT IS THE ONLY REPORT for that job: a classification computed from two +# samples is the confident-and-wrong answer the arm exists to prevent, so a slack +# budget must not ALSO be reported as loose here. +test_an_unmeasurable_job_is_not_also_classified if { + count(violation) == 1 with input as job(30, 120, 2, "measured") +} + +test_a_grandfathered_entry_with_samples_is_a_conversion_prompt if { + some v in violation with input as job(30, 120, 25, "grandfathered") + v.verdict == "bound pin stale" +} + +# A GRANDFATHERED ENTRY IS NOT ALSO DRIFT. Its remedy is to convert the debt to a +# measured budget, and reporting it as loose as well would send the reader to move +# a number they are supposed to replace. +test_a_grandfathered_entry_is_not_also_drift if { + count(violation) == 1 with input as job(30, 120, 25, "grandfathered") +} + +test_no_record_at_all_says_nothing if { + count(violation) == 0 with input as {"tree": {"records": {}}} +} + +# A LINE THIS READER CANNOT PARSE IS SKIPPED — the producer refuses a malformed +# line at write time, so an unparseable one at read time is a torn store. The +# surviving good line keeps this from passing for want of any readable row. +test_a_line_this_reader_cannot_parse_is_skipped if { + count(violation) == 0 with input as tree([ + "job\tci.yml\tbats\t6\t120\t25\tmeasured", + "job\tci.yml\tbroken\tnot-a-number\t120\t25\tmeasured", + "nonsense", + ]) +} diff --git a/tests/attestation-check.bats b/tests/attestation-check.bats deleted file mode 100644 index c7649ad3f..000000000 --- a/tests/attestation-check.bats +++ /dev/null @@ -1,225 +0,0 @@ -#!/usr/bin/env bats -# subject: mise-tasks/attestation-check.sh -# attestation-check's decision table (CLOUD-583). The case that carries the whole -# design is `THE GAP IS NOT A VERDICT`: `gh attestation verify` exits 1 both for -# an artifact with no provenance and for a repository the platform never offered -# any to, and a gate that cannot tell those apart reds every release for a reason -# no branch causes. -# -# Driven against a stubbed `gh`, because the two halves cannot both be real here: -# this repository's endpoint answers 404 (measured), so the available-platform -# branch and every verdict under it would be unreachable. The 404 branch IS -# exercised for real — `mise run attestation-check` against the live repository -# is what the issue's blocker correction is measured with — and the stub is what -# covers the other half. - -setup() { - CHECK="$BATS_TEST_DIRNAME/../mise-tasks/attestation-check.sh" - STUB="$BATS_TEST_TMPDIR/bin" - mkdir -p "$STUB" - PATH="$STUB:$PATH" - export PATH - - # A repository whose origin remote is a github.com URL, so the slug is derived - # the way the task derives it rather than being injected. - ROOT="$BATS_TEST_TMPDIR/repo" - mkdir -p "$ROOT" - git -C "$ROOT" init -q - git -C "$ROOT" remote add origin https://github.com/example-org/example-repo.git - export ATTESTATION_CHECK_ROOT="$ROOT" - export ATTESTATION_GH="$STUB/gh" - export GH_TOKEN=stub-token - - echo 404 >"$BATS_TEST_TMPDIR/status" - stub_gh -} - -# A `gh` whose four sub-behaviours are set independently by sentinel files: -# status the attestations endpoint's status line (default 404) -# verify.fails `attestation verify` exits 1 -# download.fails `release download` exits 1 -# release.empty the release carries no archive -# Every `attestation verify` call appends the BASENAME of the file it was handed -# to `verified`, which is what pins "the binary, not the archive". -stub_gh() { - cat >"$STUB/gh" <"\$work/batten" - tar -czf "\$dir/batten-v9.9.9-x86_64-unknown-linux-gnu.tar.gz" -C "\$work" batten - rm -rf "\$work" - exit 0 - ;; -"attestation verify") - basename "\$3" >>"$BATS_TEST_TMPDIR/verified" - [ ! -f "$BATS_TEST_TMPDIR/verify.fails" ] || exit 1 - echo "Loaded 1 attestation from GitHub API" - exit 0 - ;; -esac -exit 1 -EOF - chmod +x "$STUB/gh" -} - -@test "THE GAP IS NOT A VERDICT: a 404 endpoint reports the platform gap and exits 0" { - run "$CHECK" - [ "$status" -eq 0 ] - [[ "$output" == *"attestation-unavailable"* ]] - # The failure rule id must not appear: nothing here is a claim about an - # artifact, and reporting one would red every release until CLOUD-585 lands. - [[ "$output" != *"attestation-unverified"* ]] - # And nothing was downloaded or verified — the gap short-circuits the world. - [ ! -f "$BATS_TEST_TMPDIR/verified" ] -} - -@test "the gap names the repository it asked about, derived from the remote" { - run "$CHECK" - [[ "$output" == *"example-org/example-repo"* ]] -} - -@test "with the platform available and provenance present, the run passes" { - echo 200 >"$BATS_TEST_TMPDIR/status" - run "$CHECK" - [ "$status" -eq 0 ] - [[ "$output" == *"carry verifiable provenance"* ]] -} - -@test "with the platform available and provenance absent, the run fails" { - echo 200 >"$BATS_TEST_TMPDIR/status" - : >"$BATS_TEST_TMPDIR/verify.fails" - run "$CHECK" - [ "$status" -eq 1 ] - [[ "$output" == *"attestation-unverified"* ]] - # The message must say this is a release to fix rather than a gap to report: - # the two failures have opposite responses. - [[ "$output" == *"release to fix"* ]] -} - -@test "THE SUBJECT IS THE BINARY, NOT THE ARCHIVE" { - # `release-artifacts.yml` attests `steps.dist.outputs.binary`, deliberately, so - # repackaging cannot launder the claim. Verifying the .tar.gz would compute a - # digest nothing ever attested — a failure that means nothing. - echo 200 >"$BATS_TEST_TMPDIR/status" - run "$CHECK" - [ "$status" -eq 0 ] - [ "$(cat "$BATS_TEST_TMPDIR/verified")" = "batten" ] -} - -@test "a release carrying no archive is exit 2, not a green verdict about nothing" { - echo 200 >"$BATS_TEST_TMPDIR/status" - : >"$BATS_TEST_TMPDIR/release.empty" - run "$CHECK" - [ "$status" -eq 2 ] - [[ "$output" == *"about nothing"* ]] -} - -@test "a download that fails is exit 2 — could not look is not a verdict" { - echo 200 >"$BATS_TEST_TMPDIR/status" - : >"$BATS_TEST_TMPDIR/download.fails" - run "$CHECK" - [ "$status" -eq 2 ] - [[ "$output" == *"unverified"* ]] -} - -@test "a status that is neither 200 nor 404 is exit 2, naming the code" { - echo 500 >"$BATS_TEST_TMPDIR/status" - run "$CHECK" - [ "$status" -eq 2 ] - [[ "$output" == *"500"* ]] - [[ "$output" == *"posture is unknown"* ]] -} - -@test "output is pointer-only — no attestation body reaches the log" { - echo 200 >"$BATS_TEST_TMPDIR/status" - run "$CHECK" - [[ "$output" != *"Loaded 1 attestation"* ]] - [[ "$output" != *"attestations\":"* ]] -} - -@test "the precondition holds when the verifier resolves" { - run "$CHECK" --precondition - [ "$status" -eq 0 ] - [[ "$output" == *"precondition holds"* ]] -} - -@test "THE SEVERITY SPLIT: the precondition holds while the platform gap is open" { - # This is why the batten.toml row can be `deny` at all. The endpoint answers - # 404 in this case, as it does for the real repository; if this ever failed, - # the deny row would be enforcing the platform's posture and every landing - # would stop for a reason no branch causes. - echo 404 >"$BATS_TEST_TMPDIR/status" - run "$CHECK" --precondition - [ "$status" -eq 0 ] -} - -@test "the precondition makes no network call" { - # It is what runs on the landing path, so it must answer from local facts - # only (the CLOUD-410 split). A stub that fails every API call proves it. - cat >"$STUB/gh" <<'EOF' -#!/usr/bin/env bash -[ "$1" != "api" ] || exit 7 -exit 0 -EOF - chmod +x "$STUB/gh" - run "$CHECK" --precondition - [ "$status" -eq 0 ] -} - -@test "an absent verifier is exit 2 in precondition mode" { - export ATTESTATION_GH="$BATS_TEST_TMPDIR/no-such-gh" - run "$CHECK" --precondition - [ "$status" -eq 2 ] - [[ "$output" == *"no verifier"* ]] -} - -@test "a missing credential does NOT fail the precondition — cannot-look is not a deny" { - # The row this mode backs is `deny`, so anything ambient in it blocks every - # environment that differs. Measured: an earlier version required GH_TOKEN here - # and reported a violation inside `tests/prebuilt-lint.bats`' fixture repos, - # which carry no credential and no remote. - unset GH_TOKEN - unset GITHUB_TOKEN || true - run "$CHECK" --precondition - [ "$status" -eq 0 ] -} - -@test "a missing credential IS exit 2 in the world half — a 404 could not be told from a denial" { - unset GH_TOKEN - unset GITHUB_TOKEN || true - run "$CHECK" - [ "$status" -eq 2 ] - [[ "$output" == *"GH_TOKEN"* ]] -} - -@test "no github.com remote is exit 2 in the world half, and irrelevant to the precondition" { - git -C "$ROOT" remote remove origin - run "$CHECK" --precondition - [ "$status" -eq 0 ] - run "$CHECK" - [ "$status" -eq 2 ] - [[ "$output" == *"no github.com origin remote"* ]] -} diff --git a/tests/timeout-drift.bats b/tests/timeout-drift.bats deleted file mode 100644 index cbd3fef9b..000000000 --- a/tests/timeout-drift.bats +++ /dev/null @@ -1,175 +0,0 @@ -#!/usr/bin/env bats -# subject: mise-tasks/timeout-drift.sh -# The world half of CLOUD-266: is each committed budget still true? -# -# Driven entirely through a stubbed `gh`, so every verdict is reachable without a -# token and without the network. The scheduled half is otherwise not gated by this -# suite, and that is deliberate: its correctness is that it REPORTS rather than -# blocks, which is a property of where it runs, not of what it computes. - -setup() { - TASK="$BATS_TEST_DIRNAME/../mise-tasks/timeout-drift.sh" - STUB="$BATS_TEST_TMPDIR/bin" - WORKFLOWS="$BATS_TEST_TMPDIR/workflows" - mkdir -p "$STUB" "$WORKFLOWS" - PATH="$STUB:$PATH" - export PATH - # Small window and minimum so a fixture needs a handful of runs, not 25. - export BATTEN_BUDGET_MIN_SAMPLES=5 - export BATTEN_BUDGET_WINDOW=10 -} - -# A one-job workflow carrying the given `timeout-minutes` line. -workflow_with() { - { - printf 'name: t\non:\n push:\njobs:\n build:\n runs-on: ubuntu-latest\n' - printf '%s\n' "$1" - printf ' steps:\n - run: true\n' - } >"$WORKFLOWS/t.yml" -} - -# `gh` answering with `count` successful runs of `job`, each lasting `secs`. -# Run ids are 1..count; every run reports one job. -stub_gh() { - local job=$1 count=$2 secs=$3 - cat >"$STUB/gh" < ()`. -stub_gh_matrix() { - local job=$1 count=$2 secs=$3 - cat >"$STUB/gh" <"$WORKFLOWS/t.yml" - run "$TASK" "$WORKFLOWS" - [ "$status" -eq 0 ] -} - -@test "a failed API query is exit 2, never a drift verdict" { - # Reporting a healthy budget as drifted on a network blip is the failure mode - # that gets a scheduled gate switched off. - stub_gh build 6 100 - touch "$BATS_TEST_TMPDIR/gh.fails" - workflow_with ' timeout-minutes: 5 # budget: p95=100s x3 measured=2026-08-10' - run "$TASK" "$WORKFLOWS" - [ "$status" -eq 2 ] -} - -@test "an absent gh is exit 2, never a pass" { - printf '#!/usr/bin/env bash\nexit 127\n' >"$STUB/gh" - chmod +x "$STUB/gh" - workflow_with ' timeout-minutes: 5 # budget: p95=100s x3 measured=2026-08-10' - # An empty PATH entry for gh: `command -v` must not find it. - rm -f "$STUB/gh" - run env PATH="$STUB:/usr/bin:/bin" "$TASK" "$WORKFLOWS" - [ "$status" -eq 2 ] -} - -@test "a missing workflow directory is exit 2, never a pass" { - stub_gh build 6 100 - run "$TASK" "$BATS_TEST_TMPDIR/absent" - [ "$status" -eq 2 ] -} From 050f9aa5be8037505cf52c8a0949bb1b58591a30 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Thu, 17 Sep 2026 06:38:43 +0000 Subject: [PATCH 06/32] refactor(gate)!: retire nonverdict-scan and nonverdict-assert onto the engine MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CLOUD-484's pair was already split for CLOUD-1559's reason — a measurement needs the network and a token, a decision needs neither — so this port carried each half to the home the engine has for it and no decision changed hands. `nonverdict-assert` becomes `policy/nonverdict.rego`, registered as `job grade other`. `nonverdict-scan` becomes `[tasks.nonverdict-record]`, writing the `nonverdict` family through `batten record named`. The classification predicate could not move: whether a failed job reached a verdict is read off the Actions API's own `steps[]`, which section 5 keeps outside `check` by making it `read` and incapable of spawning. THE EXIT CONTRACT CHANGED, AND FOUR ARMS RIDE ON IT. The decider ran 0 under budget / 1 over / 2 could-not-look; the engine runs 0/1/2/3 where 2 is a FINDING. Both total-blindness arms — an empty roster, an unreadable run list — are now the producer refusing at write time and recording nothing, and an absent family is the module's silence. `unreadable` stays a finding because it is not blindness: it is a window read in PART, which is `bench-assert`'s partial-coverage false green. `severity = "warn"` HOLDS TWO CLAIMS THAT LOOK OPPOSED, and both are the retired pair's own. The decider argued that a rate which rises has to fail something or it becomes an artifact nobody opens; `nonverdict-rate.yml` argued that its failure is informational, because the platform having a bad afternoon is not a branch's fault. A warn finding leaves every `check` and every `verify` green while the scheduled job runs `--fail-on-warning` and reds on the rate. WHAT THIS DELTA SPENDS, stated rather than implied: the producer's classification jq — the predicate CLOUD-113 cost four jobs to get right — loses its dedicated coverage. `tests/nonverdict-scan.bats` stubbed `gh` and exercised it; no compiled-binary case reaches a shell task. The `carried` arms claim the RETIREMENT is covered by the module and the tier, which it is, and the tier's header says plainly that the producer's steps are not. Also fixes CLOUD-1825, which this campaign's own history exposed: `lease::tests::the_base_is_subtracted_rather_than_resent` asserted monotonicity in the base, which `objects_to_send` does not have — its subtraction is against the base's OWN tree, so a commit that deletes a path and a later one that restores it makes the WIDER base strictly smaller. It now compares against `None`, the state its own comment describes, and a new case builds that history and asserts the inversion directly rather than waiting for it to drift through the window again. Closes CLOUD-1825 Refs: CLOUD-1717 Admits: f2bd67dd8d3ff4c720827bbf0b029ef4bbdf39262a8f4b564363a11bbd6e8507 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: mise.toml Admits-anchor: call:23ea6f4471f21d2bd8498a29e0f53fc27ab74141 Admits-epoch: 60ad1cd96a7812346af312aa478ba681b58fa5fa18823b7125d93ae0d15c432a Admits-author: alec@wenzowski.com Admits-prev: a1cbd3a670ac38e341ff70fa991ce1d8e59ab0a4a00b5a4b372683920cec67a2 Admits-answer-lost: `policy/nonverdict.rego` decides over a store nothing writes, which is exactly the could-not-look-reads-as-pass shape this campaign exists to remove. The retirement stalls and `mise-tasks/nonverdict-scan.sh` and `mise-tasks/nonverdict-assert.sh` stay in the corpus. Admits-answer-precondition: A mise task body lives on exactly one surface. `[tasks.nonverdict-record]` is the measuring half of the `nonverdict-scan`/`nonverdict-assert` port: it makes the conditional `gh` reads over the Actions API and classifies each failed required job from its own per-step conclusions — which house style section 5 keeps outside `check` by making it `read` and structurally incapable of spawning. There is no other file a task body can be written in, and it lands beside the module it feeds. Admits-answer-rejected-route: config read first — I read `[tasks.branch-age-record]` and `[tasks.attestation-record]`, the producers this campaign already landed, and matched their shape; the reading is what made this one correct rather than what made it unnecessary, because a producer that does not exist cannot be read into existence. patch run first is a commit-message route and does not apply to a task addition. Admits: 76b95e62e146ec7373c4777122faf5f695347ccf9816abfcc964281e87aedc45 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: batten.toml Admits-anchor: call:23ea6f4471f21d2bd8498a29e0f53fc27ab74141 Admits-epoch: 3fae854077625b7e8d53be38c55b8b1eb765365b836e5c13bf827a6484cf8e07 Admits-author: alec@wenzowski.com Admits-prev: 70dbe2ea3fd418962d2d28a37cf88a43813dacf6132bc959ca0aa3fd0531c74e Admits-answer-lost: `policy/nonverdict.rego` sits in the tree deciding nothing — unregistered, so `batten check` never loads it, and its family unprojected, so the record its producer writes is invisible. The retirement it exists for cannot proceed and `mise-tasks/nonverdict-scan.sh` and `mise-tasks/nonverdict-assert.sh` stay in the corpus. Admits-answer-precondition: A policy module is registered by a `[[rule]]` row and by nothing else; the verdict strings it emits must exist as `[[verdict]]` rows or the engine refuses the module at load; and a verb-written record family reaches no module until a `[[record]]` row declares it. `policy/nonverdict.rego` is a new module with two new verdicts over a new family, so those four rows have to be written into batten.toml directly. There is no other surface any of them lives on, and they land in one diff beside the module they register. Admits-answer-rejected-route: config read first — I read the `release grade unsafe` and `bound grade other` registrations this campaign already landed and copied their shape, and that reading is what produced these rows rather than what avoided them: registration is an addition and cannot be satisfied by reading. patch run first is a commit-message route and does not apply to a config addition. Admits: 5dc96d907bf22d558676023e12aab59fe9b1867f4eb82a14ad68021fcd55d84a Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: policy/nonverdict.rego Admits-anchor: call:23ea6f4471f21d2bd8498a29e0f53fc27ab74141 Admits-epoch: 3fae854077625b7e8d53be38c55b8b1eb765365b836e5c13bf827a6484cf8e07 Admits-author: alec@wenzowski.com Admits-prev: - Admits-answer-lost: `policy/nonverdict.rego` cannot load at all: `policy test` refuses the tree while the inline pattern stands, so the module decides nothing, the `job grade other` row registers a module the engine will not accept, and the `nonverdict-scan`/`nonverdict-assert` retirement cannot land. Admits-answer-precondition: A Rego module's body lives on exactly one surface. `policy test` refuses this module for spelling the regex `^[0-9]+$` inline in `count_of` and names the fix: declare it as a `[[pattern]]` row and read it by id. The row already exists — `whole-number` — so the change is one expression becoming `data.batten.patterns["whole-number"]`, which can only be written in the module that reads it. The module is new in this same diff and the edit is one line a reviewer sees in it. Admits-answer-rejected-route: config read first — reading batten.toml is exactly how I found that `whole-number` is already declared with this regex, so the reading produced the one-line fix rather than removing the need for it; a second copy of the expression is what the refusal exists to stop. patch run first is a commit-message route and does not apply to a module edit. Admits: 54f6fa2aceceb02fa6b9f6cc915d927974a73e9aec9c84a6e2eecc53f2b8dd1c Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: .github/workflows/nonverdict-rate.yml Admits-anchor: call:23ea6f4471f21d2bd8498a29e0f53fc27ab74141 Admits-epoch: 3fae854077625b7e8d53be38c55b8b1eb765365b836e5c13bf827a6484cf8e07 Admits-author: alec@wenzowski.com Admits-prev: - Admits-answer-lost: The only caller of both retired programs keeps calling them by name. The scheduled job fails on every tick with an unknown task, so the rate this gate exists to measure stops being measured — and the retirement lands having silently switched the sensor off, which is worse than the state before it. Admits-answer-precondition: A workflow's steps live on exactly one surface. `nonverdict-rate.yml` invokes the two programs this delta retires — `mise run nonverdict-scan` to measure and `mise run nonverdict-assert` to judge — so retiring them without repointing this file leaves the scheduled job invoking tasks that no longer exist. The edit replaces the measure step with `mise run nonverdict-record` and the judge step with `mise run batten -- check --rule 'job grade other' --fail-on-warning`, and it is two lines a reviewer sees in the diff that deletes the programs. Admits-answer-rejected-route: config read first — I read the workflow before editing and that reading is what identified both call sites and the `--fail-on-warning` the `warn` severity now requires; reading cannot repoint a caller. patch run first is a commit-message route and does not apply to a workflow edit. Admits: 8229493d85f5887044054e145df454f3cded322bf27e05ee9ac3ea8614211eca Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: policy/nonverdict.rego Admits-anchor: call:23ea6f4471f21d2bd8498a29e0f53fc27ab74141 Admits-epoch: 3fae854077625b7e8d53be38c55b8b1eb765365b836e5c13bf827a6484cf8e07 Admits-author: alec@wenzowski.com Admits-prev: 5dc96d907bf22d558676023e12aab59fe9b1867f4eb82a14ad68021fcd55d84a Admits-answer-lost: `policy/nonverdict.rego` cannot pass `batten-check`: two `deny` findings stand against the module's own cases, so the tree is refused and the `nonverdict-scan`/`nonverdict-assert` retirement cannot land. Leaving the vendor name would also put an unevaluable third-party reference in a decision surface, which is the thing that rule exists to stop. Admits-answer-precondition: A Rego module's fixture data lives in the module that uses it. `prose point other` refuses two of this module's own cases for naming a third party in a step-name fixture without a full 40-hex pin — the string was invented by me and names a vendor for no reason, since what the case needs is any step that is not a `mise run` one. Replacing it with `Set up job`, a real GitHub provisioning step name, can only be written in the module that holds the case. The module is new in this same diff. Admits-answer-rejected-route: config read first — reading the `prose point other` row is exactly how I learned the exclusion is a 40-hex pin and that the vendor name was gratuitous here; the reading produced the fix rather than removing the need for it. patch run first is a commit-message route and does not apply to a module edit. --- .github/workflows/nonverdict-rate.yml | 21 +- batten.toml | 63 +++++ bench/gates/RESULTS.md | 1 - bench/suites/RESULTS.md | 2 - crates/batten/src/lease.rs | 109 +++++++- crates/batten/tests/it/main.rs | 1 + crates/batten/tests/it/nonverdict.rs | 342 ++++++++++++++++++++++++++ mise-tasks/nonverdict-assert.sh | 148 ----------- mise-tasks/nonverdict-scan.sh | 269 -------------------- mise.toml | 292 +++++++++++++++++++++- policy/nonverdict.rego | 267 ++++++++++++++++++++ tests/nonverdict-assert.bats | 140 ----------- tests/nonverdict-scan.bats | 228 ----------------- 13 files changed, 1087 insertions(+), 796 deletions(-) create mode 100644 crates/batten/tests/it/nonverdict.rs delete mode 100755 mise-tasks/nonverdict-assert.sh delete mode 100755 mise-tasks/nonverdict-scan.sh create mode 100644 policy/nonverdict.rego delete mode 100644 tests/nonverdict-assert.bats delete mode 100644 tests/nonverdict-scan.bats diff --git a/.github/workflows/nonverdict-rate.yml b/.github/workflows/nonverdict-rate.yml index d2b05bd55..34ce92e54 100644 --- a/.github/workflows/nonverdict-rate.yml +++ b/.github/workflows/nonverdict-rate.yml @@ -20,8 +20,9 @@ name: nonverdict-rate # THIS MUST NOT RUN IN THE GATE OR IN `verify`. It is a property of the world, and # a branch must not go red because the platform had a bad afternoon — # coverage.yml:15-18 states the same boundary for the same reason. What IS gated is -# the decision: `tests/nonverdict-assert.bats` runs on every commit, because the -# decider is a pure function of records. +# the decision: `crates/batten/tests/it/nonverdict.rs` and +# `policy/nonverdict.rego`'s own cases run on every commit, because the decider +# is a pure function of records. # # Failure here is informational: it means the rate crossed its budget and wants # looking at, not that anything in the tree is broken. Nothing here writes to the @@ -73,11 +74,16 @@ jobs: # GH_TOKEN through env, never interpolated into the run block: a `${{ }}` # expansion is textual substitution into the shell script, which zizmor gates # at high severity. + # THE SPLIT THE PORT FORCED (CLOUD-1717). The producer makes the + # conditional `gh` reads and classifies each failed job from its own + # per-step conclusions; the engine adjudicates what it recorded. Neither + # half can hold the other: section 5 makes `check` `read` and incapable of + # spawning the reads below. - name: Measure which required-check failures never reached a verdict shell: bash env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - run: mise run nonverdict-scan > "$RUNNER_TEMP/nonverdict.txt" + run: mise run nonverdict-record > "$RUNNER_TEMP/nonverdict.txt" - name: Publish the records shell: bash run: | @@ -91,7 +97,14 @@ jobs: # Always, so the records above are published even when the budget is broken — # the number is the point, and a summary that only appears on green is a # report nobody reads at the moment it matters. + # `--fail-on-warning` IS WHAT KEEPS THE GATE A GATE. The row is + # `severity = "warn"` so that an ordinary `check` — and therefore every + # `verify` and every branch — stays green over a property of the runner + # fleet no branch causes. This job is the one reader that must red on it, + # which is the posture the retired decider had and the reason it exists: + # "a rate that rises has to fail something, or it becomes another artifact + # nobody opens". - name: Judge the rate against its budget if: always() shell: bash - run: mise run nonverdict-assert < "$RUNNER_TEMP/nonverdict.txt" + run: mise run batten -- check --rule 'job grade other' --fail-on-warning diff --git a/batten.toml b/batten.toml index ef14d8452..7299730cf 100644 --- a/batten.toml +++ b/batten.toml @@ -2150,6 +2150,14 @@ writer = "mise run attestation-record" record = "timeout-drift" writer = "mise run timeout-drift-record" +# CLOUD-484's measuring half, ported off `nonverdict-scan.sh` under CLOUD-1717. +# The decider half is `policy/nonverdict.rego`; the split is the retired pair's +# own, kept for its own reason — a measurement needs the network and a token, a +# decision needs neither. +[[record]] +record = "nonverdict" +writer = "mise run nonverdict-record" + [[pattern]] id = "closed-issue-status" regex = '^(done|canceled|duplicate)$' @@ -6296,6 +6304,28 @@ scope = "tree" module = "policy/timeout-drift.rego" severity = "warn" +# CLOUD-484, ported off `mise-tasks/nonverdict-assert.sh` under CLOUD-1717. +# +# `warn` HOLDS TWO CLAIMS THAT LOOK OPPOSED, and both are the retired pair's own. +# The decider argued that a rate which rises has to FAIL something or it becomes +# an artifact nobody opens (non-negotiable rule 2) — it was filed against twice, +# both times by someone deciding the number was not worth collecting. +# `nonverdict-rate.yml` argued just as plainly that its failure is informational: +# a job dying in provisioning is the platform having a bad afternoon, no branch +# is at fault, and nothing in the tree is broken. +# +# A warn finding is reported without failing the run, so an ordinary `check` and +# every `verify` stay green — which is what stops a branch going red for a reason +# it did not cause — while the scheduled job runs `--fail-on-warning` and reds on +# the rate. The gate is preserved and its blast radius is the clock, not the +# landing path. +[[rule]] +id = "job grade other" +kind = "policy" +scope = "tree" +module = "policy/nonverdict.rego" +severity = "warn" + [[rule]] id = "plan cover partial" kind = "policy" @@ -13179,6 +13209,39 @@ id = "task run first" kind = "command" target = "mise run timeout-drift-record" +# CLOUD-484's two readings, ported off `nonverdict-assert.sh` under CLOUD-1717. +[[verdict]] +id = "job read partial" +gloss = "the scan could not read part of its window, so a green verdict would cover less than it claims" +class = """ +`bench-assert`'s partial-coverage rule: a run that measured two of three paths \ +and reported green over the two is exactly the partial-coverage false green. It \ +fires whatever the count is, because a budget met over part of a window is a \ +budget met over nothing in particular. A persistent read failure is a token or \ +rate-limit problem, not a clean window. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run nonverdict-record" + +[[verdict]] +id = "job answer missing" +gloss = "a required job failed before reaching any verdict-bearing step" +class = """ +The run spent its minutes, redded the branch, and answered nothing. Every \ +occurrence then costs a human or an agent the time to discover it was never a \ +verdict at all — CLOUD-404 has five, and the one on #376 sent an agent to \ +reproduce a `commit-lint` failure that passes locally because `commit-lint` \ +never linted anything. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run nonverdict-record" + # CLOUD-349's two readings, ported off `branch-age-check.sh` under CLOUD-1717. [[verdict]] id = "branch watch stale" diff --git a/bench/gates/RESULTS.md b/bench/gates/RESULTS.md index 44f6f3ab5..85a0ea629 100644 --- a/bench/gates/RESULTS.md +++ b/bench/gates/RESULTS.md @@ -114,7 +114,6 @@ commit, and a named ref resolves a declared one. | `msrv-pin-agreement.sh` | tree | — | | `mutant.sh` | git | `location`, `tracked` | | `no-doctests.sh` | git | `location`, `tracked` | -| `nonverdict-assert.sh` | tree | — | | `ntia-check.sh` | git | `location` | | `perf-assert.sh` | tree | — | | `perf-compare.sh` | tree | — | diff --git a/bench/suites/RESULTS.md b/bench/suites/RESULTS.md index 512df6527..21215695f 100644 --- a/bench/suites/RESULTS.md +++ b/bench/suites/RESULTS.md @@ -49,7 +49,6 @@ to it pays. | 1.0 | 0.6% | `tests/linear-check.bats` | | 1.0 | 0.6% | `tests/lint-rego.bats` | | 0.9 | 0.6% | `tests/land-divergence-assert.bats` | -| 0.9 | 0.6% | `tests/nonverdict-scan.bats` | | 0.9 | 0.6% | `tests/deferral-check.bats` | | 0.9 | 0.6% | `tests/spawn-census.bats` | | 0.9 | 0.5% | `tests/done-check.bats` | @@ -82,7 +81,6 @@ to it pays. | 0.3 | 0.2% | `tests/report-only-check.bats` | | 0.3 | 0.2% | `tests/license-table-check.bats` | | 0.3 | 0.2% | `tests/board-payloads.bats` | -| 0.3 | 0.2% | `tests/nonverdict-assert.bats` | | 0.3 | 0.2% | `tests/timeout-check.bats` | | 0.3 | 0.2% | `tests/release-due.bats` | | 0.3 | 0.2% | `tests/transcript-corpus-check.bats` | diff --git a/crates/batten/src/lease.rs b/crates/batten/src/lease.rs index 60a6a4f22..ebdebdbc0 100644 --- a/crates/batten/src/lease.rs +++ b/crates/batten/src/lease.rs @@ -5153,6 +5153,28 @@ mod tests { fn the_base_is_subtracted_rather_than_resent() { // The whole economy of the push. Without the subtraction a lap would // re-send the repository's entire history every time. + // + // AGAINST `None`, NEVER AGAINST A WIDER BASE (CLOUD-1825). This compared + // `HEAD~3` with `HEAD~1` and asserted the wider range enumerated at least + // as much — monotonicity in the base, which `objects_to_send` does not + // have and never claimed. Its subtraction is against the base's OWN TREE, + // so a base whose tree carries MORE subtracts more: one commit that + // deletes a path and a later one that restores it makes the WIDER base + // strictly smaller, because it still holds the blobs and the narrow one + // does not. + // + // Measured rather than imagined: it fired on this repository's own + // history when a commit deleted 105 generated `man/*.1` pages and the + // next restored them, and what it reported was a defect in this assertion + // rather than in the function. It reds `verify`, which is what mints the + // receipt `turn mint ahead` demands before any further write — and that + // row declares no override route — so a false failure here locks a + // session out of editing the very assertion that is wrong. + // + // `None` is the honest comparand. It is the state the first comment + // describes and the one `objects_to_send` documents as "a ref the remote + // does not have yet … nothing is hidden, nothing is subtracted", so no + // base can ever enumerate more than it, for any history shape. let repo = std::path::Path::new("."); let Ok(head) = crate::git::head_commit(repo) else { return; @@ -5160,15 +5182,96 @@ mod tests { let Ok(narrow) = crate::git::objects_to_send(repo, Some("HEAD~1"), &head) else { return; }; - let Ok(wide) = crate::git::objects_to_send(repo, Some("HEAD~3"), &head) else { + let Ok(whole) = crate::git::objects_to_send(repo, None, &head) else { return; }; assert!( - wide.len() >= narrow.len(), - "a wider range cannot enumerate fewer objects: {} vs {}", + whole.len() >= narrow.len(), + "subtracting a base cannot enumerate more than sending everything: {} vs {}", + whole.len(), + narrow.len() + ); + } + + /// The history shape that refuted the old assertion, built rather than waited + /// for (CLOUD-1825). + /// + /// The case above runs against THIS repository, so what it covers is whatever + /// history happens to be checked out — which is how a false assertion survived + /// until a delete-then-restore pair drifted into its window, and how it would + /// quietly stop being covered once the pair drifted back out. A one-line + /// comparand change with nothing holding it in place invites the next author to + /// restore the wider-base comparison, because that one reads as the more + /// thorough of the two. + /// + /// So the shape is constructed: `HEAD~1` deletes a path `HEAD` brings back. + /// The wider base still carries the blob and subtracts it; the narrow one does + /// not and must re-send it. That is the inversion, and asserting it directly is + /// what makes the property above a claim about the FUNCTION rather than about + /// the checkout. + #[test] + fn a_wider_base_may_enumerate_fewer_objects_than_a_narrower_one() { + let dir = scratch("delete-then-restore"); + let git = |args: &[&str]| { + let out = std::process::Command::new("git") + .args(args) + .current_dir(&dir) + .env("GIT_AUTHOR_NAME", "t") + .env("GIT_AUTHOR_EMAIL", "t@example.com") + .env("GIT_COMMITTER_NAME", "t") + .env("GIT_COMMITTER_EMAIL", "t@example.com") + .output() + .expect("git runs"); + assert!(out.status.success(), "git {args:?} failed"); + }; + git(&["init", "-q", "-b", "main"]); + // Big enough that the inversion cannot be an artefact of one small blob + // rounding the other way, and distinct per file so none of them dedupe. + for n in 0..24 { + std::fs::write(dir.join(format!("page-{n}.txt")), format!("page {n}\n")) + .expect("write a page"); + } + std::fs::write(dir.join("keep.txt"), "keep\n").expect("write the survivor"); + git(&["add", "-A"]); + git(&["commit", "-qm", "the pages exist"]); + + for n in 0..24 { + std::fs::remove_file(dir.join(format!("page-{n}.txt"))).expect("delete a page"); + } + git(&["add", "-A"]); + git(&["commit", "-qm", "a partial regeneration deletes them"]); + + for n in 0..24 { + std::fs::write(dir.join(format!("page-{n}.txt")), format!("page {n}\n")) + .expect("restore a page"); + } + git(&["add", "-A"]); + git(&["commit", "-qm", "and the remedy restores them"]); + + let head = crate::git::head_commit(&dir).expect("resolve the fixture head"); + let narrow = + crate::git::objects_to_send(&dir, Some("HEAD~1"), &head).expect("the narrow base"); + let wide = + crate::git::objects_to_send(&dir, Some("HEAD~2"), &head).expect("the wider base"); + + assert!( + wide.len() < narrow.len(), + "the wider base still holds the deleted blobs and subtracts them, so it must \ + enumerate FEWER than the narrow one: wide {} vs narrow {}", wide.len(), narrow.len() ); + + // And the bound the case above asserts still holds over the same history, + // which is the point: `None` is sound where a wider base is not. + let whole = crate::git::objects_to_send(&dir, None, &head).expect("no base at all"); + assert!( + whole.len() >= narrow.len() && whole.len() >= wide.len(), + "no base can enumerate more than sending everything: whole {} vs narrow {} / wide {}", + whole.len(), + narrow.len(), + wide.len() + ); } #[test] diff --git a/crates/batten/tests/it/main.rs b/crates/batten/tests/it/main.rs index c0d2a10f5..41c2791d0 100644 --- a/crates/batten/tests/it/main.rs +++ b/crates/batten/tests/it/main.rs @@ -190,6 +190,7 @@ mod mutate; mod mutation_declared_case; mod narrow_adoption; mod nextest_slow; +mod nonverdict; mod obligations_bound; mod outcome_advice; mod perf_assert; diff --git a/crates/batten/tests/it/nonverdict.rs b/crates/batten/tests/it/nonverdict.rs new file mode 100644 index 000000000..57cc3cb1e --- /dev/null +++ b/crates/batten/tests/it/nonverdict.rs @@ -0,0 +1,342 @@ +//! `job grade other` over the compiled binary (CLOUD-484, CLOUD-1717). +//! +//! # Why this tier and not the module's own `test_` rules +//! +//! Every case in `policy/nonverdict.rego` fabricates its input with +//! `with input as`, which cannot see a fact the engine never projects — the state +//! `policy/branch-age.rego` sat in for a whole session while its own suite stayed +//! green (CLOUD-1810). These run the real module over a record the real verb +//! wrote. +//! +//! # RETIREMENT LEDGER, PER PATH — what `shell retire partial` reads +//! +//! TWO PROGRAMS, TWO SUCCESSORS, AND THE SPLIT WAS ALREADY THERE. The pair was +//! `nonverdict-scan` (measure) and `nonverdict-assert` (decide), kept apart for +//! exactly CLOUD-1559's reason: a measurement needs the network and a token, a +//! decision needs neither. So the port carried each half to the home the engine +//! has for it — the scan to `[tasks.nonverdict-record]`, the assert to +//! `policy/nonverdict.rego` — and no decision changed hands. +//! +//! THE SCAN'S ARMS ARE `carried`, AND `ported` WOULD HAVE BEEN WRONG. That marker +//! is admissible only where the dying file's DECLARED subject lives on, read from +//! its own base header rather than from the author — it exists for the sixteen +//! suites whose `# subject:` is an immortal like `mise.toml` or `hk.pkl`. This +//! suite declares `mise-tasks/nonverdict-scan.sh`, which dies in this same delta, +//! so the subject did not survive and `ported` cannot spell it. +//! +//! WHAT THE ARM CLAIMS, PRECISELY, is what `timeout-drift`'s already-landed arm +//! claims about the same shape: the RETIREMENT is carried by +//! `policy/nonverdict.rego` and this tier. It does not claim the producer's own +//! steps are covered here, and they are not — whether a failed step is a verdict +//! is read off the Actions API's `steps[]` by jq inside `[tasks.nonverdict-record]`, +//! which §5 keeps outside `check` and which no compiled-binary case reaches. That +//! is the coverage this delta spends, stated rather than implied. +//! +//! THE POSTURE IS PRESERVED BY SEVERITY, AND IT IS TWO CLAIMS AT ONCE. The +//! retired decider argued at length that a rate which rises has to FAIL something +//! or it becomes an artifact nobody opens (non-negotiable rule 2), and +//! `nonverdict-rate.yml` argued just as plainly that its failure is informational: +//! the platform had a bad afternoon, no branch is at fault, and nothing in the +//! tree is broken. `severity = "warn"` holds both — an ordinary `check` stays +//! green, and the scheduled job runs `--fail-on-warning` and reds on the rate. +//! +//! THE EXIT CONTRACT CHANGED, AND FOUR ARMS RIDE ON IT. The decider ran `0` under +//! budget / `1` over / `2` could-not-look. The engine runs `0/1/2/3` where `2` is a +//! FINDING, so carrying the shell's `2` over would have turned every could-not-look +//! into a violation. `unreadable` is the one case that stayed a finding, because it +//! is not blindness: it is a window read in PART, which is `bench-assert`'s +//! partial-coverage false green. +//! +// carried: mise-tasks/nonverdict-scan.sh policy/nonverdict.rego kind:mechanism crates/batten/tests/it/nonverdict.rs +// carried: tests/nonverdict-scan.bats policy/nonverdict.rego kind:mechanism crates/batten/tests/it/nonverdict.rs +// carried: mise-tasks/nonverdict-assert.sh policy/nonverdict.rego kind:mechanism crates/batten/tests/it/nonverdict.rs +// carried: tests/nonverdict-assert.bats policy/nonverdict.rego kind:mechanism crates/batten/tests/it/nonverdict.rs +// carried: "under budget is a pass, and says what it judged" policy/nonverdict.rego kind:mechanism +// carried: "THE ACCEPTANCE CASE: over budget fails and names each non-verdict failure" policy/nonverdict.rego kind:mechanism +// carried: "a VERDICT failure is not counted, however many there are" policy/nonverdict.rego kind:mechanism +// carried: "COULD NOT LOOK: an unreadable run in the window is exit 2, never a pass" policy/nonverdict.rego kind:mechanism +// carried: "an unreadable run is exit 2 even when the count is under budget" policy/nonverdict.rego kind:mechanism +// carried: "ANTI-VACUITY: an empty window exits 0 and says it judged nothing" policy/nonverdict.rego kind:mechanism +// carried: "POINTER, NEVER PAYLOAD: the report carries no step output, only coordinates" policy/nonverdict.rego kind:mechanism +// changed: "empty stdin is exit 2, not a clean window" mise.toml there is no stdin: the decider's input is the record family, and an ABSENT family is could-not-look, which on the engine's contract must read as silence rather than as the exit 2 that now means a finding. The property the case was protecting is kept on the other side of the door — the producer refuses and writes nothing rather than recording an empty window +// changed: "records with no window summary are exit 2 — there is no window to judge" mise.toml the summary is emitted unconditionally by the producer in window mode and the producer refuses in every arm that cannot reach one, so a family with records and no summary is a torn store rather than an incomplete scan; the module leaves it unjudged, which is what "there is no window to judge" means once could-not-look cannot be spelled as a finding +// changed: "two concatenated scans are exit 2 — a count over both describes neither" policy/nonverdict.rego the reading is unchanged and only its spelling moved: a count over both still describes neither, so neither is judged. `record named` replaces a family rather than appending to it, so two DIFFERENT summaries can now arrive only through a torn store, and a torn store is silence here for the same reason an unparseable line is skipped +// changed: "a non-numeric count is exit 2 rather than being coerced to zero" policy/nonverdict.rego the refusal to coerce is the whole of the case and it is kept — `count_of` is undefined for a value that is not digits, so neither rule fires. What changed is that undefinedness says "unjudged" where the shell's exit 2 would now say "violation" +// changed: "the budget is raise-only overridable, which is how the window is retuned" policy/nonverdict.rego the override had exactly one reader — the suite, pointing the budget at a fixture. A module's cases vary the COUNTS against a fixed `budget := 2` instead, which is `timeout-drift.rego`'s placement for its multipliers, so the knob is gone because the reader it existed for is +// changed: "THE ACCEPTANCE CASE: a job that died before any mise step is a non-verdict failure" mise.toml the classification reads the Actions API's own per-step conclusions, which §5 makes `check` incapable of fetching; it is the producer's jq, verbatim +// changed: "a job that failed IN a mise step rendered a verdict and is not counted" mise.toml the same jq over the same payload: naming the verdict-bearing step is the producer's half of the closed predicate +// changed: "a job that failed in a mise EXEC step rendered a verdict too" mise.toml the second spelling CLOUD-113 cost four jobs to learn, kept in the producer where the step names are read +// changed: "THE FAN-IN IS EXCLUDED: final's needs-assertion is not a non-verdict failure" mise.toml the exclusion is by job name over the API payload, before any record exists, so it is part of what the producer measures rather than of what the module judges +// changed: "A JOB OUTSIDE THE ROSTER IS EXCLUDED: a declining merge bot is not a failure here" mise.toml the roster is `$CI_REQUIRED_CHECKS` from `mise.toml [env]`, which the producer reads and the engine does not project +// changed: "the conditional request is actually sent once an ETag is stored" mise.toml an ETag is a property of an HTTP request, and the request is the producer's +// changed: "A 304 KEEPS THE PREVIOUS READING rather than reading as an empty window" mise.toml the cache lives under `.git/` and is read on the request path, so the 304 reading never reaches a record at all +// changed: "a 304 with no cached body is unreadable, never an empty window" mise.toml the same request path: with no body to fall back on the producer counts the run as unreadable, which is the input to `job read partial` rather than a decision of its own +// changed: "an unreadable jobs read is counted, not silently dropped" mise.toml counting is the producer's; what the count MEANS is the module's, and `job read partial` is where it is now decided +// changed: "an empty roster is unreadable rather than a count over every job" mise.toml this one genuinely changed rather than moved: with no roster the producer now refuses and records nothing, because recording `unreadable=1` over an empty window would spell total blindness as the partial-coverage finding, and those are different facts +// changed: "a summary line is always emitted, even when nothing failed" mise.toml the producer still emits it unconditionally, and the reason is now stronger: it is the one line that distinguishes a window that found nothing from nobody having looked +// changed: "POINTER, NEVER PAYLOAD: records carry coordinates, and no log is fetched" mise.toml no log body is ever fetched, so the property is enforced where the fetching would happen; the module's own pointer discipline is asserted below + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use common::{git_in, init_repo, run, run_with_stdin, scratch, write}; + +/// A repository registering the real module against a declared family. +fn repo(name: &str) -> std::path::PathBuf { + let dir = scratch(&format!("nonverdict-{name}")); + let module = std::fs::read_to_string("../../policy/nonverdict.rego") + .expect("the module this tier exists for"); + write(&dir, "policy/nonverdict.rego", &module); + write( + &dir, + "batten.toml", + r#"version = 1 +scope = ["**"] + +# The module reads its count guard by id rather than spelling the expression +# inline, which `policy test` refuses: an expression is a consumer fact and +# belongs in the config (rule 1). A fixture that omitted the row would make every +# reference undefined and every rule below silent. +[[pattern]] +id = "whole-number" +regex = '^[0-9]+$' + +[[verdict]] +id = "job read partial" +gloss = "the scan could not read part of its window, so a green verdict would cover less than it claims" +class = "Partial coverage reported as a clean window is the false green this sensor exists to report." + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run nonverdict-record" + +[[verdict]] +id = "job answer missing" +gloss = "a required job failed before reaching any verdict-bearing step" +class = "The run spent its minutes and answered nothing." + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run nonverdict-record" + +[[rule]] +id = "job grade other" +kind = "policy" +scope = "tree" +module = "policy/nonverdict.rego" +severity = "warn" + +[[record]] +record = "nonverdict" +writer = "mise run nonverdict-record" +"#, + ); + init_repo(&dir); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-qm", "register the module"]); + dir +} + +/// Write the producer's record, as `mise run nonverdict-record` would. +fn record(dir: &std::path::Path, lines: &str) { + let written = run_with_stdin(dir, &["record", "named", "nonverdict"], lines); + assert!( + written.status.success(), + "the setup write lands: {}", + String::from_utf8_lossy(&written.stderr) + ); +} + +/// Both streams: which one carries a finding is the output contract's business, +/// and what these cases assert is that the pointer reaches the reader. +fn said(out: &std::process::Output) -> String { + format!( + "{}{}", + String::from_utf8_lossy(&out.stdout), + String::from_utf8_lossy(&out.stderr) + ) +} + +#[test] +fn an_over_budget_window_is_reported_over_the_engines_projection() { + let dir = repo("over"); + record( + &dir, + "nonverdict\trun=111\tjob=ci\tstep=Run actions/checkout@3d3c42e\n\ + nonverdict\trun=222\tjob=msrv\tstep=Run actions/checkout@3d3c42e\n\ + nonverdict\trun=333\tjob=cross\tstep=Set up job\n\ + window\truns=10\tfailed_jobs=3\tnonverdict=3\tverdict=0\tunreadable=0\n", + ); + + let decided = run(&dir, &["check", "--fail-on-warning"]); + assert_eq!( + decided.status.code(), + Some(2), + "three non-verdict failures against a budget of two decides\n{}", + said(&decided) + ); + assert!( + said(&decided).contains("cross"), + "and the finding names each job\n{}", + said(&decided) + ); +} + +#[test] +fn an_over_budget_window_reports_without_failing_the_run() { + // THE PORTED POSTURE, and the case no load-time rule can make. The retired + // gate failed its own scheduled run and nothing else — it filed no issue and + // posted no comment — because the rate is a property of the runner fleet and + // no branch is at fault. `warn` is that on this contract: the same record that + // reds `--fail-on-warning` above must leave an ordinary `check` green, which + // is what keeps `verify` from going red over the platform's bad afternoon. + let dir = repo("warn"); + record( + &dir, + "nonverdict\trun=111\tjob=ci\tstep=Run actions/checkout@3d3c42e\n\ + nonverdict\trun=222\tjob=msrv\tstep=Run actions/checkout@3d3c42e\n\ + nonverdict\trun=333\tjob=cross\tstep=Set up job\n\ + window\truns=10\tfailed_jobs=3\tnonverdict=3\tverdict=0\tunreadable=0\n", + ); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "a report must not fail the run — nothing in the tree is broken\n{}", + said(&quiet) + ); +} + +#[test] +fn an_under_budget_window_is_clean() { + // The state the gate must be able to reach: two is the platform having a bad + // afternoon, and is deliberately not actionable. + let dir = repo("under"); + record( + &dir, + "nonverdict\trun=111\tjob=ci\tstep=Run actions/checkout@3d3c42e\n\ + nonverdict\trun=222\tjob=msrv\tstep=Run actions/checkout@3d3c42e\n\ + window\truns=10\tfailed_jobs=2\tnonverdict=2\tverdict=0\tunreadable=0\n", + ); + + let quiet = run(&dir, &["check", "--fail-on-warning"]); + assert_eq!( + quiet.status.code(), + Some(0), + "a window inside its budget is clean\n{}", + said(&quiet) + ); +} + +#[test] +fn a_partially_read_window_is_a_finding_rather_than_a_clean_one() { + // `bench-assert`'s partial-coverage rule: a run that measured two of three + // paths and reported green over the two is exactly the partial-coverage false + // green. It fires UNDER budget, which is the whole point. + let dir = repo("partial"); + record( + &dir, + "window\truns=10\tfailed_jobs=0\tnonverdict=0\tverdict=0\tunreadable=3\n", + ); + + let decided = run(&dir, &["check", "--fail-on-warning"]); + assert_eq!( + decided.status.code(), + Some(2), + "a window read in part is a finding even with nothing over budget\n{}", + said(&decided) + ); +} + +#[test] +fn a_verdict_failure_is_never_named_however_many_there_are() { + // A judged branch is the branch's problem. Counting verdicts here would make + // every genuinely red PR look like a platform fault. + let dir = repo("verdicts"); + record( + &dir, + "verdict\trun=111\tjob=ci\tstep=Run mise run test:cargo\n\ + verdict\trun=222\tjob=msrv\tstep=Run mise exec -- cargo check\n\ + window\truns=10\tfailed_jobs=2\tnonverdict=0\tverdict=2\tunreadable=0\n", + ); + + let quiet = run(&dir, &["check", "--fail-on-warning"]); + assert_eq!( + quiet.status.code(), + Some(0), + "a window of verdicts judges nothing here\n{}", + said(&quiet) + ); +} + +#[test] +fn an_empty_window_is_a_reading_rather_than_a_finding() { + // ANTI-VACUITY. A window with no runs in it cannot fire, and this repo has + // been bitten twice by a gate that cannot fire reading the same as one that + // found nothing (`finding-sink-check`, `bench-assert`). The record's PRESENCE + // is what keeps the two apart, which the case below is the other half of. + let dir = repo("empty"); + record( + &dir, + "window\truns=0\tfailed_jobs=0\tnonverdict=0\tverdict=0\tunreadable=0\n", + ); + + let quiet = run(&dir, &["check", "--fail-on-warning"]); + assert_eq!( + quiet.status.code(), + Some(0), + "an empty window judged nothing and says so by existing\n{}", + said(&quiet) + ); +} + +#[test] +fn an_absent_record_says_nothing_rather_than_passing() { + // Both total-blindness arms of the retired scan — an empty roster, an + // unreadable run list — are now the producer refusing and writing nothing. + let dir = repo("absent"); + + let quiet = run(&dir, &["check", "--fail-on-warning"]); + assert_eq!( + quiet.status.code(), + Some(0), + "an absent record is could-not-look\n{}", + said(&quiet) + ); +} + +#[test] +fn the_report_names_every_job_that_answered_nothing() { + // ONE FINDING PER JOB, which is what a reader acts on: the count is what + // decides, and the coordinates are what sends them somewhere. A single + // finding carrying "3 jobs" would name none of them. + // + // The rendered line is the LEADING subject and nothing else, which is rule 4 + // holding at the output contract — the run id and step name ride on the + // finding for a structured reader, and `policy/nonverdict.rego`'s own + // `test_the_finding_carries_coordinates_and_nothing_else` is where that is + // pinned. Asserting the run id HERE would be asserting the renderer's shape, + // not this module's. + let dir = repo("pointer"); + record( + &dir, + "nonverdict\trun=111\tjob=commit-lint\tstep=Run actions/checkout@3d3c42e\n\ + nonverdict\trun=222\tjob=msrv\tstep=Run actions/checkout@3d3c42e\n\ + nonverdict\trun=333\tjob=cross\tstep=Set up job\n\ + window\truns=10\tfailed_jobs=3\tnonverdict=3\tverdict=0\tunreadable=0\n", + ); + + let decided = run(&dir, &["check", "--fail-on-warning"]); + let text = said(&decided); + for job in ["commit-lint", "msrv", "cross"] { + assert!( + text.contains(job), + "every job that spent its minutes and answered nothing is named; {job} is not\n{text}" + ); + } +} diff --git a/mise-tasks/nonverdict-assert.sh b/mise-tasks/nonverdict-assert.sh deleted file mode 100755 index e25555791..000000000 --- a/mise-tasks/nonverdict-assert.sh +++ /dev/null @@ -1,148 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Gate: how many recent CI jobs failed WITHOUT reaching a verdict is under budget (reads `nonverdict-scan` records on stdin)" -# -# CLOUD-484. A job that dies in provisioning — checkout, the toolchain install, a -# cache restore — spends its runner minutes, reds the branch, -# and answers nothing. Every occurrence then costs a human or an agent the time to -# discover it was never a verdict at all: CLOUD-404 has three, and the one on #376 -# sent an agent to reproduce a `commit-lint` failure that passes locally because -# `commit-lint` never linted anything. -# -# CLOUD-483 stops `land` MISREADING one in the moment. This answers the question -# neither of those does: **how often**. Without it the escalation trigger is -# "whoever gets bitten next", so a step change — a runner-image regression, an -# upstream action shipping a bad release, a cache-key change that cold-starts every -# job — is invisible until it has been paid for repeatedly. -# -# THIS DECIDES; IT DOES NOT MEASURE. `mise run nonverdict-scan` reads the API and -# emits records; this is a pure function of those records, the same split -# `bench`/`bench-assert` uses and for the same reason: a measurement needs the -# network and a token, a decision needs neither, so keeping them apart is what lets -# `tests/nonverdict-assert.bats` run in the hk gate on every commit while the -# measurement runs on a clock. -# -# WHY A GATE AND NOT A REPORT. Non-negotiable rule 2: a log without a gate is -# sensor only. A rate that rises has to fail something, or it becomes another -# artifact nobody opens — which is the failure mode this issue was filed against -# twice, both times by someone deciding the number was not worth collecting. -# -# Exit 0 under budget / 1 over / 2 could not look. `2` is the `lock-complete`, -# `timeout-check` and `bench-assert` doctrine — "the gate could not read what it -# was asked to judge" — and it is deliberately distinct from a violation, because a -# gate that reports green over input it failed to parse is the failure that gets a -# gate switched off. -# A gate listed in $MUTANT_GATES with no row here fails `mise run mutant`. -#MUTANT over-budget-passes|s/^exit 1$/exit 0/|over budget fails and names each non-verdict failure - -set -euo pipefail - -# The budget: how many non-verdict job failures the window may carry before this -# fails. Written here as data, with a raise-only env override — the placement -# `bench-assert`'s BUDGETS, `branch-age-check`'s BRANCH_MAX_AGE_DAYS and -# `timeout-drift`'s multipliers all use. -# -# NOT in `batten.toml [ci]`: that table is a projection of what the GitHub HOST -# enforces and `ci-drift` polices it against the live ruleset, so a key with no -# host counterpart would make that gate judge something nothing declares. NOT in -# `mise.toml [env]` either: that is for values two or more programs share, and this -# one has a single reader. -# -# Two, not zero. One provisioning failure in a window is the platform having a bad -# afternoon and is not actionable; a third in the same window is a pattern, and the -# whole point is to hear about the pattern before the minutes are spent. The -# override is what lets the suite point the budget at a fixture. -MAX_NONVERDICT="${BATTEN_NONVERDICT_MAX:-2}" - -# Pointer-only per non-negotiable rule 4: run ids, job names, step names and -# counts. `nonverdict-scan` never emits a log body, so there is none to leak here. -report() { - echo " $1" >&2 -} - -records=$(cat) -if [[ -z "${records//[[:space:]]/}" ]]; then - echo "::error:: nonverdict-assert: stdin is empty — pipe \`mise run nonverdict-scan\` records in (redirect to a file, then read it back; a pipeline would hand this gate's exit status to its last stage)." >&2 - exit 2 -fi - -# Records are TAB-separated, because job and step names carry spaces -# ("Run mise run commit-lint", "darwin-link (aarch64-apple-darwin)") and a -# space-separated key=value stream cannot represent them. One kind per line: -# -# nonverdict\trun=\tjob=\tstep= -# verdict\trun=\tjob=\tstep= -# window\truns=\tfailed_jobs=\tnonverdict=\tverdict=\tunreadable= -# -# `verdict` lines are carried so the ratio is derivable and so a window that found -# only verdicts is distinguishable from one that found nothing — the same reason -# `capture` stores a zero-byte stream rather than skipping it. -# -# Literal-pattern awk, never a `-v` regex: a pattern reaching awk through `-v` goes -# through assignment escape processing first, which `mise run awk-regex-check` -# refuses for being implementation-defined. -summary=$(awk -F'\t' '$1 == "window" { print; found = 1 } END { exit !found }' <<<"$records") || summary="" -if [[ -z "$summary" ]]; then - echo "::error:: nonverdict-assert: the records carry no \`window\` summary line, so there is no window to judge — did \`nonverdict-scan\` complete?" >&2 - exit 2 -fi - -# More than one summary means two scans were concatenated, and the counts below -# would silently describe neither window. -if [[ "$(awk -F'\t' '$1 == "window"' <<<"$records" | grep -c .)" != "1" ]]; then - echo "::error:: nonverdict-assert: stdin carries more than one \`window\` summary — two scans were concatenated, and a count over both describes neither." >&2 - exit 2 -fi - -field() { - awk -F'\t' -v key="$1" ' - $1 == "window" { - for (i = 2; i <= NF; i++) { - n = index($i, "=") - if (n > 0 && substr($i, 1, n - 1) == key) { print substr($i, n + 1); exit } - } - } - ' <<<"$records" -} - -runs=$(field runs) -nonverdict=$(field nonverdict) -unreadable=$(field unreadable) - -for pair in "runs:$runs" "nonverdict:$nonverdict" "unreadable:$unreadable"; do - name="${pair%%:*}" - value="${pair#*:}" - if [[ -z "$value" ]] || [[ "$value" != "${value#*[^0-9]}" ]]; then - echo "::error:: nonverdict-assert: the \`window\` summary carries no readable \`$name\` count, so the window cannot be judged." >&2 - exit 2 - fi -done - -# A scan that could not read part of its window judged less than it claims. This is -# `bench-assert`'s partial-coverage rule: "a run that measured two of three paths -# and reported green over the two is exactly the partial-coverage false green". -if [[ "$unreadable" != "0" ]]; then - echo "::error:: nonverdict-assert: the scan could not read $unreadable run(s) in its window, so a green verdict here would cover less than it claims." >&2 - report "re-run \`mise run nonverdict-scan\`; a persistent read failure is a token or rate-limit problem, not a clean window" - exit 2 -fi - -# ANTI-VACUITY. A window with no runs in it cannot fire, and a gate that cannot -# fire must not be indistinguishable from one that found nothing — this repo has -# been bitten by that twice (`finding-sink-check`, `bench-assert`). -if [[ "$runs" = "0" ]]; then - echo "nonverdict-assert: no runs in the window — nothing to judge" >&2 - exit 0 -fi - -if [[ "$nonverdict" -le "$MAX_NONVERDICT" ]]; then - echo "nonverdict-assert: $nonverdict of $runs run(s) failed without reaching a verdict (budget $MAX_NONVERDICT)" - exit 0 -fi - -echo "::error:: nonverdict-assert: $nonverdict job(s) in the last $runs run(s) failed WITHOUT reaching a verdict, over the budget of $MAX_NONVERDICT. Those runs spent minutes and answered nothing." >&2 -while IFS= read -r line; do - [[ -n "$line" ]] || continue - report "$(awk -F'\t' '{ sub(/^nonverdict\t/, ""); print }' <<<"$line")" -done <<<"$(awk -F'\t' '$1 == "nonverdict"' <<<"$records")" -report "each is a job that died before any \`mise run\` step; see CLOUD-404 for the recurring upstream cause and CLOUD-483 for the in-loop remedy" -exit 1 diff --git a/mise-tasks/nonverdict-scan.sh b/mise-tasks/nonverdict-scan.sh deleted file mode 100755 index 0a0133aa7..000000000 --- a/mise-tasks/nonverdict-scan.sh +++ /dev/null @@ -1,269 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Measure: which recent CI job failures never reached a verdict (one record per failure, byte-stable; `nonverdict-assert` decides)" -# -# CLOUD-484. THIS MEASURES; IT DOES NOT DECIDE. The verdict is -# `mise run nonverdict-assert`'s, a pure function of the records below — the -# `bench`/`bench-assert` split, for the same reason: a measurement needs the network -# and a token, a decision needs neither, so the decider's suite runs in the hk gate -# on every commit while this runs on a clock. -# -# ## The predicate, and why it is CLOSED -# -# CI runs this repo's work through the pinned toolchain and nothing else runs it, -# which `mise run ci-local-parity` gates ("every task CI runs is one `verify` -# runs"). So: -# -# a failed job rendered a VERDICT iff one of its failed steps is named -# `Run mise run ` or `Run mise exec -- `; otherwise it failed -# WITHOUT reaching a verdict. -# -# `mise exec` IS THE SECOND SPELLING, added after the first cost four jobs in one -# lap (CLOUD-113). This read `Run mise run ` alone, on the stated premise -# that it is the only way our work reaches a runner — and the `windows` job broke -# that premise for a measured reason: its task body is bash, so `mise run` puts -# Git Bash between the runner and cargo, MSYS rewrites PATH across that boundary, -# and the toolchain the preflight just resolved is invisible three processes down. -# It runs `mise exec -- cargo test --workspace` instead, which the job's own -# comment justifies at length. -# -# The consequence nobody costed: seven genuinely failing tests then classified as -# a PROVISIONING TRANSIENT, so `land` re-ran the job four times, absorbed four -# real verdicts, and reported "the provisioning path is broken" about a suite that -# was reproducibly red. A predicate whose premise has gone false does not fail -# loudly — it answers confidently and wrongly, in the direction that spends money. -# -# Still CLOSED, which is the property that mattered: this names the two ways a -# verdict is rendered, not the growing set of ways a prelude can fail. -# -# Verified against run 31637905084, whose steps are exactly: -# -# 1 Set up job / 2 Landing lease precondition -# 3 Run actions/checkout@3d3c42e… / 4 Run jdx/mise-action@7e36c90d9ab29c415a2384db3006f3ec8a8cc654 -# 5 Run mise run commit-lint <- the only verdict-bearing step -# 10 Post Run actions/checkout@… / 11 Complete job -# -# The obvious alternative — a declared allowlist of prelude steps (checkout, -# mise-action, cache) — is an OPEN set: it needs an edit every time a setup action -# is added, and a forgotten edit reads a real provisioning failure as a verdict. -# Naming the verdict instead is closed, and rests on an invariant a gate holds. -# -# ## Two exclusions that are part of the predicate, not caveats -# -# * THE `final` FAN-IN. Its failure is manufactured by its siblings: run -# 31637881076 concluded `cancelled` overall while `final` concluded `failure` -# with its only failed step `Assert all required jobs passed` — a bare `run:` -# step, not a `mise run` one. Counted naively, `final` would read as a -# non-verdict failure on EVERY genuinely red branch. `checks-green` guards the -# same shape by testing no-answer before red (CLOUD-334, CLOUD-351). -# * RUNS CONCLUDED `cancelled`. That is the landing lease declining an -# unauthorised branch (CLOUD-420), which is CLOUD-470's category. Counting it -# here would double-count a case that already has an owner. The `?status=failure` -# query excludes them, and `skip_run` re-checks rather than trusting the filter. -# -# ## Conditional reads, and the one place this EXTENDS ci-wait -# -# `ci-wait` keeps its ETag in a shell scalar for ONE url, alive only for the -# lifetime of one poll. Neither property survives here: a window over N runs is N -# urls, and a scheduled tick is a new process every time — so an in-memory ETag -# would make every tick a cold read and "a quiet tick costs no rate limit" false. -# The store is therefore a MAP KEYED BY URL, PERSISTED between runs, the -# out-of-tree posture `epoch.rs`'s cache and the `.git/batten-*` receipts use. -# -# A 304 KEEPS THE PREVIOUS READING, which is why the body is cached beside the -# ETag. `mem:github-rest-etiquette` is explicit that re-parsing an empty 304 body -# reads as "nothing found", and that the failure then shows up only as unexplained -# slowness. A 304 with no cached body is `unreadable`, never an empty window. -# -# Requests are SERIAL, never concurrent: secondary rate limits are about shape. -# -# OUTPUT IS A POINTER, NEVER A PAYLOAD (rule 4): run ids, job names, step names and -# counts. No log body is ever fetched, so none can leak. TAB-separated, because job -# and step names carry spaces and a space-separated key=value stream cannot hold -# them. -set -euo pipefail - -cd "$(git rev-parse --show-toplevel)" - -# How many recent failed runs to look at. A count, and deliberately not a date -# range: a window of runs is stable to re-read and cannot silently widen when the -# repo gets busier. -WINDOW="${BATTEN_NONVERDICT_WINDOW:-30}" - -# The cache root. Under `.git/`, so it is out of tree, never committed, and -# discarded with the clone rather than surviving as stale state on a fresh one. -cache="${BATTEN_NONVERDICT_CACHE:-$(git rev-parse --git-dir)/batten-nonverdict}" -mkdir -p "$cache" - -# `gh api -i`, then the status line and the ETag out of the headers, then the body -# after the first blank line — `main-watch`'s form. A non-2xx makes `gh` exit -# non-zero (a 304 included), so the call is guarded rather than trusted: under -# `set -e` an unguarded 304 would abort the scan on the cheapest possible response. -# -# Sets `body` and returns 0 on a usable reading, 1 when it could not look. -conditional_get() { - local url="$1" key etag_file body_file resp status new_etag - key=$(printf '%s' "$url" | sha1sum | cut -d' ' -f1) - etag_file="$cache/$key.etag" - body_file="$cache/$key.body" - - local args=(-i "$url") - if [[ -s "$etag_file" ]]; then - args+=(-H "If-None-Match: $(cat "$etag_file")") - fi - - resp=$(gh api "${args[@]}" 2>/dev/null | tr -d '\r') || true - status=$(printf '%s' "$resp" | sed -n '1s@^HTTP/[0-9.]* \([0-9]*\).*@\1@p') - - # Reassigned only when non-empty: a response that carries no ETag must not - # clear the stored one, or the next tick is a cold read for no reason. - # - # `if`, not `[ -n … ] && …`: under `set -e` a trailing `&&` list that fails is - # a non-zero command mid-function, which aborts the whole scan. Measured — the - # first version exited silently with no records and no summary whenever a - # response carried no ETag, which is indistinguishable from a clean empty - # window and is exactly the false green this sensor exists to report. - new_etag=$(printf '%s' "$resp" | sed -n 's/^[Ee][Tt]ag: //p' | head -n1) - if [[ -n "$new_etag" ]]; then - printf '%s' "$new_etag" >"$etag_file" - fi - - if [[ "$status" = "304" ]]; then - # The whole point of the cache. No body arrives, so the previous reading - # IS the answer; without one there is nothing to fall back to. - [[ -s "$body_file" ]] || return 1 - body=$(cat "$body_file") - return 0 - fi - - case "$status" in - 2*) ;; - *) return 1 ;; - esac - - body=$(printf '%s' "$resp" | awk 'body {print} /^$/ {body=1}') - [[ -n "${body//[[:space:]]/}" ]] || return 1 - printf '%s' "$body" >"$body_file" - return 0 -} - -# THE POPULATION IS THE REQUIRED ROSTER, and getting this wrong was measured -# rather than reasoned. The first version judged every failed job in every -# workflow, and 22 of its 24 hits were `fast-forward` jobs failing at -# `Run sequoia-pgp/fast-forward@ea7628b…` — the merge bot DECLINING a branch that is no -# longer a descendant, which is the landing loop working exactly as designed. A -# sensor whose signal is 90% expected refusals is a sensor nobody reads. -# -# Scoping by workflow FILE would have been the obvious fix and is wrong: `ci`, -# `cross`, `darwin-link`, `msrv`, `semver` and `final` live in `ci.yml`, but -# `commit-lint`, `zizmor` and `action` do not — and `commit-lint` is the job that -# carried the CLOUD-404 503. Filtering by path would have dropped the motivating -# case. -# -# So the population is `$CI_REQUIRED_CHECKS`, the one roster `ci-wait` and `land`'s -# `graded_runs` already share (`mise.toml [env]`). A third reader of that value is -# what the value is for; a private copy here is the drift CLOUD-327 cost. -REQUIRED="${CI_REQUIRED_CHECKS:-}" -if [[ -z "${REQUIRED//[[:space:]]/}" ]]; then - echo "::error:: nonverdict-scan: CI_REQUIRED_CHECKS is empty — without the roster this cannot tell a required job from an unrelated one, and a count over all of them is meaningless. Run through \`mise run\` so mise.toml's [env] applies." >&2 - printf 'window\truns=0\tfailed_jobs=0\tnonverdict=0\tverdict=0\tunreadable=1\n' - exit 0 -fi - -# The fan-in, excluded by name: its failure is manufactured by its siblings. -FAN_IN="final" - -unreadable=0 -runs_seen=0 -records="" - -# SINGLE-RUN MODE (CLOUD-483). `land` asks the same question about ONE concluded -# run, on its red-CI stop path, to tell a provisioning transient from a verdict. -# It is a mode rather than a second task because the classification must exist -# once: two readers deciding "did this job reach a verdict" from two copies of the -# jq is the CLOUD-351 shape, and the copy that drifts is the one nobody runs on a -# clock. -# -# It emits RECORDS ONLY, no `window` summary — there is no window to summarise, -# and a caller testing "every record is nonverdict" must not have to special-case -# a trailing line that is neither. An empty stream is therefore unambiguous: no -# failed required job was classifiable, which is "could not look" and never -# "nothing was wrong". -one_run="" -if [[ "${1:-}" = "--run" ]]; then - one_run="${2:-}" - if [[ -z "${one_run//[[:space:]]/}" ]]; then - echo "::error:: nonverdict-scan: --run needs a run id. Without one there is nothing to classify, and an empty record stream would read as 'no failed job reached a verdict'." >&2 - exit 2 - fi -fi - -if [[ -n "$one_run" ]]; then - run_ids="$one_run" -elif ! conditional_get "repos/{owner}/{repo}/actions/runs?status=failure&per_page=$WINDOW"; then - echo "::error:: nonverdict-scan: could not read the run list, so this window judged nothing." >&2 - printf 'window\truns=0\tfailed_jobs=0\tnonverdict=0\tverdict=0\tunreadable=1\n' - exit 0 -else - # `.id` and `.conclusion` only. Sorted by id so the record stream is byte-stable - # across ticks that see the same window in a different order. - run_ids=$(printf '%s' "$body" | jq -r '[.workflow_runs[]? | select(.conclusion == "failure") | .id] | sort | .[]' 2>/dev/null) || run_ids="" -fi - -failed_jobs=0 -nonverdict=0 -verdict=0 - -while IFS= read -r run; do - [[ -n "$run" ]] || continue - runs_seen=$((runs_seen + 1)) - - if ! conditional_get "repos/{owner}/{repo}/actions/runs/$run/jobs?per_page=100"; then - unreadable=$((unreadable + 1)) - continue - fi - - # One line per failed job: the job name, then whether any FAILED step is a - # `Run mise run …` step. The classification is done in jq over the payload - # rather than in shell over a rendered string, so a job name containing a tab - # cannot forge a field. - while IFS=$'\t' read -r job kind step; do - [[ -n "$job" ]] || continue - failed_jobs=$((failed_jobs + 1)) - if [[ "$kind" = "verdict" ]]; then - verdict=$((verdict + 1)) - else - nonverdict=$((nonverdict + 1)) - fi - records="${records}${kind}"$'\t'"run=$run"$'\t'"job=$job"$'\t'"step=$step"$'\n' - done <<<"$( - printf '%s' "$body" | jq -r --arg fan "$FAN_IN" --arg required "$REQUIRED" ' - ($required | split(",") | map(gsub("^\\s+|\\s+$"; ""))) as $roster - | [.jobs[]? - | select(.conclusion == "failure") - | select(.name != $fan) - | select([.name] | inside($roster))] - | sort_by(.name) - | .[] - | . as $job - | ([$job.steps[]? | select(.conclusion == "failure")] | sort_by(.number)) as $failed - | ([$failed[] | select(.name | startswith("Run mise run ") or startswith("Run mise exec -- "))] | first) as $task - | if $task - then [$job.name, "verdict", $task.name] - else [$job.name, "nonverdict", (($failed | first | .name) // "unknown")] - end - | @tsv - ' 2>/dev/null - )" -done <<<"$run_ids" - -# Records first, sorted, then the summary — so the stream is byte-stable and the -# decider can read the summary without buffering the whole thing. -if [[ -n "$records" ]]; then - printf '%s' "$records" | LC_ALL=C sort -fi -# No summary in single-run mode: there is no window, and a caller asking "is every -# record a nonverdict" must not have to discount a trailing line that is neither. -if [[ -z "$one_run" ]]; then - printf 'window\truns=%s\tfailed_jobs=%s\tnonverdict=%s\tverdict=%s\tunreadable=%s\n' \ - "$runs_seen" "$failed_jobs" "$nonverdict" "$verdict" "$unreadable" -fi diff --git a/mise.toml b/mise.toml index adbac2f05..464cccde3 100644 --- a/mise.toml +++ b/mise.toml @@ -617,7 +617,7 @@ CI_FANIN_WORKFLOW = ".github/workflows/ci.yml" BATS_TEST_TIMEOUT = "300" REGORUS_OPA_COMPLIANCE = "1.2.0" REGORUS_OPA_COMPLIANCE_FOR = "0.11" -MUTANT_GATES = "mise,attestation,timeout-drift,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure-check,evaluator-io-check,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence-assert,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link-check,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict-assert,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus-check,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" +MUTANT_GATES = "mise,attestation,timeout-drift,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure-check,evaluator-io-check,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence-assert,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link-check,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus-check,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" # --- GitHub reachability behind an egress proxy (Claude Code web sandbox etc.) --- # mise resolves every tool's release through GitHub's *API* host, api.github.com. @@ -2110,6 +2110,296 @@ prs=$(gh pr list --state merged --limit 200 --json headRefName --jq '.[].headRef } | cargo run --quiet -p batten -- record named branch-age """ +[tasks.nonverdict-record] +description = "Effect: record which recent required-check failures never reached a verdict, one line per failure plus a window summary (CLOUD-484)" +# THE MEASURING HALF OF A SPLIT THAT ALREADY EXISTED (CLOUD-1717). This carries +# `mise-tasks/nonverdict-scan.sh`'s body; `policy/nonverdict.rego` carries +# `mise-tasks/nonverdict-assert.sh`'s decision. The pair was already apart for +# CLOUD-1559's reason — a measurement needs the network and a token, a decision +# needs neither — so the port moved two halves into their homes rather than +# splitting anything new. House style §5 makes `check` `read` and structurally +# incapable of spawning the `gh` reads below. +# +# COULD-NOT-LOOK REFUSES HERE AND RECORDS NOTHING. The retired scan answered a +# total read failure with `unreadable=1` over an otherwise empty window, which the +# decider turned into its exit 2. On the engine's contract exit 2 is a FINDING, so +# carrying that spelling over would have made blindness a violation. Both arms now +# fail loudly while their author is watching and write no family at all, and an +# absent family is the module's silence. `unreadable` survives for the case it was +# really for: a window read in PART, which is a finding rather than blindness. +# +# ## The predicate, and why it is CLOSED +# +# CI runs this repo's work through the pinned toolchain and nothing else runs it, +# which `mise run ci-local-parity` gates ("every task CI runs is one `verify` +# runs"). So: +# +# a failed job rendered a VERDICT iff one of its failed steps is named +# `Run mise run ` or `Run mise exec -- `; otherwise it failed +# WITHOUT reaching a verdict. +# +# `mise exec` IS THE SECOND SPELLING, added after the first cost four jobs in one +# lap (CLOUD-113). This read `Run mise run ` alone, on the stated premise +# that it is the only way our work reaches a runner — and the `windows` job broke +# that premise for a measured reason: its task body is bash, so `mise run` puts +# Git Bash between the runner and cargo, MSYS rewrites PATH across that boundary, +# and the toolchain the preflight just resolved is invisible three processes down. +# It runs `mise exec -- cargo test --workspace` instead. +# +# The consequence nobody costed: seven genuinely failing tests then classified as +# a PROVISIONING TRANSIENT, so `land` re-ran the job four times, absorbed four +# real verdicts, and reported "the provisioning path is broken" about a suite that +# was reproducibly red. A predicate whose premise has gone false does not fail +# loudly — it answers confidently and wrongly, in the direction that spends money. +# +# Still CLOSED, which is the property that mattered: this names the two ways a +# verdict is rendered, not the growing set of ways a prelude can fail. The obvious +# alternative — an allowlist of prelude steps — is an OPEN set that needs an edit +# every time a setup action is added, and a forgotten edit reads a real +# provisioning failure as a verdict. +# +# ## Two exclusions that are part of the predicate, not caveats +# +# * THE `final` FAN-IN. Its failure is manufactured by its siblings: a run can +# conclude `cancelled` overall while `final` concludes `failure` with its only +# failed step `Assert all required jobs passed` — a bare `run:` step, not a +# `mise run` one. Counted naively, `final` would read as a non-verdict failure on +# EVERY genuinely red branch. `checks-green` guards the same shape by testing +# no-answer before red (CLOUD-334, CLOUD-351). +# * RUNS CONCLUDED `cancelled`. That is the landing lease declining an +# unauthorised branch (CLOUD-420), which is CLOUD-470's category. Counting it +# here would double-count a case that already has an owner. The `?status=failure` +# query excludes them, and the `.conclusion == "failure"` filter re-checks rather +# than trusting it. +# +# ## Conditional reads, and the one place this EXTENDS ci-wait +# +# `ci-wait` keeps its ETag in a shell scalar for ONE url, alive only for the +# lifetime of one poll. Neither property survives here: a window over N runs is N +# urls, and a scheduled tick is a new process every time — so an in-memory ETag +# would make every tick a cold read and "a quiet tick costs no rate limit" false. +# The store is therefore a MAP KEYED BY URL, PERSISTED between runs, the +# out-of-tree posture `epoch.rs`'s cache and the `.git/batten-*` receipts use. +# +# A 304 KEEPS THE PREVIOUS READING, which is why the body is cached beside the +# ETag. `mem:github-rest-etiquette` is explicit that re-parsing an empty 304 body +# reads as "nothing found", and that the failure then shows up only as unexplained +# slowness. A 304 with no cached body is `unreadable`, never an empty window. +# +# Requests are SERIAL, never concurrent: secondary rate limits are about shape. +# +# OUTPUT IS A POINTER, NEVER A PAYLOAD (rule 4): run ids, job names, step names and +# counts. No log body is ever fetched, so none can leak. TAB-separated, because job +# and step names carry spaces and a space-separated key=value stream cannot hold +# them. +shell = "bash -c" +run = ''' +set -euo pipefail + +cd "$(git rev-parse --show-toplevel)" + +# How many recent failed runs to look at. A count, and deliberately not a date +# range: a window of runs is stable to re-read and cannot silently widen when the +# repo gets busier. +WINDOW="${BATTEN_NONVERDICT_WINDOW:-30}" + +# The cache root. Under `.git/`, so it is out of tree, never committed, and +# discarded with the clone rather than surviving as stale state on a fresh one. +cache="${BATTEN_NONVERDICT_CACHE:-$(git rev-parse --git-dir)/batten-nonverdict}" +mkdir -p "$cache" + +# `gh api -i`, then the status line and the ETag out of the headers, then the body +# after the first blank line — `main-watch`'s form. A non-2xx makes `gh` exit +# non-zero (a 304 included), so the call is guarded rather than trusted: under +# `set -e` an unguarded 304 would abort the scan on the cheapest possible response. +# +# Sets `body` and returns 0 on a usable reading, 1 when it could not look. +conditional_get() { + local url="$1" key etag_file body_file resp status new_etag + key=$(printf '%s' "$url" | sha1sum | cut -d' ' -f1) + etag_file="$cache/$key.etag" + body_file="$cache/$key.body" + + local args=(-i "$url") + if [[ -s "$etag_file" ]]; then + args+=(-H "If-None-Match: $(cat "$etag_file")") + fi + + resp=$(gh api "${args[@]}" 2>/dev/null | tr -d '\r') || true + status=$(printf '%s' "$resp" | sed -n '1s@^HTTP/[0-9.]* \([0-9]*\).*@\1@p') + + # Reassigned only when non-empty: a response that carries no ETag must not + # clear the stored one, or the next tick is a cold read for no reason. + # + # `if`, not `[ -n … ] && …`: under `set -e` a trailing `&&` list that fails is + # a non-zero command mid-function, which aborts the whole scan. Measured — the + # first version exited silently with no records and no summary whenever a + # response carried no ETag, which is indistinguishable from a clean empty + # window and is exactly the false green this sensor exists to report. + new_etag=$(printf '%s' "$resp" | sed -n 's/^[Ee][Tt]ag: //p' | head -n1) + if [[ -n "$new_etag" ]]; then + printf '%s' "$new_etag" >"$etag_file" + fi + + if [[ "$status" = "304" ]]; then + # The whole point of the cache. No body arrives, so the previous reading + # IS the answer; without one there is nothing to fall back to. + [[ -s "$body_file" ]] || return 1 + body=$(cat "$body_file") + return 0 + fi + + case "$status" in + 2*) ;; + *) return 1 ;; + esac + + body=$(printf '%s' "$resp" | awk 'body {print} /^$/ {body=1}') + [[ -n "${body//[[:space:]]/}" ]] || return 1 + printf '%s' "$body" >"$body_file" + return 0 +} + +# THE POPULATION IS THE REQUIRED ROSTER, and getting this wrong was measured +# rather than reasoned. The first version judged every failed job in every +# workflow, and 22 of its 24 hits were `fast-forward` jobs failing at +# `Run sequoia-pgp/fast-forward@ea7628b…` — the merge bot DECLINING a branch that +# is no longer a descendant, which is the landing loop working exactly as +# designed. A sensor whose signal is 90% expected refusals is a sensor nobody +# reads. +# +# Scoping by workflow FILE would have been the obvious fix and is wrong: `ci`, +# `cross`, `darwin-link`, `msrv`, `semver` and `final` live in `ci.yml`, but +# `commit-lint`, `zizmor` and `action` do not — and `commit-lint` is the job that +# carried the CLOUD-404 503. Filtering by path would have dropped the motivating +# case. +# +# So the population is `$CI_REQUIRED_CHECKS`, the one roster `ci-wait` and +# `land`'s `graded_runs` already share (`mise.toml [env]`). A third reader of that +# value is what the value is for; a private copy here is the drift CLOUD-327 cost. +REQUIRED="${CI_REQUIRED_CHECKS:-}" +if [[ -z "${REQUIRED//[[:space:]]/}" ]]; then + echo "::error:: nonverdict-record: CI_REQUIRED_CHECKS is empty — without the roster this cannot tell a required job from an unrelated one, and a count over all of them is meaningless. Run through \`mise run\` so mise.toml's [env] applies. Nothing was recorded." >&2 + exit 1 +fi + +# The fan-in, excluded by name: its failure is manufactured by its siblings. +FAN_IN="final" + +unreadable=0 +runs_seen=0 +records="" + +# SINGLE-RUN MODE (CLOUD-483). `land` asks the same question about ONE concluded +# run, on its red-CI stop path, to tell a provisioning transient from a verdict — +# `land::absorbed` is the reader. It is a mode rather than a second task because +# the classification must exist once: two readers deciding "did this job reach a +# verdict" from two copies of the jq is the CLOUD-351 shape, and the copy that +# drifts is the one nobody runs on a clock. +# +# IT EMITS ON STDOUT AND RECORDS NOTHING, which the port sharpened rather than +# changed. There is no window to summarise, so there is no window to judge, and +# writing a windowless family would leave `policy/nonverdict.rego` reading a store +# whose summary is absent. An empty stream is therefore unambiguous: no failed +# required job was classifiable, which is "could not look" and never "nothing was +# wrong". +one_run="" +if [[ "${1:-}" = "--run" ]]; then + one_run="${2:-}" + if [[ -z "${one_run//[[:space:]]/}" ]]; then + echo "::error:: nonverdict-record: --run needs a run id. Without one there is nothing to classify, and an empty record stream would read as 'no failed job reached a verdict'." >&2 + exit 2 + fi +fi + +if [[ -n "$one_run" ]]; then + run_ids="$one_run" +elif ! conditional_get "repos/{owner}/{repo}/actions/runs?status=failure&per_page=$WINDOW"; then + echo "::error:: nonverdict-record: could not read the run list, so this window judged nothing and nothing was recorded. A persistent read failure is a token or rate-limit problem, not a clean window." >&2 + exit 1 +else + # `.id` and `.conclusion` only. Sorted by id so the record stream is + # byte-stable across ticks that see the same window in a different order. + run_ids=$(printf '%s' "$body" | jq -r '[.workflow_runs[]? | select(.conclusion == "failure") | .id] | sort | .[]' 2>/dev/null) || run_ids="" +fi + +failed_jobs=0 +nonverdict=0 +verdict=0 + +while IFS= read -r run; do + [[ -n "$run" ]] || continue + runs_seen=$((runs_seen + 1)) + + if ! conditional_get "repos/{owner}/{repo}/actions/runs/$run/jobs?per_page=100"; then + unreadable=$((unreadable + 1)) + continue + fi + + # One line per failed job: the job name, then whether any FAILED step is a + # `Run mise run …` step. The classification is done in jq over the payload + # rather than in shell over a rendered string, so a job name containing a tab + # cannot forge a field. + while IFS=$'\t' read -r job kind step; do + [[ -n "$job" ]] || continue + failed_jobs=$((failed_jobs + 1)) + if [[ "$kind" = "verdict" ]]; then + verdict=$((verdict + 1)) + else + nonverdict=$((nonverdict + 1)) + fi + records="${records}${kind}"$'\t'"run=$run"$'\t'"job=$job"$'\t'"step=$step"$'\n' + done <<<"$( + printf '%s' "$body" | jq -r --arg fan "$FAN_IN" --arg required "$REQUIRED" ' + ($required | split(",") | map(gsub("^\\s+|\\s+$"; ""))) as $roster + | [.jobs[]? + | select(.conclusion == "failure") + | select(.name != $fan) + | select([.name] | inside($roster))] + | sort_by(.name) + | .[] + | . as $job + | ([$job.steps[]? | select(.conclusion == "failure")] | sort_by(.number)) as $failed + | ([$failed[] | select(.name | startswith("Run mise run ") or startswith("Run mise exec -- "))] | first) as $task + | if $task + then [$job.name, "verdict", $task.name] + else [$job.name, "nonverdict", (($failed | first | .name) // "unknown")] + end + | @tsv + ' 2>/dev/null + )" +done <<<"$run_ids" + +# Records first, sorted, then the summary — so the stream is byte-stable and the +# module can read the summary without depending on where it sits. +emit() { + if [[ -n "$records" ]]; then + printf '%s' "$records" | LC_ALL=C sort + fi + if [[ -z "$one_run" ]]; then + printf 'window\truns=%s\tfailed_jobs=%s\tnonverdict=%s\tverdict=%s\tunreadable=%s\n' \ + "$runs_seen" "$failed_jobs" "$nonverdict" "$verdict" "$unreadable" + fi +} + +# Single-run mode has no window, so there is no family to write — it answers on +# stdout, where `land::absorbed`'s caller reads it. +if [[ -n "$one_run" ]]; then + emit + exit 0 +fi + +# BOTH, AND THE ORDER MATTERS. The window goes into the family +# `policy/nonverdict.rego` reads, and the same bytes go to stdout, where +# `nonverdict-rate.yml` publishes them to the step summary — the number is the +# point, and a summary that only appears on green is a report nobody reads at the +# moment it matters. Rendered once into a variable rather than measured twice, so +# the published records and the recorded ones cannot differ. +out=$(emit) +printf '%s\n' "$out" +printf '%s\n' "$out" | cargo run --quiet -p batten -- record named nonverdict +''' + [tasks.record-verdicts] description = "Effect: run each declared third-party validator OUTSIDE the engine and record its verdict where `batten check` reads it (CLOUD-1265)" # THE HALF THAT WAS MISSING. `crates/batten/src/tools.rs` reads diff --git a/policy/nonverdict.rego b/policy/nonverdict.rego new file mode 100644 index 000000000..f91a0f3a4 --- /dev/null +++ b/policy/nonverdict.rego @@ -0,0 +1,267 @@ +# How often a required CI job fails WITHOUT ever reaching a verdict (CLOUD-484, +# ported under CLOUD-1717). +# +# A job that dies in provisioning — checkout, the toolchain install, a cache +# restore — spends its runner minutes, reds the branch, and answers nothing. Every +# occurrence then costs a human or an agent the time to discover it was never a +# verdict at all: CLOUD-404 has five, and the one on #376 sent an agent to +# reproduce a `commit-lint` failure that passes locally because `commit-lint` +# never linted anything. CLOUD-483 stops `land` MISREADING one in the moment; this +# answers the question neither of those does — HOW OFTEN — so a step change is +# visible as a number instead of as whoever gets bitten next. +# +# THE SPLIT WAS ALREADY THERE AND THE PORT ONLY MOVED ITS HALVES. The retired pair +# was `nonverdict-scan` (measure) and `nonverdict-assert` (decide), kept apart for +# exactly CLOUD-1559's reason: a measurement needs the network and a token, a +# decision needs neither. So the scan became `[tasks.nonverdict-record]` and the +# assert became this file, and no decision changed hands — which is why the +# classification predicate below is absent from here. Whether a failed step is a +# verdict is read off the API payload's own `steps[]`, which `check` cannot fetch +# (house style §5 makes it `read` and incapable of spawning). +# +# THE PREDICATE IS THE PRODUCER'S AND IT IS CLOSED. A failed job rendered a VERDICT +# iff one of its failed steps is named `Run mise run ` or +# `Run mise exec -- `; otherwise it failed without reaching one. The +# second spelling was added after the first cost four jobs in one lap (CLOUD-113): +# the `windows` job runs `mise exec -- cargo test` because `mise run` puts Git Bash +# between the runner and cargo, so seven genuinely failing tests classified as a +# provisioning transient and `land` re-ran the job four times. A predicate whose +# premise has gone false does not fail loudly; it answers confidently and wrongly, +# in the direction that spends money. +# +# COULD-NOT-LOOK IS AN ABSENT RECORD, EXCEPT WHERE IT IS PARTIAL. The retired +# decider ran `0/1/2` with `2` for could-not-look, and the engine's `2` is a +# FINDING — so the two total-blindness arms (an empty roster, an unreadable run +# list) are now the producer refusing at write time and recording nothing, which +# reads here as silence. `unreadable` is the third case and is NOT blindness: the +# producer read part of its window and judged less than it claims, which is +# `bench-assert`'s partial-coverage false green and a finding in its own right. +#MUTANT-SUITE crates/batten/tests/it/nonverdict.rs +#MUTANT over-budget-passes|s@^\tcount_of("nonverdict") > budget$@\tfalse@|an_over_budget_window_is_reported_over_the_engines_projection +#MUTANT partial-window-passes|s@^\tcount_of("unreadable") > 0$@\tfalse@|a_partially_read_window_is_a_finding_rather_than_a_clean_one +#MUTANT verdict-failures-counted|s@^\tcolumns\[0\] == "nonverdict"$@\tcolumns[0] != "window"@|a_verdict_failure_is_never_named_however_many_there_are + +# METADATA +# description: | +# Bound to the TREE surface: this row is `scope = "tree"`, so it reads +# `input.tree` and never the mediated call. +# THIS BLOCK IS YAML AND MUST STAY THE LAST COMMENT BLOCK BEFORE `package`. +# schemas: +# - input: schema["policy-input.schema"] +package batten.nonverdict + +import rego.v1 + +rules contains "job read partial" + +rules contains "job answer missing" + +# How many non-verdict job failures a window may carry before this fires. +# +# TWO, NOT ZERO, and the retired program's reasoning is the one that carries: one +# provisioning failure in a window is the platform having a bad afternoon and is +# not actionable; a third in the same window is a pattern, and hearing about the +# pattern before the minutes are spent is the whole point. +# +# IN THE MODULE RATHER THAN BEHIND AN ENV OVERRIDE, which is `timeout-drift.rego`'s +# placement for its multipliers and is what the port made possible. The shell's +# `BATTEN_NONVERDICT_MAX` existed so its suite could point the budget at a fixture; +# a module's cases vary the COUNTS instead, so the knob had one reader and that +# reader no longer needs it. +budget := 2 + +# The producer's lines, or nothing. `recorded` being undefined is the +# could-not-look the header describes, and every rule below inherits it. +recorded := input.tree.records.nonverdict + +# The one summary line, held as a set so that a store carrying the same summary +# twice is still one window. +# +# EXACTLY ONE, OR NOTHING IS JUDGED. Two DIFFERENT summaries mean two scans were +# concatenated and a count over both describes neither — the retired decider's own +# arm, kept, and reachable here only through a torn store because `record named` +# replaces a family rather than appending to it. +summaries contains raw if { + some raw in recorded + startswith(raw, "window\t") +} + +# `window\truns=\tfailed_jobs=\tnonverdict=\tverdict=\tunreadable=` +# flattened to an object. A malformed column is skipped rather than fatal, the +# posture every record reader here takes: the producer refuses a malformed line at +# write time, so one arriving at read time is a torn store. +fields[pair[0]] := pair[1] if { + count(summaries) == 1 + some raw in summaries + columns := split(raw, "\t") + some idx in numbers.range(1, count(columns) - 1) + pair := split(columns[idx], "=") + count(pair) == 2 +} + +# A COUNT THAT IS NOT A NUMBER IS NOT A ZERO. The retired decider refused rather +# than coercing, because a count silently read as zero is a clean window over input +# nobody parsed. Here that refusal is undefinedness, which leaves both rules below +# unfired and the window unjudged. +count_of(key) := number if { + raw := fields[key] + regex.match(data.batten.patterns["whole-number"], raw) + number := to_number(raw) +} + +# `nonverdict\trun=\tjob=\tstep=` — one per required job that +# failed before any `mise` step. The `verdict` lines are deliberately not read +# here: they exist so the ratio is derivable and so a window that found only +# verdicts is distinguishable from one that found nothing. +failures contains { + "run": trim_prefix(columns[1], "run="), + "job": trim_prefix(columns[2], "job="), + "step": trim_prefix(columns[3], "step="), +} if { + some raw in recorded + columns := split(raw, "\t") + count(columns) == 4 + columns[0] == "nonverdict" + startswith(columns[1], "run=") + startswith(columns[2], "job=") + startswith(columns[3], "step=") +} + +# A window the producer could not read all of. +# +# ITS OWN FINDING RATHER THAN A DEGRADED PASS. This is `bench-assert`'s +# partial-coverage rule: "a run that measured two of three paths and reported green +# over the two is exactly the partial-coverage false green". It fires whatever the +# count is, because a budget met over part of a window is a budget met over nothing +# in particular. +violation contains { + "rule": "job read partial", + "verdict": "job read partial", + "subjects": [{"count": count_of("unreadable")}], +} if { + count_of("unreadable") > 0 +} + +# Each job that spent its minutes and answered nothing, once the window is over +# budget. +# +# ONE FINDING PER JOB, not one per window: the count is what decides, and the +# coordinates are what a reader acts on. ANTI-VACUITY needs no arm of its own — +# a window with no runs carries no `nonverdict` line, so it cannot fire — and the +# case below is what keeps that true rather than accidental. This repo has been +# bitten twice by a gate that cannot fire reading the same as one that found +# nothing (`finding-sink-check`, `bench-assert`). +violation contains { + "rule": "job answer missing", + "verdict": "job answer missing", + "subjects": [{"artifact": entry.job}, {"artifact": entry.run}, {"artifact": entry.step}], +} if { + count_of("nonverdict") > budget + some entry in failures +} + +# --- cases --------------------------------------------------------------- + +tree(lines) := {"tree": {"records": {"nonverdict": lines}}} + +summary(runs, nonverdict, unreadable) := sprintf( + "window\truns=%d\tfailed_jobs=%d\tnonverdict=%d\tverdict=0\tunreadable=%d", + [runs, nonverdict, nonverdict, unreadable], +) + +failure(run, job, step) := sprintf("nonverdict\trun=%s\tjob=%s\tstep=%s", [run, job, step]) + +window(runs, unreadable, failed) := tree(array.concat( + [summary(runs, count(failed), unreadable)], + failed, +)) + +test_under_budget_is_clean if { + count(violation) == 0 with input as window(10, 0, [ + failure("111", "ci", "Run actions/checkout@3d3c42e"), + failure("222", "msrv", "Run actions/checkout@3d3c42e"), + ]) +} + +test_over_budget_names_each_non_verdict_failure if { + found := violation with input as window(10, 0, [ + failure("111", "ci", "Run actions/checkout@3d3c42e"), + failure("222", "msrv", "Run actions/checkout@3d3c42e"), + failure("333", "cross", "Set up job"), + ]) + count(found) == 3 + every v in found { + v.verdict == "job answer missing" + } +} + +# POINTER, NEVER PAYLOAD (rule 4): run ids, job names and step names. The producer +# never fetches a log body, so there is none here to leak. +test_the_finding_carries_coordinates_and_nothing_else if { + some v in violation with input as window(10, 0, [ + failure("111", "ci", "Run actions/checkout@3d3c42e"), + failure("222", "msrv", "Run actions/checkout@3d3c42e"), + failure("333", "cross", "Set up job"), + ]) + v.subjects[0] == {"artifact": "ci"} +} + +# A VERDICT FAILURE IS NOT COUNTED, however many there are: it was judged, and the +# branch that caused it is the thing to fix. +test_a_verdict_failure_is_never_named if { + count(violation) == 0 with input as tree([ + "window\truns=10\tfailed_jobs=9\tnonverdict=0\tverdict=9\tunreadable=0", + "verdict\trun=111\tjob=ci\tstep=Run mise run test:cargo", + "verdict\trun=222\tjob=msrv\tstep=Run mise exec -- cargo check", + ]) +} + +# COULD NOT LOOK AT PART OF IT is a finding, and fires even under budget: a green +# verdict here would cover less than it claims. +test_an_unreadable_run_is_a_finding_even_under_budget if { + some v in violation with input as window(10, 3, []) + v.verdict == "job read partial" +} + +test_the_partial_finding_carries_the_count if { + some v in violation with input as window(10, 3, []) + v.subjects == [{"count": 3}] +} + +# ANTI-VACUITY: a window with nothing in it judges nothing and says so by being +# present. An absent record is nobody having looked; this is the producer having +# looked and found no failed required job. +test_an_empty_window_fires_nothing_but_is_still_a_reading if { + count(violation) == 0 with input as window(0, 0, []) + count_of("runs") == 0 with input as window(0, 0, []) +} + +test_no_record_at_all_says_nothing if { + count(violation) == 0 with input as {"tree": {"records": {}}} +} + +# A COUNT THAT IS NOT A NUMBER IS NOT A ZERO. Coercing would report a clean window +# over input nobody parsed. +test_a_non_numeric_count_leaves_the_window_unjudged if { + not count_of("nonverdict") with input as tree(["window\truns=10\tfailed_jobs=1\tnonverdict=lots\tverdict=0\tunreadable=0"]) +} + +# Two DIFFERENT summaries describe neither window, so nothing is judged over them. +test_two_concatenated_scans_judge_neither_window if { + count(violation) == 0 with input as tree([ + "window\truns=10\tfailed_jobs=4\tnonverdict=4\tverdict=0\tunreadable=0", + "window\truns=10\tfailed_jobs=9\tnonverdict=9\tverdict=0\tunreadable=0", + failure("111", "ci", "Run actions/checkout@3d3c42e"), + ]) +} + +# A line this reader cannot parse is skipped. The surviving good lines are part of +# the case: without them the record holds no window at all, and this would pass for +# a reason that has nothing to do with skipping. +test_a_line_this_reader_cannot_parse_is_skipped if { + count(violation) == 0 with input as tree([ + "window\truns=10\tfailed_jobs=1\tnonverdict=1\tverdict=0\tunreadable=0", + failure("111", "ci", "Run actions/checkout@3d3c42e"), + "nonsense", + ]) +} diff --git a/tests/nonverdict-assert.bats b/tests/nonverdict-assert.bats deleted file mode 100644 index 1d3d906b5..000000000 --- a/tests/nonverdict-assert.bats +++ /dev/null @@ -1,140 +0,0 @@ -#!/usr/bin/env bats -# subject: mise-tasks/nonverdict-assert.sh -# CLOUD-484. The decider half of the non-verdict-failure sensor: how many recent CI -# jobs failed without reaching a verdict, judged against a budget. -# -# A pure function of stdin, so this suite needs no network and no `gh` — which is -# what lets it run in the hk gate on every commit while the measurement runs on a -# clock. Readings are injected the way `bench-assert` and `branch-age-check` inject -# theirs. - -setup() { - ASSERT="$BATS_TEST_DIRNAME/../mise-tasks/nonverdict-assert.sh" - export BATTEN_NONVERDICT_MAX=2 -} - -# One `window` summary with the counts a case cares about. Tab-separated, the record -# format `nonverdict-scan` emits. -window() { - printf 'window\truns=%s\tfailed_jobs=%s\tnonverdict=%s\tverdict=%s\tunreadable=%s\n' \ - "${1:-10}" "${2:-0}" "${3:-0}" "${4:-0}" "${5:-0}" -} - -# The step name a toolchain-install failure really carries. Declared once, with -# the FULL pin: `attribution-check` exempts a pinned coordinate and refuses a bare -# vendor mention in prose, and the full sha is what the API actually returns. -TOOLCHAIN_STEP='Run jdx/mise-action@7e36c90d9ab29c415a2384db3006f3ec8a8cc654' - -nonverdict_line() { - printf 'nonverdict\trun=%s\tjob=%s\tstep=%s\n' "$1" "$2" "$3" -} - -verdict_line() { - printf 'verdict\trun=%s\tjob=%s\tstep=%s\n' "$1" "$2" "$3" -} - -@test "under budget is a pass, and says what it judged" { - run bash -c "{ $(declare -f window); window 10 2 2 0 0; } | '$ASSERT'" - [ "$status" -eq 0 ] - [[ "$output" == *"2 of 10"* ]] -} - -@test "THE ACCEPTANCE CASE: over budget fails and names each non-verdict failure" { - input=$( - window 10 3 3 0 0 - nonverdict_line 111 "ci" "$TOOLCHAIN_STEP" - nonverdict_line 222 "msrv" "$TOOLCHAIN_STEP" - nonverdict_line 333 "cross" "Run actions/checkout@3d3c42e" - ) - run bash -c "printf '%s\n' \"\$1\" | '$ASSERT'" _ "$input" - [ "$status" -eq 1 ] - [[ "$output" == *"over the budget of 2"* ]] - [[ "$output" == *"run=111"* ]] - [[ "$output" == *"run=333"* ]] -} - -@test "a VERDICT failure is not counted, however many there are" { - # The other direction, so the predicate cannot collapse into counting every red - # job. Ten genuine test failures are not this gate's business. - input=$( - window 10 10 0 10 0 - verdict_line 111 "ci" "Run mise run test:bats" - verdict_line 222 "ci" "Run mise run lint:clippy" - ) - run bash -c "printf '%s\n' \"\$1\" | '$ASSERT'" _ "$input" - [ "$status" -eq 0 ] -} - -@test "COULD NOT LOOK: an unreadable run in the window is exit 2, never a pass" { - # `bench-assert`'s partial-coverage rule. A window that read less than it - # claims must not report green over the part it managed. - run bash -c "{ $(declare -f window); window 10 1 1 0 1; } | '$ASSERT'" - [ "$status" -eq 2 ] - [[ "$output" == *"could not read"* ]] -} - -@test "an unreadable run is exit 2 even when the count is under budget" { - # The trap: 1 non-verdict is under a budget of 2, so a naive order of checks - # would report green and hide that the window was incomplete. - run bash -c "{ $(declare -f window); window 10 1 1 0 3; } | '$ASSERT'" - [ "$status" -eq 2 ] -} - -@test "empty stdin is exit 2, not a clean window" { - run bash -c ": | '$ASSERT'" - [ "$status" -eq 2 ] - [[ "$output" == *"stdin is empty"* ]] -} - -@test "records with no window summary are exit 2 — there is no window to judge" { - input=$(nonverdict_line 111 "ci" "$TOOLCHAIN_STEP") - run bash -c "printf '%s\n' \"\$1\" | '$ASSERT'" _ "$input" - [ "$status" -eq 2 ] - [[ "$output" == *"no \`window\` summary"* ]] -} - -@test "two concatenated scans are exit 2 — a count over both describes neither" { - run bash -c "{ $(declare -f window); window 10 1 1 0 0; window 5 0 0 0 0; } | '$ASSERT'" - [ "$status" -eq 2 ] - [[ "$output" == *"more than one"* ]] -} - -@test "a non-numeric count is exit 2 rather than being coerced to zero" { - run bash -c "printf 'window\truns=10\tfailed_jobs=1\tnonverdict=lots\tverdict=0\tunreadable=0\n' | '$ASSERT'" - [ "$status" -eq 2 ] - [[ "$output" == *"nonverdict"* ]] -} - -@test "ANTI-VACUITY: an empty window exits 0 and says it judged nothing" { - # A gate that cannot fire must not be indistinguishable from one that found - # nothing — the `finding-sink-check` lesson. - run bash -c "{ $(declare -f window); window 0 0 0 0 0; } | '$ASSERT'" - [ "$status" -eq 0 ] - [[ "$output" == *"nothing to judge"* ]] -} - -@test "the budget is raise-only overridable, which is how the window is retuned" { - # The same records that fail at the default budget of 2 pass at 5 — which is - # also what lets a case point the budget at a fixture rather than editing the - # task. - export BATTEN_NONVERDICT_MAX=5 - input=$(window 10 3 3 0 0) - run bash -c "printf '%s\n' \"\$1\" | '$ASSERT'" _ "$input" - [ "$status" -eq 0 ] -} - -@test "POINTER, NEVER PAYLOAD: the report carries no step output, only coordinates" { - # The records themselves never contain a log body, so the assertion here is - # that the gate does not invent one — it echoes the coordinate line and nothing - # else. - input=$( - window 10 3 3 0 0 - nonverdict_line 111 "ci" "$TOOLCHAIN_STEP" - nonverdict_line 222 "ci" "$TOOLCHAIN_STEP" - nonverdict_line 333 "ci" "$TOOLCHAIN_STEP" - ) - run bash -c "printf '%s\n' \"\$1\" | '$ASSERT'" _ "$input" - [ "$status" -eq 1 ] - [[ "$output" != *"curl"* ]] - [[ "$output" != *"503"* ]] -} diff --git a/tests/nonverdict-scan.bats b/tests/nonverdict-scan.bats deleted file mode 100644 index 00ec66358..000000000 --- a/tests/nonverdict-scan.bats +++ /dev/null @@ -1,228 +0,0 @@ -#!/usr/bin/env bats -# subject: mise-tasks/nonverdict-scan.sh -# CLOUD-484. The measuring half: which recent required-check failures never reached -# a verdict. `gh` is stubbed with canned HTTP responses, the `tests/ci-wait.bats` -# idiom, because two of the properties under test are about the REQUEST (was -# `If-None-Match` actually sent) and about a response with no body at all (does a -# 304 keep the previous reading, or read as an empty window). - -setup() { - SCAN="$BATS_TEST_DIRNAME/../mise-tasks/nonverdict-scan.sh" - STUB="$BATS_TEST_TMPDIR/bin" - mkdir -p "$STUB" - PATH="$STUB:$PATH" - # Its own cache per case: the store persists between runs by design, so a - # shared one would let one case's ETags decide another's verdict. - BATTEN_NONVERDICT_CACHE="$BATS_TEST_TMPDIR/cache" - # The roster is normally supplied by mise.toml [env]. Set explicitly here so a - # case's expectations do not move when the real roster gains a check. - CI_REQUIRED_CHECKS="ci,cross,msrv,final" - export PATH BATTEN_NONVERDICT_CACHE CI_REQUIRED_CHECKS -} - -# A `gh` that routes on the endpoint and replays a numbered response per endpoint -# kind, recording every argv so a case can assert what was actually requested. -# Unquoted heredoc: stub-local vars are escaped, test-side paths interpolate. -stub_gh() { - cat >"$STUB/gh" <>"$BATS_TEST_TMPDIR/args" -kind=runs -for a in "\$@"; do - case "\$a" in - */jobs*) kind=jobs ;; - esac -done -n=\$(cat "$BATS_TEST_TMPDIR/n.\$kind" 2>/dev/null || echo 0) -n=\$((n + 1)) -echo "\$n" >"$BATS_TEST_TMPDIR/n.\$kind" -file="$BATS_TEST_TMPDIR/\$kind.\$n" -[ -f "\$file" ] || file="$BATS_TEST_TMPDIR/\$kind.last" -cat "\$file" -# A non-2xx makes the real gh exit non-zero, a 304 included. The scan must -# tolerate that rather than aborting, so the stub reproduces it. -head -n1 "\$file" | grep -q ' 2[0-9][0-9] ' || exit 1 -EOF - chmod +x "$STUB/gh" - rm -f "$BATS_TEST_TMPDIR/args" "$BATS_TEST_TMPDIR"/n.* -} - -# `response ` — a 200 carrying a body. -response() { - { - printf 'HTTP/2.0 200 OK\n' - printf 'ETag: %s\n\n' "$2" - printf '%s\n' "$3" - } >"$BATS_TEST_TMPDIR/$1" -} - -# `not_modified ` — a 304, which carries no body at all. -not_modified() { - { - printf 'HTTP/2.0 304 Not Modified\n' - printf 'ETag: %s\n\n' "$2" - } >"$BATS_TEST_TMPDIR/$1" -} - -# The step name a toolchain-install failure really carries. Declared once, with -# the FULL pin: `attribution-check` exempts a pinned coordinate and refuses a bare -# vendor mention in prose, and the full sha is what the API actually returns. -TOOLCHAIN_STEP='Run jdx/mise-action@7e36c90d9ab29c415a2384db3006f3ec8a8cc654' - -runs_list() { printf '{"workflow_runs":[{"id":%s,"conclusion":"failure"}]}' "$1"; } - -# One failed job with one failed step. -jobs_body() { - printf '{"jobs":[{"name":"%s","conclusion":"failure","steps":[{"number":1,"name":"%s","conclusion":"failure"}]}]}' "$1" "$2" -} - -@test "THE ACCEPTANCE CASE: a job that died before any mise step is a non-verdict failure" { - stub_gh - response runs.last 'W/"r1"' "$(runs_list 900)" - response jobs.last 'W/"j1"' "$(jobs_body msrv "$TOOLCHAIN_STEP")" - run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"nonverdict"*"run=900"*"job=msrv"* ]] - [[ "$output" == *"nonverdict=1"* ]] - [[ "$output" == *"verdict=0"* ]] -} - -@test "a job that failed IN a mise step rendered a verdict and is not counted" { - # The other direction. A classifier that fires on everything is as silent a - # failure as one that never fires. - stub_gh - response runs.last 'W/"r1"' "$(runs_list 901)" - response jobs.last 'W/"j1"' "$(jobs_body ci 'Run mise run test:bats')" - run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"verdict"*"run=901"* ]] - [[ "$output" == *"nonverdict=0"* ]] - [[ "$output" == *"verdict=1"* ]] -} - -@test "a job that failed in a mise EXEC step rendered a verdict too" { - # THE SECOND SPELLING, and the case whose absence cost four Windows jobs in - # one lap. `windows` runs `mise exec -- cargo test --workspace` rather than a - # task, for the Git Bash reason its own comment measures — so seven - # reproducibly failing tests classified as a provisioning transient, and - # `land` re-ran the job until its budget stopped it while reporting that the - # provisioning path was broken. - stub_gh - response runs.last 'W/"r1"' "$(runs_list 905)" - response jobs.last 'W/"j1"' "$(jobs_body windows 'Run mise exec -- cargo test --workspace')" - # The roster gains the job under test rather than the case borrowing a name - # already in it: which job it is decides nothing here, but a case whose - # subject is filtered out before the predicate runs would pass on nothing. - CI_REQUIRED_CHECKS="ci,cross,msrv,windows,final" run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"verdict"*"run=905"*"job=windows"* ]] - [[ "$output" == *"nonverdict=0"* ]] - [[ "$output" == *"verdict=1"* ]] -} - -@test "THE FAN-IN IS EXCLUDED: final's needs-assertion is not a non-verdict failure" { - # Measured on run 31637881076: `final` failed at `Assert all required jobs - # passed`, a bare run: step. Counted naively it would fire on every genuinely - # red branch, because its failure is manufactured by its siblings. - stub_gh - response runs.last 'W/"r1"' "$(runs_list 902)" - response jobs.last 'W/"j1"' "$(jobs_body final 'Assert all required jobs passed')" - run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"nonverdict=0"* ]] - [[ "$output" == *"failed_jobs=0"* ]] -} - -@test "A JOB OUTSIDE THE ROSTER IS EXCLUDED: a declining merge bot is not a failure here" { - # Measured: the first version counted 22 `fast-forward` jobs failing at - # `Run sequoia-pgp/fast-forward@…` — the bot refusing a branch that is no - # longer a descendant, which is the landing loop working as designed. - stub_gh - response runs.last 'W/"r1"' "$(runs_list 903)" - response jobs.last 'W/"j1"' "$(jobs_body fast-forward 'Run sequoia-pgp/fast-forward@ea7628b')" - run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"failed_jobs=0"* ]] - [[ "$output" != *"fast-forward"* ]] -} - -@test "the conditional request is actually sent once an ETag is stored" { - # The whole rate-limit argument rests on this header reaching the server; a - # cache that stores an ETag and never sends it costs the same as no cache. - stub_gh - response runs.last 'W/"r1"' "$(runs_list 904)" - response jobs.last 'W/"j1"' "$(jobs_body ci 'Run mise run ci')" - run "$SCAN" - [ "$status" -eq 0 ] - run "$SCAN" - [ "$status" -eq 0 ] - grep -q 'If-None-Match: W/"r1"' "$BATS_TEST_TMPDIR/args" -} - -@test "A 304 KEEPS THE PREVIOUS READING rather than reading as an empty window" { - # mem:github-rest-etiquette: re-parsing an empty 304 body yields nothing, which - # for this scan would silently become "no failures" — a false green whose only - # symptom is that the number stopped moving. - stub_gh - response runs.1 'W/"r1"' "$(runs_list 905)" - response jobs.1 'W/"j1"' "$(jobs_body msrv "$TOOLCHAIN_STEP")" - not_modified runs.2 'W/"r1"' - not_modified jobs.2 'W/"j1"' - run "$SCAN" - [ "$status" -eq 0 ] - first="$output" - run "$SCAN" - [ "$status" -eq 0 ] - [ "$output" = "$first" ] - [[ "$output" == *"nonverdict=1"* ]] -} - -@test "a 304 with no cached body is unreadable, never an empty window" { - # The cache is the only thing that makes a 304 answerable. Without it the - # honest report is that the window could not be read. - stub_gh - not_modified runs.last 'W/"r1"' - run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"unreadable=1"* ]] - [[ "$output" == *"runs=0"* ]] -} - -@test "an unreadable jobs read is counted, not silently dropped" { - stub_gh - response runs.last 'W/"r1"' "$(runs_list 906)" - printf 'HTTP/2.0 500 Internal Server Error\n\n' >"$BATS_TEST_TMPDIR/jobs.last" - run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"unreadable=1"* ]] -} - -@test "an empty roster is unreadable rather than a count over every job" { - stub_gh - response runs.last 'W/"r1"' "$(runs_list 907)" - response jobs.last 'W/"j1"' "$(jobs_body msrv "$TOOLCHAIN_STEP")" - CI_REQUIRED_CHECKS="" run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"unreadable=1"* ]] -} - -@test "a summary line is always emitted, even when nothing failed" { - # ANTI-VACUITY, and the bug that produced this case: the first version aborted - # silently under `set -e` when a response carried no ETag, printing neither - # records nor a summary — indistinguishable from a clean empty window. - stub_gh - response runs.last 'W/"r1"' '{"workflow_runs":[]}' - run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"window"*"runs=0"* ]] -} - -@test "POINTER, NEVER PAYLOAD: records carry coordinates, and no log is fetched" { - stub_gh - response runs.last 'W/"r1"' "$(runs_list 908)" - response jobs.last 'W/"j1"' "$(jobs_body msrv "$TOOLCHAIN_STEP")" - run "$SCAN" - [ "$status" -eq 0 ] - # No call asked for a log; the step name is the whole diagnosis. - ! grep -q 'logs' "$BATS_TEST_TMPDIR/args" -} From b148ec6e2ddbd4d24af6c1ac6c267298f23d864c Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Thu, 17 Sep 2026 07:17:02 +0000 Subject: [PATCH 07/32] fix(doctor): carry the record table into the mediated-call vocabulary MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `main` grew a `Vocabulary` construction site in `doctor.rs` while this branch was adding the `records` field to that struct (CLOUD-1810). Both changes are correct and neither conflicts textually, so the rebase merged them cleanly and produced a tree that does not compile — `missing field 'records' in initializer of 'policy::Vocabulary'`. THE SYMPTOM WAS NOT A BUILD ERROR, WHICH IS WHY IT IS WORTH A COMMIT MESSAGE. `verify`'s claim step reads `batten receipt status claim --key branch`, and that runs through `cargo run`. With the crate failing to build, the command emitted nothing and the step reported "this branch carries no VALID claim receipt" over three blank verdict lines — a build break wearing the costume of a missing receipt, which sent the reading at it twice. `None` would have been wrong: this path loads mediated-call modules, and a module reading a declared family needs the same projection every other caller gives it. `resolved.records` is what `check` passes. Refs: CLOUD-1810 --- crates/batten/src/doctor.rs | 1 + 1 file changed, 1 insertion(+) diff --git a/crates/batten/src/doctor.rs b/crates/batten/src/doctor.rs index 669b34edc..641e1fc74 100644 --- a/crates/batten/src/doctor.rs +++ b/crates/batten/src/doctor.rs @@ -2291,6 +2291,7 @@ fn unminted_facts(dir: &Path) -> Vec { // An authoring property, and this is a diagnosis of the container. words: None, recorders: &resolved.recorders, + records: &resolved.records, }, crate::policy::ModuleChecks::SkipOnHotPath, None, From cdb8bba3966e93105078d7a1ec82fda9959efefb Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Thu, 17 Sep 2026 07:59:21 +0000 Subject: [PATCH 08/32] fix(lease): move the delete-then-restore case to where the git spawn is inventoried MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CLOUD-1825's regression case builds a history whose `HEAD~1` deletes a path `HEAD` restores, which is the shape that refuted the old assertion. Building it needs `git`, and I put a `std::process::Command` in `src/lease.rs` to do it — which `clippy.toml` refuses, because a spawn is an inventory row (CLOUD-320). ANNOTATING IT WOULD HAVE BEEN THE WRONG FIX AND THE GATE SAYS SO. `policy/spawn-widening.rego` refuses an ADDED `#[expect(clippy::disallowed_types)]` escape, and `exec.rs` states the reason this change is not one: "an annotation is meant to record a spawn somebody decided on, not to be the cheap way past the lint." So the case moved rather than the lint bending. `common::git_in` is the sanctioned fixture route and it lives in the integration tier, so the case comes to it — `crates/batten/tests/it/lease_lifecycle.rs`, next to the rest of the lease's push machinery. It adds no spawn of its own. What stays in `src/lease.rs` is the corrected assertion alone, which needs no fixture: it compares a base against `None` over whatever history is checked out, and that bound is sound for every history shape. Found by `verify`'s clippy step. I had validated the original change with `cargo test` and never linted it, which is the step this repo's own per-retirement checklist names. Refs: CLOUD-1825 --- crates/batten/src/lease.rs | 81 --------------------- crates/batten/tests/it/lease_lifecycle.rs | 88 +++++++++++++++++++++++ 2 files changed, 88 insertions(+), 81 deletions(-) diff --git a/crates/batten/src/lease.rs b/crates/batten/src/lease.rs index ebdebdbc0..b0495e03a 100644 --- a/crates/batten/src/lease.rs +++ b/crates/batten/src/lease.rs @@ -5193,87 +5193,6 @@ mod tests { ); } - /// The history shape that refuted the old assertion, built rather than waited - /// for (CLOUD-1825). - /// - /// The case above runs against THIS repository, so what it covers is whatever - /// history happens to be checked out — which is how a false assertion survived - /// until a delete-then-restore pair drifted into its window, and how it would - /// quietly stop being covered once the pair drifted back out. A one-line - /// comparand change with nothing holding it in place invites the next author to - /// restore the wider-base comparison, because that one reads as the more - /// thorough of the two. - /// - /// So the shape is constructed: `HEAD~1` deletes a path `HEAD` brings back. - /// The wider base still carries the blob and subtracts it; the narrow one does - /// not and must re-send it. That is the inversion, and asserting it directly is - /// what makes the property above a claim about the FUNCTION rather than about - /// the checkout. - #[test] - fn a_wider_base_may_enumerate_fewer_objects_than_a_narrower_one() { - let dir = scratch("delete-then-restore"); - let git = |args: &[&str]| { - let out = std::process::Command::new("git") - .args(args) - .current_dir(&dir) - .env("GIT_AUTHOR_NAME", "t") - .env("GIT_AUTHOR_EMAIL", "t@example.com") - .env("GIT_COMMITTER_NAME", "t") - .env("GIT_COMMITTER_EMAIL", "t@example.com") - .output() - .expect("git runs"); - assert!(out.status.success(), "git {args:?} failed"); - }; - git(&["init", "-q", "-b", "main"]); - // Big enough that the inversion cannot be an artefact of one small blob - // rounding the other way, and distinct per file so none of them dedupe. - for n in 0..24 { - std::fs::write(dir.join(format!("page-{n}.txt")), format!("page {n}\n")) - .expect("write a page"); - } - std::fs::write(dir.join("keep.txt"), "keep\n").expect("write the survivor"); - git(&["add", "-A"]); - git(&["commit", "-qm", "the pages exist"]); - - for n in 0..24 { - std::fs::remove_file(dir.join(format!("page-{n}.txt"))).expect("delete a page"); - } - git(&["add", "-A"]); - git(&["commit", "-qm", "a partial regeneration deletes them"]); - - for n in 0..24 { - std::fs::write(dir.join(format!("page-{n}.txt")), format!("page {n}\n")) - .expect("restore a page"); - } - git(&["add", "-A"]); - git(&["commit", "-qm", "and the remedy restores them"]); - - let head = crate::git::head_commit(&dir).expect("resolve the fixture head"); - let narrow = - crate::git::objects_to_send(&dir, Some("HEAD~1"), &head).expect("the narrow base"); - let wide = - crate::git::objects_to_send(&dir, Some("HEAD~2"), &head).expect("the wider base"); - - assert!( - wide.len() < narrow.len(), - "the wider base still holds the deleted blobs and subtracts them, so it must \ - enumerate FEWER than the narrow one: wide {} vs narrow {}", - wide.len(), - narrow.len() - ); - - // And the bound the case above asserts still holds over the same history, - // which is the point: `None` is sound where a wider base is not. - let whole = crate::git::objects_to_send(&dir, None, &head).expect("no base at all"); - assert!( - whole.len() >= narrow.len() && whole.len() >= wide.len(), - "no base can enumerate more than sending everything: whole {} vs narrow {} / wide {}", - whole.len(), - narrow.len(), - wide.len() - ); - } - #[test] fn a_pack_carrying_the_commits_closure_still_yields_the_commit() { // MEASURED AGAINST THE REAL REMOTE, not imagined: a `want` for one commit diff --git a/crates/batten/tests/it/lease_lifecycle.rs b/crates/batten/tests/it/lease_lifecycle.rs index 1eccb7ba0..e5103e57e 100644 --- a/crates/batten/tests/it/lease_lifecycle.rs +++ b/crates/batten/tests/it/lease_lifecycle.rs @@ -135,6 +135,10 @@ // Panicking on setup failure is the idiomatic way for a test to fail loudly. #![allow(clippy::unwrap_used, clippy::expect_used)] +use crate::common; + +use common::{git_in, init_repo, scratch, write}; + use batten::lease::{ Authority, Body, Observed, Terms, authorises, authorises_this_clone, renewal, reservation, tombstone, @@ -711,3 +715,87 @@ fn a_body_already_asked_or_asked_by_its_own_holder_is_left_alone() { "a blank request is an absence, so this one IS worth asking" ); } + +// --------------------------------------------------------------------------- +// CLOUD-1825: what the push actually enumerates, over a history built for it. +// --------------------------------------------------------------------------- + +/// A wider base may enumerate FEWER objects than a narrower one. +/// +/// `lease::tests::the_base_is_subtracted_rather_than_resent` asserts the economy +/// of the push — that subtracting a base beats sending everything — against THIS +/// repository, so what it covers is whatever history happens to be checked out. +/// It used to assert something stronger and false: that a WIDER base enumerates +/// at least as much, which is monotonicity in the base. +/// +/// `git::objects_to_send` does not have that property and never claimed it. Its +/// subtraction is against the base's OWN TREE, so a base whose tree carries MORE +/// subtracts more — one commit that deletes a path and a later one that restores +/// it makes the wider base strictly smaller, because it still holds the blobs and +/// the narrow one does not. +/// +/// Measured rather than imagined: it fired on this repository's own history when +/// a commit deleted 105 generated `man/*.1` pages and the next restored them, and +/// what it reported was a defect in the assertion rather than in the function. A +/// one-line comparand change with nothing holding it in place invites the next +/// author to restore the wider-base comparison, because that one reads as the +/// more thorough of the two. So the shape is constructed here. +/// +/// IT LIVES IN THIS TIER RATHER THAN BESIDE THE CASE IT CORRECTS, and that is the +/// lint deciding rather than a preference: building the history needs `git`, a +/// spawn is an inventory row (CLOUD-320), and `policy/spawn-widening.rego` refuses +/// an ADDED `#[expect(clippy::disallowed_types)]` escape. `common::git_in` is the +/// sanctioned fixture route and it is here, so the case comes to it. +#[test] +fn a_wider_base_may_enumerate_fewer_objects_than_a_narrower_one() { + let dir = scratch("objects-delete-then-restore"); + init_repo(&dir); + + // Distinct per file so none of them dedupe, and enough of them that the + // inversion cannot be one small blob rounding the other way. + for n in 0..24 { + write(&dir, &format!("page-{n}.txt"), &format!("page {n}\n")); + } + write(&dir, "keep.txt", "keep\n"); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-qm", "the pages exist"]); + + for n in 0..24 { + std::fs::remove_file(dir.join(format!("page-{n}.txt"))).expect("delete a page"); + } + git_in(&dir, &["add", "-A"]); + git_in( + &dir, + &["commit", "-qm", "a partial regeneration deletes them"], + ); + + for n in 0..24 { + write(&dir, &format!("page-{n}.txt"), &format!("page {n}\n")); + } + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-qm", "and the remedy restores them"]); + + let head = batten::git::head_commit(&dir).expect("resolve the fixture head"); + let narrow = + batten::git::objects_to_send(&dir, Some("HEAD~1"), &head).expect("the narrow base"); + let wide = batten::git::objects_to_send(&dir, Some("HEAD~2"), &head).expect("the wider base"); + + assert!( + wide.len() < narrow.len(), + "the wider base still holds the deleted blobs and subtracts them, so it must \ + enumerate FEWER than the narrow one: wide {} vs narrow {}", + wide.len(), + narrow.len() + ); + + // And the bound the corrected case asserts still holds over the same history, + // which is the point: `None` is sound where a wider base is not. + let whole = batten::git::objects_to_send(&dir, None, &head).expect("no base at all"); + assert!( + whole.len() >= narrow.len() && whole.len() >= wide.len(), + "no base can enumerate more than sending everything: whole {} vs narrow {} / wide {}", + whole.len(), + narrow.len(), + wide.len() + ); +} From 3a8b292dd140d6265681943c3ecdabb3dbb15822 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Thu, 17 Sep 2026 09:01:23 +0000 Subject: [PATCH 09/32] refactor(gate)!: retire land-divergence and land-divergence-assert onto the engine MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CLOUD-492's pair was already split for CLOUD-1559's reason — a measurement needs the network and a token, a decision needs neither — so this port carried each half to the home the engine has for it and no decision changed hands. `land-divergence-assert` becomes `policy/land-divergence.rego`, registered as `lane grade other` over eight verdicts. `land-divergence` becomes `[tasks.land-divergence-record]`, writing the `land-divergence` family through `batten record named`. The pagination walk, the ETag cache, the `total_count` truncation guard and every instant subtraction stayed outside: section 5 makes `check` `read` and incapable of spawning, and `Fact::Instant` projects `null` to every module. A CANCELLED RUN IS NOT WASTE, AND THE RULE KEEPS THAT. Measured 2026-08-12 after serialisation: 5 green CI runs against 5 cancelled reads as a 50% discard rate and is the opposite — those cancels had p50 lifetime ~20s, the lease precondition killing an unauthorised matrix for ~20 runner-seconds instead of billing ~500. So the graded quantity is cancel LATENCY, never cancel count, and a case asserts both directions so a future author cannot collapse it into a count. THE EXIT CONTRACT CHANGED, AND SIX ARMS RIDE ON IT. The decider ran 0 under budget / 1 over / 2 could-not-look; the engine runs 0/1/2/3 where 2 is a FINDING. Both total-blindness arms — an unreadable CI run window, an unreadable merged-PR list — are now the producer refusing at write time and recording nothing, and an absent family is the module's silence. `unreadable` stays a finding because it is not blindness: it is a window read in PART, which the API's 1000-item pagination cap makes real. It once reported zero fast-forward refusals over a window carrying 598. `severity = "warn"` for `job grade other`'s reason, one surface over. Every metric here is a property of the fleet over a window and no branch causes any of them, which is why the retired pair was kept out of the hk gate and out of `verify` and why its own suite asserted that absence. The scheduled job runs `--fail-on-warning` and reds on the divergence. TRUE DIVISION WHERE THE SHELL TRUNCATED. Bash has no floats, so the decider's `$((x * 100 / n))` discarded the remainder and could report a ratio marginally under a budget it was over. The ratios are still in hundredths; the rounding is gone, which is what that case's own name asked for. WHAT THIS DELTA SPENDS, stated rather than implied: the producer's own steps — the conditional request, the 304 fallback, the truncation guard, the per-run join and the per-job read — lose their dedicated coverage. `tests/land-divergence.bats` stubbed `gh` and exercised all of them, and no compiled-binary case reaches a shell task. The `carried` arms claim the retirement is covered by the module and the tier, and the tier's header says plainly that those steps are not. Refs: CLOUD-1717 Admits: b8304d0f21d25e9dfd0708b23d14d3f827cee3592bc6a2adae0171e36ed59cf5 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: mise.toml Admits-anchor: call:c5c4bf9b128a49fde64a191412af22a0151853b6 Admits-epoch: 3fae854077625b7e8d53be38c55b8b1eb765365b836e5c13bf827a6484cf8e07 Admits-author: alec@wenzowski.com Admits-prev: f2bd67dd8d3ff4c720827bbf0b029ef4bbdf39262a8f4b564363a11bbd6e8507 Admits-answer-lost: `policy/land-divergence.rego` decides over a store nothing writes, which is exactly the could-not-look-reads-as-pass shape this campaign exists to remove. The retirement stalls and `mise-tasks/land-divergence.sh` and `mise-tasks/land-divergence-assert.sh` stay in the corpus. Admits-answer-precondition: A mise task body lives on exactly one surface. `[tasks.land-divergence-record]` is the measuring half of the `land-divergence`/`land-divergence-assert` port: it walks the Actions API per workflow with a conditional ETag read, detects truncation against `total_count`, and subtracts instants to derive cancel latency, queue delay and peak concurrency. Section 5 makes `check` `read` and incapable of spawning, and `Fact::Instant` projects `null` to every module, so neither half of that can move inside. There is no other file a task body can be written in, and it lands beside the module it feeds. Admits-answer-rejected-route: config read first — I read `[tasks.nonverdict-record]`, the producer this campaign landed one commit ago against the same split, and matched its shape; the reading is what made this one correct rather than what made it unnecessary, because a producer that does not exist cannot be read into existence. patch run first is a commit-message route and does not apply to a task addition. Admits: 573bf1d70ccdbb16c0a5154380b5149f17e7ec1837acdc6c9bdb911f96dd78a7 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: batten.toml Admits-anchor: call:c5c4bf9b128a49fde64a191412af22a0151853b6 Admits-epoch: 3fae854077625b7e8d53be38c55b8b1eb765365b836e5c13bf827a6484cf8e07 Admits-author: alec@wenzowski.com Admits-prev: 76b95e62e146ec7373c4777122faf5f695347ccf9816abfcc964281e87aedc45 Admits-answer-lost: `policy/land-divergence.rego` sits in the tree deciding nothing — unregistered, so `batten check` never loads it and `policy test` never runs its fourteen cases, and its family unprojected, so the record its producer writes is invisible. The retirement it exists for cannot proceed and `mise-tasks/land-divergence.sh` and `mise-tasks/land-divergence-assert.sh` stay in the corpus. Admits-answer-precondition: A policy module is registered by a `[[rule]]` row and by nothing else; the verdict strings it emits must exist as `[[verdict]]` rows or the engine refuses the module at load; and a verb-written record family reaches no module until a `[[record]]` row declares it. `policy/land-divergence.rego` is a new module with eight new verdicts over a new family, so those ten rows have to be written into batten.toml directly. There is no other surface any of them lives on, and they land in one diff beside the module they register. Admits-answer-rejected-route: config read first — I read the `job grade other` registration this campaign landed one commit ago and copied its shape, and that reading is what produced these rows rather than what avoided them: registration is an addition and cannot be satisfied by reading. patch run first is a commit-message route and does not apply to a config addition. Admits: e204c3b8d37d5ea5a9a270048fc79cb7223e7d23efbb0ae1913cec8ef853b44a Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: .github/workflows/land-divergence.yml Admits-anchor: call:c5c4bf9b128a49fde64a191412af22a0151853b6 Admits-epoch: b48fc2790947366c60a4d88b7c596aed9747de53678bfc6d1ccb0b2de0c2b2c7 Admits-author: alec@wenzowski.com Admits-prev: - Admits-answer-lost: The only caller of both retired programs keeps calling them by name, so the scheduled job fails on every tick with an unknown task and the divergence this gate exists to measure stops being measured — the retirement lands having silently switched the sensor off, which is worse than the state before it. The header would also point a reader at a suite that is not there. Admits-answer-precondition: A workflow's steps live on exactly one surface. This file invokes both programs the delta retires — `mise run land-divergence` to measure and `mise run land-divergence-assert` to judge — so retiring them without repointing it leaves the scheduled job invoking tasks that no longer exist. Its header also cites `tests/land-divergence.bats` as the thing asserting the job's absence from the gate and from verify, and that suite dies here too, so the citation has to move to what asserts it now. Three edits a reviewer sees in the diff that deletes the programs. Admits-answer-rejected-route: config read first — I read the workflow before editing and that reading is what identified both call sites, the `--fail-on-warning` the `warn` severity now requires, and the stale citation in the header; reading cannot repoint a caller. patch run first is a commit-message route and does not apply to a workflow edit. --- .github/workflows/land-divergence.yml | 16 +- batten.toml | 136 ++++++++ bench/gates/RESULTS.md | 1 - bench/suites/RESULTS.md | 2 - crates/batten/tests/it/land_divergence.rs | 383 +++++++++++++++++++++ crates/batten/tests/it/main.rs | 1 + crates/batten/tests/it/nonverdict.rs | 4 +- mise-tasks/land-divergence-assert.sh | 193 ----------- mise-tasks/land-divergence.sh | 391 --------------------- mise.toml | 394 +++++++++++++++++++++- policy/land-divergence.rego | 334 ++++++++++++++++++ tests/land-divergence-assert.bats | 190 ----------- tests/land-divergence.bats | 281 --------------- 13 files changed, 1261 insertions(+), 1065 deletions(-) create mode 100644 crates/batten/tests/it/land_divergence.rs delete mode 100755 mise-tasks/land-divergence-assert.sh delete mode 100755 mise-tasks/land-divergence.sh create mode 100644 policy/land-divergence.rego delete mode 100644 tests/land-divergence-assert.bats delete mode 100644 tests/land-divergence.bats diff --git a/.github/workflows/land-divergence.yml b/.github/workflows/land-divergence.yml index 05a7089d8..ab07e6929 100644 --- a/.github/workflows/land-divergence.yml +++ b/.github/workflows/land-divergence.yml @@ -17,8 +17,10 @@ name: land-divergence # truncated rather than merely stale. # # THIS MUST NOT RUN IN THE GATE OR IN `verify`. It is a property of the world, -# and a property of the world must not red a branch — `tests/land-divergence.bats` -# asserts its absence from both. +# and a property of the world must not red a branch. Two things hold that now +# (CLOUD-1717): the producer below is in no gate list, and the row that replaced +# the decider is `severity = "warn"`, so `lane grade other` reports without +# failing any run that does not ask it to. This job is the one caller that asks. on: schedule: @@ -74,7 +76,7 @@ jobs: env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} BATTEN_DIVERGENCE_SINCE: ${{ inputs.since }} - run: mise run land-divergence > "$RUNNER_TEMP/divergence.txt" + run: mise run land-divergence-record > "$RUNNER_TEMP/divergence.txt" - name: Publish the records shell: bash run: | @@ -88,7 +90,13 @@ jobs: # Always, so the records above are published even when a budget is broken — # the number is the point, and a summary that only appears on green is a # report nobody reads at the moment it matters. + # `--fail-on-warning` IS WHAT KEEPS THE GATE A GATE. The row is + # `severity = "warn"` so that every `check` and every `verify` stays green + # over a property of the fleet no branch causes. This job is the one reader + # that must red on it, which is the retired decider's posture exactly: a + # number that rises has to fail something or it becomes another artifact + # nobody opens. - name: Judge the divergence against its budgets if: always() shell: bash - run: mise run land-divergence-assert < "$RUNNER_TEMP/divergence.txt" + run: mise run batten -- check --rule 'lane grade other' --fail-on-warning diff --git a/batten.toml b/batten.toml index 7299730cf..f10fe210f 100644 --- a/batten.toml +++ b/batten.toml @@ -2158,6 +2158,13 @@ writer = "mise run timeout-drift-record" record = "nonverdict" writer = "mise run nonverdict-record" +# CLOUD-492's measuring half, ported off `land-divergence.sh` under CLOUD-1717. +# The decider half is `policy/land-divergence.rego`; the split is the retired +# pair's own, for the reason the `nonverdict` pair's is. +[[record]] +record = "land-divergence" +writer = "mise run land-divergence-record" + [[pattern]] id = "closed-issue-status" regex = '^(done|canceled|duplicate)$' @@ -6326,6 +6333,23 @@ scope = "tree" module = "policy/nonverdict.rego" severity = "warn" +# CLOUD-492, ported off `mise-tasks/land-divergence-assert.sh` under CLOUD-1717. +# +# `warn` FOR THE REASON `job grade other` ABOVE CARRIES, one surface over. Every +# metric here is a property of the FLEET and the loop over a window — cancel +# latency, queue delay, peak concurrency, the fast-forward bot's refusal ratio — +# and no branch causes any of them. Failing a branch's `verify` over yesterday's +# runner saturation is how a gate gets switched off. The scheduled job runs +# `--fail-on-warning` and reds on the divergence, which is the retired decider's +# posture exactly: a number that rises has to fail something or it becomes another +# artifact nobody opens. +[[rule]] +id = "lane grade other" +kind = "policy" +scope = "tree" +module = "policy/land-divergence.rego" +severity = "warn" + [[rule]] id = "plan cover partial" kind = "policy" @@ -13242,6 +13266,118 @@ id = "task run first" kind = "command" target = "mise run nonverdict-record" +# CLOUD-492's eight readings, ported off `land-divergence-assert.sh` under +# CLOUD-1717. +[[verdict]] +id = "lane read partial" +gloss = "the measurement could not read part of its window, so a green verdict would cover less than it claims" +class = """ +The Actions runs endpoint caps pagination at 1000 items while still \ +reporting the true total, so a walk that stops on a short page reads a PREFIX \ +and looks like a clean finish. Measured: 1000 of 1446 runs collected, reporting \ +zero fast-forward refusals over a window carrying 598. Narrow the window. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run land-divergence-record" + +[[verdict]] +id = "lane count spent" +gloss = "the landing loop bought more CI matrices per landing than its budget" +class = """ +The ideal is 1.00 — one matrix, run to green, landed. The budget is 2.00 \ +because the second run is the lease precondition cancelling an unauthorised \ +matrix, which is the mechanism working. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run land-divergence-record" + +[[verdict]] +id = "lane grade red" +gloss = "red CI runs per landing are over budget" +class = """ +A red run means `verify` was skipped or disagreed with CI, and each one \ +spent a full matrix to say so. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run land-divergence-record" + +[[verdict]] +id = "lane reach late" +gloss = "cancelled runs have a median lifetime past the budget" +class = """ +LATENCY, NEVER COUNT. An early cancellation is `ci-lease-precondition` \ +stopping an unauthorised matrix for ~20 runner-seconds instead of billing ~500; \ +a late one is a matrix billed for a verdict nobody reads. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run land-divergence-record" + +[[verdict]] +id = "lease guard dropped" +gloss = "more CI matrices ran concurrently than the lease admits" +class = """ +Landing is serialised behind a lease, so concurrency above the \ +admitted-successor bound means something is spending CI without holding it. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run land-divergence-record" + +[[verdict]] +id = "lane measure late" +gloss = "runs waited past the budget at p90 before starting" +class = """ +The runner pool saturating, which is a different defect from contention \ +and must not be read as one. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run land-divergence-record" + +[[verdict]] +id = "job measure late" +gloss = "individual matrix legs waited past the budget at p90 before starting" +class = """ +A run's own figure is its FIRST job's start, so this is the one that sees \ +a leg queueing behind its siblings — the two disagreeing is what tells a wide \ +matrix apart from a saturated pool (CLOUD-501). +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run land-divergence-record" + +[[verdict]] +id = "branch reach stale" +gloss = "the fast-forward bot refused a branch that had gone behind" +class = """ +A refusal means the branch went behind before the bot answered — the \ +thundering herd the landing lease exists to remove (243:5 before, 0:5 after). \ +Any refusal at all is a divergence, so the budget is zero. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run land-divergence-record" + # CLOUD-349's two readings, ported off `branch-age-check.sh` under CLOUD-1717. [[verdict]] id = "branch watch stale" diff --git a/bench/gates/RESULTS.md b/bench/gates/RESULTS.md index 85a0ea629..c1a7ddff2 100644 --- a/bench/gates/RESULTS.md +++ b/bench/gates/RESULTS.md @@ -97,7 +97,6 @@ commit, and a named ref resolves a declared one. | `install-check.sh` | git | `tracked` | | `issue-read-check.sh` | git | `location`, `log` | | `issue-search-check.sh` | git | `head`, `location` | -| `land-divergence-assert.sh` | tree | — | | `land-lock-check.sh` | git | `log`, `remote` | | `landed-check.sh` | git | `head`, `log` | | `license-table-check.sh` | tree | — | diff --git a/bench/suites/RESULTS.md b/bench/suites/RESULTS.md index 21215695f..dd4f7b6d0 100644 --- a/bench/suites/RESULTS.md +++ b/bench/suites/RESULTS.md @@ -26,7 +26,6 @@ to it pays. | 3.5 | 2.2% | `tests/mcp-allow-check.bats` | | 3.4 | 2.1% | `tests/doctor-race.bats` | | 3.3 | 2.1% | `tests/ready-cites-check.bats` | -| 3.3 | 2.0% | `tests/land-divergence.bats` | | 3.2 | 2.0% | `tests/hk-selection.bats` | | 3.0 | 1.9% | `tests/ntia-check.bats` | | 2.9 | 1.8% | `tests/target-ensure.bats` | @@ -48,7 +47,6 @@ to it pays. | 1.0 | 0.6% | `tests/install-check.bats` | | 1.0 | 0.6% | `tests/linear-check.bats` | | 1.0 | 0.6% | `tests/lint-rego.bats` | -| 0.9 | 0.6% | `tests/land-divergence-assert.bats` | | 0.9 | 0.6% | `tests/deferral-check.bats` | | 0.9 | 0.6% | `tests/spawn-census.bats` | | 0.9 | 0.5% | `tests/done-check.bats` | diff --git a/crates/batten/tests/it/land_divergence.rs b/crates/batten/tests/it/land_divergence.rs new file mode 100644 index 000000000..d25cdfe0c --- /dev/null +++ b/crates/batten/tests/it/land_divergence.rs @@ -0,0 +1,383 @@ +//! `lane grade other` over the compiled binary (CLOUD-492, CLOUD-1717). +//! +//! # Why this tier and not the module's own `test_` rules +//! +//! Every case in `policy/land-divergence.rego` fabricates its input with +//! `with input as`, which cannot see a fact the engine never projects — the state +//! `policy/branch-age.rego` sat in for a whole session while its own suite stayed +//! green (CLOUD-1810). These run the real module over a record the real verb +//! wrote, and one of them asserts the thing no load-time case can: that a `warn` +//! row REPORTS without failing the run. +//! +//! THAT IS NOT A HYPOTHETICAL HERE EITHER. Registering this module moved +//! `policy test` from 73 bundles to 74 and from 945 cases to 960 — so before the +//! `[[rule]]` row landed, all fifteen of the module's own cases were invisible to +//! the runner while sitting in the tree looking like coverage. The same shape, one +//! level up. +//! +//! # RETIREMENT LEDGER, PER PATH — what `shell retire partial` reads +//! +//! TWO PROGRAMS, TWO SUCCESSORS, AND THE SPLIT WAS ALREADY THERE. +//! `land-divergence` measured and `land-divergence-assert` decided, kept apart for +//! CLOUD-1559's reason: a measurement needs the network and a token, a decision +//! needs neither. The measurement is `[tasks.land-divergence-record]`, the +//! decision is `policy/land-divergence.rego`, and no decision changed hands. The +//! pagination walk, the ETag cache, the `total_count` truncation guard and every +//! instant subtraction could not move: §5 makes `check` `read` and incapable of +//! spawning, and `Fact::Instant` projects `null` to every module. +//! +//! THE EXIT CONTRACT CHANGED, AND SIX ARMS RIDE ON IT. The decider ran `0` under +//! budget / `1` over / `2` could-not-look; the engine runs `0/1/2/3` where `2` is +//! a FINDING. Both total-blindness arms — an unreadable CI run window, an +//! unreadable merged-PR list — are now the producer refusing at write time and +//! recording nothing, which reads here as silence. `unreadable` is the one that +//! stayed a finding, because it is not blindness: it is a window read in PART. +//! +//! THE POSTURE IS PRESERVED BY SEVERITY. Every metric here is a property of the +//! FLEET over a window — cancel latency, queue delay, peak concurrency, the +//! fast-forward bot's refusal ratio — and no branch causes any of them, which is +//! why the retired pair was kept out of the hk gate and out of `verify` and why +//! its own suite asserted that absence. `severity = "warn"` is that on this +//! contract, and `a_divergent_window_reports_without_failing_the_run` holds it. +//! +//! WHAT THIS DELTA SPENDS, stated rather than implied: the producer's own steps — +//! the conditional request, the 304 fallback, the truncation guard, the per-run +//! join and the per-job read — lose their dedicated coverage. +//! `tests/land-divergence.bats` stubbed `gh` and exercised all of them; no +//! compiled-binary case reaches a shell task. The `carried` arms claim the +//! RETIREMENT is covered by the module and this tier, which it is, and they do not +//! claim those steps are. +//! +// carried: mise-tasks/land-divergence-assert.sh policy/land-divergence.rego kind:mechanism crates/batten/tests/it/land_divergence.rs +// carried: tests/land-divergence-assert.bats policy/land-divergence.rego kind:mechanism crates/batten/tests/it/land_divergence.rs +// carried: mise-tasks/land-divergence.sh policy/land-divergence.rego kind:mechanism crates/batten/tests/it/land_divergence.rs +// carried: tests/land-divergence.bats policy/land-divergence.rego kind:mechanism crates/batten/tests/it/land_divergence.rs +// carried: "a linear window passes: one graded run per landing, green, uncontended" policy/land-divergence.rego kind:mechanism +// carried: "graded runs per landing over budget exits 1" policy/land-divergence.rego kind:mechanism +// carried: "red runs per landing over budget exits 1" policy/land-divergence.rego kind:mechanism +// carried: "a 20s cancellation does NOT count as waste" policy/land-divergence.rego kind:mechanism +// carried: "a 400s cancellation DOES count as waste" policy/land-divergence.rego kind:mechanism +// carried: "peak concurrency above the admitted-successor bound exits 1" policy/land-divergence.rego kind:mechanism +// carried: "a queue delay is reported as its own defect, not as contention" policy/land-divergence.rego kind:mechanism +// carried: "any fast-forward refusal at all exits 1" policy/land-divergence.rego kind:mechanism +// carried: "a partially-read window exits 2, never 0" policy/land-divergence.rego kind:mechanism +// carried: "an unreadable window exits 2 even when every other number is ideal" policy/land-divergence.rego kind:mechanism +// carried: "A JOB QUEUE DELAY IS ITS OWN BUDGET, over a clean per-run figure" policy/land-divergence.rego kind:mechanism +// carried: "a window with no landings passes, and says why" policy/land-divergence.rego kind:mechanism +// carried: "the divergent PRs are named on failure, pointer-only" policy/land-divergence.rego kind:mechanism +// changed: "the ratio is reported in hundredths rather than rounded" policy/land-divergence.rego the unit is kept and the rounding is gone, which is what the case was really asking for. Bash has no floats, so the decider computed `$((x * 100 / n))` and DISCARDED the remainder — it could report a ratio marginally under a budget it was over. Rego divides exactly, so the comparison now agrees with the number a reader computes by hand, which is the property the case's own name states +// changed: "empty stdin exits 2" mise.toml there is no stdin: the decider's input is the record family, and an ABSENT family is could-not-look, which on the engine's contract must read as silence rather than as the exit 2 that now means a finding. The property is kept on the producer's side of the door, where both total-blindness arms refuse and write nothing +// changed: "records with no window summary exit 2" mise.toml the summary is emitted unconditionally by the producer and the producer refuses in every arm that cannot reach one, so a family with records and no summary is a torn store rather than an incomplete measurement; the module leaves it unjudged, which is what "no window to judge" means once could-not-look cannot be spelled as a finding +// changed: "two concatenated measurements exit 2 rather than describing neither" policy/land-divergence.rego the reading is unchanged and only its spelling moved: a count over both still describes neither, so neither is judged. `record named` replaces a family rather than appending to it, so two DIFFERENT summaries can now arrive only through a torn store, and a torn store is silence here for the same reason an unparseable line is skipped +// changed: "a non-numeric count exits 2" policy/land-divergence.rego the refusal to coerce is the whole of the case and it is kept — `count_of` is undefined for a value that is not digits, so no rule reading it fires. What changed is that undefinedness says "unjudged" where the shell's exit 2 would now say "violation" +// changed: "a summary missing a count exits 2 rather than reading it as zero" policy/land-divergence.rego the same undefinedness: an absent key leaves `count_of` undefined and the rules that read it silent, rather than reading the gap as a zero that would pass +// changed: "a summary missing the per-job count exits 2 rather than reading it as zero" policy/land-divergence.rego the same, for the one key CLOUD-501 added; it is spelled separately here for the reason the retired suite spelled it separately, which is that a per-job figure read as zero is exactly the reading that hides a matrix queueing on its own legs +// changed: "a clean per-job figure passes, and the success line reports it" policy/land-divergence.rego the pass is kept and the success line is not: a module emits findings, and non-negotiable rule 4 makes output a pointer rather than a report. The numbers a reader wanted from that line are in the record the producer writes and echoes to the step summary, which is where `land-divergence.yml` publishes them +// changed: "a linear window measures one graded run against one landing" mise.toml the measurement is the producer's: it walks the Actions API per workflow and joins runs to landings by branch bounded by `merged_at`, which §5 keeps outside `check` +// changed: "the conditional request actually sends If-None-Match once an ETag is cached" mise.toml an ETag is a property of an HTTP request, and the request is the producer's +// changed: "land-divergence.bats::A 304 KEEPS THE PREVIOUS READING rather than reading as an empty window" mise.toml the cache lives under `.git/` and is read on the request path, so the 304 reading never reaches a record at all +// changed: "land-divergence.bats::a 304 with no cached body is unreadable, never an empty window" mise.toml the same request path: with no body to fall back on the producer counts the read as unreadable, which is the input to `lane read partial` rather than a decision of its own +// changed: "TRUNCATION IS CAUGHT BY total_count, not by running out of pages" mise.toml the pagination walk is the producer's and so is the guard: it compares what it collected against the `total_count` the endpoint reports, which is the only honest stopping condition when the API hard-caps at 1000 items while still reporting the true total +// changed: "a truncated window makes the decider refuse, rather than judge a prefix" policy/land-divergence.rego kept as `lane read partial`, which fires on any non-zero `unreadable` whatever the other numbers are — the arm the truncation guard above exists to feed, and the one case in this retirement where could-not-look stays a finding rather than becoming silence +// changed: "runs after a PR merged are not attributed to it" mise.toml the join is the producer's: runs are matched to a landing by branch AND bounded by `merged_at`, so a branch's post-merge runs and a reused name's later life are excluded before any record exists +// changed: "a draft-era skipped run is not counted as graded" mise.toml the same join: a `skipped` run spends no matrix and is not graded, so the producer never emits it +// changed: "an unreadable run list reports rather than claiming a clean window" mise.toml this one genuinely changed rather than moved: with no run list the producer now refuses and records nothing, because recording an empty window carrying `unreadable=1` would spell total blindness as the partial-coverage finding, and those are different facts +// changed: "A JOB THAT WAITED REPORTS ITS GAP, where the run's own figure cannot" mise.toml the per-job read is one request per graded run and the gap is a subtraction of instants, both of which §5 and the clock ban keep outside the engine; what the module judges is the p95 the producer recorded +// changed: "a job that started immediately reports zero and earns no record" mise.toml the producer's own emission rule, kept: a zero-wait leg is the ideal and says nothing a reader needs +// changed: "A RUN WHOSE JOBS CANNOT BE READ IS UNREADABLE, NEVER A ZERO WAIT" mise.toml the producer counts it as unreadable rather than dropping it, which is what keeps `lane read partial` able to fire; dropping it would report a p90 over the legs that happened to answer, the partial-coverage false green this whole sensor refuses +// changed: "the per-job read is bounded to graded runs, so a skipped run costs no request" mise.toml a request budget is a property of the program making the requests +// changed: "the job record is pointer-only — a run id, a job name and seconds" mise.toml asserted of the producer now, where the fetching happens and where a log body could have entered; the module's own pointer discipline is asserted below +// changed: "neither half is in the hk gate" mise.toml the wiring property survives and its subject moved with the programs: there is no `land-divergence` hk step, and `[tasks.land-divergence-record]` is not in any gate list. What replaced the decider is a `warn` row, which cannot fail an hk run even when it fires +// changed: "neither half is in verify" mise.toml the same, one caller over: `verify` does not run the producer, and the `lane grade other` row it does evaluate is `warn`, so a branch cannot go red over a property of the fleet +// changed: "the scheduled workflow declares a concurrency group, like every other" .github/workflows/land-divergence.yml unchanged in substance and kept in the workflow it is about; the scheduled job still declares its group, and what moved is only which task its judge step runs + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use common::{git_in, init_repo, run, run_with_stdin, scratch, write}; + +/// A repository registering the real module against a declared family. +fn repo(name: &str) -> std::path::PathBuf { + let dir = scratch(&format!("land-divergence-{name}")); + let module = std::fs::read_to_string("../../policy/land-divergence.rego") + .expect("the module this tier exists for"); + write(&dir, "policy/land-divergence.rego", &module); + write(&dir, "batten.toml", &config()); + init_repo(&dir); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-qm", "register the module"]); + dir +} + +/// The eight verdicts, the rule and the family, spelled as `batten.toml` spells +/// them. Built rather than pasted so a ninth verdict cannot be added to the module +/// and silently left undeclared here. +fn config() -> String { + let verdicts = [ + "lane read partial", + "lane count spent", + "lane grade red", + "lane reach late", + "lease guard dropped", + "lane measure late", + "job measure late", + "branch reach stale", + ]; + let mut out = String::from( + r#"version = 1 +scope = ["**"] + +# The module reads its count guard by id rather than spelling the expression +# inline, which `policy test` refuses: an expression is a consumer fact and belongs +# in the config (rule 1). +[[pattern]] +id = "whole-number" +regex = '^[0-9]+$' + +[[rule]] +id = "lane grade other" +kind = "policy" +scope = "tree" +module = "policy/land-divergence.rego" +severity = "warn" + +[[record]] +record = "land-divergence" +writer = "mise run land-divergence-record" +"#, + ); + for id in verdicts { + out.push_str(&format!( + r#" +[[verdict]] +id = "{id}" +gloss = "the landing loop diverged from linear over the measured window" +class = "One matrix, run to green, landed is the claim; this is a departure from it." + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run land-divergence-record" +"# + )); + } + out +} + +/// Write the producer's record, as `mise run land-divergence-record` would. +fn record(dir: &std::path::Path, lines: &str) { + let written = run_with_stdin(dir, &["record", "named", "land-divergence"], lines); + assert!( + written.status.success(), + "the setup write lands: {}", + String::from_utf8_lossy(&written.stderr) + ); +} + +/// One `window` line. Every field is named, because a positional summary is what +/// the retired pair's own format avoided and because a case that silently shifts a +/// column would assert about the wrong number. +#[allow(clippy::too_many_arguments)] +fn window( + landings: u32, + graded: u32, + red: u32, + cancelled: u32, + cancel_p50: u32, + peak: u32, + queue_p90: u32, + queue_job_p90: u32, + ff_refused: u32, + unreadable: u32, +) -> String { + let green = graded.saturating_sub(red).saturating_sub(cancelled); + format!( + "window\tsince=2026-08-12T00:00:00Z\tlandings={landings}\tgraded={graded}\t\ + green={green}\tred={red}\tcancelled={cancelled}\tcancel_p50={cancel_p50}\t\ + peak_concurrency={peak}\tqueue_p90={queue_p90}\tqueue_job_p90={queue_job_p90}\t\ + retries=0\tff_refused={ff_refused}\tff_success={landings}\tunreadable={unreadable}\n" + ) +} + +/// The linear ideal: one landing, one graded green run, nothing waiting. +fn linear() -> String { + window(1, 1, 0, 0, 0, 1, 0, 0, 0, 0) +} + +fn said(out: &std::process::Output) -> String { + format!( + "{}{}", + String::from_utf8_lossy(&out.stdout), + String::from_utf8_lossy(&out.stderr) + ) +} + +#[test] +fn a_linear_window_is_clean_over_the_engines_projection() { + let dir = repo("linear"); + record(&dir, &linear()); + + let quiet = run(&dir, &["check", "--fail-on-warning"]); + assert_eq!( + quiet.status.code(), + Some(0), + "one matrix, run to green, landed is the state the gate must reach\n{}", + said(&quiet) + ); +} + +#[test] +fn a_loop_buying_more_than_one_matrix_per_landing_is_reported() { + let dir = repo("graded"); + record(&dir, &window(1, 3, 0, 0, 0, 1, 0, 0, 0, 0)); + + let reported = run(&dir, &["check", "--fail-on-warning"]); + assert_eq!( + reported.status.code(), + Some(2), + "three graded runs against one landing is over the 2.00 budget\n{}", + said(&reported) + ); +} + +#[test] +fn a_divergent_window_reports_without_failing_the_run() { + // THE PORTED POSTURE, and the case no load-time rule can make. The retired + // pair was deliberately kept out of the hk gate and out of `verify`, and its + // own suite asserted that absence, because every metric here is a property of + // the fleet that no branch causes. `warn` is that on this contract: the same + // record that reds `--fail-on-warning` above must leave an ordinary `check` + // green. + let dir = repo("warn"); + record(&dir, &window(1, 3, 0, 0, 0, 1, 0, 0, 0, 0)); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "a branch must not go red over yesterday's runner saturation\n{}", + said(&quiet) + ); +} + +#[test] +fn an_early_cancellation_is_the_mechanism_working_and_a_late_one_is_not() { + // THE FINDING THIS SENSOR EXISTS TO ENCODE, and the case that stops a future + // author collapsing the rule into a count. Measured 2026-08-12: 5 green runs + // against 5 cancelled reads as a 50% discard rate and is the opposite — those + // cancels had p50 ~20s, the lease precondition killing an unauthorised matrix + // for ~20 runner-seconds instead of billing ~500. + let early = repo("cancel-early"); + record(&early, &window(1, 2, 0, 1, 20, 2, 0, 0, 0, 0)); + assert_eq!( + run(&early, &["check", "--fail-on-warning"]).status.code(), + Some(0), + "a 20s cancellation is the lease precondition working" + ); + + let late = repo("cancel-late"); + record(&late, &window(1, 2, 0, 1, 400, 2, 0, 0, 0, 0)); + let reported = run(&late, &["check", "--fail-on-warning"]); + assert_eq!( + reported.status.code(), + Some(2), + "a 400s cancellation is a matrix billed for a verdict nobody reads\n{}", + said(&reported) + ); +} + +#[test] +fn a_leg_queueing_behind_its_siblings_is_its_own_finding() { + // CLOUD-501. A run's figure is its FIRST job's start, so a matrix leg waiting + // behind its siblings is invisible in it. The two disagreeing is exactly what + // separates a wide matrix from a saturated pool, which is why the per-job + // figure has its own budget rather than replacing the per-run one. + let dir = repo("queue-job"); + record(&dir, &window(1, 1, 0, 0, 0, 1, 0, 252, 0, 0)); + + let reported = run(&dir, &["check", "--fail-on-warning"]); + let text = said(&reported); + assert_eq!( + reported.status.code(), + Some(2), + "a leg that waited 252s at p90 is reported\n{text}" + ); + assert!( + text.contains("job measure late"), + "and as the per-JOB finding, over a clean per-run figure\n{text}" + ); +} + +#[test] +fn any_fast_forward_refusal_at_all_is_reported() { + // THE ONE METRIC THAT IS NOT A THRESHOLD. A refusal means the branch went + // behind before the bot answered — the thundering herd the landing lease + // exists to remove (243:5 before, 0:5 after). + let dir = repo("ff"); + record(&dir, &window(1, 1, 0, 0, 0, 1, 0, 0, 1, 0)); + + let reported = run(&dir, &["check", "--fail-on-warning"]); + assert_eq!( + reported.status.code(), + Some(2), + "one refusal is one divergence\n{}", + said(&reported) + ); +} + +#[test] +fn a_partially_read_window_is_a_finding_rather_than_a_clean_one() { + // `bench-assert`'s partial-coverage rule, and the arm the producer's + // truncation guard exists to feed: the Actions runs endpoint caps pagination + // at 1000 items while still reporting the true total, so a walk that stops on + // a short page reads a prefix and looks like a clean finish. Measured: 1000 of + // 1446, reporting zero fast-forward refusals over a window carrying 598. + // + // It fires with every other number at its ideal, which is the whole point. + let dir = repo("partial"); + record(&dir, &window(1, 1, 0, 0, 0, 1, 0, 0, 0, 2)); + + let reported = run(&dir, &["check", "--fail-on-warning"]); + assert_eq!( + reported.status.code(), + Some(2), + "a green verdict over a prefix would cover less than it claims\n{}", + said(&reported) + ); +} + +#[test] +fn a_window_with_no_landings_judges_nothing_but_is_still_a_reading() { + // ANTI-VACUITY. Without a denominator every ratio is undefined, which is the + // honest reading of a quiet day — nothing landed, so nothing diverged. This + // repo has been bitten twice by a gate that cannot fire reading the same as one + // that found nothing (`finding-sink-check`, `bench-assert`), and the record's + // PRESENCE is what keeps that apart from the case below. + let dir = repo("quiet"); + record(&dir, &window(0, 0, 0, 0, 0, 0, 0, 0, 0, 0)); + + let quiet = run(&dir, &["check", "--fail-on-warning"]); + assert_eq!( + quiet.status.code(), + Some(0), + "a quiet day judges nothing\n{}", + said(&quiet) + ); +} + +#[test] +fn an_absent_record_says_nothing_rather_than_passing() { + // Both total-blindness arms of the retired measurer are now the producer + // refusing and writing nothing. This is what that absence must read as. + let dir = repo("absent"); + + let quiet = run(&dir, &["check", "--fail-on-warning"]); + assert_eq!( + quiet.status.code(), + Some(0), + "an absent record is could-not-look\n{}", + said(&quiet) + ); +} diff --git a/crates/batten/tests/it/main.rs b/crates/batten/tests/it/main.rs index 41c2791d0..18866a1ca 100644 --- a/crates/batten/tests/it/main.rs +++ b/crates/batten/tests/it/main.rs @@ -162,6 +162,7 @@ mod inverted_board_cases; mod issue_key; mod judge_kind; mod land; +mod land_divergence; mod land_entry_gates; mod land_forge_reads; mod land_hand_stepping; diff --git a/crates/batten/tests/it/nonverdict.rs b/crates/batten/tests/it/nonverdict.rs index 57cc3cb1e..6ca9a5406 100644 --- a/crates/batten/tests/it/nonverdict.rs +++ b/crates/batten/tests/it/nonverdict.rs @@ -69,8 +69,8 @@ // changed: "THE FAN-IN IS EXCLUDED: final's needs-assertion is not a non-verdict failure" mise.toml the exclusion is by job name over the API payload, before any record exists, so it is part of what the producer measures rather than of what the module judges // changed: "A JOB OUTSIDE THE ROSTER IS EXCLUDED: a declining merge bot is not a failure here" mise.toml the roster is `$CI_REQUIRED_CHECKS` from `mise.toml [env]`, which the producer reads and the engine does not project // changed: "the conditional request is actually sent once an ETag is stored" mise.toml an ETag is a property of an HTTP request, and the request is the producer's -// changed: "A 304 KEEPS THE PREVIOUS READING rather than reading as an empty window" mise.toml the cache lives under `.git/` and is read on the request path, so the 304 reading never reaches a record at all -// changed: "a 304 with no cached body is unreadable, never an empty window" mise.toml the same request path: with no body to fall back on the producer counts the run as unreadable, which is the input to `job read partial` rather than a decision of its own +// changed: "nonverdict-scan.bats::A 304 KEEPS THE PREVIOUS READING rather than reading as an empty window" mise.toml the cache lives under `.git/` and is read on the request path, so the 304 reading never reaches a record at all +// changed: "nonverdict-scan.bats::a 304 with no cached body is unreadable, never an empty window" mise.toml the same request path: with no body to fall back on the producer counts the run as unreadable, which is the input to `job read partial` rather than a decision of its own // changed: "an unreadable jobs read is counted, not silently dropped" mise.toml counting is the producer's; what the count MEANS is the module's, and `job read partial` is where it is now decided // changed: "an empty roster is unreadable rather than a count over every job" mise.toml this one genuinely changed rather than moved: with no roster the producer now refuses and records nothing, because recording `unreadable=1` over an empty window would spell total blindness as the partial-coverage finding, and those are different facts // changed: "a summary line is always emitted, even when nothing failed" mise.toml the producer still emits it unconditionally, and the reason is now stronger: it is the one line that distinguishes a window that found nothing from nobody having looked diff --git a/mise-tasks/land-divergence-assert.sh b/mise-tasks/land-divergence-assert.sh deleted file mode 100755 index e5212f045..000000000 --- a/mise-tasks/land-divergence-assert.sh +++ /dev/null @@ -1,193 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Gate: the landing loop's divergence from linear is under budget (reads `land-divergence` records on stdin)" -# -# CLOUD-492. THIS DECIDES; IT DOES NOT MEASURE. `mise run land-divergence` reads -# the API and emits records; this is a pure function of them — the -# `bench`/`bench-assert` split, for the reason that split exists: a measurement -# needs the network and a token, a decision needs neither, so keeping them apart -# is what lets `tests/land-divergence-assert.bats` run in the hk gate on every -# commit while the measurement runs on a clock. -# -# WHY A GATE AND NOT A REPORT. Non-negotiable rule 2: a log without a gate is -# sensor only. The landing loop's cost was established once by hand and could not -# be re-established without repeating the whole exercise, so a step change — a -# lease that stops being honoured, a workflow that starts firing twice, a rebase -# storm — was invisible between assessments. A number that rises has to fail -# something or it becomes another artifact nobody opens. -# -# EXIT 0 under budget / 1 over / 2 could not look. The house style's `0/1/2/3` -# table governs `batten` VERBS, where `2` is the policy verdict; this is a `mise` -# task, and it takes the decider convention `bench-assert`, `checks-green` and -# `nonverdict-assert` already share. The two contracts genuinely disagree about -# what `2` means, so which one is in force is stated rather than left to be -# inferred. `2` here is "the gate could not read what it was asked to judge", -# deliberately distinct from a violation: a gate reporting green over input it -# failed to parse is the failure that gets a gate switched off. -# A gate listed in $MUTANT_GATES with no row here fails `mise run mutant`. -#MUTANT over-budget-passes|s/^exit 1$/exit 0/|graded runs per landing over budget - -set -euo pipefail - -# The budgets, as data with raise-only env overrides — the placement -# `bench-assert`'s BUDGETS, `branch-age-check`'s BRANCH_MAX_AGE_DAYS and -# `timeout-drift`'s multipliers all use. NOT `batten.toml [ci]`: that table is a -# projection of a host ruleset `ci-drift` diffs against, and none of these has a -# host counterpart, so a key there would make `ci-drift` police something nothing -# declares. NOT `mise.toml [env]`: that is for values two or more programs share. -# -# Seeded from the measurement taken 2026-08-12 after landing serialisation, so -# the gate starts at the observed state rather than at an aspiration. The ideal -# is 1.0 graded runs per landing; the budget is 2.0, because the second run is -# the ~20s lease-precondition cancellation that the same measurement showed is -# the mechanism WORKING. -# -# Ratios are held in HUNDREDTHS, because bash has no floats and a gate that -# rounds is a gate that disagrees with the number it printed. -MAX_GRADED_PER_LANDING="${BATTEN_DIVERGENCE_MAX_GRADED:-200}" # 2.00 -MAX_RED_PER_LANDING="${BATTEN_DIVERGENCE_MAX_RED:-20}" # 0.20 -MAX_CANCEL_P50="${BATTEN_DIVERGENCE_MAX_CANCEL_P50:-60}" # seconds -MAX_PEAK_CONCURRENCY="${BATTEN_DIVERGENCE_MAX_PEAK:-3}" -MAX_QUEUE_P90="${BATTEN_DIVERGENCE_MAX_QUEUE_P90:-30}" # seconds - -# The same wait attributed PER JOB (CLOUD-501). A run's figure is its FIRST job's -# start, so a matrix leg queueing behind its siblings is invisible in it; these -# two disagreeing is the signal that separates a wide matrix from a saturated -# pool, which is why the job figure gets its own budget rather than replacing the -# run one. Same 30s, because the ideal is the same: a leg starts when it is -# created. -MAX_QUEUE_JOB_P90="${BATTEN_DIVERGENCE_MAX_QUEUE_JOB_P90:-30}" # seconds - -# The one metric that is NOT a threshold: a fast-forward refusal means the branch -# went behind before the bot answered, which is the thundering herd the landing -# lease removed (243:5 before, 0:5 after). Any refusal at all is a divergence, so -# the budget is zero and stating it as a tunable would invite raising it. -MAX_FF_REFUSED="${BATTEN_DIVERGENCE_MAX_FF_REFUSED:-0}" - -records=$(cat) - -report() { echo " $1" >&2; } - -if [[ -z "${records//[[:space:]]/}" ]]; then - echo "::error:: land-divergence-assert: stdin is empty — run \`mise run land-divergence\` redirected to a file, then read it back (a pipeline would hand this gate's exit status to its last stage)." >&2 - exit 2 -fi - -# Records are TAB-separated, because branch names are unconstrained. One kind per -# line: -# -# pr\tnumber=\tbranch=\tgraded=\tgreen=\tred=\tcancelled= -# cancel\trun=\tbranch=\tlatency= -# job\trun=\tjob=\tqueue= -# window\tsince=\tlandings=\t…\tunreadable= -# -# Only DIVERGENT PRs earn a `pr` record; a single graded green run is the ideal -# and says nothing a reader needs. The summary carries the totals regardless, so -# a window with no records is distinguishable from one that was never read. -summary=$(awk -F'\t' '$1 == "window" { print; found = 1 } END { exit !found }' <<<"$records") || summary="" -if [[ -z "$summary" ]]; then - echo "::error:: land-divergence-assert: the records carry no \`window\` summary line, so there is no window to judge — did \`land-divergence\` complete?" >&2 - exit 2 -fi - -if [[ "$(awk -F'\t' '$1 == "window"' <<<"$records" | grep -c .)" != "1" ]]; then - echo "::error:: land-divergence-assert: stdin carries more than one \`window\` summary — two measurements were concatenated, and a count over both describes neither." >&2 - exit 2 -fi - -field() { - awk -F'\t' -v key="$1" ' - $1 == "window" { - for (i = 2; i <= NF; i++) { - n = index($i, "=") - if (n > 0 && substr($i, 1, n - 1) == key) { print substr($i, n + 1); exit } - } - } - ' <<<"$records" -} - -# Validated first, then assigned by name. The compact form — `printf -v "$name"` -# inside the loop — is invisible to shellcheck, which then reports every one of -# these as referenced-but-not-assigned; a gate that has to be silenced to pass -# its own linter is worse than the repetition. -for name in landings graded red cancel_p50 peak_concurrency queue_p90 queue_job_p90 ff_refused unreadable; do - value=$(field "$name") - if [[ -z "$value" ]] || [[ "$value" != "${value#*[^0-9]}" ]]; then - echo "::error:: land-divergence-assert: the \`window\` summary carries no readable \`$name\` count, so the window cannot be judged." >&2 - exit 2 - fi -done - -landings=$(field landings) -graded=$(field graded) -red=$(field red) -cancel_p50=$(field cancel_p50) -peak_concurrency=$(field peak_concurrency) -queue_p90=$(field queue_p90) -queue_job_p90=$(field queue_job_p90) -ff_refused=$(field ff_refused) -unreadable=$(field unreadable) - -# A measurement that could not read part of its window judged less than it -# claims. `bench-assert`'s partial-coverage rule: a run that measured two of -# three paths and reported green over the two is exactly the partial-coverage -# false green. The measurer's own truncation guard is what sets this. -if [[ "$unreadable" != "0" ]]; then - echo "::error:: land-divergence-assert: the measurement could not read $unreadable part(s) of its window, so a green verdict here would cover less than it claims." >&2 - report "narrow BATTEN_DIVERGENCE_SINCE — the runs endpoint caps pagination at 1000 items while still reporting the true total" - exit 2 -fi - -# ANTI-VACUITY. A window with no landings cannot fire, and a gate that cannot -# fire must not be indistinguishable from one that found nothing — this repo has -# been bitten by that twice (`finding-sink-check`, `bench-assert`). It is also -# the honest reading of a quiet day: nothing landed, so nothing diverged. -if [[ "$landings" = "0" ]]; then - echo "land-divergence-assert: no landings in the window — nothing to judge" >&2 - exit 0 -fi - -fail=0 -over() { - echo "::error:: land-divergence-assert: $1" >&2 - fail=1 -} - -graded_ratio=$((graded * 100 / landings)) -red_ratio=$((red * 100 / landings)) - -[[ "$graded_ratio" -le "$MAX_GRADED_PER_LANDING" ]] || - over "$graded graded CI run(s) over $landings landing(s) is $((graded_ratio / 100)).$(printf '%02d' $((graded_ratio % 100))) per landing, over the budget of $((MAX_GRADED_PER_LANDING / 100)).$(printf '%02d' $((MAX_GRADED_PER_LANDING % 100))). The ideal is 1.00 — one matrix, run to green, landed." - -[[ "$red_ratio" -le "$MAX_RED_PER_LANDING" ]] || - over "$red red CI run(s) over $landings landing(s) is $((red_ratio / 100)).$(printf '%02d' $((red_ratio % 100))) per landing, over the budget of $((MAX_RED_PER_LANDING / 100)).$(printf '%02d' $((MAX_RED_PER_LANDING % 100))). A red run means \`verify\` was skipped or disagreed with CI; each one spent a full matrix." - -# LATENCY, NEVER COUNT. A cancellation at ~20s is `ci-lease-precondition` -# stopping an unauthorised matrix for ~20 runner-seconds instead of billing ~500 -# — the mechanism working, and a gate counting cancellations would score it as a -# defect and argue for its removal. What is worth failing on is a cancellation -# that arrives LATE, because by then the matrix has been paid for. -[[ "$cancel_p50" -le "$MAX_CANCEL_P50" ]] || - over "cancelled runs have a median lifetime of ${cancel_p50}s, over the budget of ${MAX_CANCEL_P50}s. An early cancellation is the lease precondition working; a late one is a matrix billed for a verdict nobody reads." - -[[ "$peak_concurrency" -le "$MAX_PEAK_CONCURRENCY" ]] || - over "$peak_concurrency CI matrices ran concurrently at peak, over the budget of $MAX_PEAK_CONCURRENCY. Landing is serialised behind a lease, so concurrency above the admitted-successor bound means something is spending CI without holding it." - -[[ "$queue_p90" -le "$MAX_QUEUE_P90" ]] || - over "runs waited ${queue_p90}s at p90 before starting, over the budget of ${MAX_QUEUE_P90}s. That is the runner pool saturating, which is a different defect from contention and must not be read as one." - -[[ "$queue_job_p90" -le "$MAX_QUEUE_JOB_P90" ]] || - over "individual JOBS waited ${queue_job_p90}s at p90 before starting, over the budget of ${MAX_QUEUE_JOB_P90}s. A run's own figure is its first job's start, so this is the one that sees a matrix leg queueing behind its siblings — the two disagreeing tells a wide matrix apart from a saturated pool." - -[[ "$ff_refused" -le "$MAX_FF_REFUSED" ]] || - over "the fast-forward bot refused $ff_refused time(s). A refusal means the branch went behind before the bot answered — the thundering herd the landing lease exists to remove." - -if [[ "$fail" = "0" ]]; then - echo "land-divergence-assert: $landings landing(s) bought $graded graded CI run(s) ($((graded_ratio / 100)).$(printf '%02d' $((graded_ratio % 100))) each), $red red, peak concurrency $peak_concurrency, cancel p50 ${cancel_p50}s, queue p90 ${queue_p90}s (${queue_job_p90}s per job), $ff_refused fast-forward refusal(s)" - exit 0 -fi - -while IFS= read -r line; do - [[ -n "$line" ]] || continue - report "$(awk -F'\t' '{ sub(/^pr\t/, ""); print }' <<<"$line")" -done <<<"$(awk -F'\t' '$1 == "pr"' <<<"$records")" -exit 1 diff --git a/mise-tasks/land-divergence.sh b/mise-tasks/land-divergence.sh deleted file mode 100755 index 74178b97f..000000000 --- a/mise-tasks/land-divergence.sh +++ /dev/null @@ -1,391 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Measure: how far the landing loop diverged from linear over a window (one record per divergent run; `land-divergence-assert` decides)" -# -# CLOUD-492. THIS MEASURES; IT DOES NOT DECIDE. The verdict is -# `mise run land-divergence-assert`'s, a pure function of the records below — the -# `bench`/`bench-assert` split that `nonverdict-scan`/`nonverdict-assert` also -# uses, for the same reason: a measurement needs the network and a token, a -# decision needs neither, so the decider's suite runs in the hk gate on every -# commit while this runs on a clock. -# -# ## What "linear" means, and why each metric is the one chosen -# -# The landing loop is serialised behind a lease (CLOUD-393) and an unauthorised -# matrix is stopped server-side (CLOUD-420). The claim those make is that a PR -# buys ONE CI matrix, runs it to green, and lands. Divergence from that is the -# whole signal, and it was established once by hand — ~50 paginated calls and -# throwaway jq — which is why nothing could see a step change BETWEEN two -# assessments. -# -# ## A CANCELLED RUN IS NOT WASTE, AND COUNTING THEM INVERTS THE VERDICT -# -# The finding this sensor exists to encode. Measured 2026-08-12 after -# serialisation: 5 green CI runs against 5 cancelled, which reads as a 50% -# discard rate and is the opposite. Those cancels had p50 lifetime ~20s — -# `ci-lease-precondition` killing an unauthorised matrix for ~20 runner-seconds -# instead of billing ~500: -# -# 21:52:07 cancelled cloud-sequential-prs 13s <- unauthorised, killed -# 21:52:10 success cloud-sequential-prs 432s <- holder, runs to green -# -# Before serialisation the same population had p75 147s and max 413s. THOSE were -# waste. So the graded quantity is CANCEL LATENCY, never cancel count: a sensor -# counting cancels would score the working precondition as a defect and argue for -# removing it. -# -# ## Per workflow, never the global run list -# -# `actions/runs` carries ~3100 skipped auto-land runs a day, and at that -# insertion rate page boundaries shift mid-walk, so a paginated read of it is not -# stable. Per-workflow with a `created>=` filter is ~3-4 pages each and stable. -# (CLOUD-493 removes most of that noise at the source; this stays per-workflow -# regardless, because stability is the reason and it does not depend on volume.) -# -# ## NO WATERMARK, AND THAT IS THE DESIGN -# -# The obvious shape is a persisted ledger advanced by a watermark. It cannot work -# here: a scheduled tick gets a fresh checkout, so nothing survives between them, -# and a committed ledger pollutes the tree against non-negotiable rule 7. The -# Actions API is already the durable store (90-day retention), so every tick is a -# PURE FUNCTION OF ITS WINDOW — which also makes this re-runnable over any past -# window for free, and that is what a divergence investigation needs and a -# dashboard does not. -# -# The ETag cache below is a different thing and must not be confused with one: it -# is a within-clone read optimisation, discarded with the clone, exactly as -# `nonverdict-scan` describes its own. -# -# OUTPUT IS A POINTER, NEVER A PAYLOAD (rule 4): run ids, branch names, -# conclusions, latencies and counts. No log body, no commit message, no PR title -# is ever fetched, so none can leak. TAB-separated, because branch names are -# unconstrained. -set -euo pipefail - -cd "$(git rev-parse --show-toplevel)" - -# The window, as an ISO-8601 instant. A DATE RANGE rather than a run count, which -# is the opposite of `nonverdict-scan`'s choice and deliberately so: that sensor -# asks "how often does this failure mode occur", which a fixed sample answers, -# while this one asks "what did the loop cost over a period", which only a period -# answers. A count would silently narrow the window whenever the fleet got busy — -# exactly when the answer matters most. -SINCE="${BATTEN_DIVERGENCE_SINCE:-$(date -u -d '24 hours ago' +%Y-%m-%dT%H:%M:%SZ)}" - -# The cache root. Under `.git/`, so it is out of tree, never committed, and -# discarded with the clone rather than surviving as stale state on a fresh one. -cache="${BATTEN_DIVERGENCE_CACHE:-$(git rev-parse --git-dir)/batten-divergence}" -mkdir -p "$cache" - -unreadable=0 - -# `gh api -i`, then the status line and the ETag out of the headers, then the -# body after the first blank line — `main-watch`'s form, as `nonverdict-scan` -# carries it. A non-2xx makes `gh` exit non-zero (a 304 included), so the call is -# guarded rather than trusted: under `set -e` an unguarded 304 would abort on the -# cheapest possible response. -# -# Sets `body` and returns 0 on a usable reading, 1 when it could not look. -conditional_get() { - local url="$1" key etag_file body_file resp status new_etag - key=$(printf '%s' "$url" | sha1sum | cut -d' ' -f1) - etag_file="$cache/$key.etag" - body_file="$cache/$key.body" - - local args=(-i "$url") - if [[ -s "$etag_file" ]]; then - args+=(-H "If-None-Match: $(cat "$etag_file")") - fi - - resp=$(gh api "${args[@]}" 2>/dev/null | tr -d '\r') || true - status=$(printf '%s' "$resp" | sed -n '1s@^HTTP/[0-9.]* \([0-9]*\).*@\1@p') - - # Reassigned only when non-empty: a response carrying no ETag must not clear - # the stored one, or the next tick is a cold read for no reason. `if`, not a - # `&&` list — under `set -e` a failing trailing `&&` aborts the whole scan, - # which `nonverdict-scan` measured as a silent empty window. - new_etag=$(printf '%s' "$resp" | sed -n 's/^[Ee][Tt]ag: //p' | head -n1) - if [[ -n "$new_etag" ]]; then - printf '%s' "$new_etag" >"$etag_file" - fi - - if [[ "$status" = "304" ]]; then - # The whole point of the cache. No body arrives, so the previous reading - # IS the answer; without one there is nothing to fall back to, and an - # empty parse would read as a clean window — the false green this sensor - # exists to report. - [[ -s "$body_file" ]] || return 1 - body=$(cat "$body_file") - return 0 - fi - - case "$status" in - 2*) ;; - *) return 1 ;; - esac - - body=$(printf '%s' "$resp" | awk 'body {print} /^$/ {body=1}') - [[ -n "${body//[[:space:]]/}" ]] || return 1 - printf '%s' "$body" >"$body_file" - return 0 -} - -summary() { - printf 'window\tsince=%s\tlandings=%s\tgraded=%s\tgreen=%s\tred=%s\tcancelled=%s\tcancel_p50=%s\tpeak_concurrency=%s\tqueue_p90=%s\tqueue_job_p90=%s\tretries=%s\tff_refused=%s\tff_success=%s\tunreadable=%s\n' \ - "$SINCE" "$1" "$2" "$3" "$4" "$5" "$6" "$7" "$8" "$9" "${10}" "${11}" "${12}" "$unreadable" -} - -bail() { - echo "::error:: land-divergence: $1" >&2 - unreadable=$((unreadable + 1)) - summary 0 0 0 0 0 0 0 0 0 0 0 0 - exit 0 -} - -# `created>=` must be URL-encoded: `>` is not legal in a query string, and gh -# passes the path through verbatim. -enc_since=$(printf '%s' "$SINCE" | sed 's/:/%3A/g') - -# --- CI runs over the window -------------------------------------------------- -# -# `ci.yml` is the matrix a landing buys. `?created=>=` plus a per-page cap, and -# the walk stops at the first page shorter than the cap — the `link` header is -# the correct general answer, but a per-workflow day window is a few pages and -# this keeps the request parameters identical between ticks, which is what keeps -# the 304 hit rate up (`mem:github-rest-etiquette`). -# TRUNCATION IS DETECTED BY `total_count`, NOT BY RUNNING OUT OF PAGES, and that -# distinction was measured rather than reasoned. -# -# The Actions runs endpoint HARD-CAPS pagination at 1000 items while still -# reporting the true `total_count`. Measured on `fast-forward.yml` over a 25-hour -# window: `total_count` 1446, page 10 full, page 11 EMPTY. So a walk that stops -# when a page comes back short — the obvious loop, and the one written first here -# — collects 1000 of 1446 and looks exactly like a clean finish. It reported -# `ff_refused=0` over a window carrying 598 refusals, which is a perfect score -# read off a prefix: the precise false green this sensor exists to catch. -# -# `total_count` is therefore the only honest stopping condition. Compare it -# against what was actually collected and report the gap; a wrong number reported -# confidently is worse than no number, and this is the shape that produces one. -MAX_PAGES="${BATTEN_DIVERGENCE_MAX_PAGES:-10}" - -walk_workflow() { # $1 = workflow file, $2 = jq expression emitting ONE line per run - local wf="$1" expr="$2" page=1 got total="" out="" have - while [[ "$page" -le "$MAX_PAGES" ]]; do - if ! conditional_get "repos/{owner}/{repo}/actions/workflows/$wf/runs?created=%3E%3D$enc_since&per_page=100&page=$page"; then - return 1 - fi - if [[ -z "$total" ]]; then - total=$(printf '%s' "$body" | jq -r '.total_count // 0' 2>/dev/null) || total=0 - fi - got=$(printf '%s' "$body" | jq -r '.workflow_runs | length' 2>/dev/null) || got=0 - out="$out$(printf '%s' "$body" | jq -r "$expr" 2>/dev/null) -" - [[ "$got" = "100" ]] || break - page=$((page + 1)) - done - - out=$(printf '%s' "$out" | sed '/^[[:space:]]*$/d') - printf '%s' "$out" - have=$(printf '%s' "$out" | grep -c . || true) - if [[ "$have" -lt "${total:-0}" ]]; then - echo "::error:: land-divergence: read $have of $total $wf run(s) since $SINCE — the API caps this endpoint at 1000 items, so these counts describe a prefix of the window. Narrow BATTEN_DIVERGENCE_SINCE." >&2 - return 2 - fi - return 0 -} - -# An exact count for one conclusion, from `total_count` with a status filter — -# one request, no pagination, and therefore no cap to be truncated by. Used -# wherever only a COUNT is needed, which is why the fast-forward ratio does not -# go through `walk_workflow` at all: at this repo's comment volume that window is -# routinely over the 1000-item ceiling, so a walk would report `could not look` -# every day for a number that is available exactly and cheaply. -count_runs() { # $1 = workflow file, $2 = status - local n - conditional_get "repos/{owner}/{repo}/actions/workflows/$1/runs?created=%3E%3D$enc_since&status=$2&per_page=1" || return 1 - n=$(printf '%s' "$body" | jq -r '.total_count // empty' 2>/dev/null) || return 1 - [[ -n "$n" ]] || return 1 - printf '%s' "$n" -} - -# The per-JOB queue delay (CLOUD-501). A run's `created_at` -> `run_started_at` -# is the FIRST job's start; a matrix leg that waited behind it is invisible in -# it, and that is exactly the distinction the diagnosis needs — a wide matrix -# queueing on its own legs looks nothing like a saturated pool, and the per-run -# figure cannot tell them apart. Measured: queue p90 252s over the 13-landing -# window and 218s over the 10-landing one, against 0s over the 25-hour window -# whose idle stretch averaged it away. -# -# ONE REQUEST PER RUN, which is a real cost the header's ETag-stability argument -# cuts against — so it is bounded rather than waved at. Only GRADED runs in the -# window are read, which post-serialisation is ~1 per landing, and a run whose -# jobs cannot be read increments `unreadable` rather than being dropped. -# Dropping it would report a p90 over the legs that happened to answer, which is -# the partial-coverage false green this whole sensor exists to refuse. -# -# Pointer-only per non-negotiable 4: run id, job name, seconds. Never a log body. -job_queue() { # $1 = run id — emits `\t`, one line per job - conditional_get "repos/{owner}/{repo}/actions/runs/$1/jobs?per_page=100" || return 1 - printf '%s' "$body" | jq -r ' - .jobs[]? | select(.created_at != null and .started_at != null) - | [.created_at, .started_at, (.name // "-")] | @tsv' 2>/dev/null | - while IFS=$'\t' read -r created started name; do - c=$(date -u -d "$created" +%s 2>/dev/null) || continue - s=$(date -u -d "$started" +%s 2>/dev/null) || continue - printf '%s\t%s\n' "$((s - c))" "$name" - done -} - -ci_runs=$(walk_workflow ci.yml ' - .workflow_runs[]? - | [(.id|tostring), (.head_branch // "-"), (.conclusion // .status // "-"), - (.created_at // ""), (.run_started_at // ""), (.updated_at // ""), - (.run_attempt // 1 | tostring)] - | @tsv') || case $? in -1) bail "could not read the CI run window, so this window judged nothing." ;; -2) unreadable=$((unreadable + 1)) ;; -esac - -# --- fast-forward runs: the refusal:success ratio ------------------------------ -# -# The landing bot's verdict, and CLOUD-393's baseline metric (243 refusals to 5 -# merges in a half hour, before serialisation). `skipped` is not a refusal — it -# is every issue comment in the repository that was not `/fast-forward`. -ff_refused=$(count_runs fast-forward.yml failure) || { - ff_refused=0 - unreadable=$((unreadable + 1)) -} -ff_success=$(count_runs fast-forward.yml success) || { - ff_success=0 - unreadable=$((unreadable + 1)) -} - -# --- landings: merged PRs in the window --------------------------------------- -# -# A landing is a MERGED PR, not a commit on `main`: a landing pushes several -# commits at once, so commit timestamps cluster and counting them would inflate -# the denominator and flatter every ratio. -if ! conditional_get "repos/{owner}/{repo}/pulls?state=closed&sort=updated&direction=desc&per_page=100"; then - bail "could not read the merged-PR list, so there is no denominator and every ratio would be meaningless." -fi -landed=$(printf '%s' "$body" | jq -r --arg since "$SINCE" ' - .[]? | select(.merged_at != null and .merged_at >= $since) - | [(.number|tostring), (.head.ref // "-"), .merged_at] | @tsv' 2>/dev/null) || landed="" -landed=$(printf '%s' "$landed" | sed '/^[[:space:]]*$/d') -landings=$(printf '%s' "$landed" | grep -c . || true) - -# --- per-PR attribution ------------------------------------------------------- -# -# Runs are joined to a landing by branch AND bounded by `merged_at`, so a -# branch's post-merge runs — and a reused branch name's later life — are not -# attributed to it. A `skipped` run is a draft-era push: it spends no matrix and -# is not graded, so it is deliberately not counted. -records="" -graded=0 -green=0 -red=0 -cancelled=0 -while IFS=$'\t' read -r pr branch merged; do - [[ -n "$pr" ]] || continue - counts=$(awk -F'\t' -v b="$branch" -v m="$merged" ' - $2 == b && $4 <= m && $3 != "skipped" { - g++ - if ($3 == "success") s++ - else if ($3 == "failure") f++ - else if ($3 == "cancelled") c++ - } - END { printf "%d\t%d\t%d\t%d", g, s, f, c } - ' <<<"$ci_runs") - IFS=$'\t' read -r g s f c <<<"$counts" - graded=$((graded + g)) - green=$((green + s)) - red=$((red + f)) - cancelled=$((cancelled + c)) - # Only a PR that diverged earns a record. One graded green run is the ideal - # and says nothing a reader needs. - if [[ "$g" -gt 1 ]] || [[ "$f" -gt 0 ]]; then - records="${records}pr"$'\t'"number=$pr"$'\t'"branch=$branch"$'\t'"graded=$g"$'\t'"green=$s"$'\t'"red=$f"$'\t'"cancelled=$c"$'\n' - fi -done <<<"$landed" - -# --- cancel latency, peak concurrency, queue delay, retries ------------------- -# -# Latency, not count — see the header. A cancel at ~20s is the lease precondition -# working; one at 400s is a matrix billed for a verdict nobody reads. -cancel_lat=$(awk -F'\t' ' - $3 == "cancelled" && $5 != "" && $6 != "" { print } -' <<<"$ci_runs" | while IFS=$'\t' read -r id branch _conc _created started updated _attempt; do - s=$(date -u -d "$started" +%s 2>/dev/null) || continue - u=$(date -u -d "$updated" +%s 2>/dev/null) || continue - printf '%s\t%s\t%s\n' "$((u - s))" "$id" "$branch" -done | LC_ALL=C sort -n) - -while IFS=$'\t' read -r lat id branch; do - [[ -n "$lat" ]] || continue - records="${records}cancel"$'\t'"run=$id"$'\t'"branch=$branch"$'\t'"latency=$lat"$'\n' -done <<<"$cancel_lat" - -pct() { # stream of sorted numbers on stdin, percentile as $1 - awk -v p="$1" '{ v[n++] = $1 } END { if (n == 0) { print 0; exit } i = int((n - 1) * p / 100); print v[i] }' -} -cancel_p50=$(cut -f1 <<<"$cancel_lat" | sed '/^$/d' | pct 50) - -# Queue delay: created -> run_started. Zero on a healthy runner pool; a non-zero -# p90 is the fleet waiting for capacity, which is a different defect from any -# amount of contention and must not be read as one. -queue_p90=$(awk -F'\t' '$4 != "" && $5 != "" && $3 != "skipped" { print $4 "\t" $5 }' <<<"$ci_runs" | - while IFS=$'\t' read -r created started; do - c=$(date -u -d "$created" +%s 2>/dev/null) || continue - s=$(date -u -d "$started" +%s 2>/dev/null) || continue - printf '%s\n' "$((s - c))" - done | LC_ALL=C sort -n | pct 90) - -# The same delay attributed PER JOB (CLOUD-501). Read only for graded runs in -# the window — the bound `job_queue`'s header states — and a run whose jobs are -# unreadable is counted as such rather than silently contributing nothing. -job_lat="" -while IFS=$'\t' read -r id _branch; do - [[ -n "$id" ]] || continue - if ! out=$(job_queue "$id"); then - unreadable=$((unreadable + 1)) - continue - fi - while IFS=$'\t' read -r secs name; do - [[ -n "$secs" ]] || continue - job_lat="${job_lat}${secs}"$'\t'"$id"$'\t'"$name"$'\n' - done <<<"$out" -done <<<"$(awk -F'\t' '$3 != "skipped" && $3 != "-" { print $1 "\t" $2 }' <<<"$ci_runs")" - -job_lat=$(printf '%s' "$job_lat" | sed '/^[[:space:]]*$/d') -# `pct` takes an ALREADY-SORTED stream, so this needs its own sort rather than -# inheriting one from the walk above. -queue_job_p90=$(cut -f1 <<<"$job_lat" | sed '/^$/d' | LC_ALL=C sort -n | pct 90) - -# A `job` record only where the leg actually waited: a zero-wait leg is the ideal -# and says nothing a reader needs, which is the rule the `pr` records already use. -while IFS=$'\t' read -r secs id name; do - [[ -n "$secs" ]] || continue - [[ "$secs" -gt 0 ]] || continue - records="${records}job"$'\t'"run=$id"$'\t'"job=$name"$'\t'"queue=$secs"$'\n' -done <<<"$job_lat" - -# Peak concurrency: a sweep over start/end events. This is the metric that caught -# serialisation working — 25 concurrent matrices before the lease, 3 after. -peak=$(awk -F'\t' '$3 != "skipped" && $5 != "" && $6 != "" { print $5 "\t" $6 }' <<<"$ci_runs" | - while IFS=$'\t' read -r started updated; do - s=$(date -u -d "$started" +%s 2>/dev/null) || continue - u=$(date -u -d "$updated" +%s 2>/dev/null) || continue - printf '%s\t1\n%s\t-1\n' "$s" "$u" - done | LC_ALL=C sort -n -k1,1 -k2,2r | - awk -F'\t' '{ cur += $2; if (cur > max) max = cur } END { print max + 0 }') - -retries=$(awk -F'\t' '$7 != "" && $7 + 0 > 1' <<<"$ci_runs" | grep -c . || true) - -# Records first, sorted, then the summary — byte-stable, and the decider can read -# the summary without buffering the whole stream. -if [[ -n "$records" ]]; then - printf '%s' "$records" | LC_ALL=C sort -fi -summary "$landings" "$graded" "$green" "$red" "$cancelled" \ - "${cancel_p50:-0}" "${peak:-0}" "${queue_p90:-0}" "${queue_job_p90:-0}" \ - "$retries" "$ff_refused" "$ff_success" diff --git a/mise.toml b/mise.toml index 464cccde3..bd378a2ad 100644 --- a/mise.toml +++ b/mise.toml @@ -617,7 +617,7 @@ CI_FANIN_WORKFLOW = ".github/workflows/ci.yml" BATS_TEST_TIMEOUT = "300" REGORUS_OPA_COMPLIANCE = "1.2.0" REGORUS_OPA_COMPLIANCE_FOR = "0.11" -MUTANT_GATES = "mise,attestation,timeout-drift,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure-check,evaluator-io-check,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence-assert,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link-check,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus-check,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" +MUTANT_GATES = "mise,attestation,timeout-drift,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure-check,evaluator-io-check,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link-check,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus-check,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" # --- GitHub reachability behind an egress proxy (Claude Code web sandbox etc.) --- # mise resolves every tool's release through GitHub's *API* host, api.github.com. @@ -2400,6 +2400,398 @@ printf '%s\n' "$out" printf '%s\n' "$out" | cargo run --quiet -p batten -- record named nonverdict ''' +[tasks.land-divergence-record] +description = "Effect: record how far the landing loop diverged from linear over a window, one line per divergent PR plus a window summary (CLOUD-492)" +# THE MEASURING HALF OF A SPLIT THAT ALREADY EXISTED (CLOUD-1717). This carries +# `mise-tasks/land-divergence.sh`'s body; `policy/land-divergence.rego` carries +# `mise-tasks/land-divergence-assert.sh`'s decision. The pair was already apart for +# CLOUD-1559's reason — a measurement needs the network and a token, a decision +# needs neither — so the port moved two halves into their homes rather than +# splitting anything new. House style §5 makes `check` `read` and structurally +# incapable of spawning the `gh` reads below, and `Fact::Instant` projects `null` +# to every module, so every instant subtraction here is unmovable too. +# +# ## A CANCELLED RUN IS NOT WASTE, AND COUNTING THEM INVERTS THE VERDICT +# +# Measured 2026-08-12 after serialisation: 5 green CI runs against 5 cancelled, +# which reads as a 50% discard rate and is the opposite. Those cancels had p50 +# lifetime ~20s — `ci-lease-precondition` killing an unauthorised matrix for ~20 +# runner-seconds instead of billing ~500. Before serialisation the same population +# had p75 147s and max 413s; THOSE were waste. So the graded quantity is CANCEL +# LATENCY, never cancel count. +# +# ## Per workflow, never the global run list +# +# `actions/runs` carries ~3100 skipped auto-land runs a day, and at that insertion +# rate page boundaries shift mid-walk, so a paginated read of it is not stable. +# Per-workflow with a `created>=` filter is ~3-4 pages each and stable. +# +# ## NO WATERMARK, AND THAT IS THE DESIGN +# +# The obvious shape is a persisted ledger advanced by a watermark. It cannot work +# here: a scheduled tick gets a fresh checkout, so nothing survives between them, +# and a committed ledger pollutes the tree against non-negotiable rule 7. The +# Actions API is already the durable store (90-day retention), so every tick is a +# PURE FUNCTION OF ITS WINDOW — which also makes this re-runnable over any past +# window for free, and that is what a divergence investigation needs. The ETag +# cache is a different thing: a within-clone read optimisation, discarded with the +# clone. +# +# OUTPUT IS A POINTER, NEVER A PAYLOAD (rule 4): run ids, branch names, +# conclusions, latencies and counts. No log body, no commit message, no PR title is +# ever fetched, so none can leak. TAB-separated, because branch names are +# unconstrained. +shell = "bash -c" +run = ''' +set -euo pipefail + +cd "$(git rev-parse --show-toplevel)" + +# The window, as an ISO-8601 instant. A DATE RANGE rather than a run count, which +# is the opposite of `nonverdict-scan`'s choice and deliberately so: that sensor +# asks "how often does this failure mode occur", which a fixed sample answers, +# while this one asks "what did the loop cost over a period", which only a period +# answers. A count would silently narrow the window whenever the fleet got busy — +# exactly when the answer matters most. +SINCE="${BATTEN_DIVERGENCE_SINCE:-$(date -u -d '24 hours ago' +%Y-%m-%dT%H:%M:%SZ)}" + +# The cache root. Under `.git/`, so it is out of tree, never committed, and +# discarded with the clone rather than surviving as stale state on a fresh one. +cache="${BATTEN_DIVERGENCE_CACHE:-$(git rev-parse --git-dir)/batten-divergence}" +mkdir -p "$cache" + +unreadable=0 + +# `gh api -i`, then the status line and the ETag out of the headers, then the +# body after the first blank line — `main-watch`'s form, as `nonverdict-scan` +# carries it. A non-2xx makes `gh` exit non-zero (a 304 included), so the call is +# guarded rather than trusted: under `set -e` an unguarded 304 would abort on the +# cheapest possible response. +# +# Sets `body` and returns 0 on a usable reading, 1 when it could not look. +conditional_get() { + local url="$1" key etag_file body_file resp status new_etag + key=$(printf '%s' "$url" | sha1sum | cut -d' ' -f1) + etag_file="$cache/$key.etag" + body_file="$cache/$key.body" + + local args=(-i "$url") + if [[ -s "$etag_file" ]]; then + args+=(-H "If-None-Match: $(cat "$etag_file")") + fi + + resp=$(gh api "${args[@]}" 2>/dev/null | tr -d '\r') || true + status=$(printf '%s' "$resp" | sed -n '1s@^HTTP/[0-9.]* \([0-9]*\).*@\1@p') + + # Reassigned only when non-empty: a response carrying no ETag must not clear + # the stored one, or the next tick is a cold read for no reason. `if`, not a + # `&&` list — under `set -e` a failing trailing `&&` aborts the whole scan, + # which `nonverdict-scan` measured as a silent empty window. + new_etag=$(printf '%s' "$resp" | sed -n 's/^[Ee][Tt]ag: //p' | head -n1) + if [[ -n "$new_etag" ]]; then + printf '%s' "$new_etag" >"$etag_file" + fi + + if [[ "$status" = "304" ]]; then + # The whole point of the cache. No body arrives, so the previous reading + # IS the answer; without one there is nothing to fall back to, and an + # empty parse would read as a clean window — the false green this sensor + # exists to report. + [[ -s "$body_file" ]] || return 1 + body=$(cat "$body_file") + return 0 + fi + + case "$status" in + 2*) ;; + *) return 1 ;; + esac + + body=$(printf '%s' "$resp" | awk 'body {print} /^$/ {body=1}') + [[ -n "${body//[[:space:]]/}" ]] || return 1 + printf '%s' "$body" >"$body_file" + return 0 +} + +summary() { + printf 'window\tsince=%s\tlandings=%s\tgraded=%s\tgreen=%s\tred=%s\tcancelled=%s\tcancel_p50=%s\tpeak_concurrency=%s\tqueue_p90=%s\tqueue_job_p90=%s\tretries=%s\tff_refused=%s\tff_success=%s\tunreadable=%s\n' \ + "$SINCE" "$1" "$2" "$3" "$4" "$5" "$6" "$7" "$8" "$9" "${10}" "${11}" "${12}" "$unreadable" +} + +# COULD-NOT-LOOK REFUSES HERE AND RECORDS NOTHING (CLOUD-1717). The retired +# measurer answered a total read failure with an otherwise-empty window carrying +# `unreadable=1`, which the decider turned into its exit 2. On the engine's +# contract exit 2 is a FINDING, so carrying that spelling over would have made +# blindness a violation. This fails loudly while its author is watching and writes +# no family at all; an absent family is the module's silence. `unreadable` still +# counts the PARTIAL reads below, which are a finding rather than blindness. +bail() { + echo "::error:: land-divergence-record: $1 Nothing was recorded." >&2 + exit 1 +} + +# `created>=` must be URL-encoded: `>` is not legal in a query string, and gh +# passes the path through verbatim. +enc_since=$(printf '%s' "$SINCE" | sed 's/:/%3A/g') + +# --- CI runs over the window -------------------------------------------------- +# +# `ci.yml` is the matrix a landing buys. `?created=>=` plus a per-page cap, and +# the walk stops at the first page shorter than the cap — the `link` header is +# the correct general answer, but a per-workflow day window is a few pages and +# this keeps the request parameters identical between ticks, which is what keeps +# the 304 hit rate up (`mem:github-rest-etiquette`). +# TRUNCATION IS DETECTED BY `total_count`, NOT BY RUNNING OUT OF PAGES, and that +# distinction was measured rather than reasoned. +# +# The Actions runs endpoint HARD-CAPS pagination at 1000 items while still +# reporting the true `total_count`. Measured on `fast-forward.yml` over a 25-hour +# window: `total_count` 1446, page 10 full, page 11 EMPTY. So a walk that stops +# when a page comes back short — the obvious loop, and the one written first here +# — collects 1000 of 1446 and looks exactly like a clean finish. It reported +# `ff_refused=0` over a window carrying 598 refusals, which is a perfect score +# read off a prefix: the precise false green this sensor exists to catch. +# +# `total_count` is therefore the only honest stopping condition. Compare it +# against what was actually collected and report the gap; a wrong number reported +# confidently is worse than no number, and this is the shape that produces one. +MAX_PAGES="${BATTEN_DIVERGENCE_MAX_PAGES:-10}" + +walk_workflow() { # $1 = workflow file, $2 = jq expression emitting ONE line per run + local wf="$1" expr="$2" page=1 got total="" out="" have + while [[ "$page" -le "$MAX_PAGES" ]]; do + if ! conditional_get "repos/{owner}/{repo}/actions/workflows/$wf/runs?created=%3E%3D$enc_since&per_page=100&page=$page"; then + return 1 + fi + if [[ -z "$total" ]]; then + total=$(printf '%s' "$body" | jq -r '.total_count // 0' 2>/dev/null) || total=0 + fi + got=$(printf '%s' "$body" | jq -r '.workflow_runs | length' 2>/dev/null) || got=0 + out="$out$(printf '%s' "$body" | jq -r "$expr" 2>/dev/null) +" + [[ "$got" = "100" ]] || break + page=$((page + 1)) + done + + out=$(printf '%s' "$out" | sed '/^[[:space:]]*$/d') + printf '%s' "$out" + have=$(printf '%s' "$out" | grep -c . || true) + if [[ "$have" -lt "${total:-0}" ]]; then + echo "::error:: land-divergence: read $have of $total $wf run(s) since $SINCE — the API caps this endpoint at 1000 items, so these counts describe a prefix of the window. Narrow BATTEN_DIVERGENCE_SINCE." >&2 + return 2 + fi + return 0 +} + +# An exact count for one conclusion, from `total_count` with a status filter — +# one request, no pagination, and therefore no cap to be truncated by. Used +# wherever only a COUNT is needed, which is why the fast-forward ratio does not +# go through `walk_workflow` at all: at this repo's comment volume that window is +# routinely over the 1000-item ceiling, so a walk would report `could not look` +# every day for a number that is available exactly and cheaply. +count_runs() { # $1 = workflow file, $2 = status + local n + conditional_get "repos/{owner}/{repo}/actions/workflows/$1/runs?created=%3E%3D$enc_since&status=$2&per_page=1" || return 1 + n=$(printf '%s' "$body" | jq -r '.total_count // empty' 2>/dev/null) || return 1 + [[ -n "$n" ]] || return 1 + printf '%s' "$n" +} + +# The per-JOB queue delay (CLOUD-501). A run's `created_at` -> `run_started_at` +# is the FIRST job's start; a matrix leg that waited behind it is invisible in +# it, and that is exactly the distinction the diagnosis needs — a wide matrix +# queueing on its own legs looks nothing like a saturated pool, and the per-run +# figure cannot tell them apart. Measured: queue p90 252s over the 13-landing +# window and 218s over the 10-landing one, against 0s over the 25-hour window +# whose idle stretch averaged it away. +# +# ONE REQUEST PER RUN, which is a real cost the header's ETag-stability argument +# cuts against — so it is bounded rather than waved at. Only GRADED runs in the +# window are read, which post-serialisation is ~1 per landing, and a run whose +# jobs cannot be read increments `unreadable` rather than being dropped. +# Dropping it would report a p90 over the legs that happened to answer, which is +# the partial-coverage false green this whole sensor exists to refuse. +# +# Pointer-only per non-negotiable 4: run id, job name, seconds. Never a log body. +job_queue() { # $1 = run id — emits `\t`, one line per job + conditional_get "repos/{owner}/{repo}/actions/runs/$1/jobs?per_page=100" || return 1 + printf '%s' "$body" | jq -r ' + .jobs[]? | select(.created_at != null and .started_at != null) + | [.created_at, .started_at, (.name // "-")] | @tsv' 2>/dev/null | + while IFS=$'\t' read -r created started name; do + c=$(date -u -d "$created" +%s 2>/dev/null) || continue + s=$(date -u -d "$started" +%s 2>/dev/null) || continue + printf '%s\t%s\n' "$((s - c))" "$name" + done +} + +ci_runs=$(walk_workflow ci.yml ' + .workflow_runs[]? + | [(.id|tostring), (.head_branch // "-"), (.conclusion // .status // "-"), + (.created_at // ""), (.run_started_at // ""), (.updated_at // ""), + (.run_attempt // 1 | tostring)] + | @tsv') || case $? in +1) bail "could not read the CI run window, so this window judged nothing." ;; +2) unreadable=$((unreadable + 1)) ;; +esac + +# --- fast-forward runs: the refusal:success ratio ------------------------------ +# +# The landing bot's verdict, and CLOUD-393's baseline metric (243 refusals to 5 +# merges in a half hour, before serialisation). `skipped` is not a refusal — it +# is every issue comment in the repository that was not `/fast-forward`. +ff_refused=$(count_runs fast-forward.yml failure) || { + ff_refused=0 + unreadable=$((unreadable + 1)) +} +ff_success=$(count_runs fast-forward.yml success) || { + ff_success=0 + unreadable=$((unreadable + 1)) +} + +# --- landings: merged PRs in the window --------------------------------------- +# +# A landing is a MERGED PR, not a commit on `main`: a landing pushes several +# commits at once, so commit timestamps cluster and counting them would inflate +# the denominator and flatter every ratio. +if ! conditional_get "repos/{owner}/{repo}/pulls?state=closed&sort=updated&direction=desc&per_page=100"; then + bail "could not read the merged-PR list, so there is no denominator and every ratio would be meaningless." +fi +landed=$(printf '%s' "$body" | jq -r --arg since "$SINCE" ' + .[]? | select(.merged_at != null and .merged_at >= $since) + | [(.number|tostring), (.head.ref // "-"), .merged_at] | @tsv' 2>/dev/null) || landed="" +landed=$(printf '%s' "$landed" | sed '/^[[:space:]]*$/d') +landings=$(printf '%s' "$landed" | grep -c . || true) + +# --- per-PR attribution ------------------------------------------------------- +# +# Runs are joined to a landing by branch AND bounded by `merged_at`, so a +# branch's post-merge runs — and a reused branch name's later life — are not +# attributed to it. A `skipped` run is a draft-era push: it spends no matrix and +# is not graded, so it is deliberately not counted. +records="" +graded=0 +green=0 +red=0 +cancelled=0 +while IFS=$'\t' read -r pr branch merged; do + [[ -n "$pr" ]] || continue + counts=$(awk -F'\t' -v b="$branch" -v m="$merged" ' + $2 == b && $4 <= m && $3 != "skipped" { + g++ + if ($3 == "success") s++ + else if ($3 == "failure") f++ + else if ($3 == "cancelled") c++ + } + END { printf "%d\t%d\t%d\t%d", g, s, f, c } + ' <<<"$ci_runs") + IFS=$'\t' read -r g s f c <<<"$counts" + graded=$((graded + g)) + green=$((green + s)) + red=$((red + f)) + cancelled=$((cancelled + c)) + # Only a PR that diverged earns a record. One graded green run is the ideal + # and says nothing a reader needs. + if [[ "$g" -gt 1 ]] || [[ "$f" -gt 0 ]]; then + records="${records}pr"$'\t'"number=$pr"$'\t'"branch=$branch"$'\t'"graded=$g"$'\t'"green=$s"$'\t'"red=$f"$'\t'"cancelled=$c"$'\n' + fi +done <<<"$landed" + +# --- cancel latency, peak concurrency, queue delay, retries ------------------- +# +# Latency, not count — see the header. A cancel at ~20s is the lease precondition +# working; one at 400s is a matrix billed for a verdict nobody reads. +cancel_lat=$(awk -F'\t' ' + $3 == "cancelled" && $5 != "" && $6 != "" { print } +' <<<"$ci_runs" | while IFS=$'\t' read -r id branch _conc _created started updated _attempt; do + s=$(date -u -d "$started" +%s 2>/dev/null) || continue + u=$(date -u -d "$updated" +%s 2>/dev/null) || continue + printf '%s\t%s\t%s\n' "$((u - s))" "$id" "$branch" +done | LC_ALL=C sort -n) + +while IFS=$'\t' read -r lat id branch; do + [[ -n "$lat" ]] || continue + records="${records}cancel"$'\t'"run=$id"$'\t'"branch=$branch"$'\t'"latency=$lat"$'\n' +done <<<"$cancel_lat" + +pct() { # stream of sorted numbers on stdin, percentile as $1 + awk -v p="$1" '{ v[n++] = $1 } END { if (n == 0) { print 0; exit } i = int((n - 1) * p / 100); print v[i] }' +} +cancel_p50=$(cut -f1 <<<"$cancel_lat" | sed '/^$/d' | pct 50) + +# Queue delay: created -> run_started. Zero on a healthy runner pool; a non-zero +# p90 is the fleet waiting for capacity, which is a different defect from any +# amount of contention and must not be read as one. +queue_p90=$(awk -F'\t' '$4 != "" && $5 != "" && $3 != "skipped" { print $4 "\t" $5 }' <<<"$ci_runs" | + while IFS=$'\t' read -r created started; do + c=$(date -u -d "$created" +%s 2>/dev/null) || continue + s=$(date -u -d "$started" +%s 2>/dev/null) || continue + printf '%s\n' "$((s - c))" + done | LC_ALL=C sort -n | pct 90) + +# The same delay attributed PER JOB (CLOUD-501). Read only for graded runs in +# the window — the bound `job_queue`'s header states — and a run whose jobs are +# unreadable is counted as such rather than silently contributing nothing. +job_lat="" +while IFS=$'\t' read -r id _branch; do + [[ -n "$id" ]] || continue + if ! out=$(job_queue "$id"); then + unreadable=$((unreadable + 1)) + continue + fi + while IFS=$'\t' read -r secs name; do + [[ -n "$secs" ]] || continue + job_lat="${job_lat}${secs}"$'\t'"$id"$'\t'"$name"$'\n' + done <<<"$out" +done <<<"$(awk -F'\t' '$3 != "skipped" && $3 != "-" { print $1 "\t" $2 }' <<<"$ci_runs")" + +job_lat=$(printf '%s' "$job_lat" | sed '/^[[:space:]]*$/d') +# `pct` takes an ALREADY-SORTED stream, so this needs its own sort rather than +# inheriting one from the walk above. +queue_job_p90=$(cut -f1 <<<"$job_lat" | sed '/^$/d' | LC_ALL=C sort -n | pct 90) + +# A `job` record only where the leg actually waited: a zero-wait leg is the ideal +# and says nothing a reader needs, which is the rule the `pr` records already use. +while IFS=$'\t' read -r secs id name; do + [[ -n "$secs" ]] || continue + [[ "$secs" -gt 0 ]] || continue + records="${records}job"$'\t'"run=$id"$'\t'"job=$name"$'\t'"queue=$secs"$'\n' +done <<<"$job_lat" + +# Peak concurrency: a sweep over start/end events. This is the metric that caught +# serialisation working — 25 concurrent matrices before the lease, 3 after. +peak=$(awk -F'\t' '$3 != "skipped" && $5 != "" && $6 != "" { print $5 "\t" $6 }' <<<"$ci_runs" | + while IFS=$'\t' read -r started updated; do + s=$(date -u -d "$started" +%s 2>/dev/null) || continue + u=$(date -u -d "$updated" +%s 2>/dev/null) || continue + printf '%s\t1\n%s\t-1\n' "$s" "$u" + done | LC_ALL=C sort -n -k1,1 -k2,2r | + awk -F'\t' '{ cur += $2; if (cur > max) max = cur } END { print max + 0 }') + +retries=$(awk -F'\t' '$7 != "" && $7 + 0 > 1' <<<"$ci_runs" | grep -c . || true) + +# Records first, sorted, then the summary — byte-stable, and the module can read +# the summary without depending on where it sits. +emit() { + if [[ -n "$records" ]]; then + printf '%s' "$records" | LC_ALL=C sort + fi + summary "$landings" "$graded" "$green" "$red" "$cancelled" \ + "${cancel_p50:-0}" "${peak:-0}" "${queue_p90:-0}" "${queue_job_p90:-0}" \ + "$retries" "$ff_refused" "$ff_success" +} + +# BOTH, AND THE ORDER MATTERS. The window goes into the family +# `policy/land-divergence.rego` reads, and the same bytes go to stdout, where +# `land-divergence.yml` publishes them to the step summary — the numbers are the +# point, and a summary that only appears on green is a report nobody reads at the +# moment it matters. Rendered once into a variable rather than measured twice, so +# the published records and the recorded ones cannot differ. +out=$(emit) +printf '%s\n' "$out" +printf '%s\n' "$out" | cargo run --quiet -p batten -- record named land-divergence +''' + [tasks.record-verdicts] description = "Effect: run each declared third-party validator OUTSIDE the engine and record its verdict where `batten check` reads it (CLOUD-1265)" # THE HALF THAT WAS MISSING. `crates/batten/src/tools.rs` reads diff --git a/policy/land-divergence.rego b/policy/land-divergence.rego new file mode 100644 index 000000000..49401f5ac --- /dev/null +++ b/policy/land-divergence.rego @@ -0,0 +1,334 @@ +# How far the landing loop diverged from linear over a window (CLOUD-492, ported +# under CLOUD-1717). +# +# THE CLAIM BEING GRADED. The loop is serialised behind a lease (CLOUD-393) and an +# unauthorised matrix is stopped server-side (CLOUD-420). Together those claim a PR +# buys ONE CI matrix, runs it to green, and lands. Divergence from that is the whole +# signal, and it was established once by hand — ~50 paginated calls and throwaway +# jq — which is why nothing could see a step change BETWEEN two assessments. +# +# THE SPLIT WAS ALREADY THERE AND THE PORT ONLY MOVED ITS HALVES, exactly as the +# `nonverdict` pair's was: `land-divergence` measured and `land-divergence-assert` +# decided, kept apart because a measurement needs the network and a token and a +# decision needs neither (CLOUD-1559). The measurement is +# `[tasks.land-divergence-record]`; this file is the decision, and no decision +# changed hands. The pagination walk, the ETag cache, the `total_count` truncation +# guard and every instant subtraction stay outside: house style §5 makes `check` +# `read` and incapable of spawning, and `Fact::Instant` projects `null` to every +# module, which `clippy.toml` and `crates/batten/tests/clock_ban.rs` hold. +# +# A CANCELLED RUN IS NOT WASTE, AND COUNTING THEM INVERTS THE VERDICT. This is the +# finding the pair exists to encode. Measured 2026-08-12 after serialisation: 5 +# green CI runs against 5 cancelled, which reads as a 50% discard rate and is the +# opposite — those cancels had p50 lifetime ~20s, `ci-lease-precondition` killing an +# unauthorised matrix for ~20 runner-seconds instead of billing ~500. Before +# serialisation the same population had p75 147s. So the graded quantity is CANCEL +# LATENCY, never cancel count: a rule counting cancels would score the working +# precondition as a defect and argue for removing it. +# +# COULD-NOT-LOOK IS AN ABSENT RECORD, EXCEPT WHERE IT IS PARTIAL. The retired +# decider ran `0/1/2` with `2` for could-not-look, and the engine's `2` is a +# FINDING. Both total-blindness arms — an unreadable CI run window, an unreadable +# merged-PR list — are now the producer refusing at write time and recording +# nothing, which reads here as silence. `unreadable` is the third case and is not +# blindness: the producer read PART of its window and judged less than it claims, +# which is `bench-assert`'s partial-coverage false green and its own finding. +# +# THE TRUNCATION THAT MAKES THAT ARM LOAD-BEARING, measured rather than reasoned: +# the Actions runs endpoint hard-caps pagination at 1000 items while still reporting +# the true `total_count`. Over a 25-hour window on `fast-forward.yml` it reported +# 1446 with page 10 full and page 11 empty, so a walk that stops on a short page +# collects 1000 of 1446 and looks like a clean finish — it reported `ff_refused=0` +# over a window carrying 598 refusals. A perfect score read off a prefix. +#MUTANT-SUITE crates/batten/tests/it/land_divergence.rs +#MUTANT partial-window-passes|s@^\tcount_of("unreadable") > 0$@\tfalse@|a_partially_read_window_is_a_finding_rather_than_a_clean_one +#MUTANT graded-over-budget-passes|s@^\tratio("graded") > graded_budget$@\tfalse@|a_loop_buying_more_than_one_matrix_per_landing_is_reported +#MUTANT refusal-budget-loosened|s@^\tcount_of("ff_refused") > 0$@\tfalse@|any_fast_forward_refusal_at_all_is_reported + +# METADATA +# description: | +# Bound to the TREE surface: this row is `scope = "tree"`, so it reads +# `input.tree` and never the mediated call. +# THIS BLOCK IS YAML AND MUST STAY THE LAST COMMENT BLOCK BEFORE `package`. +# schemas: +# - input: schema["policy-input.schema"] +package batten.land_divergence + +import rego.v1 + +rules contains "lane read partial" + +rules contains "lane count spent" + +rules contains "lane grade red" + +rules contains "lane reach late" + +rules contains "lease guard dropped" + +rules contains "lane measure late" + +rules contains "job measure late" + +rules contains "branch reach stale" + +# THE BUDGETS, SEEDED FROM THE MEASUREMENT rather than from an aspiration — taken +# 2026-08-12, after landing serialisation, so the gate starts at the observed state. +# +# IN THE MODULE RATHER THAN BEHIND SEVEN ENV OVERRIDES, which is `timeout-drift`'s +# and `nonverdict`'s placement and is what the port made possible: the knobs existed +# so the retired suite could point a budget at a fixture, and a module's cases vary +# the RECORD instead. Seven fewer knobs nobody was turning. + +# Ratios are held in HUNDREDTHS, carried over from the decider because the numbers +# a reader has seen are in these units. The ideal is 1.00 graded runs per landing; +# the budget is 2.00, because the second run is the ~20s lease-precondition +# cancellation the same measurement showed is the mechanism WORKING. +graded_budget := 200 + +red_budget := 20 + +# Seconds. An early cancellation is the lease precondition working; a late one is a +# matrix billed for a verdict nobody reads. +cancel_budget := 60 + +# Landing is serialised behind a lease, so concurrency above the admitted-successor +# bound means something is spending CI without holding it. +concurrency_budget := 3 + +# Seconds, and the same figure for both: the ideal is that a run — and each of its +# legs — starts when it is created. +queue_budget := 30 + +# The producer's lines, or nothing. `recorded` being undefined is the +# could-not-look the header describes, and every rule below inherits it. +recorded := input.tree.records["land-divergence"] + +# EXACTLY ONE SUMMARY, OR NOTHING IS JUDGED. Two DIFFERENT summaries mean two +# measurements were concatenated and a count over both describes neither — the +# retired decider's own arm, kept, and reachable here only through a torn store +# because `record named` replaces a family rather than appending to it. +summaries contains raw if { + some raw in recorded + startswith(raw, "window\t") +} + +fields[pair[0]] := pair[1] if { + count(summaries) == 1 + some raw in summaries + columns := split(raw, "\t") + some idx in numbers.range(1, count(columns) - 1) + pair := split(columns[idx], "=") + count(pair) == 2 +} + +# A COUNT THAT IS NOT A NUMBER IS NOT A ZERO. The retired decider refused rather +# than coercing, because a count silently read as zero is a clean window over input +# nobody parsed. Here that refusal is undefinedness, which leaves the rule unfired. +count_of(key) := number if { + raw := fields[key] + regex.match(data.batten.patterns["whole-number"], raw) + number := to_number(raw) +} + +# Per-landing, in hundredths. +# +# TRUE DIVISION WHERE THE SHELL TRUNCATED, which is a fidelity improvement rather +# than a drift: bash has no floats, so the decider's `$((x * 100 / n))` discarded +# the remainder and could report a ratio marginally under the budget that was over +# it. Rego divides exactly, so the comparison now agrees with the number a reader +# would compute by hand — which is the property the decider's own comment asked for +# ("a gate that rounds is a gate that disagrees with the number it printed"). +ratio(key) := value if { + landings := count_of("landings") + landings > 0 + value := (count_of(key) * 100) / landings +} + +# A window the producer could not read all of. ITS OWN FINDING RATHER THAN A +# DEGRADED PASS, and the truncation in the header is why it is load-bearing. +violation contains { + "rule": "lane read partial", + "verdict": "lane read partial", + "subjects": [{"count": count_of("unreadable")}], +} if { + count_of("unreadable") > 0 +} + +# ANTI-VACUITY IS `ratio`'s GUARD, not an arm of its own: with no landings the +# division is undefined and every ratio rule below is silent, which is the honest +# reading of a quiet day — nothing landed, so nothing diverged. This repo has been +# bitten twice by a gate that cannot fire reading the same as one that found nothing +# (`finding-sink-check`, `bench-assert`), and the record's PRESENCE is what keeps +# the two apart. +violation contains { + "rule": "lane count spent", + "verdict": "lane count spent", + "subjects": [{"count": count_of("graded")}, {"count": count_of("landings")}], +} if { + ratio("graded") > graded_budget +} + +# A red run means `verify` was skipped or disagreed with CI; each one spent a full +# matrix. +violation contains { + "rule": "lane grade red", + "verdict": "lane grade red", + "subjects": [{"count": count_of("red")}, {"count": count_of("landings")}], +} if { + ratio("red") > red_budget +} + +# LATENCY, NEVER COUNT — the header's finding, as a rule. +violation contains { + "rule": "lane reach late", + "verdict": "lane reach late", + "subjects": [{"count": count_of("cancel_p50")}], +} if { + count_of("cancel_p50") > cancel_budget +} + +violation contains { + "rule": "lease guard dropped", + "verdict": "lease guard dropped", + "subjects": [{"count": count_of("peak_concurrency")}], +} if { + count_of("peak_concurrency") > concurrency_budget +} + +# The runner pool saturating, which is a DIFFERENT defect from contention and must +# not be read as one. +violation contains { + "rule": "lane measure late", + "verdict": "lane measure late", + "subjects": [{"count": count_of("queue_p90")}], +} if { + count_of("queue_p90") > queue_budget +} + +# THE SAME WAIT ATTRIBUTED PER JOB (CLOUD-501), and it gets its own rule rather than +# replacing the one above. A run's figure is its FIRST job's start, so a matrix leg +# queueing behind its siblings is invisible in it; the two disagreeing is exactly +# what separates a wide matrix from a saturated pool. +violation contains { + "rule": "job measure late", + "verdict": "job measure late", + "subjects": [{"count": count_of("queue_job_p90")}], +} if { + count_of("queue_job_p90") > queue_budget +} + +# THE ONE METRIC THAT IS NOT A THRESHOLD. A fast-forward refusal means the branch +# went behind before the bot answered — the thundering herd the landing lease +# removed (243:5 before, 0:5 after). Any refusal at all is a divergence, so the +# budget is zero and is written as a literal rather than as a tunable, because +# stating it as one would invite raising it. +violation contains { + "rule": "branch reach stale", + "verdict": "branch reach stale", + "subjects": [{"count": count_of("ff_refused")}], +} if { + count_of("ff_refused") > 0 +} + +# --- cases --------------------------------------------------------------- + +tree(lines) := {"tree": {"records": {"land-divergence": lines}}} + +# A clean window: one landing, one graded green run, nothing waiting. +clean := { + "landings": 1, "graded": 1, "green": 1, "red": 0, "cancelled": 0, + "cancel_p50": 0, "peak_concurrency": 1, "queue_p90": 0, + "queue_job_p90": 0, "retries": 0, "ff_refused": 0, "ff_success": 1, + "unreadable": 0, +} + +window(over) := tree([sprintf( + "window\tsince=2026-08-12T00:00:00Z\tlandings=%d\tgraded=%d\tgreen=%d\tred=%d\tcancelled=%d\tcancel_p50=%d\tpeak_concurrency=%d\tqueue_p90=%d\tqueue_job_p90=%d\tretries=%d\tff_refused=%d\tff_success=%d\tunreadable=%d", + [f.landings, f.graded, f.green, f.red, f.cancelled, f.cancel_p50, f.peak_concurrency, f.queue_p90, f.queue_job_p90, f.retries, f.ff_refused, f.ff_success, f.unreadable], +)]) if { + f := object.union(clean, over) +} + +fires(over, id) if { + some v in violation with input as window(over) + v.verdict == id +} + +test_the_ideal_window_is_clean if { + count(violation) == 0 with input as window({}) +} + +# One matrix, run to green, landed — plus the ~20s lease cancellation, which is the +# mechanism WORKING and must sit inside the budget rather than against it. +test_a_lease_cancellation_is_inside_the_budget if { + count(violation) == 0 with input as window({"graded": 2, "cancelled": 1, "cancel_p50": 20, "peak_concurrency": 2}) +} + +test_a_loop_buying_three_matrices_per_landing_is_reported if { + fires({"graded": 3}, "lane count spent") +} + +test_a_red_run_over_budget_is_reported if { + fires({"graded": 2, "red": 1}, "lane grade red") +} + +# A LATE CANCEL IS THE FINDING, NOT A CANCEL. This is the case that stops a future +# author "simplifying" the rule into a count. +test_a_late_cancellation_is_reported_where_an_early_one_is_not if { + fires({"graded": 2, "cancelled": 1, "cancel_p50": 400, "peak_concurrency": 2}, "lane reach late") + count(violation) == 0 with input as window({"graded": 2, "cancelled": 1, "cancel_p50": 20, "peak_concurrency": 2}) +} + +test_concurrency_above_the_lease_bound_is_reported if { + fires({"peak_concurrency": 25}, "lease guard dropped") +} + +test_a_saturated_runner_pool_is_reported if { + fires({"queue_p90": 252}, "lane measure late") +} + +# THE TWO QUEUE FIGURES ARE SEPARATE RULES, and this is why: a wide matrix whose +# legs queue behind each other shows a clean run figure and a bad job figure. +test_a_leg_queueing_behind_its_siblings_is_its_own_finding if { + fires({"queue_job_p90": 252}, "job measure late") + not fires({"queue_job_p90": 252}, "lane measure late") +} + +test_any_fast_forward_refusal_is_reported if { + fires({"ff_refused": 1}, "branch reach stale") +} + +test_a_partially_read_window_is_a_finding if { + fires({"unreadable": 2}, "lane read partial") +} + +# ANTI-VACUITY: a window with no landings judges nothing, because every ratio is +# undefined without a denominator. It is still a reading — the record is present. +test_a_window_with_no_landings_judges_nothing if { + count(violation) == 0 with input as window({"landings": 0, "graded": 0, "green": 0, "ff_success": 0}) + count_of("landings") == 0 with input as window({"landings": 0, "graded": 0, "green": 0, "ff_success": 0}) +} + +test_no_record_at_all_says_nothing if { + count(violation) == 0 with input as {"tree": {"records": {}}} +} + +test_a_non_numeric_count_leaves_the_window_unjudged if { + not count_of("graded") with input as tree(["window\tsince=2026-08-12T00:00:00Z\tlandings=1\tgraded=lots\tgreen=1\tred=0\tcancelled=0\tcancel_p50=0\tpeak_concurrency=1\tqueue_p90=0\tqueue_job_p90=0\tretries=0\tff_refused=0\tff_success=1\tunreadable=0"]) +} + +test_two_concatenated_measurements_judge_neither_window if { + lines := array.concat( + window({"graded": 9}).tree.records["land-divergence"], + window({"graded": 3, "landings": 2}).tree.records["land-divergence"], + ) + count(violation) == 0 with input as tree(lines) +} + +# A line this reader cannot parse is skipped; the summary survives, so this passes +# for the reason it says rather than because the window went missing. +test_a_line_this_reader_cannot_parse_is_skipped if { + lines := array.concat(window({}).tree.records["land-divergence"], ["nonsense"]) + count(violation) == 0 with input as tree(lines) +} diff --git a/tests/land-divergence-assert.bats b/tests/land-divergence-assert.bats deleted file mode 100644 index 98267563a..000000000 --- a/tests/land-divergence-assert.bats +++ /dev/null @@ -1,190 +0,0 @@ -#!/usr/bin/env bats -# subject: mise-tasks/land-divergence-assert.sh -# CLOUD-492. The deciding half: is the landing loop's divergence from linear under -# budget? A pure function of records on stdin — no network, no token — which is -# what lets it run in the hk gate on every commit while the measurement runs on a -# clock. - -setup() { - ASSERT="$BATS_TEST_DIRNAME/../mise-tasks/land-divergence-assert.sh" -} - -# A `window` summary with everything at its ideal, so each case varies exactly one -# field and nothing else can explain the verdict. -window() { - local landings="${1:-10}" graded="${2:-10}" red="${3:-0}" cancel_p50="${4:-15}" \ - peak="${5:-1}" queue="${6:-0}" ff="${7:-0}" unreadable="${8:-0}" job_queue="${9:-0}" - printf 'window\tsince=2026-08-12T00:00:00Z\tlandings=%s\tgraded=%s\tgreen=%s\tred=%s\tcancelled=0\tcancel_p50=%s\tpeak_concurrency=%s\tqueue_p90=%s\tqueue_job_p90=%s\tretries=0\tff_refused=%s\tff_success=5\tunreadable=%s\n' \ - "$landings" "$graded" "$((graded - red))" "$red" "$cancel_p50" "$peak" "$queue" "$job_queue" "$ff" "$unreadable" -} - -@test "a linear window passes: one graded run per landing, green, uncontended" { - run "$ASSERT" <<<"$(window)" - [ "$status" -eq 0 ] - [[ "$output" == *"1.00 each"* ]] -} - -# --- direction 1: over the ratio --------------------------------------------- - -@test "graded runs per landing over budget exits 1" { - # The headline divergence. 30 graded runs over 10 landings is 3.00 each, - # against a budget of 2.00 — the pre-serialisation shape, measured at 6.95. - run "$ASSERT" <<<"$(window 10 30)" - [ "$status" -eq 1 ] - [[ "$output" == *"3.00 per landing"* ]] - [[ "$output" == *"budget of 2.00"* ]] -} - -@test "the ratio is reported in hundredths rather than rounded" { - # A gate that rounds is a gate that disagrees with the number it printed. - run "$ASSERT" <<<"$(window 64 422)" - [ "$status" -eq 1 ] - [[ "$output" == *"6.59 per landing"* ]] -} - -@test "red runs per landing over budget exits 1" { - run "$ASSERT" <<<"$(window 10 10 5)" - [ "$status" -eq 1 ] - [[ "$output" == *"0.50 per landing"* ]] - [[ "$output" == *"verify\` was skipped"* ]] -} - -# --- directions 3 and 4: cancel LATENCY, never cancel count ------------------- - -@test "a 20s cancellation does NOT count as waste" { - # The load-bearing case. Measured post-serialisation: 5 green against 5 - # cancelled, p50 ~20s — `ci-lease-precondition` stopping an unauthorised - # matrix for ~20 runner-seconds instead of billing ~500. A gate counting - # cancellations would score the working mechanism as a defect and argue for - # removing it, so the graded quantity is latency. - run "$ASSERT" <<<"$(window 10 10 0 20)" - [ "$status" -eq 0 ] -} - -@test "a 400s cancellation DOES count as waste" { - # The other direction, and the one that makes the metric discriminate: by - # 400s the matrix has been paid for, so the cancellation saved nothing. - run "$ASSERT" <<<"$(window 10 10 0 400)" - [ "$status" -eq 1 ] - [[ "$output" == *"median lifetime of 400s"* ]] -} - -# --- the remaining budgets ---------------------------------------------------- - -@test "peak concurrency above the admitted-successor bound exits 1" { - # Landing is serialised behind a lease that admits one successor, so anything - # above that is something spending CI without holding it. Measured 25 before - # the lease, 3 after. - run "$ASSERT" <<<"$(window 10 10 0 15 25)" - [ "$status" -eq 1 ] - [[ "$output" == *"25 CI matrices ran concurrently"* ]] -} - -@test "a queue delay is reported as its own defect, not as contention" { - run "$ASSERT" <<<"$(window 10 10 0 15 1 300)" - [ "$status" -eq 1 ] - [[ "$output" == *"runner pool saturating"* ]] -} - -@test "any fast-forward refusal at all exits 1" { - # 243:5 before the lease, 0:5 after. A refusal means the branch went behind - # before the bot answered, so the budget is zero rather than tunable. - run "$ASSERT" <<<"$(window 10 10 0 15 1 0 1)" - [ "$status" -eq 1 ] - [[ "$output" == *"refused 1 time(s)"* ]] -} - -# --- direction 2: could not look is never a pass ------------------------------ - -@test "a partially-read window exits 2, never 0" { - # bench-assert's partial-coverage rule: a run that measured two of three paths - # and reported green over the two is exactly the partial-coverage false green. - run "$ASSERT" <<<"$(window 10 10 0 15 1 0 0 1)" - [ "$status" -eq 2 ] - [[ "$output" == *"cover less than it claims"* ]] -} - -@test "an unreadable window exits 2 even when every other number is ideal" { - # The ordering matters: if the budgets were judged first, a perfect prefix of - # a bad window would exit 0 before anything noticed it was a prefix. - run "$ASSERT" <<<"$(window 10 10 0 15 1 0 0 3)" - [ "$status" -eq 2 ] -} - -@test "empty stdin exits 2" { - run "$ASSERT" <<<"" - [ "$status" -eq 2 ] - [[ "$output" == *"stdin is empty"* ]] -} - -@test "records with no window summary exit 2" { - run "$ASSERT" <<<"$(printf 'pr\tnumber=1\tbranch=b\tgraded=9\tgreen=9\tred=0\tcancelled=0\n')" - [ "$status" -eq 2 ] - [[ "$output" == *"no \`window\` summary"* ]] -} - -@test "two concatenated measurements exit 2 rather than describing neither" { - run "$ASSERT" <<<"$( - window - window 10 30 - )" - [ "$status" -eq 2 ] - [[ "$output" == *"more than one"* ]] -} - -@test "A JOB QUEUE DELAY IS ITS OWN BUDGET, over a clean per-run figure" { - # CLOUD-501, and the case the whole per-job attribution exists for: a run's - # `created_at` -> `run_started_at` is its FIRST job's start, so a matrix leg - # queueing behind its siblings is invisible in it. Per-run 0s, per-job 300s is - # a wide matrix contending with itself, and a gate reading only the run figure - # would call that window ideal. - run "$ASSERT" <<<"$(window 10 10 0 15 1 0 0 0 300)" - [ "$status" -eq 1 ] - [[ "$output" == *"individual JOBS waited 300s"* ]] -} - -@test "a clean per-job figure passes, and the success line reports it" { - run "$ASSERT" <<<"$(window)" - [ "$status" -eq 0 ] - [[ "$output" == *"per job"* ]] -} - -@test "a summary missing the per-job count exits 2 rather than reading it as zero" { - # A measurer that predates the field must not be judged as if it had reported - # a perfect one — that is the partial-coverage false green in miniature. - run "$ASSERT" <<<"$(printf 'window\tsince=x\tlandings=10\tgraded=10\tred=0\tcancel_p50=1\tpeak_concurrency=1\tqueue_p90=0\tff_refused=0\tunreadable=0\n')" - [ "$status" -eq 2 ] - [[ "$output" == *"queue_job_p90"* ]] -} - -@test "a summary missing a count exits 2 rather than reading it as zero" { - run "$ASSERT" <<<"$(printf 'window\tsince=x\tlandings=10\tgraded=10\n')" - [ "$status" -eq 2 ] - [[ "$output" == *"cannot be judged"* ]] -} - -@test "a non-numeric count exits 2" { - run "$ASSERT" <<<"$(printf 'window\tsince=x\tlandings=lots\tgraded=10\tred=0\tcancel_p50=1\tpeak_concurrency=1\tqueue_p90=0\tqueue_job_p90=0\tff_refused=0\tunreadable=0\n')" - [ "$status" -eq 2 ] -} - -# --- anti-vacuity ------------------------------------------------------------- - -@test "a window with no landings passes, and says why" { - # A gate that cannot fire must not be indistinguishable from one that found - # nothing — and a quiet day is the honest reading here, not a defect. - run "$ASSERT" <<<"$(window 0 0)" - [ "$status" -eq 0 ] - [[ "$output" == *"no landings in the window"* ]] -} - -@test "the divergent PRs are named on failure, pointer-only" { - # Rule 4: the record carries a number and a branch, never a title or a body. - run "$ASSERT" <<<"$( - printf 'pr\tnumber=366\tbranch=claude/landing-lease-optimization\tgraded=16\tgreen=12\tred=2\tcancelled=2\n' - window 10 30 - )" - [ "$status" -eq 1 ] - [[ "$output" == *"number=366"* ]] - [[ "$output" == *"graded=16"* ]] -} diff --git a/tests/land-divergence.bats b/tests/land-divergence.bats deleted file mode 100644 index be19b325d..000000000 --- a/tests/land-divergence.bats +++ /dev/null @@ -1,281 +0,0 @@ -#!/usr/bin/env bats -# subject: mise-tasks/land-divergence.sh -# CLOUD-492. The measuring half. `gh` is stubbed with canned HTTP responses, the -# `tests/ci-wait.bats` idiom, because the properties under test are about the -# REQUEST (was `If-None-Match` actually sent), about a response with no body at -# all (does a 304 keep the previous reading), and about a response that is honest -# about being a prefix (does `total_count` catch the API's 1000-item cap). - -setup() { - SCAN="$BATS_TEST_DIRNAME/../mise-tasks/land-divergence.sh" - ASSERT="$BATS_TEST_DIRNAME/../mise-tasks/land-divergence-assert.sh" - ROOT="$BATS_TEST_DIRNAME/.." - STUB="$BATS_TEST_TMPDIR/bin" - mkdir -p "$STUB" - PATH="$STUB:$PATH" - # Its own cache per case: the ETag store persists within a clone by design, so - # a shared one would let one case's cached body decide another's verdict. - BATTEN_DIVERGENCE_CACHE="$BATS_TEST_TMPDIR/cache" - BATTEN_DIVERGENCE_SINCE="2026-08-12T00:00:00Z" - export PATH BATTEN_DIVERGENCE_CACHE BATTEN_DIVERGENCE_SINCE -} - -# A `gh` that routes on the endpoint and replays a numbered response per endpoint -# kind, recording every argv so a case can assert what was actually requested. -# Unquoted heredoc: stub-local vars are escaped, test-side paths interpolate. -stub_gh() { - cat >"$STUB/gh" <>"$BATS_TEST_TMPDIR/args" -kind=ci -for a in "\$@"; do - case "\$a" in - *fast-forward.yml*) kind=ff ;; - */pulls*) kind=pulls ;; - */jobs*) kind=jobs ;; - esac -done -n=\$(cat "$BATS_TEST_TMPDIR/n.\$kind" 2>/dev/null || echo 0) -n=\$((n + 1)) -printf '%s' "\$n" >"$BATS_TEST_TMPDIR/n.\$kind" -f="$BATS_TEST_TMPDIR/resp.\$kind.\$n" -[ -f "\$f" ] || f="$BATS_TEST_TMPDIR/resp.\$kind.1" -head -n1 "\$f" | grep -q ' 2' || { cat "\$f"; exit 1; } -cat "\$f" -EOF - chmod +x "$STUB/gh" -} - -# `ok ` — a 200 with a body. -ok() { - { - printf 'HTTP/2.0 200 OK\n' - [ -n "$2" ] && printf 'ETag: %s\n' "$2" - printf '\n%s\n' "$3" - } >"$1" -} - -# `not_modified ` — a 304, which carries no body at all. -not_modified() { - { - printf 'HTTP/2.0 304 Not Modified\n' - printf 'ETag: %s\n' "$2" - printf '\n' - } >"$1" -} - -# One CI run, shaped as the runs endpoint returns it. -ci_payload() { # $1 = total_count, $2 = conclusion, $3 = branch - printf '{"total_count":%s,"workflow_runs":[{"id":1,"head_branch":"%s","conclusion":"%s","created_at":"2026-08-12T01:00:00Z","run_started_at":"2026-08-12T01:00:00Z","updated_at":"2026-08-12T01:05:00Z","run_attempt":1}]}' \ - "$1" "$3" "$2" -} - -pulls_payload() { # $1 = branch - printf '[{"number":1,"merged_at":"2026-08-12T02:00:00Z","head":{"ref":"%s"}}]' "$1" -} - -# One job of a run, as the per-run jobs endpoint returns it. `created` is when the -# leg was queued and `started` when a runner picked it up; the gap between them is -# the whole measurement (CLOUD-501). -jobs_payload() { # $1 = created, $2 = started, $3 = name - printf '{"total_count":1,"jobs":[{"id":7,"name":"%s","created_at":"%s","started_at":"%s","conclusion":"success"}]}' \ - "$3" "$1" "$2" -} - -prime() { # the happy path every case starts from - stub_gh - ok "$BATS_TEST_TMPDIR/resp.ci.1" 'W/"c1"' "$(ci_payload 1 success feat)" - ok "$BATS_TEST_TMPDIR/resp.ff.1" 'W/"f1"' '{"total_count":0,"workflow_runs":[]}' - ok "$BATS_TEST_TMPDIR/resp.pulls.1" 'W/"p1"' "$(pulls_payload feat)" - # The jobs endpoint is read once per graded run. Without a default here every - # existing case would fall through to the `ci` canned response, because the - # stub's unmatched-URL branch is `kind=ci`. - ok "$BATS_TEST_TMPDIR/resp.jobs.1" 'W/"j1"' \ - "$(jobs_payload 2026-08-12T01:00:00Z 2026-08-12T01:00:00Z ci)" -} - -@test "a linear window measures one graded run against one landing" { - prime - run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"landings=1"* ]] - [[ "$output" == *"graded=1"* ]] - [[ "$output" == *"unreadable=0"* ]] -} - -@test "the conditional request actually sends If-None-Match once an ETag is cached" { - # mem:github-rest-etiquette: a 304 does not count against the primary rate - # limit, and that is what makes a daily walk affordable. Asserted on the - # REQUEST, because a cache nobody sends is not a cache. - prime - run "$SCAN" - [ "$status" -eq 0 ] - rm -f "$BATS_TEST_TMPDIR"/n.ci "$BATS_TEST_TMPDIR"/n.ff "$BATS_TEST_TMPDIR"/n.pulls "$BATS_TEST_TMPDIR"/n.jobs - run "$SCAN" - [ "$status" -eq 0 ] - grep -q 'If-None-Match: W/"c1"' "$BATS_TEST_TMPDIR/args" -} - -@test "A 304 KEEPS THE PREVIOUS READING rather than reading as an empty window" { - # Re-parsing an empty 304 body yields nothing, which reads as "no runs" — a - # perfect score off a response that carried no data. The failure then shows up - # only as unexplained quiet, which is why this is asserted rather than assumed. - prime - run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"graded=1"* ]] - - rm -f "$BATS_TEST_TMPDIR"/n.ci "$BATS_TEST_TMPDIR"/n.ff "$BATS_TEST_TMPDIR"/n.pulls "$BATS_TEST_TMPDIR"/n.jobs - not_modified "$BATS_TEST_TMPDIR/resp.ci.1" 'W/"c1"' - run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"graded=1"* ]] - [[ "$output" == *"unreadable=0"* ]] -} - -@test "a 304 with no cached body is unreadable, never an empty window" { - stub_gh - not_modified "$BATS_TEST_TMPDIR/resp.ci.1" 'W/"c1"' - ok "$BATS_TEST_TMPDIR/resp.ff.1" 'W/"f1"' '{"total_count":0,"workflow_runs":[]}' - ok "$BATS_TEST_TMPDIR/resp.pulls.1" 'W/"p1"' "$(pulls_payload feat)" - run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"unreadable=1"* ]] -} - -@test "TRUNCATION IS CAUGHT BY total_count, not by running out of pages" { - # The measured defect, and the reason this property exists. The runs endpoint - # hard-caps pagination at 1000 items while still reporting the true - # total_count: measured on fast-forward.yml, total_count 1446, page 10 full, - # page 11 EMPTY. A walk that stops when a page comes back short collects a - # prefix and looks exactly like a clean finish — it reported ff_refused=0 over - # a window carrying 598 refusals. - prime - ok "$BATS_TEST_TMPDIR/resp.ci.1" 'W/"c1"' "$(ci_payload 1446 success feat)" - run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"unreadable=1"* ]] - [[ "$output" == *"read 1 of 1446"* ]] -} - -@test "a truncated window makes the decider refuse, rather than judge a prefix" { - # The two halves together: the measurer reports the prefix honestly and the - # decider turns that into `could not look`, never a pass. - prime - ok "$BATS_TEST_TMPDIR/resp.ci.1" 'W/"c1"' "$(ci_payload 1446 success feat)" - "$SCAN" >"$BATS_TEST_TMPDIR/records" 2>/dev/null - run "$ASSERT" <"$BATS_TEST_TMPDIR/records" - [ "$status" -eq 2 ] -} - -@test "runs after a PR merged are not attributed to it" { - # A branch name is reused, and a landing's denominator must not collect the - # runs of its next life. Bounded by merged_at, so this run is out of scope. - prime - ok "$BATS_TEST_TMPDIR/resp.ci.1" 'W/"c1"' \ - '{"total_count":1,"workflow_runs":[{"id":9,"head_branch":"feat","conclusion":"failure","created_at":"2026-08-12T09:00:00Z","run_started_at":"2026-08-12T09:00:00Z","updated_at":"2026-08-12T09:05:00Z","run_attempt":1}]}' - run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"graded=0"* ]] -} - -@test "a draft-era skipped run is not counted as graded" { - # CI does not run on drafts; the run object still exists and concluded - # `skipped`. It spent no matrix, so counting it would inflate every ratio. - prime - ok "$BATS_TEST_TMPDIR/resp.ci.1" 'W/"c1"' "$(ci_payload 1 skipped feat)" - run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"graded=0"* ]] -} - -@test "an unreadable run list reports rather than claiming a clean window" { - stub_gh - printf 'HTTP/2.0 500 Server Error\n\n' >"$BATS_TEST_TMPDIR/resp.ci.1" - ok "$BATS_TEST_TMPDIR/resp.ff.1" 'W/"f1"' '{"total_count":0,"workflow_runs":[]}' - ok "$BATS_TEST_TMPDIR/resp.pulls.1" 'W/"p1"' "$(pulls_payload feat)" - run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"unreadable=1"* ]] -} - -# --- a property of the world must not red a branch ---------------------------- - -@test "neither half is in the hk gate" { - # Non-negotiable: this measures the world, not the commit. A gate step that - # reaches the network on the landing path is what CLOUD-410 cost. - run grep -c 'land-divergence' "$ROOT/hk.pkl" - [ "$output" = "0" ] -} - -@test "neither half is in verify" { - run bash -c "awk '/^\[tasks.verify\]/{p=1;next} /^\[/{p=0} p' '$ROOT/mise.toml' | grep -c land-divergence" - [ "$output" = "0" ] -} - -@test "the scheduled workflow declares a concurrency group, like every other" { - # ci-local-parity property 8 requires it; asserted here too so the workflow - # this task ships with cannot be the one that breaks the gate it landed after. - run grep -c '^concurrency:' "$ROOT/.github/workflows/land-divergence.yml" - [ "$output" = "1" ] -} - -# --- CLOUD-501: the queue delay, attributed per JOB ---------------------------- - -@test "A JOB THAT WAITED REPORTS ITS GAP, where the run's own figure cannot" { - # The measured defect. A run's `created_at` -> `run_started_at` is the FIRST - # job's start, so a matrix leg that waited behind its siblings contributes - # nothing to it — and telling a wide matrix from a saturated pool is exactly - # what the diagnosis needs. - prime - ok "$BATS_TEST_TMPDIR/resp.jobs.1" 'W/"j1"' \ - "$(jobs_payload 2026-08-12T01:00:00Z 2026-08-12T01:04:00Z ci-linux)" - run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"queue_job_p90=240"* ]] - [[ "$output" == *"job"*"job=ci-linux"*"queue=240"* ]] -} - -@test "a job that started immediately reports zero and earns no record" { - # The other direction, and the anti-noise rule the `pr` records already use: a - # zero-wait leg is the ideal and says nothing a reader needs. - prime - run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"queue_job_p90=0"* ]] - [[ "$output" != *"job"$'\t'"run="* ]] -} - -@test "A RUN WHOSE JOBS CANNOT BE READ IS UNREADABLE, NEVER A ZERO WAIT" { - # Dropping it would report a p90 over the legs that happened to answer, which - # is the partial-coverage false green this whole sensor exists to refuse — and - # the decider turns `unreadable` into `could not look` rather than a pass. - prime - printf 'HTTP/2.0 500 Server Error\n\n' >"$BATS_TEST_TMPDIR/resp.jobs.1" - run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" == *"unreadable=1"* ]] -} - -@test "the per-job read is bounded to graded runs, so a skipped run costs no request" { - # `job_queue` is one request per run, which cuts against the ETag-stability - # argument in the header — so the bound is asserted rather than described. - prime - ok "$BATS_TEST_TMPDIR/resp.ci.1" 'W/"c1"' "$(ci_payload 1 skipped feat)" - run "$SCAN" - [ "$status" -eq 0 ] - run grep -c '/jobs' "$BATS_TEST_TMPDIR/args" - [ "$output" = "0" ] -} - -@test "the job record is pointer-only — a run id, a job name and seconds" { - # Non-negotiable 4. A jobs payload carries step detail and log urls; none of it - # reaches the record. - prime - ok "$BATS_TEST_TMPDIR/resp.jobs.1" 'W/"j1"' \ - "$(jobs_payload 2026-08-12T01:00:00Z 2026-08-12T01:00:30Z ci-linux)" - run "$SCAN" - [ "$status" -eq 0 ] - [[ "$output" != *"conclusion"* ]] - [[ "$output" != *"total_count"* ]] -} From 890054cfc67affdd845d94a2fc07a0b4fb3964b1 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Thu, 17 Sep 2026 09:17:42 +0000 Subject: [PATCH 10/32] fix(prune): rescale the warm floor for the eleven tiers this campaign added MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `target-prune` refused the tree: `[prune.warm.basis]` declared 255 test stems against a live 266, eleven past a tolerance of 10. That is the gate working. CLOUD-1717's corpus campaign owes a `crates/batten/tests/it/*.rs` per retired gate and has landed eleven of them, which is exactly the drift the basis exists to notice. Scaled by the stem model the block documents: 11642 at 255 is 45.65 per stem, and 45.65 x 266 is 12144. Derived rather than independently measured, and said so — an honest warm number needs a minimal post-prune tree, and the basis block asks that a reader who needs it exact take it rather than trust the line. `mb`, `worst_mb` and `measured` move together, and so do BOTH `basis.count` values, because refreshing one arm makes the next lap refuse on the other with warm never breached. THE LIVE COUNT IS 266 WHERE `git ls-files` SAYS 265, and that is not a discrepancy to average away. The engine's `Selector` sets `literal_separator(true)`, so its `**` matches zero directories as well as many and reaches `crates/batten/tests/policy_modules.rs`; git's pathspec does not. `count` is compared against the engine's reading, so it is the engine's. AND IT WRITES THE ROW THE BLOCK HAS OWED SINCE ITS SIXTH MISREPORT. Seven times `verify` has rendered a stem-count refusal as "not enough disk to run the gate, and pruning did not recover it" — this time over 18781MB free against an 11642MB floor, with `target-prune` printing that margin two lines above the caller's claim there was none. The sixth entry says the fix is a declared `[[verify_environment_pattern]]` and that "the pattern belongs with the measurement that justifies it". This is a measurement, so the row lands with it: `prune-basis-stale`, matching the CALLEE's sentence rather than the caller's wrapper, because keying on the wrapper would carry one specific remedy for every environment refusal the task emits. Refs: CLOUD-1717 Admits: 1524f84ee02b9a55c887949e65b2b9ab89b6fb2ea38dbc798f90e557fb6c68b4 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: batten.toml Admits-anchor: call:0120188cdfe22249719c36d6b0e6965df94b2514 Admits-epoch: b48fc2790947366c60a4d88b7c596aed9747de53678bfc6d1ccb0b2de0c2b2c7 Admits-author: alec@wenzowski.com Admits-prev: 573bf1d70ccdbb16c0a5154380b5149f17e7ec1837acdc6c9bdb911f96dd78a7 Admits-answer-lost: `mise run verify` cannot reach a receipt at all: `target-prune` refuses before any gate runs, so nothing on this branch can be verified, readied or landed, and the twelve retirements already committed sit unlandable. The refusal is also not what it says — it renders as "not enough disk" over 18781MB free against an 11642MB floor, so a reader who trusts the sentence goes looking for a full disk that does not exist. Admits-answer-precondition: The prune floor and its basis live in `batten.toml` and nowhere else. `target-prune` refuses this tree because `[prune.warm.basis]` declares 255 test stems against a live 266 — eleven past a tolerance of 10 — which is this campaign moving the basis, since every retirement adds a tier under `crates/batten/tests/it/`. The file's own entries state the remedy and its shape: scale warm by the stem model, move `mb`, `worst_mb` and `measured` together, and move both bases so refreshing one does not make the next lap refuse on the other arm. There is no other surface any of those four numbers lives on. Admits-answer-rejected-route: config read first — reading the prune block is exactly how I learned the stem model, the per-stem figure and the rule that both bases move together, and that reading produced the numbers rather than removing the need to write them; a stale floor cannot be re-measured by being read. patch run first is a commit-message route and does not apply to a config measurement. --- batten.toml | 48 +++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 43 insertions(+), 5 deletions(-) diff --git a/batten.toml b/batten.toml index f10fe210f..3fcbbd952 100644 --- a/batten.toml +++ b/batten.toml @@ -8301,11 +8301,27 @@ keep = 2 # $GIT_DIR/batten-prune/laps.json and already governs — so the declared number is # not raised to meet it. A declared floor that chases the worst observed lap is the # floor-nobody-can-satisfy this file warns about, two blocks down. +# THE 2026-09-17 WARM RESCALE (CLOUD-1717's corpus campaign). The campaign owes a +# `crates/batten/tests/it/*.rs` per retired gate and had landed eleven of them, +# taking the live count to 266 against a basis of 255 with a tolerance of 10 — +# this gate working, exactly as the 2026-09-12 entry above describes it working. +# +# WARM SCALES BY THE STEM MODEL, as every move above it did: 11642 at 255 is 45.65 +# per stem, and 45.65 x 266 is 12144. Derived rather than independently measured, +# and said so for the reason the basis block insists on: an honest warm number +# needs a minimal post-prune tree, and a reader who needs it exact should take it +# rather than trust this line. +# +# THE LIVE COUNT IS 266 AND `git ls-files` SAYS 265, which is not a discrepancy to +# reconcile away. The engine's `Selector` sets `literal_separator(true)`, so its +# `**` matches zero directories as well as many and the glob reaches +# `crates/batten/tests/policy_modules.rs` at the top level; git's pathspec `**` +# does not. `count` is the number the GATE compares against, so it is the engine's. [prune.warm] -mb = 11642 -worst_mb = 11642 +mb = 12144 +worst_mb = 12144 multiplier = 1 -measured = "2026-09-12" +measured = "2026-09-17" [prune.cold] mb = 21455 @@ -8760,7 +8776,7 @@ measured = "2026-09-07" # makes the very next lap refuse on the other arm with warm never breached. [prune.warm.basis] glob = "crates/batten/tests/**/*.rs" -count = 255 +count = 266 tolerance = 10 # COLD'S BASIS MOVES AND COLD'S FLOOR DOES NOT, and that is the entry above @@ -8780,7 +8796,7 @@ tolerance = 10 # together. [prune.cold.basis] glob = "crates/batten/tests/**/*.rs" -count = 255 +count = 266 tolerance = 10 # THE REGROWABLE ROOTS THE ESCALATION MAY DROP (CLOUD-1157), in the order it drops @@ -9294,6 +9310,28 @@ pattern = "No space left on device" stream = "both" reason = "the disk filled, so this is the environment rather than your branch. Run `mise run target-prune`, and consider building with CARGO_INCREMENTAL=0." +# THE ROW THE PRUNE BLOCK HAS OWED SINCE ITS SIXTH MISREPORT (CLOUD-1717). Seven +# times now `verify` has rendered a STEM-COUNT refusal as "not enough disk to run +# the gate, and pruning did not recover it", every time over ample free space, and +# twice with the callee printing the margin two lines above the caller's claim +# there was none. The measurements are in the `[prune]` block; this is the remedy +# they justify. +# +# IT MATCHES THE CALLEE, NOT THE CALLER. `target-prune` prints this sentence and +# `verify` wraps it, so keying on the wrapper would match every environment +# refusal the task emits and carry a specific remedy for all of them — the opposite +# of the point. The callee's line reaches both streams through the wrapper. +# +# `Environment` RATHER THAN A FAILING GATE is the classification that matters: the +# tree is fine, the branch is fine, and "reproduce and fix locally" sends a reader +# after a defect that is not there. What is stale is a number in the committed +# authority, and the remedy is to move it. +[[verify_environment_pattern]] +id = "prune-basis-stale" +pattern = "was measured against a tree that no longer exists" +stream = "both" +reason = "the prune floor's basis is stale, not the disk. Nothing is wrong with this branch: the declared stem count has drifted past its tolerance, which is what a campaign adding test files does. Re-measure `[prune.warm]` by the stem model the block documents, and move `mb`, `worst_mb`, `measured` and BOTH `basis.count` values together." + [lease] landing_paths = [ "mise-tasks/land.sh", From 43142287499b2bfe89fb045b2806ffa0c657e194 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Fri, 18 Sep 2026 02:22:43 +0000 Subject: [PATCH 11/32] fix(test): write! rather than push_str(format!) in the land-divergence fixture MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two clippy errors in the tier's `config()` builder, both caught by `verify` rather than by me: `format_push_string` on the per-verdict append, and a missing backtick on `ETag` in the module doc. `write!` over the `fmt::Write` impl for `String` is the fix clippy names. It is infallible there, so the `expect` cannot fire — said in the comment so the next reader does not take it for an unhandled error path. I ran `cargo test` on this tier and not `mise run lint:clippy`, which is the step the per-retirement recipe lists. A 31-minute `verify` found what a seconds-long local lint would have. Refs: CLOUD-1717 --- crates/batten/tests/it/land_divergence.rs | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/crates/batten/tests/it/land_divergence.rs b/crates/batten/tests/it/land_divergence.rs index d25cdfe0c..5766f284e 100644 --- a/crates/batten/tests/it/land_divergence.rs +++ b/crates/batten/tests/it/land_divergence.rs @@ -22,7 +22,7 @@ //! CLOUD-1559's reason: a measurement needs the network and a token, a decision //! needs neither. The measurement is `[tasks.land-divergence-record]`, the //! decision is `policy/land-divergence.rego`, and no decision changed hands. The -//! pagination walk, the ETag cache, the `total_count` truncation guard and every +//! pagination walk, the `ETag` cache, the `total_count` truncation guard and every //! instant subtraction could not move: §5 makes `check` `read` and incapable of //! spawning, and `Fact::Instant` projects `null` to every module. //! @@ -94,6 +94,8 @@ // Panicking on setup failure is the idiomatic way for a test to fail loudly. #![allow(clippy::unwrap_used, clippy::expect_used)] +use std::fmt::Write as _; + use crate::common; use common::{git_in, init_repo, run, run_with_stdin, scratch, write}; @@ -149,7 +151,11 @@ writer = "mise run land-divergence-record" "#, ); for id in verdicts { - out.push_str(&format!( + // `write!` rather than `push_str(&format!(..))`: clippy's + // `format_push_string` refuses the second for the extra allocation, and + // `fmt::Write` for `String` is infallible, so the `expect` cannot fire. + write!( + out, r#" [[verdict]] id = "{id}" @@ -161,7 +167,8 @@ id = "task run first" kind = "command" target = "mise run land-divergence-record" "# - )); + ) + .expect("writing to a String cannot fail"); } out } From 22148ffbd72bc45f53e245b7ea5407cbb1578a6f Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Fri, 18 Sep 2026 03:02:56 +0000 Subject: [PATCH 12/32] fix(test): mint the read receipt before each adjudication `only_the_move_to_in_review_is_this_rows_business` minted one `issue-read` receipt and then made eleven adjudications against it. The receipt carries a recency bound, so on a loaded runner the later calls read a receipt that has already aged out and an ungated column (`Todo`) refuses with 2 where 0 is expected. The window, not the verdict, was the variable. Mint fresh before each call: what the case asserts is which column is this row's business, not how long a receipt lives. --- crates/batten/tests/it/board_receipts.rs | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/crates/batten/tests/it/board_receipts.rs b/crates/batten/tests/it/board_receipts.rs index 9e96add5d..877dece6a 100644 --- a/crates/batten/tests/it/board_receipts.rs +++ b/crates/batten/tests/it/board_receipts.rs @@ -1218,10 +1218,24 @@ fn an_adjudication_of_one_row_does_not_authorise_another() { /// The three spellings are one move because the tracker's parameter takes a type, /// a name or an id; the engine folds case and drops spaces, underscores and /// hyphens. +/// +/// # The receipt is minted before EACH adjudication, not once +/// +/// `issue-read` carries a recency bound, and this case makes eleven mediated +/// calls. Minting once made every later assertion a function of how long the case +/// took to run rather than of the rule it is about: measured failing under a +/// loaded `test:musl` at 318s wall, where the same case passes in 109s alone, and +/// reproduced deterministically by minting the receipt 400s old — the `Todo` +/// assertion then flips from `0` to `2` because the row refuses a call whose read +/// has gone stale, which is the row working. +/// +/// A fresh mint per call is what makes the case assert which COLUMN this row +/// governs. It is not a slower suite being accommodated: a case whose verdict +/// depends on the wall clock is one that reports a defect nobody introduced, and +/// the campaign that lengthens the suite is what made this reachable. #[test] fn only_the_move_to_in_review_is_this_rows_business() { let repo = repo("row3-columns"); - mint_read_receipt(&repo, "CLOUD-1", 5); for spelling in [ "In Review", "in review", @@ -1229,6 +1243,7 @@ fn only_the_move_to_in_review_is_this_rows_business() { "in_review", "IN-REVIEW", ] { + mint_read_receipt(&repo, "CLOUD-1", 5); assert_eq!( verdict( &repo, @@ -1240,6 +1255,7 @@ fn only_the_move_to_in_review_is_this_rows_business() { ); } for column in ["Todo", "In Progress", "Done", "Backlog", "Canceled"] { + mint_read_receipt(&repo, "CLOUD-1", 5); assert_eq!( verdict( &repo, @@ -1250,6 +1266,7 @@ fn only_the_move_to_in_review_is_this_rows_business() { "this column has a different owner and is not gated here: {column}" ); } + mint_read_receipt(&repo, "CLOUD-1", 5); assert_eq!( verdict( &repo, From 94cc5702e746df837d048031e8fa87d0aa97542c Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Fri, 18 Sep 2026 07:41:07 +0000 Subject: [PATCH 13/32] feat(policy): a workflow's callers resolve, and five that did not MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A retirement deletes a program and no clause asked whether its CALLERS still resolve. `workflow run unknown` is that clause, and the tree it first ran over had five dead ones. ## What it found `batten check --rule 'workflow run unknown'`, over this branch before the repoint, exit 2 with five findings: .github/workflows/branch-hygiene.yml:63 .github/workflows/land-divergence.yml:102 .github/workflows/nonverdict-rate.yml:110 .github/workflows/release-assets.yml:135 .github/workflows/timeout-drift.yml:73 Two name tasks CLOUD-1717 deleted while porting their programs — `attestation-check` and `timeout-drift`. The other three run `mise run batten -- check --rule …`, and there has never been a `[tasks.batten]` at all. That is the more interesting half: the spelling was invented once, copied into two more files, and then read back as an established precedent by the next author to need it. Nothing about a dead caller decays. It was wrong on the commit that introduced it, and the only reason all three survived is that no surface was reading. ## Why none of them was ever going to be noticed Every one of those five steps sits in a job triggered by `workflow_run`, `schedule` or `workflow_dispatch`. None reaches a pull request: `ci-local-parity` holds its properties over `pull_request` workflows and `land` watches a PR's check-runs. So a workflow failing on 100% of its invocations is indistinguishable from one that has never fired — which is CLOUD-258's measurement exactly, and `release-assets.yml` is the same file that paid for it then and for CLOUD-1777 after. ## The predicate's third population, not a second authority `command-task-defined` asks "does this task exist" of a `[[rule]]`'s `check`; `verdict-routes-resolve` asks it of a `[[verdict.route]]`'s `target`; this asks it of a workflow step's `run`. Each owns its population and they share the `defined` shape deliberately, for the reason `verdict-routes-resolve`'s own header gives: two spellings of "what tasks exist" is the drift a shared question does not survive. One arm is new rather than copied. A file task's name is its path under `mise-tasks/` with the separator spelled `:`, so `mise-tasks/render/cli.sh` is `mise run render:cli` — which `release-artifacts.yml` calls. Taking the last path component alone resolves it as `cli`, leaves `render:cli` undefined, and refuses a caller that works. That is a false positive on the gate's first run, which `verdict-routes-resolve`'s header names as the shape that gets an exception written for it, and the exception is what rots. `#MUTANT nested-task-unreachable` is the row that keeps the arm honest. ## The parsed scalar decides; the lines only place the pointer These files carry long comments naming tasks in order to explain that they are ABSENT — `timeout-drift.yml:7` is one — and a gate that fires on its own documentation is a gate people delete. That is `ci-parity`'s rule and it binds here: the decision reads `step.run` from the parsed document, and the line index is consulted afterwards, for a task the parsed reading already decided on. Both tiers carry the case. An interpolated name abstains structurally. `mise run ${{ matrix.task }}` is not decidable from a committed document, and the shared `mise-run-task` pattern requires a lowercase letter where the name begins, so the fragment never matches. No guard is written for it; a conjunct excluded by the pattern before it ran is a surviving mutant rather than a safeguard. ## The tier is not the module's own suite Eleven load-time cases, every one fabricating its input with `with input as` — the shape `rules/policy-modules.md` warns about, and the shape two live dead gates in this repository were found in. `crates/batten/tests/it/task_callable.rs` runs the real module over real documents through the engine's own projection, and one of its seven cases asserts the finding lands on `probe.yml:9`: the pointer is the half most likely to be silently absent, because `line_sources` is a separate acquisition from `sources`. ## The repoint `[tasks.batten]` is a passthrough rather than a task per rule, and `lock-complete`'s comment is why that needs saying: its drift argument is about a TRIPLE — an hk step name, a task and a rule id, which hk selects between by name — and a workflow step names its task directly, so there is no second selector to drift against. Four near-identical wrapper rows would be the duplication rather than the guard. `timeout-drift.yml` and `release-assets.yml` take the two-step producer-then- engine shape `branch-hygiene.yml` already uses, because the split is forced: section 5 makes `check` `read` and incapable of spawning, so the `gh` reads are the producer's and the verdict is the engine's. After the repoint the same command exits 0. Closes CLOUD-1833 Refs: CLOUD-1717 Admits: e7a1ce538a0db7e05318c2990973539df8d334f1db4fa2e42705e8b4e528c904 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: batten.toml Admits-anchor: call:22148ffbd72bc45f53e245b7ea5407cbb1578a6f Admits-epoch: afc3642509e5113fc44540010d92262d59c71440140b360424f99a002df6037c Admits-author: alec@wenzowski.com Admits-prev: - Admits-answer-lost: CLOUD-1833's mechanism cannot ship. Measured on this branch right now, five workflow steps across four files call mise tasks that do not exist — release-assets.yml calls the deleted attestation-check, timeout-drift.yml calls the deleted timeout-drift, and branch-hygiene.yml, land-divergence.yml and nonverdict-rate.yml each call a `mise run batten` that has never existed. All five fire only on schedule or workflow_run, so no PR and no reviewer ever sees them fail. Without the row the module loads nothing and the four dead callers stay dead and silent. Admits-answer-precondition: batten.toml IS the surface that declares [[rule]] and [[verdict]] rows; a new policy module is unreachable by the engine until a [[rule]] row names it, and no other file can carry that row. The write is a rule and verdict registration for policy/task-callable.rego (CLOUD-1833), and it lands in PR #962 where a reviewer reads it as an ordinary diff hunk. Admits-answer-rejected-route: `config read first` is a read of batten.toml, which I have done — it is how I found ci-cache-declared's rule row and copied its sources/line_sources shape; it does not write the registration the engine needs. `patch run first` is `git restore`, which reverts the working tree; there is nothing here to revert, since this is a new registration rather than a botched edit. Admits: a671f0bf508457d53339d0effc969ff4e219451573672285f3de37a6d35624e8 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: batten.toml Admits-anchor: call:22148ffbd72bc45f53e245b7ea5407cbb1578a6f Admits-epoch: 3e2f9971c083a22192eadf5934ddf98f049ccdbafdf955badfd1e3151d931ab2 Admits-author: alec@wenzowski.com Admits-prev: e7a1ce538a0db7e05318c2990973539df8d334f1db4fa2e42705e8b4e528c904 Admits-answer-lost: The module already placed cannot load at all: `task run unknown` is raised by policy/task-callable.rego and declared nowhere, which is a load-time refusal. Leaving it here would leave the tree in a state where batten itself will not start — strictly worse than before the rule row landed. Admits-answer-precondition: batten.toml is the only surface that carries [[verdict]] rows. policy/task-callable.rego raises the token `task run unknown`, and the engine refuses at load both a module raising a token no row declares and a row nothing raises — so the module and this row are one change that cannot be split across files. It lands in PR #962 as an ordinary diff hunk beside the [[rule]] row admitted immediately before it. Admits-answer-rejected-route: `config read first` is a read of batten.toml; I have read it, which is how I found that `workflow read unread` and `task resolve missing` already exist and can be reused instead of declared twice. It cannot write the one row that is genuinely new. `patch run first` is `git restore`, which would discard the [[rule]] row and the module together rather than complete them. Admits: fde50924f01c8cf8458646ff778c0b7aab15a8c21cf0337a8e7cffd40a18cd73 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: .github/workflows/timeout-drift.yml Admits-anchor: call:22148ffbd72bc45f53e245b7ea5407cbb1578a6f Admits-epoch: 88f57cc183d508e8081b071986ae78ec62612b992add1a34f0dd8d77d7cbc5a5 Admits-author: alec@wenzowski.com Admits-prev: - Admits-answer-lost: The weekly timeout-budget sweep stays dead and stays silent. `bound grade other` is declared in batten.toml and policy/timeout-drift.rego implements it, but nothing invokes it: the one caller names a task that does not exist, so a budget that has drifted is never reported by anything. The new `workflow run unknown` gate reports this file today and will keep the branch red until the caller is repointed, so declining also blocks the PR. Admits-answer-precondition: The dead caller IS a line in this workflow file; nothing else can carry the fix. `.github/workflows/timeout-drift.yml:73` runs `mise run timeout-drift`, a task CLOUD-1717 deleted when it ported the program to policy/timeout-drift.rego. The job fires on `schedule` only, so it has been dying at that step with no pull request and no reviewer able to see it. The replacement is the two-step producer-then-engine shape `branch-hygiene.yml` already uses, and it lands in PR #962 as an ordinary diff hunk. Admits-answer-rejected-route: `config read first` is a read of batten.toml; I read it to find that the rule id is `bound grade other` and that its severity is `warn`, which is why the new step carries `--fail-on-warning`. Reading cannot repoint the caller. `patch run first` is `git restore`, which would revert a file whose committed state is the defect — restoring it reinstates the dead call. Admits: 533ece3f4841b5308dc1108f4ab70d4b58aab6ce9bbd2b3b5b75a76149682cf8 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: .github/workflows/release-assets.yml Admits-anchor: call:22148ffbd72bc45f53e245b7ea5407cbb1578a6f Admits-epoch: 88f57cc183d508e8081b071986ae78ec62612b992add1a34f0dd8d77d7cbc5a5 Admits-author: alec@wenzowski.com Admits-prev: - Admits-answer-lost: The release provenance gate stays dead and stays silent, which is the failure this file has already paid for twice. The job fires on `workflow_run`, `schedule` and `workflow_dispatch` only, so it reaches no pull request and no reviewer: CLOUD-258 recorded `release-artifacts.yml` failing every run from v0.0.31 to v0.0.36 with nothing able to say so, and this is the same composition on the same file. Declining also leaves the branch red, since the new `workflow run unknown` gate reports this line. Admits-answer-precondition: The dead caller IS a line in this workflow file. `.github/workflows/release-assets.yml:135` runs `mise run attestation-check "$TAG"`, a task CLOUD-1717 deleted when it ported the program to the `attestation-record` producer plus the `release grade unsafe` rule. This is CLOUD-1833's own subject — the row was filed for exactly this line — and the fix is the two-step producer-then-engine shape `branch-hygiene.yml` already uses. It lands in PR #962, which carries `Closes CLOUD-1833`. Admits-answer-rejected-route: `config read first` is a read of batten.toml; I read it to find that the ported rule is `release grade unsafe` and that `[tasks.attestation-record]` is the producer half. Reading cannot repoint the caller. `patch run first` is `git restore`, which would revert a file whose committed state is the defect — restoring it reinstates the call to the deleted task. --- .github/workflows/release-assets.yml | 15 +- .github/workflows/timeout-drift.yml | 17 +- batten.toml | 54 ++++ crates/batten/tests/it/main.rs | 1 + crates/batten/tests/it/task_callable.rs | 300 ++++++++++++++++++ mise.toml | 26 +- policy/task-callable.rego | 393 ++++++++++++++++++++++++ 7 files changed, 801 insertions(+), 5 deletions(-) create mode 100644 crates/batten/tests/it/task_callable.rs create mode 100644 policy/task-callable.rego diff --git a/.github/workflows/release-assets.yml b/.github/workflows/release-assets.yml index eb5963b9e..51ca556b0 100644 --- a/.github/workflows/release-assets.yml +++ b/.github/workflows/release-assets.yml @@ -127,9 +127,20 @@ jobs: # offering the feature at all, and the task exits 0 having judged no # artifact. It starts verifying on its own the moment CLOUD-585 makes the # repository public, with no edit here. - - name: Check the release's binaries carry build provenance + # + # THE SPLIT THE PORT FORCED (CLOUD-1717), and this caller is what CLOUD-1833 + # was filed for. `attestation-check` was one program making the probe and + # adjudicating it; section 5 makes `check` `read` and incapable of spawning, + # so the `gh` reads are the producer's and the verdict is the engine's. The + # step above went on calling the deleted task for as long as nothing asked + # whether a workflow's callers still resolve — `workflow run unknown` is + # that clause, and it reported this line. + - name: Record the platform's attestation posture and each archive's verdict shell: bash env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} TAG: ${{ inputs.tag }} - run: mise run attestation-check "$TAG" + run: mise run attestation-record -- "$TAG" + - name: Check the release's binaries carry build provenance + shell: bash + run: mise run batten -- check --rule 'release grade unsafe' diff --git a/.github/workflows/timeout-drift.yml b/.github/workflows/timeout-drift.yml index 3d050c60e..63002d761 100644 --- a/.github/workflows/timeout-drift.yml +++ b/.github/workflows/timeout-drift.yml @@ -66,8 +66,21 @@ jobs: # `${{ }}` expansion is textual substitution into the shell script, and # zizmor gates that at high severity. All logic is in the task; nothing # here but the invocation. - - name: Report budgets that no longer match their measurement + # THE SPLIT THE PORT FORCED (CLOUD-1717), and the shape `branch-hygiene.yml` + # takes: the producer makes the forge reads, the engine adjudicates what it + # recorded. Neither half can hold the other — section 5 makes `check` `read` + # and incapable of spawning, and the engine calls no clock on any evaluation + # path. + # + # `--fail-on-warning` because `bound grade other` is declared `warn`: a + # budget drifting is a report about the world rather than a verdict on a + # branch, and this scheduled job is the one caller that should still go red + # for it. + - name: Record each job's measured duration against its declared budget shell: bash env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - run: mise run timeout-drift + run: mise run timeout-drift-record + - name: Report budgets that no longer match their measurement + shell: bash + run: mise run batten -- check --rule 'bound grade other' --fail-on-warning diff --git a/batten.toml b/batten.toml index 3fcbbd952..267521613 100644 --- a/batten.toml +++ b/batten.toml @@ -7554,6 +7554,34 @@ line_sources = [".github/workflows/*.yml"] module = "policy/ci-cache-declared.rego" severity = "deny" +# CLOUD-1833. A retirement deletes a program and nothing asked whether its +# CALLERS still resolve. Measured when this row was written: five steps across +# four workflows named tasks that do not exist — two deleted by CLOUD-1717's own +# commits, and three spelling a `mise run batten` that has never existed at all. +# Every one fires on `workflow_run`, `schedule` or `workflow_dispatch`, so it +# reaches no pull request and no reviewer, and a workflow failing on 100% of its +# invocations is indistinguishable from one that has never fired. +# +# THE THIRD POPULATION OF A PREDICATE THIS TREE ALREADY HAS TWICE. +# `command-task-defined` asks it of a `[[rule]]`'s `check`, +# `verdict-routes-resolve` of a `[[verdict.route]]`'s `target`, and this of a +# workflow step's `run`. Each owns its population; the shared `defined` shape is +# deliberate, for the reason `verdict-routes-resolve`'s header states. +# +# The two declared sources are `ci-cache-declared`'s pair and for its reason: the +# question is about mise's TASK GRAPH, which it owns and this reads as a parsed +# document rather than re-parsing text. `line_sources` is the workflows only — +# every pointer this module emits is into one, and nothing it decides needs a +# line in `mise.toml`. +[[rule]] +id = "workflow run unknown" +kind = "policy" +scope = "tree" +sources = [".github/workflows/*.yml", "mise.toml"] +line_sources = [".github/workflows/*.yml"] +module = "policy/task-callable.rego" +severity = "deny" + # CLOUD-1431. The two pinned-tool surfaces this repository has, compared for the # first time: what `release-artifacts.yml` PUBLISHES against what a # `[[provision]]` row can RESOLVE. @@ -12256,6 +12284,32 @@ id = "workflow read first" kind = "document" target = ".github/workflows/release-assets.yml" +[[verdict]] +id = "task run unknown" +gloss = "a workflow step runs a mise task that neither the manifest nor a program under mise-tasks/ defines" +class = """ +The step will fail the moment it runs, and it will fail invisibly. A workflow \ +triggered by `schedule`, `workflow_run` or `workflow_dispatch` reaches no pull \ +request and no reviewer, so a run that dies at the step is indistinguishable \ +from one that never fired — which is how five such callers accumulated across \ +four files before anything asked the question. Nothing about a dead caller \ +decays over time: it was wrong on the commit that introduced it, and the only \ +reason it survived is that no surface was reading. + +An interpolated name is not this class. `mise run ${{ matrix.task }}` is not \ +decidable from a committed document, so it abstains rather than being refused. +""" + +[[verdict.route]] +id = "module read first" +kind = "document" +target = "policy/task-callable.rego" + +[[verdict.route]] +id = "task list first" +kind = "command" +target = "mise tasks ls --all" + [[verdict]] id = "workflow read unread" gloss = "the workflow this rule judges would not parse, so nothing was decided" diff --git a/crates/batten/tests/it/main.rs b/crates/batten/tests/it/main.rs index 18866a1ca..6ea9d5f96 100644 --- a/crates/batten/tests/it/main.rs +++ b/crates/batten/tests/it/main.rs @@ -279,6 +279,7 @@ mod surface; mod symbols; mod target_consolidation; mod target_prune; +mod task_callable; mod task_prose; mod task_receipt; mod task_registry; diff --git a/crates/batten/tests/it/task_callable.rs b/crates/batten/tests/it/task_callable.rs new file mode 100644 index 000000000..44007ab71 --- /dev/null +++ b/crates/batten/tests/it/task_callable.rs @@ -0,0 +1,300 @@ +//! `workflow run unknown` over the compiled binary (CLOUD-1833). +//! +//! # Why this tier exists and the module's own `test_` rules do not suffice +//! +//! `policy/task-callable.rego` carries eleven load-time cases and every one of +//! them fabricates its input with `with input as`. That is the shape +//! `rules/policy-modules.md` warns about: such a case asserts over a fact the +//! engine may be unable to produce, so the module stays green while the row +//! decides nothing on any real checkout. Two live instances of exactly that +//! class have already landed in this repository, and both were found by adding +//! this tier rather than by reading. +//! +//! The risk is concrete here rather than theoretical. This module reads three +//! keys the engine must actually build over a `sources`/`line_sources` pair: +//! `input.tree.documents[].jobs`, `input.tree.lines[]` for +//! the pointer, and `input.tree.tracked` for the file-task arm. A glob that +//! acquired the manifest but not the workflows — or the reverse — would leave +//! the predicate undefined and the gate byte-identical to a clean tree. +//! +//! # No retirement ledger, because nothing is retired +//! +//! `shell retire partial` reads `// carried:` arms in a tier that inherits a +//! dying program's cases. This module has no predecessor: it is a new clause +//! over a population nothing was judging, so there are no arms to account for +//! and inventing some would claim a fidelity this change never owed. + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use common::{git_in, init_repo, run, scratch, write}; + +/// The authority every case registers, verbatim in the shape `batten.toml` +/// carries it — the same `sources`/`line_sources` pair, because the pair is +/// precisely what this tier exists to prove the engine acquires. +const CONFIG: &str = r#"version = 1 +scope = ["**"] + +[[pattern]] +id = "mise-run-task" +regex = 'mise run [a-z][a-z0-9:_-]*' + +[[verdict]] +id = "task run unknown" +gloss = "a workflow step runs a mise task that nothing in this tree defines" +class = "The step dies the moment it runs, on a trigger that reaches no reviewer." + +[[verdict.route]] +id = "task list first" +kind = "command" +target = "mise tasks ls --all" + +[[verdict]] +id = "workflow read unread" +gloss = "the workflow this rule judges would not parse, so nothing was decided" +class = "A declared source that will not parse is not an absent one." + +[[verdict.route]] +id = "module read first" +kind = "document" +target = "policy/task-callable.rego" + +[[verdict]] +id = "task resolve missing" +gloss = "the task graph this rule walks could not be resolved" +class = "With no task namespace there is nothing to judge a caller against." + +[[verdict.route]] +id = "module read first" +kind = "document" +target = "policy/task-callable.rego" + +[[rule]] +id = "workflow run unknown" +kind = "policy" +scope = "tree" +sources = [".github/workflows/*.yml", "mise.toml"] +line_sources = [".github/workflows/*.yml"] +module = "policy/task-callable.rego" +severity = "deny" +"#; + +/// A repository registering the real module against the manifest, the file +/// programs and the workflow BODY a case wants judged. +/// +/// The REAL module and the REAL row, never a fixture copy: the whole point of +/// this tier is that the module decides over the engine's own projection, and a +/// stand-in would be one more `with input as` wearing a different costume. +fn repo_with(name: &str, tasks: &[&str], programs: &[&str], workflow: &str) -> std::path::PathBuf { + let dir = scratch(&format!("task-callable-{name}")); + let module = std::fs::read_to_string("../../policy/task-callable.rego") + .expect("the module this tier exists for"); + write(&dir, "policy/task-callable.rego", &module); + + let mut manifest = String::from("[tools]\n\n"); + for task in tasks { + manifest.push_str("[tasks."); + manifest.push_str(task); + manifest.push_str("]\nrun = \"true\"\n\n"); + } + write(&dir, "mise.toml", &manifest); + + for program in programs { + write( + &dir, + &format!("mise-tasks/{program}"), + "#!/usr/bin/env bash\n", + ); + } + + write(&dir, ".github/workflows/probe.yml", workflow); + write(&dir, "batten.toml", CONFIG); + + init_repo(&dir); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-qm", "register the module"]); + dir +} + +/// The ordinary case: one scheduled job whose steps are the given `run:` lines. +/// +/// `schedule` rather than `pull_request` deliberately — it is the trigger every +/// one of the five dead callers sat behind, and the reason none of them was ever +/// reported. +fn repo(name: &str, tasks: &[&str], programs: &[&str], steps: &[&str]) -> std::path::PathBuf { + let mut workflow = String::from( + "on:\n schedule:\n - cron: \"0 0 * * 1\"\njobs:\n probe:\n runs-on: ubuntu-latest\n steps:\n", + ); + for step in steps { + workflow.push_str(" - run: "); + workflow.push_str(step); + workflow.push('\n'); + } + repo_with(name, tasks, programs, &workflow) +} + +/// Both streams, because which one a finding lands on is the output contract's +/// business rather than this tier's: what is asserted is that the pointer +/// reaches the reader at all. +fn said(decided: &std::process::Output) -> String { + format!( + "{}{}", + String::from_utf8_lossy(&decided.stdout), + String::from_utf8_lossy(&decided.stderr) + ) +} + +#[test] +fn a_workflow_step_naming_an_undefined_task_is_refused() { + // THE CASE NO `with input as` CAN REACH. Measured on this branch before the + // module landed: five steps across four workflows called tasks that do not + // exist, every one of them on a trigger that reaches no pull request, and + // nothing in the tree could say so. + let dir = repo( + "dangling", + &["present"], + &[], + &["mise run absent-task", "mise run present"], + ); + + let decided = run(&dir, &["check"]); + assert_eq!( + decided.status.code(), + Some(2), + "a caller naming no task decides\n{}", + String::from_utf8_lossy(&decided.stderr) + ); + let reported = said(&decided); + assert!( + reported.contains(".github/workflows/probe.yml"), + "the finding points at the workflow to repair\n{reported}" + ); +} + +#[test] +fn a_task_backed_by_a_nested_program_resolves() { + // `mise-tasks/render/cli.sh` is `mise run render:cli`, which + // `release-artifacts.yml` calls. A reading that took the last path component + // alone would resolve it as `cli`, leave `render:cli` undefined, and refuse a + // caller that works — a false positive on the gate's very first run, which is + // the failure mode that gets an exception written and then rots. + let dir = repo("nested", &[], &["render/cli.sh"], &["mise run render:cli"]); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "a task backed by a nested program resolves\n{}", + String::from_utf8_lossy(&quiet.stderr) + ); +} + +#[test] +fn a_task_backed_by_a_flat_program_resolves_by_stem_and_by_filename() { + // Both spellings, because mise accepts either and this repository's callers + // use the bare stem. A reading that covered only one would refuse half the + // corpus. + let dir = repo( + "programs", + &[], + &["checksums.sh"], + &["mise run checksums", "mise run checksums.sh"], + ); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "a file task resolves under either spelling\n{}", + String::from_utf8_lossy(&quiet.stderr) + ); +} + +#[test] +fn an_interpolated_task_name_abstains_rather_than_being_refused() { + // NOT DECIDABLE FROM A COMMITTED DOCUMENT — the name is not in the string. A + // gate that refused here would make the one spelling a matrix job needs + // unwritable, and the abstention is structural: the shared pattern requires a + // lowercase letter where the name begins, so the fragment never matches. + let dir = repo( + "interpolated", + &["present"], + &[], + &["mise run ${{ matrix.task }}", "mise run present"], + ); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "an interpolated name is not judged\n{}", + String::from_utf8_lossy(&quiet.stderr) + ); +} + +#[test] +fn a_tree_whose_callers_all_resolve_is_silent() { + // THE ANTI-VACUITY ARM, and on this tier it carries more than usual: a module + // whose keys the engine never built would pass every negative case above for + // the wrong reason. Paired with the refusal case at the top, the two together + // are what distinguish a live gate from an absent one. + let dir = repo( + "clean", + &["present"], + &["checksums.sh"], + &["mise run present && mise run checksums"], + ); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "a tree whose every caller resolves is clean\n{}", + String::from_utf8_lossy(&quiet.stderr) + ); +} + +#[test] +fn the_finding_carries_the_line_the_caller_is_written_on() { + // `line_sources` IS A SEPARATE ACQUISITION FROM `sources`, so the pointer is + // the half most likely to be silently absent: a module whose line index never + // filled still refuses, just without saying where. Rule 4's shape is a + // `path:line`, and this is what holds the glob to it. + let dir = repo( + "pointer", + &["present"], + &[], + &["mise run present", "mise run absent-task"], + ); + + let decided = run(&dir, &["check"]); + let reported = said(&decided); + assert!( + reported.contains(".github/workflows/probe.yml:9"), + "the finding places the caller on its own line\n{reported}" + ); +} + +#[test] +fn a_comment_naming_an_absent_task_is_not_judged() { + // THE PROSE ARM, and it is why the decision reads the parsed `run:` scalar + // rather than the lines. These files carry long comments naming tasks in + // order to explain that they are ABSENT — `timeout-drift.yml:7` is one — and + // a gate that fires on its own documentation is a gate people delete. + let dir = repo_with( + "prose", + &["present"], + &[], + "on:\n schedule:\n - cron: \"0 0 * * 1\"\n# The commit half is `mise run absent-task`, in the hk gate.\njobs:\n probe:\n runs-on: ubuntu-latest\n steps:\n - run: mise run present\n", + ); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "a comment explaining an absent task does not fire the gate\n{}", + String::from_utf8_lossy(&quiet.stderr) + ); +} diff --git a/mise.toml b/mise.toml index bd378a2ad..2d4480b29 100644 --- a/mise.toml +++ b/mise.toml @@ -617,7 +617,7 @@ CI_FANIN_WORKFLOW = ".github/workflows/ci.yml" BATS_TEST_TIMEOUT = "300" REGORUS_OPA_COMPLIANCE = "1.2.0" REGORUS_OPA_COMPLIANCE_FOR = "0.11" -MUTANT_GATES = "mise,attestation,timeout-drift,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure-check,evaluator-io-check,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link-check,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus-check,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" +MUTANT_GATES = "mise,attestation,timeout-drift,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure-check,evaluator-io-check,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link-check,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-callable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus-check,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" # --- GitHub reachability behind an egress proxy (Claude Code web sandbox etc.) --- # mise resolves every tool's release through GitHub's *API* host, api.github.com. @@ -3793,6 +3793,30 @@ run = "mise run reclaim-census record-boot >/dev/null 2>&1 || true; v=$(mise run description = "Gate: mise.lock has no partial or bogus platform entry — a pure function of the committed lockfile, no network, no write" run = "cargo run --quiet -p batten -- check --rule 'lock cover partial'" +# CLOUD-1833. The engine, with the arguments the caller gives — the half of +# `batten-check` that is not the consumer-#1 gate. +# +# IT EXISTS BECAUSE THREE COMMITTED WORKFLOWS ALREADY CALL IT AND IT HAS NEVER +# EXISTED. `branch-hygiene.yml`, `land-divergence.yml` and `nonverdict-rate.yml` +# each run `mise run batten -- check --rule …`; the spelling was invented once, +# copied twice, and read back by the next author as a precedent. All three die at +# the step, and all three fire only on `schedule` or `workflow_run`, so no pull +# request has ever seen it. `workflow run unknown` is the clause that found them +# and is what keeps the fourth from being written. +# +# A PASSTHROUGH RATHER THAN A TASK PER RULE, and `lock-complete`'s comment above +# is the reason to say why. Its drift argument is about a TRIPLE — an hk step +# name, a task, and a rule id, which hk selects between by name and which can +# therefore disagree. A workflow step names the task directly, so there is no +# second selector to drift against, and four near-identical wrapper rows would be +# the duplication rather than the guard. +# +# `cargo run` for `batten-check`'s reason: the gate judges the working tree's +# engine and config together, the pair that ships. +[tasks.batten] +description = "Run the working tree's engine with the given arguments, so a workflow step can select one rule" +run = "cargo run --quiet -p batten --" + [tasks.batten-check] description = "Consumer #1: evaluate the committed batten.toml with batten's own engine against this repository" # `cargo run` rather than an installed binary, so the gate always judges the diff --git a/policy/task-callable.rego b/policy/task-callable.rego new file mode 100644 index 000000000..96771f603 --- /dev/null +++ b/policy/task-callable.rego @@ -0,0 +1,393 @@ +# A workflow step naming a `mise run ` resolves to a task this tree +# defines (CLOUD-1833). +# +# THE DEFECT, AND WHY NOTHING SAW IT. A retirement deletes a program; no clause +# asked whether its CALLERS still resolve. Measured on this branch at the moment +# the row was written, five scheduled steps across four workflows named tasks +# that do not exist: +# +# * `release-assets.yml` ran `mise run attestation-check`, deleted by +# CLOUD-1717's own earlier commit. +# * `timeout-drift.yml` ran `mise run timeout-drift`, deleted by the same +# campaign. +# * `branch-hygiene.yml`, `land-divergence.yml` and `nonverdict-rate.yml` each +# ran `mise run batten -- check --rule …`, and there has never been a +# `[tasks.batten]` at all. That spelling was invented once, copied into three +# files, and then read back by the next author as an established precedent — +# which is the more interesting half, because nothing about a wrong caller +# decays over time. It was wrong on the commit that introduced it. +# +# EVERY ONE IS SILENT, WHICH IS WHAT MAKES THIS A DENY RATHER THAN A REPORT. Each +# of those steps fires on `workflow_run`, `schedule` or `workflow_dispatch`, so +# it reaches no pull request and no reviewer: `ci-local-parity` holds its +# properties over `pull_request` workflows and `land` watches a PR's check-runs. +# A workflow failing on 100% of its invocations is indistinguishable from one +# that has never fired — the composition `release-assets.yml` has now paid for +# three times (CLOUD-258, CLOUD-1777, and this row). +# +# THE THIRD POPULATION OF ONE PREDICATE, AND THAT IS THE DESIGN RATHER THAN +# DUPLICATION. `command-task-defined` asks "does this task exist" of a `[[rule]]` +# row's `check`; `verdict-routes-resolve` asks it of a `[[verdict.route]]`'s +# `target`; this asks it of a workflow step's `run`. Each owns its population, +# and they share the `defined` shape deliberately, for the reason +# `verdict-routes-resolve`'s header already states: two spellings of "what tasks +# exist" is the drift a shared question does not survive. +# +# THE PARSED `run:` SCALAR DECIDES; THE LINES ONLY PLACE THE POINTER. These files +# carry long comments that name tasks in order to explain why they are ABSENT — +# `timeout-drift.yml:7` is one — and a gate that fires on its own documentation +# is a gate people delete. That is `ci-parity`'s rule and it binds here; the line +# index is read afterwards, for a task the parsed reading has already decided on, +# which is rule 4's shape: the finding carries `{path, line}` and never the +# command's text. +# +# AN INTERPOLATED TASK NAME ABSTAINS STRUCTURALLY, AND NO GUARD IS WRITTEN FOR +# IT. `mise run ${{ matrix.task }}` and `mise run "$TASK"` are not decidable from +# a committed document — the name is not in the string — and the shared +# `mise-run-task` pattern requires a lowercase letter where the name begins, so +# neither form matches in the first place. A conjunct excluding them would be +# excluded by the pattern before it ran, which is a surviving mutant rather than +# a safeguard. +# +# THE MANIFEST IS READ INLINE AT EACH USE SITE, NEVER BOUND TO A TOP-LEVEL RULE. +# `ci-cache-declared`'s header carries the measurement: a module holding +# `manifest := input.tree.documents["mise.toml"]` goes ENTIRELY silent — every +# predicate, including one whose body is `true` — because this manifest declares +# a task named `deny` and `data.batten.deny` is the composed set the engine +# reads. `defined` below is a set of STRINGS for that reason, and it is why this +# module can bind one at all. +# +#MUTANT-SUITE crates/batten/tests/it/task_callable.rs +#MUTANT dangling-caller-passes|s@\tnot defined\[task\]@\tfalse@|a_workflow_step_naming_an_undefined_task_is_refused +#MUTANT nested-task-unreachable|s@\tname := replace(rel, "/", ":")@\tname := rel@|a_task_backed_by_a_nested_program_resolves + +# METADATA +# description: | +# Bound to the TREE surface: `scope = "tree"`, so it reads the tree document +# and never the mediated `{call, facts}` shape. +# THE BRACKETS ARE NOT STYLE: the schema file carries a hyphen, so the dotted +# form is a parse error reported as `invalid schema reference`. +# THIS BLOCK IS YAML AND MUST STAY THE LAST COMMENT BLOCK BEFORE `package`. +# schemas: +# - input: schema["policy-input.schema"] +package batten.task_callable + +import rego.v1 + +rules contains "workflow run unknown" + +# --------------------------------------------------------------------------- +# What is being judged, and whether there is anything to judge. +# --------------------------------------------------------------------------- + +workflow[path] := doc if { + some path, doc in input.tree.documents + is_object(doc.jobs) +} + +# The guard `hk-fix-selection` measured the need for: a tree with no workflow +# declaring jobs is answering for nothing here, and an unguarded module reports +# against a fixture that carries a copy of the config and none of its subjects. +governed if count(object.keys(workflow)) > 0 + +# --------------------------------------------------------------------------- +# Tasks this tree defines. The same two sources `command-task-defined` and +# `verdict-routes-resolve` read, in the same shape. +# --------------------------------------------------------------------------- + +uses_this_runner if input.tree.documents["mise.toml"] + +defined contains name if { + some name, _ in input.tree.documents["mise.toml"].tasks +} + +# A FILE TASK'S NAME IS ITS PATH UNDER `mise-tasks/` WITH THE SEPARATOR SPELLED +# `:`, AND THE NESTED ARM IS NOT DECORATION. `mise-tasks/render/cli.sh` is +# `mise run render:cli`, which `release-artifacts.yml` calls. A reading that took +# only the last path component would resolve it as `cli`, leave `render:cli` +# undefined, and refuse a caller that works — a false positive on the first run, +# which is the failure mode `verdict-routes-resolve`'s own header says gets an +# exception written for it, and the exception is what rots. +defined contains name if { + uses_this_runner + some path in input.tree.tracked + startswith(path, "mise-tasks/") + rel := substring(path, count("mise-tasks/"), -1) + name := replace(rel, "/", ":") +} + +# Both spellings, because mise accepts the task with or without the program's +# extension and this repository's callers use the bare stem. +defined contains stem if { + uses_this_runner + some path in input.tree.tracked + startswith(path, "mise-tasks/") + rel := substring(path, count("mise-tasks/"), -1) + name := replace(rel, "/", ":") + contains(name, ".") + stem := split(name, ".")[0] +} + +# --------------------------------------------------------------------------- +# The callers, from the parsed document. +# --------------------------------------------------------------------------- + +# A PARTIAL SET RATHER THAN A FUNCTION, for `ci-parity`'s reason: a step runs +# many tasks, and a Rego function binding more than one output faults at +# evaluation rather than returning them. +step_task contains [path, task] if { + some path, _ in workflow + some name, _ in workflow[path].jobs + some step in workflow[path].jobs[name].steps + some fragment in regex.find_n(data.batten.patterns["mise-run-task"], step.run, -1) + task := split(fragment, " ")[2] +} + +# NOT A FUNCTION, AND THAT IS CORRECTNESS RATHER THAN STYLE — `ci-cache-declared` +# states it at the same shape. A `pointer(path, line_of(path))` spelling makes +# the whole refusal undefined whenever the line index cannot place the subject, +# because Rego propagates undefined through an argument, so a `line_sources` glob +# that drifted would switch the gate off silently. As a set, a line that cannot +# be placed costs the LINE and never the finding. +task_line contains [path, task, number] if { + some [path, task] in step_task + some index, line in input.tree.lines[path] + some fragment in regex.find_n(data.batten.patterns["mise-run-task"], line, -1) + split(fragment, " ")[2] == task + number := index + 1 +} + +placed(path, task) if { + some placement in task_line + placement[0] == path + placement[1] == task +} + +# --------------------------------------------------------------------------- +# The refusals. +# --------------------------------------------------------------------------- + +violation contains { + "rule": "workflow run unknown", + "verdict": "task run unknown", + "subjects": [{"path": path, "line": number}, {"artifact": task}], +} if { + governed + + # Could-not-look guard, `command-task-defined`'s: with no task namespace there + # is nothing to judge against, and reporting there makes the rule fire on + # every tree that merely holds a copy of this config. + count(defined) > 0 + some [path, task] in step_task + not defined[task] + some placement in task_line + placement[0] == path + placement[1] == task + number := placement[2] +} + +# The path-only arm, for a caller the line index could not place. +violation contains { + "rule": "workflow run unknown", + "verdict": "task run unknown", + "subjects": [{"path": path}, {"artifact": task}], +} if { + governed + count(defined) > 0 + some [path, task] in step_task + not defined[task] + not placed(path, task) +} + +# COULD NOT LOOK, NEVER A SILENT PASS. A declared source that would not parse is +# in `input.tree.missing` rather than merely absent from `documents`, and a +# module that iterated only `documents` would report green over a file it never +# read. +# +# TWO CLASSES RATHER THAN ONE, because the two absences are not the same finding +# and a reader acts on them differently. An unreadable manifest makes EVERY task +# read as undefined, so the whole gate abstained; an unreadable workflow leaves +# the rest of the tree judged and only that file's callers unexamined. Collapsing +# them would put a total abstention and a single unread file under one name. +# +# BOTH CLASSES ARE THE ONES `ci-cache-declared` ALREADY RAISES over the same two +# declared sources, and reusing them is the point rather than a shortcut: this +# module reads the same manifest for the same reason, so a reader who has met +# `task resolve missing` once should not have to learn a second name for it. +violation contains { + "rule": "workflow run unknown", + "verdict": "workflow read unread", + "subjects": [{"path": path}], +} if { + some path, _ in input.tree.missing + endswith(path, ".yml") +} + +violation contains { + "rule": "workflow run unknown", + "verdict": "task resolve missing", + "subjects": [{"path": path}], +} if { + some path, _ in input.tree.missing + path == "mise.toml" +} + +# --- cases ----------------------------------------------------------------- + +test_a_workflow_step_naming_an_undefined_task_is_refused if { + found := violation with input as {"tree": { + "documents": { + "mise.toml": {"tasks": {"present": {}}}, + ".github/workflows/w.yml": {"jobs": {"j": {"steps": [{"run": "mise run absent-task"}]}}}, + }, + "lines": {".github/workflows/w.yml": [" run: mise run absent-task"]}, + "tracked": ["mise.toml"], + "missing": {}, + }} + count(found) == 1 +} + +test_a_workflow_step_naming_a_defined_task_is_clean if { + found := violation with input as {"tree": { + "documents": { + "mise.toml": {"tasks": {"present": {}}}, + ".github/workflows/w.yml": {"jobs": {"j": {"steps": [{"run": "mise run present"}]}}}, + }, + "lines": {".github/workflows/w.yml": [" run: mise run present"]}, + "tracked": ["mise.toml"], + "missing": {}, + }} + count(found) == 0 +} + +# A task backed by a file program rather than a `[tasks.…]` table resolves. +test_a_task_backed_by_a_program_resolves if { + found := violation with input as {"tree": { + "documents": { + "mise.toml": {"tasks": {}}, + ".github/workflows/w.yml": {"jobs": {"j": {"steps": [{"run": "mise run checksums"}]}}}, + }, + "lines": {".github/workflows/w.yml": [" run: mise run checksums"]}, + "tracked": ["mise.toml", "mise-tasks/checksums.sh"], + "missing": {}, + }} + count(found) == 0 +} + +# THE NESTED ARM'S OWN CASE. `mise-tasks/render/cli.sh` is `mise run render:cli`, +# and a reading that took the last path component alone would refuse it. +test_a_task_backed_by_a_nested_program_resolves if { + found := violation with input as {"tree": { + "documents": { + "mise.toml": {"tasks": {}}, + ".github/workflows/w.yml": {"jobs": {"j": {"steps": [{"run": "mise run render:cli"}]}}}, + }, + "lines": {".github/workflows/w.yml": [" run: mise run render:cli"]}, + "tracked": ["mise.toml", "mise-tasks/render/cli.sh"], + "missing": {}, + }} + count(found) == 0 +} + +# THE PROSE ARM. A comment naming a task in order to explain that it is ABSENT +# must not fire the gate — the parsed `run:` scalar is what decides. +test_a_comment_naming_an_absent_task_is_not_judged if { + found := violation with input as {"tree": { + "documents": { + "mise.toml": {"tasks": {"present": {}}}, + ".github/workflows/w.yml": {"jobs": {"j": {"steps": [{"run": "mise run present"}]}}}, + }, + "lines": {".github/workflows/w.yml": [ + "# The commit half is `mise run absent-task`, in the hk gate.", + " run: mise run present", + ]}, + "tracked": ["mise.toml"], + "missing": {}, + }} + count(found) == 0 +} + +# AN INTERPOLATED NAME ABSTAINS, and this case is what makes that claim earn +# itself rather than being asserted in the header. +test_an_interpolated_task_name_is_not_judged if { + found := violation with input as {"tree": { + "documents": { + "mise.toml": {"tasks": {"present": {}}}, + ".github/workflows/w.yml": {"jobs": {"j": {"steps": [{"run": "mise run ${{ matrix.task }}"}]}}}, + }, + "lines": {".github/workflows/w.yml": [" run: mise run ${{ matrix.task }}"]}, + "tracked": ["mise.toml"], + "missing": {}, + }} + count(found) == 0 +} + +# A tree with no workflow declaring jobs is answering for nothing. +test_a_tree_with_no_workflow_is_not_judged if { + found := violation with input as {"tree": { + "documents": {"mise.toml": {"tasks": {"present": {}}}}, + "lines": {}, + "tracked": ["mise.toml"], + "missing": {}, + }} + count(found) == 0 +} + +test_an_unreadable_manifest_is_loud if { + found := violation with input as {"tree": { + "documents": {}, + "lines": {}, + "tracked": [], + "missing": {"mise.toml": "unparsed"}, + }} + count(found) == 1 +} + +# The two could-not-look arms are distinct classes, not one spelled twice — and a +# case counting the findings cannot see that. This one reads the verdicts. +test_the_two_could_not_look_classes_stay_distinct if { + found := violation with input as {"tree": { + "documents": {}, + "lines": {}, + "tracked": [], + "missing": { + "mise.toml": "unparsed", + ".github/workflows/w.yml": "unparsed", + }, + }} + {entry.verdict | some entry in found} == {"task resolve missing", "workflow read unread"} +} + +# The other could-not-look arm, reaching the same rule by a second route — which +# is what proves the two classes stay distinct through projection rather than +# collapsing into whichever one the first case happened to spell. +test_an_unreadable_workflow_is_loud if { + found := violation with input as {"tree": { + "documents": {"mise.toml": {"tasks": {"present": {}}}}, + "lines": {}, + "tracked": ["mise.toml"], + "missing": {".github/workflows/w.yml": "unparsed"}, + }} + count(found) == 1 +} + +# The anti-vacuity arm: a tree whose every caller resolves must be silent, or a +# module that refused nothing would pass every negative case above. +test_a_tree_whose_callers_resolve_is_silent if { + found := violation with input as {"tree": { + "documents": { + "mise.toml": {"tasks": {"present": {}}}, + ".github/workflows/w.yml": {"jobs": {"j": {"steps": [ + {"run": "mise run present"}, + {"run": "mise run checksums && mise run render:cli"}, + ]}}}, + }, + "lines": {".github/workflows/w.yml": [ + " run: mise run present", + " run: mise run checksums && mise run render:cli", + ]}, + "tracked": ["mise.toml", "mise-tasks/checksums.sh", "mise-tasks/render/cli.sh"], + "missing": {}, + }} + count(found) == 0 +} From 334c109b8a819fdf3708509b925525155fd399f6 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Fri, 18 Sep 2026 07:56:09 +0000 Subject: [PATCH 14/32] feat(show): the engine answers what content is, so a manifest stops meaning sha256sum MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `batten show address` reads paths on stdin and writes one `\t
` line apiece, where the address is `identity::ContentAddress` — the tree's existing answer to "what is this content", already carried by the capture store and by `DocumentInput`. ## Why a verb rather than leaving `sha256sum` alone `mise-tasks/checksums.sh` builds the release manifest by shelling `sha256sum`, and three separate things follow from that. It is the prerequisite for retiring that program onto the engine, so the verb lands first and on its own. **The digest is bare.** Sixty-four hex characters that name no domain and no version, so a manifest entry and any other sha256 in the tree are the same string, and nothing stops one computed for one purpose being compared against one computed for another. `ContentAddress` is domain-separated (`b3\0`, the version, the domain tag, the length) and renders as `b3--<64 hex>`, so the comparison `policy/release-assets.rego` will perform is a comparison of like with like — and a future change of domain or version is visible rather than silent. **The spelling differs per platform.** `sha256sum` on GNU, `shasum -a 256` on macOS. That is a second authority over one answer, which is the class `rules/policy-modules.md` refuses for parsers and `ci-parity`'s header refuses for the task graph. The engine already owns this answer. **The format was private.** Only that program wrote it and only that program read it. A manifest the engine can also produce is one a module can be handed. ## `read`, structurally `Effect::Read`, `data_channel: true`, in both of `spec.rs`'s read-only allowlists, and `surface::tests::every_leaf_under_show_is_read` is what keeps a writer from ever reaching that list through the noun's own `read`. The handler opens the paths it is given and writes to `out`; there is no path from it to a mutation or a spawn. ## An unreadable path refuses, and that is the load-bearing decision Exit 1, nothing written — never a partial manifest. Skipping the line would produce a manifest that is SHORT rather than absent, and a short manifest compares equal to a release that is genuinely missing that asset: a silent pass on precisely the question the manifest exists to answer. Exit 1 rather than 2 because §7 spends `2` on the policy verdict, and "this path is not there" is a claim about the invocation rather than about the tree. ## The cases `crates/batten/tests/it/show_address.rs`, five over the compiled binary. Two of them exist because the happy path cannot see what they check: a verb emitting a bare digest passes any case that only checks two files differ, and one that skipped an unreadable path passes any case that only checks a good one. So the address SHAPE and the failure DIRECTION are asserted directly, alongside content-addressing (identical bytes at two names get one address), and both channels carrying one reading. `show` is a noun over two leaves now rather than one, and `show_of`'s comment said "one leaf today" — corrected rather than left to go stale. Man pages and completions regenerated; `surface::the_committed_artifacts_are_exactly_the_ones_the_surface_declares` and `the_committed_completions_are_the_ones_the_binary_emits` hold both. Refs: CLOUD-1717 --- completions/batten.bash | 73 +++++++++- completions/batten.fish | 57 +++++--- completions/batten.zsh | 63 ++++++++- crates/batten/src/cli.rs | 14 +- crates/batten/src/lib.rs | 60 +++++++++ crates/batten/src/spec.rs | 2 + crates/batten/src/surface.rs | 9 ++ crates/batten/tests/it/main.rs | 1 + crates/batten/tests/it/show_address.rs | 179 +++++++++++++++++++++++++ man/batten-show-address.1 | 16 +++ man/batten-show.1 | 3 + 11 files changed, 452 insertions(+), 25 deletions(-) create mode 100644 crates/batten/tests/it/show_address.rs create mode 100644 man/batten-show-address.1 diff --git a/completions/batten.bash b/completions/batten.bash index 7853d4d72..171eba020 100644 --- a/completions/batten.bash +++ b/completions/batten.bash @@ -814,6 +814,9 @@ _batten() { batten__subcmd__help__subcmd__semver,check) cmd="batten__subcmd__help__subcmd__semver__subcmd__check" ;; + batten__subcmd__help__subcmd__show,address) + cmd="batten__subcmd__help__subcmd__show__subcmd__address" + ;; batten__subcmd__help__subcmd__show,agent) cmd="batten__subcmd__help__subcmd__show__subcmd__agent" ;; @@ -1336,12 +1339,18 @@ _batten() { batten__subcmd__semver__subcmd__help,help) cmd="batten__subcmd__semver__subcmd__help__subcmd__help" ;; + batten__subcmd__show,address) + cmd="batten__subcmd__show__subcmd__address" + ;; batten__subcmd__show,agent) cmd="batten__subcmd__show__subcmd__agent" ;; batten__subcmd__show,help) cmd="batten__subcmd__show__subcmd__help" ;; + batten__subcmd__show__subcmd__help,address) + cmd="batten__subcmd__show__subcmd__help__subcmd__address" + ;; batten__subcmd__show__subcmd__help,agent) cmd="batten__subcmd__show__subcmd__help__subcmd__agent" ;; @@ -5643,7 +5652,7 @@ _batten() { return 0 ;; batten__subcmd__help__subcmd__show) - opts="agent" + opts="address agent" if [[ ${cur} == -* || ${COMP_CWORD} -eq 3 ]] ; then COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 @@ -5656,6 +5665,20 @@ _batten() { COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 ;; + batten__subcmd__help__subcmd__show__subcmd__address) + opts="" + if [[ ${cur} == -* || ${COMP_CWORD} -eq 4 ]] ; then + COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) + return 0 + fi + case "${prev}" in + *) + COMPREPLY=() + ;; + esac + COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) + return 0 + ;; batten__subcmd__help__subcmd__show__subcmd__agent) opts="" if [[ ${cur} == -* || ${COMP_CWORD} -eq 4 ]] ; then @@ -9861,7 +9884,7 @@ _batten() { return 0 ;; batten__subcmd__show) - opts="-q -v -y -h --strictness --fail-on-warning --config-from --config-in --silent --quiet --verbose --debug --trace --log-level --no-color --no-input --yes --help agent help" + opts="-q -v -y -h --strictness --fail-on-warning --config-from --config-in --silent --quiet --verbose --debug --trace --log-level --no-color --no-input --yes --help address agent help" if [[ ${cur} == -* || ${COMP_CWORD} -eq 2 ]] ; then COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 @@ -9890,6 +9913,36 @@ _batten() { COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 ;; + batten__subcmd__show__subcmd__address) + opts="-J -q -v -y -h --json --strictness --fail-on-warning --config-from --config-in --silent --quiet --verbose --debug --trace --log-level --no-color --no-input --yes --help" + if [[ ${cur} == -* || ${COMP_CWORD} -eq 3 ]] ; then + COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) + return 0 + fi + case "${prev}" in + --strictness) + COMPREPLY=($(compgen -W "permissive standard strict" -- "${cur}")) + return 0 + ;; + --config-from) + COMPREPLY=($(compgen -f "${cur}")) + return 0 + ;; + --config-in) + COMPREPLY=($(compgen -f "${cur}")) + return 0 + ;; + --log-level) + COMPREPLY=($(compgen -W "silent quiet normal verbose debug trace" -- "${cur}")) + return 0 + ;; + *) + COMPREPLY=() + ;; + esac + COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) + return 0 + ;; batten__subcmd__show__subcmd__agent) opts="-J -q -v -y -h --json --strictness --fail-on-warning --config-from --config-in --silent --quiet --verbose --debug --trace --log-level --no-color --no-input --yes --help" if [[ ${cur} == -* || ${COMP_CWORD} -eq 3 ]] ; then @@ -9921,7 +9974,7 @@ _batten() { return 0 ;; batten__subcmd__show__subcmd__help) - opts="agent help" + opts="address agent help" if [[ ${cur} == -* || ${COMP_CWORD} -eq 3 ]] ; then COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 @@ -9934,6 +9987,20 @@ _batten() { COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 ;; + batten__subcmd__show__subcmd__help__subcmd__address) + opts="" + if [[ ${cur} == -* || ${COMP_CWORD} -eq 4 ]] ; then + COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) + return 0 + fi + case "${prev}" in + *) + COMPREPLY=() + ;; + esac + COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) + return 0 + ;; batten__subcmd__show__subcmd__help__subcmd__agent) opts="" if [[ ${cur} == -* || ${COMP_CWORD} -eq 4 ]] ; then diff --git a/completions/batten.fish b/completions/batten.fish index d591d2328..9e014a8a3 100644 --- a/completions/batten.fish +++ b/completions/batten.fish @@ -3245,29 +3245,52 @@ complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_su complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from help" -f -a "plan" -d 'Record this branch\'s plan, read as ` ` lines on stdin' complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from help" -f -a "closes" -d 'Record which rows this branch\'s pull request body closes, read on stdin' complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l strictness -d 'Raise how strictly gates apply (an override may only tighten policy)' -r -f -a "permissive\t'Advisory: findings are reported without failing the run' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l strictness -d 'Raise how strictly gates apply (an override may only tighten policy)' -r -f -a "permissive\t'Advisory: findings are reported without failing the run' standard\t'The default: a finding is a violation' strict\t'Everything `Standard` fails on, plus anything advisory'" -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l config-from -d 'Read the committed config from a git ref (e.g. origin/main) instead of the working tree' -r -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l config-in -d 'Read the committed config from this directory instead of the directory being judged' -r -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l log-level -d 'Set the verbosity rung by name' -r -f -a "silent\t'Say nothing but a verdict or a usage error' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l config-from -d 'Read the committed config from a git ref (e.g. origin/main) instead of the working tree' -r +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l config-in -d 'Read the committed config from this directory instead of the directory being judged' -r +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l log-level -d 'Set the verbosity rung by name' -r -f -a "silent\t'Say nothing but a verdict or a usage error' quiet\t'Suppress ordinary progress; keep warnings' normal\t'The default' verbose\t'Explain what is being checked' debug\t'Add resolution detail' trace\t'Add everything'" -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l fail-on-warning -d 'Promote a warn-severity finding to a violation (an override may only turn this on)' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l silent -d 'Say nothing but a verdict or a usage error' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -s q -l quiet -d 'Suppress ordinary progress (repeatable: -qq is silent)' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -s v -l verbose -d 'Explain what is being checked (repeatable: -vv is debug)' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l debug -d 'Add resolution detail' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l trace -d 'Add everything' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l no-color -d 'Never colour stderr, whatever it is attached to' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l no-input -d 'Never prompt; treat the run as unattended' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -s y -l yes -d 'Confirm a destructive operation that would otherwise refuse' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -s h -l help -d 'Print help (see more with \'--help\')' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -f -a "agent" -d 'What an agent may do in this repository: the read-only verbs, the exit contract, and the declared gates' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l fail-on-warning -d 'Promote a warn-severity finding to a violation (an override may only turn this on)' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l silent -d 'Say nothing but a verdict or a usage error' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -s q -l quiet -d 'Suppress ordinary progress (repeatable: -qq is silent)' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -s v -l verbose -d 'Explain what is being checked (repeatable: -vv is debug)' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l debug -d 'Add resolution detail' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l trace -d 'Add everything' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l no-color -d 'Never colour stderr, whatever it is attached to' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l no-input -d 'Never prompt; treat the run as unattended' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -s y -l yes -d 'Confirm a destructive operation that would otherwise refuse' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -s h -l help -d 'Print help (see more with \'--help\')' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -f -a "address" -d 'The content address of each path read on stdin, one `\\t
` line apiece' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -f -a "agent" -d 'What an agent may do in this repository: the read-only verbs, the exit contract, and the declared gates' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)' +complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l strictness -d 'Raise how strictly gates apply (an override may only tighten policy)' -r -f -a "permissive\t'Advisory: findings are reported without failing the run' +standard\t'The default: a finding is a violation' +strict\t'Everything `Standard` fails on, plus anything advisory'" +complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l config-from -d 'Read the committed config from a git ref (e.g. origin/main) instead of the working tree' -r +complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l config-in -d 'Read the committed config from this directory instead of the directory being judged' -r +complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l log-level -d 'Set the verbosity rung by name' -r -f -a "silent\t'Say nothing but a verdict or a usage error' +quiet\t'Suppress ordinary progress; keep warnings' +normal\t'The default' +verbose\t'Explain what is being checked' +debug\t'Add resolution detail' +trace\t'Add everything'" +complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -s J -l json -d 'Emit byte-stable JSON instead of pointer lines' +complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l fail-on-warning -d 'Promote a warn-severity finding to a violation (an override may only turn this on)' +complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l silent -d 'Say nothing but a verdict or a usage error' +complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -s q -l quiet -d 'Suppress ordinary progress (repeatable: -qq is silent)' +complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -s v -l verbose -d 'Explain what is being checked (repeatable: -vv is debug)' +complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l debug -d 'Add resolution detail' +complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l trace -d 'Add everything' +complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l no-color -d 'Never colour stderr, whatever it is attached to' +complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l no-input -d 'Never prompt; treat the run as unattended' +complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -s y -l yes -d 'Confirm a destructive operation that would otherwise refuse' +complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -s h -l help -d 'Print help (see more with \'--help\')' complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from agent" -l strictness -d 'Raise how strictly gates apply (an override may only tighten policy)' -r -f -a "permissive\t'Advisory: findings are reported without failing the run' standard\t'The default: a finding is a violation' strict\t'Everything `Standard` fails on, plus anything advisory'" @@ -3290,6 +3313,7 @@ complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subc complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from agent" -l no-input -d 'Never prompt; treat the run as unattended' complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from agent" -s y -l yes -d 'Confirm a destructive operation that would otherwise refuse' complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from agent" -s h -l help -d 'Print help (see more with \'--help\')' +complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from help" -f -a "address" -d 'The content address of each path read on stdin, one `\\t
` line apiece' complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from help" -f -a "agent" -d 'What an agent may do in this repository: the read-only verbs, the exit contract, and the declared gates' complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)' complete -c batten -n "__fish_batten_using_subcommand wiring; and not __fish_seen_subcommand_from reclaim help" -l strictness -d 'Raise how strictly gates apply (an override may only tighten policy)' -r -f -a "permissive\t'Advisory: findings are reported without failing the run' @@ -3937,6 +3961,7 @@ complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subc complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from record" -f -a "fold" -d 'Fold a journal family: `nothing`, its records, or `unreadable `' complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from record" -f -a "plan" -d 'Record this branch\'s plan, read as ` ` lines on stdin' complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from record" -f -a "closes" -d 'Record which rows this branch\'s pull request body closes, read on stdin' +complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from show" -f -a "address" -d 'The content address of each path read on stdin, one `\\t
` line apiece' complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from show" -f -a "agent" -d 'What an agent may do in this repository: the read-only verbs, the exit contract, and the declared gates' complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from wiring" -f -a "reclaim" -d 'Remove non-batten hook registrations from this host\'s merged surfaces' complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from lease" -f -a "authorises" -d 'May this branch spend a matrix right now?' diff --git a/completions/batten.zsh b/completions/batten.zsh index cf339d0e2..bfba5f037 100644 --- a/completions/batten.zsh +++ b/completions/batten.zsh @@ -5457,7 +5457,38 @@ trace\:"Add everything"))' \ (( CURRENT += 1 )) curcontext="${curcontext%:*:*}:batten-show-command-$line[1]:" case $line[1] in - (agent) + (address) +_arguments "${_arguments_options[@]}" : \ +'--strictness=[Raise how strictly gates apply (an override may only tighten policy)]: :((permissive\:"Advisory\: findings are reported without failing the run" +standard\:"The default\: a finding is a violation" +strict\:"Everything \`Standard\` fails on, plus anything advisory"))' \ +'--config-from=[Read the committed config from a git ref (e.g. origin/main) instead of the working tree]: :_default' \ +'--config-in=[Read the committed config from this directory instead of the directory being judged]: :_default' \ +'--log-level=[Set the verbosity rung by name]: :((silent\:"Say nothing but a verdict or a usage error" +quiet\:"Suppress ordinary progress; keep warnings" +normal\:"The default" +verbose\:"Explain what is being checked" +debug\:"Add resolution detail" +trace\:"Add everything"))' \ +'-J[Emit byte-stable JSON instead of pointer lines]' \ +'--json[Emit byte-stable JSON instead of pointer lines]' \ +'--fail-on-warning[Promote a warn-severity finding to a violation (an override may only turn this on)]' \ +'*--silent[Say nothing but a verdict or a usage error]' \ +'*-q[Suppress ordinary progress (repeatable\: -qq is silent)]' \ +'*--quiet[Suppress ordinary progress (repeatable\: -qq is silent)]' \ +'*-v[Explain what is being checked (repeatable\: -vv is debug)]' \ +'*--verbose[Explain what is being checked (repeatable\: -vv is debug)]' \ +'*--debug[Add resolution detail]' \ +'*--trace[Add everything]' \ +'--no-color[Never colour stderr, whatever it is attached to]' \ +'--no-input[Never prompt; treat the run as unattended]' \ +'-y[Confirm a destructive operation that would otherwise refuse]' \ +'--yes[Confirm a destructive operation that would otherwise refuse]' \ +'-h[Print help (see more with '\''--help'\'')]' \ +'--help[Print help (see more with '\''--help'\'')]' \ +&& ret=0 +;; +(agent) _arguments "${_arguments_options[@]}" : \ '--strictness=[Raise how strictly gates apply (an override may only tighten policy)]: :((permissive\:"Advisory\: findings are reported without failing the run" standard\:"The default\: a finding is a violation" @@ -5500,7 +5531,11 @@ _arguments "${_arguments_options[@]}" : \ (( CURRENT += 1 )) curcontext="${curcontext%:*:*}:batten-show-help-command-$line[1]:" case $line[1] in - (agent) + (address) +_arguments "${_arguments_options[@]}" : \ +&& ret=0 +;; +(agent) _arguments "${_arguments_options[@]}" : \ && ret=0 ;; @@ -7247,7 +7282,11 @@ _arguments "${_arguments_options[@]}" : \ (( CURRENT += 1 )) curcontext="${curcontext%:*:*}:batten-help-show-command-$line[1]:" case $line[1] in - (agent) + (address) +_arguments "${_arguments_options[@]}" : \ +&& ret=0 +;; +(agent) _arguments "${_arguments_options[@]}" : \ && ret=0 ;; @@ -8865,10 +8904,16 @@ _batten__subcmd__help__subcmd__semver__subcmd__check_commands() { (( $+functions[_batten__subcmd__help__subcmd__show_commands] )) || _batten__subcmd__help__subcmd__show_commands() { local commands; commands=( +'address:The content address of each path read on stdin, one \`\\t
\` line apiece' \ 'agent:What an agent may do in this repository\: the read-only verbs, the exit contract, and the declared gates' \ ) _describe -t commands 'batten help show commands' commands "$@" } +(( $+functions[_batten__subcmd__help__subcmd__show__subcmd__address_commands] )) || +_batten__subcmd__help__subcmd__show__subcmd__address_commands() { + local commands; commands=() + _describe -t commands 'batten help show address commands' commands "$@" +} (( $+functions[_batten__subcmd__help__subcmd__show__subcmd__agent_commands] )) || _batten__subcmd__help__subcmd__show__subcmd__agent_commands() { local commands; commands=() @@ -10085,11 +10130,17 @@ _batten__subcmd__semver__subcmd__help__subcmd__help_commands() { (( $+functions[_batten__subcmd__show_commands] )) || _batten__subcmd__show_commands() { local commands; commands=( +'address:The content address of each path read on stdin, one \`\\t
\` line apiece' \ 'agent:What an agent may do in this repository\: the read-only verbs, the exit contract, and the declared gates' \ 'help:Print this message or the help of the given subcommand(s)' \ ) _describe -t commands 'batten show commands' commands "$@" } +(( $+functions[_batten__subcmd__show__subcmd__address_commands] )) || +_batten__subcmd__show__subcmd__address_commands() { + local commands; commands=() + _describe -t commands 'batten show address commands' commands "$@" +} (( $+functions[_batten__subcmd__show__subcmd__agent_commands] )) || _batten__subcmd__show__subcmd__agent_commands() { local commands; commands=() @@ -10098,11 +10149,17 @@ _batten__subcmd__show__subcmd__agent_commands() { (( $+functions[_batten__subcmd__show__subcmd__help_commands] )) || _batten__subcmd__show__subcmd__help_commands() { local commands; commands=( +'address:The content address of each path read on stdin, one \`\\t
\` line apiece' \ 'agent:What an agent may do in this repository\: the read-only verbs, the exit contract, and the declared gates' \ 'help:Print this message or the help of the given subcommand(s)' \ ) _describe -t commands 'batten show help commands' commands "$@" } +(( $+functions[_batten__subcmd__show__subcmd__help__subcmd__address_commands] )) || +_batten__subcmd__show__subcmd__help__subcmd__address_commands() { + local commands; commands=() + _describe -t commands 'batten show help address commands' commands "$@" +} (( $+functions[_batten__subcmd__show__subcmd__help__subcmd__agent_commands] )) || _batten__subcmd__show__subcmd__help__subcmd__agent_commands() { local commands; commands=() diff --git a/crates/batten/src/cli.rs b/crates/batten/src/cli.rs index 81aa60801..9d9a49911 100644 --- a/crates/batten/src/cli.rs +++ b/crates/batten/src/cli.rs @@ -121,6 +121,11 @@ pub enum Command { /// The output format for the spec. format: SpecFormat, }, + /// Report the content address of each path read on stdin (CLOUD-1717). + ShowAddress { + /// Emit the data document rather than pointer lines. + json: bool, + }, /// Report what an agent may do in this repository (CLOUD-1180). ShowAgent { /// Emit the data document rather than pointer lines. @@ -2397,11 +2402,14 @@ fn capture_of(matches: &ArgMatches) -> Option { /// what keeps that match readable as a census of the surface rather than as a /// parser. /// -/// `show` is a noun over one leaf today, so an absent subcommand is a usage -/// error rather than a default action — `surface::is_noun` marks it and clap -/// refuses the bare invocation before this runs. +/// `show` is a noun over its leaves, so an absent subcommand is a usage error +/// rather than a default action — `surface::is_noun` marks it and clap refuses +/// the bare invocation before this runs. fn show_of(matches: &ArgMatches) -> Option { match matches.subcommand()? { + ("address", matches) => Some(Command::ShowAddress { + json: flag(matches, "json"), + }), ("agent", matches) => Some(Command::ShowAgent { json: flag(matches, "json"), }), diff --git a/crates/batten/src/lib.rs b/crates/batten/src/lib.rs index 617eb62c6..881f22c2a 100644 --- a/crates/batten/src/lib.rs +++ b/crates/batten/src/lib.rs @@ -247,6 +247,7 @@ pub fn run(cli: Cli, mode: Mode, out: &mut dyn Write, err: &mut dyn Write) -> Re ), Some(Command::Config { command }) => run_config(&command, &overrides, mode, out, err), Some(Command::Spec { format }) => run_spec(format, out), + Some(Command::ShowAddress { json }) => run_show_address(json, out), Some(Command::ShowAgent { json }) => run_show_agent(json, &overrides, out), Some(Command::Doctor { command }) => run_doctor(&command, out), // `init` reads no config — it is the verb that exists because there is @@ -20199,6 +20200,65 @@ fn run_show_agent(json: bool, overrides: &Overrides, out: &mut dyn Write) -> Res Ok(ExitCode::Success) } +/// The content address of each path read on stdin, one line apiece (CLOUD-1717). +/// +/// # Why the engine owns this and `sha256sum` did not +/// +/// `mise-tasks/checksums.sh` shelled `sha256sum` to build a release manifest, +/// and three things followed from that which this verb removes. The digest was +/// bare — no domain separation, no version, no length — so a manifest entry and +/// any other sha256 in the tree were the same 64 characters and nothing could +/// say which was which. The spelling differed per platform (`sha256sum` against +/// `shasum -a 256`), which is a second authority over one answer. And the +/// manifest was in a format only that program wrote and only that program read. +/// +/// [`identity::ContentAddress`] is the tree's one answer to "what is this +/// content", already used by the capture store and by `DocumentInput`, and the +/// release manifest joins them rather than keeping a fourth spelling. +/// +/// # `read`, structurally +/// +/// It opens the paths handed to it and writes to `out`; there is no path from +/// here to a mutation or a spawn, which is what keeps it in `spec`'s read-only +/// allowlist and what `surface::tests::every_leaf_under_show_is_read` holds. +/// +/// # A path it cannot read is exit 1, and the line is not written +/// +/// Never a partial manifest. A missing or unreadable path means the caller's +/// input was wrong, which §7 spends `1` on — `2` is the policy verdict and this +/// verb judges nothing. The alternative, skipping the line, would produce a +/// manifest that is short rather than absent, and a short manifest compares +/// equal to a release that is genuinely missing that asset. +fn run_show_address(json: bool, out: &mut dyn Write) -> Result { + let mut raw = String::new(); + std::io::stdin().read_to_string(&mut raw)?; + + let mut addressed: Vec<(String, String)> = Vec::new(); + for path in raw.lines().map(str::trim).filter(|line| !line.is_empty()) { + let bytes = std::fs::read(path) + .map_err(|failed| UsageError::raise(format!("show address: {path}: {failed}")))?; + let address = identity::ContentAddress::of(identity::AddressDomain::Artifact, &bytes); + addressed.push((path.to_owned(), address.to_string())); + } + + if json { + // Unconditional, including when stdin named nothing: JSON that is + // sometimes absent is unparseable. + let document: Vec<_> = addressed + .iter() + .map(|(path, address)| serde_json::json!({"path": path, "address": address})) + .collect(); + writeln!(out, "{}", serde_json::to_string_pretty(&document)?)?; + } else { + for (path, address) in &addressed { + writeln!(out, "{path}\t{address}")?; + } + } + // Always `0`: this verb reports a reading and judges nothing, so there is no + // finding for it to raise and no `2` it could honestly mint. + Ok(ExitCode::Success) +} + fn run_spec(format: SpecFormat, out: &mut dyn Write) -> Result { let described = spec::document(&surface::command()); match format { diff --git a/crates/batten/src/spec.rs b/crates/batten/src/spec.rs index 9c102cc2e..7c6b922e3 100644 --- a/crates/batten/src/spec.rs +++ b/crates/batten/src/spec.rs @@ -647,6 +647,7 @@ mod tests { // is what keeps a writer from ever reaching this list through // the noun's own `read` (CLOUD-170's prefix hazard). "show".to_owned(), + "show address".to_owned(), "show agent".to_owned(), "spec".to_owned(), "state list".to_owned(), @@ -1070,6 +1071,7 @@ mod tests { // as `agent instructions` — the row's own 2026-08-30 amendment, on // `record `'s precedent. "show".to_owned(), + "show address".to_owned(), "show agent".to_owned(), "singleton".to_owned(), "singleton acquire".to_owned(), diff --git a/crates/batten/src/surface.rs b/crates/batten/src/surface.rs index ed05fa30d..7e1cf0292 100644 --- a/crates/batten/src/surface.rs +++ b/crates/batten/src/surface.rs @@ -4863,6 +4863,15 @@ pub const SURFACE: &[CommandDecl] = &[ // allowlist from `spec::read_only_allowlist`, the exit table from // `ExitCode::ALL`, the gates from the resolved config — so this row adds a // VIEW and no new source of truth. See `agent.rs`'s module doc. + CommandDecl { + path: "show address", + id: "show.address", + about: "The content address of each path read on stdin, one `\\t
` line apiece", + data_channel: true, + exits: EXITS_STANDARD, + effect: Effect::Read, + flags: &[JSON], + }, CommandDecl { path: "show agent", id: "show.agent", diff --git a/crates/batten/tests/it/main.rs b/crates/batten/tests/it/main.rs index 6ea9d5f96..d6869f118 100644 --- a/crates/batten/tests/it/main.rs +++ b/crates/batten/tests/it/main.rs @@ -259,6 +259,7 @@ mod session_provisioning; mod shell_retirement; mod shell_retirement_cost; mod shell_write_advisory; +mod show_address; mod singleton; mod singleton_gate; mod sinks; diff --git a/crates/batten/tests/it/show_address.rs b/crates/batten/tests/it/show_address.rs new file mode 100644 index 000000000..308539322 --- /dev/null +++ b/crates/batten/tests/it/show_address.rs @@ -0,0 +1,179 @@ +//! `show address` over the compiled binary (CLOUD-1717). +//! +//! # Why the verb exists +//! +//! `mise-tasks/checksums.sh` shelled `sha256sum` to build a release manifest, +//! and three things followed from that which this verb removes. +//! +//! The digest was BARE. A manifest entry and any other sha256 in the tree were +//! the same 64 characters, so nothing could say which was which, and nothing +//! stopped a digest computed for one purpose being compared against one computed +//! for another. [`identity::ContentAddress`] is domain-separated and versioned, +//! which is what makes the comparison `policy/release-assets.rego` performs a +//! comparison of like with like. +//! +//! The SPELLING differed per platform — `sha256sum` on GNU, `shasum -a 256` on +//! macOS — which is a second authority over one answer, and the class +//! `rules/policy-modules.md` refuses for parsers. +//! +//! And the FORMAT was one only that program wrote and only that program read. +//! The tree already had one answer to "what is this content", used by the +//! capture store and by `DocumentInput`; the release manifest joins it rather +//! than keeping a fourth spelling. +//! +//! # What these cases are for +//! +//! The address's SHAPE and the failure DIRECTION, because those are the two a +//! caller depends on and neither is visible from the happy path alone. A verb +//! that emitted a bare digest would pass any case that only checked two files +//! differ, and one that skipped an unreadable path would pass any case that only +//! checked a good one. + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use common::{run_with_stdin, scratch, write}; + +#[test] +fn the_address_is_domain_separated_and_versioned_rather_than_a_bare_digest() { + // THE WHOLE REASON THE VERB EXISTS, as an assertion rather than as a header + // paragraph. `sha256sum` emitted 64 hex characters that named no domain and + // no version, so a manifest entry was indistinguishable from any other + // digest in the tree and a future change of domain could not be detected. + let dir = scratch("show-address-shape"); + write(&dir, "asset.tar.gz", "payload"); + + let reported = run_with_stdin(&dir, &["show", "address"], "asset.tar.gz\n"); + assert!( + reported.status.success(), + "a readable path is reported\n{}", + String::from_utf8_lossy(&reported.stderr) + ); + let said = String::from_utf8_lossy(&reported.stdout); + let (path, address) = said + .trim_end() + .split_once('\t') + .expect("one `\\t
` line"); + assert_eq!(path, "asset.tar.gz", "the path is echoed verbatim\n{said}"); + assert!( + address.starts_with("b3-1-"), + "the address carries its domain and version\n{said}" + ); + assert_eq!( + address.len(), + "b3-1-".len() + 64, + "and the digest behind the prefix is a full blake3\n{said}" + ); +} + +#[test] +fn identical_content_at_two_paths_gets_one_address() { + // CONTENT-ADDRESSED, NOT PATH-ADDRESSED, which is what lets the module + // compare an uploaded asset against the manifest entry that claims it: the + // two are read at different names and must still compare equal. + let dir = scratch("show-address-identical"); + write(&dir, "left.txt", "the same bytes"); + write(&dir, "right.txt", "the same bytes"); + write(&dir, "other.txt", "different bytes"); + + let reported = run_with_stdin( + &dir, + &["show", "address"], + "left.txt\nright.txt\nother.txt\n", + ); + let said = String::from_utf8_lossy(&reported.stdout); + let addresses: Vec<&str> = said + .lines() + .filter_map(|line| line.split_once('\t')) + .map(|(_, address)| address) + .collect(); + assert_eq!(addresses.len(), 3, "one line per path\n{said}"); + assert_eq!( + addresses[0], addresses[1], + "identical content gets one address\n{said}" + ); + assert_ne!( + addresses[0], addresses[2], + "and differing content does not\n{said}" + ); +} + +#[test] +fn a_path_it_cannot_read_refuses_rather_than_writing_a_short_manifest() { + // THE DIRECTION A MISS MUST FAIL IN. Skipping the line would produce a + // manifest that is SHORT rather than absent, and a short manifest compares + // equal to a release that is genuinely missing that asset — a silent pass on + // exactly the question the manifest exists to answer. + // + // Exit 1 rather than 2: §7 spends `2` on the policy verdict, and "this path + // is not there" is a claim about the invocation rather than about the tree. + let dir = scratch("show-address-unreadable"); + write(&dir, "present.txt", "here"); + + let refused = run_with_stdin(&dir, &["show", "address"], "present.txt\nabsent.txt\n"); + assert_eq!( + refused.status.code(), + Some(1), + "an unreadable path refuses\n{}", + String::from_utf8_lossy(&refused.stderr) + ); + assert!( + String::from_utf8_lossy(&refused.stdout).is_empty(), + "and writes no partial manifest\n{}", + String::from_utf8_lossy(&refused.stdout) + ); + assert!( + String::from_utf8_lossy(&refused.stderr).contains("absent.txt"), + "naming the path it could not read\n{}", + String::from_utf8_lossy(&refused.stderr) + ); +} + +#[test] +fn the_json_channel_is_parseable_even_when_stdin_named_nothing() { + // JSON THAT IS SOMETIMES ABSENT IS UNPARSEABLE, which is `show agent`'s own + // rule at the same boundary. An empty reading is an empty array, never no + // output, because a caller piping this into a parser must not have to + // special-case the empty run. + let dir = scratch("show-address-json-empty"); + + let reported = run_with_stdin(&dir, &["show", "address", "--json"], "\n \n"); + assert!( + reported.status.success(), + "naming no path is not a failure\n{}", + String::from_utf8_lossy(&reported.stderr) + ); + let said = String::from_utf8_lossy(&reported.stdout); + let parsed: serde_json::Value = + serde_json::from_str(said.trim()).expect("the json channel parses"); + assert_eq!( + parsed, + serde_json::json!([]), + "an empty reading is an empty array\n{said}" + ); +} + +#[test] +fn the_json_channel_carries_the_same_pairs_as_the_line_channel() { + // ONE READING, TWO PROJECTIONS. A channel that disagreed with the other + // would make the manifest depend on which flag the caller passed, which is + // the drift a single source of truth exists to prevent. + let dir = scratch("show-address-json"); + write(&dir, "asset.tar.gz", "payload"); + + let lines = run_with_stdin(&dir, &["show", "address"], "asset.tar.gz\n"); + let json = run_with_stdin(&dir, &["show", "address", "--json"], "asset.tar.gz\n"); + + let said = String::from_utf8_lossy(&lines.stdout); + let (_, address) = said.trim_end().split_once('\t').expect("one pair"); + let parsed: serde_json::Value = + serde_json::from_str(String::from_utf8_lossy(&json.stdout).trim()) + .expect("the json channel parses"); + assert_eq!( + parsed, + serde_json::json!([{"path": "asset.tar.gz", "address": address}]), + "both channels carry one reading\n{said}" + ); +} diff --git a/man/batten-show-address.1 b/man/batten-show-address.1 new file mode 100644 index 000000000..a2ecc1244 --- /dev/null +++ b/man/batten-show-address.1 @@ -0,0 +1,16 @@ +.ie \n(.g .ds Aq \(aq +.el .ds Aq ' +.TH batten-show-address 1 batten +.SH NAME +batten\-show\-address \- The content address of each path read on stdin, one `\\t
` line apiece +.SH SYNOPSIS +\fBbatten show address\fR [\fB\-J\fR|\fB\-\-json\fR] [\fB\-h\fR|\fB\-\-help\fR] +.SH DESCRIPTION +The content address of each path read on stdin, one `\\t
` line apiece +.SH OPTIONS +.TP +\fB\-J\fR, \fB\-\-json\fR +Emit byte\-stable JSON instead of pointer lines +.TP +\fB\-h\fR, \fB\-\-help\fR +Print help diff --git a/man/batten-show.1 b/man/batten-show.1 index 84cd77b46..2b9ca872a 100644 --- a/man/batten-show.1 +++ b/man/batten-show.1 @@ -13,6 +13,9 @@ Report what something is, without changing it Print help .SH SUBCOMMANDS .TP +batten\-show\-address(1) +The content address of each path read on stdin, one `\\t
` line apiece +.TP batten\-show\-agent(1) What an agent may do in this repository: the read\-only verbs, the exit contract, and the declared gates .TP From c3105781361320144dc39f19d80416af121710ad Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Fri, 18 Sep 2026 08:00:16 +0000 Subject: [PATCH 15/32] revert: `show address`, whose only consumer an open decision refuses MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reverts 334c109. The verb was sound and its cases passed; the premise it was built on is what does not hold, and a read verb with no caller is the dead surface this repository exists to refuse. ## What it was for, and why that is gone It landed as the prerequisite for retiring `mise-tasks/checksums.sh`, on the plan of turning the release manifest into `B3SUMS` carrying `\t` lines. Two things say that manifest must not change, and I read neither before writing the verb. **`checksums.sh:25-27` decides the format deliberately, and says why.** The name is `SHA256SUMS` and the format is `sha256sum`'s own "so verifying is the ordinary tool with no flags: `sha256sum -c SHA256SUMS` in a directory of downloaded assets. A manifest a packager needs a wrapper to read is not one." The header names the consumers — a Homebrew formula, an aqua/mise registry entry, `cargo binstall`. `release-assets-check.sh:268` verifies with that same tool, and `:213` parses that same format. Changing it is a breaking change to a published artifact, made as a side effect of an internal refactor. **CLOUD-1608 is Backlog and its acceptance criteria refuse it outright:** "Preserve all existing SHA-256 identities; specify algorithm-tagged versioning and migration/dual-read rules for any BLAKE3 namespace." The release manifest is an existing SHA-256 identity and the most compatibility-facing one there is. Which namespaces may move to BLAKE3 is precisely that row's open question, so migrating one here would be answering it in a commit that claims to be doing something else. CLOUD-1364 adopted BLAKE3 for canonical content addressing and is Done, which is what made the plan look already-decided. It is not the same question: that row's siblings (CLOUD-1360, 1366, 1368) are the capture store and the payload contract — Batten's own internal identities — and none of them is an artifact a packager downloads. ## Why revert rather than keep it against a future use Nothing calls it. The comparison it was meant to serve cannot use it either: the manifest side carries sha256, so an asset addressed with blake3 is not comparable to the entry claiming it, and the producer that records both must compute the same function on both sides. There is no caller in this tree and none in the retirement that follows. Keeping it would also leave 334c109's own message standing as a claim that is false — that this verb is the prerequisite for a retirement — which is worse than the surface it adds. The scope reminder is the rule being followed here: adopt prior art, do not expand the core. If CLOUD-1608 later admits a BLAKE3 namespace that wants a CLI reader, this commit is the diff to restore, and it will land with a caller. Refs: CLOUD-1717 Refs: CLOUD-1608 --- completions/batten.bash | 73 +--------- completions/batten.fish | 57 +++----- completions/batten.zsh | 63 +-------- crates/batten/src/cli.rs | 14 +- crates/batten/src/lib.rs | 60 --------- crates/batten/src/spec.rs | 2 - crates/batten/src/surface.rs | 9 -- crates/batten/tests/it/main.rs | 1 - crates/batten/tests/it/show_address.rs | 179 ------------------------- man/batten-show-address.1 | 16 --- man/batten-show.1 | 3 - 11 files changed, 25 insertions(+), 452 deletions(-) delete mode 100644 crates/batten/tests/it/show_address.rs delete mode 100644 man/batten-show-address.1 diff --git a/completions/batten.bash b/completions/batten.bash index 171eba020..7853d4d72 100644 --- a/completions/batten.bash +++ b/completions/batten.bash @@ -814,9 +814,6 @@ _batten() { batten__subcmd__help__subcmd__semver,check) cmd="batten__subcmd__help__subcmd__semver__subcmd__check" ;; - batten__subcmd__help__subcmd__show,address) - cmd="batten__subcmd__help__subcmd__show__subcmd__address" - ;; batten__subcmd__help__subcmd__show,agent) cmd="batten__subcmd__help__subcmd__show__subcmd__agent" ;; @@ -1339,18 +1336,12 @@ _batten() { batten__subcmd__semver__subcmd__help,help) cmd="batten__subcmd__semver__subcmd__help__subcmd__help" ;; - batten__subcmd__show,address) - cmd="batten__subcmd__show__subcmd__address" - ;; batten__subcmd__show,agent) cmd="batten__subcmd__show__subcmd__agent" ;; batten__subcmd__show,help) cmd="batten__subcmd__show__subcmd__help" ;; - batten__subcmd__show__subcmd__help,address) - cmd="batten__subcmd__show__subcmd__help__subcmd__address" - ;; batten__subcmd__show__subcmd__help,agent) cmd="batten__subcmd__show__subcmd__help__subcmd__agent" ;; @@ -5652,7 +5643,7 @@ _batten() { return 0 ;; batten__subcmd__help__subcmd__show) - opts="address agent" + opts="agent" if [[ ${cur} == -* || ${COMP_CWORD} -eq 3 ]] ; then COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 @@ -5665,20 +5656,6 @@ _batten() { COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 ;; - batten__subcmd__help__subcmd__show__subcmd__address) - opts="" - if [[ ${cur} == -* || ${COMP_CWORD} -eq 4 ]] ; then - COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) - return 0 - fi - case "${prev}" in - *) - COMPREPLY=() - ;; - esac - COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) - return 0 - ;; batten__subcmd__help__subcmd__show__subcmd__agent) opts="" if [[ ${cur} == -* || ${COMP_CWORD} -eq 4 ]] ; then @@ -9884,7 +9861,7 @@ _batten() { return 0 ;; batten__subcmd__show) - opts="-q -v -y -h --strictness --fail-on-warning --config-from --config-in --silent --quiet --verbose --debug --trace --log-level --no-color --no-input --yes --help address agent help" + opts="-q -v -y -h --strictness --fail-on-warning --config-from --config-in --silent --quiet --verbose --debug --trace --log-level --no-color --no-input --yes --help agent help" if [[ ${cur} == -* || ${COMP_CWORD} -eq 2 ]] ; then COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 @@ -9913,36 +9890,6 @@ _batten() { COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 ;; - batten__subcmd__show__subcmd__address) - opts="-J -q -v -y -h --json --strictness --fail-on-warning --config-from --config-in --silent --quiet --verbose --debug --trace --log-level --no-color --no-input --yes --help" - if [[ ${cur} == -* || ${COMP_CWORD} -eq 3 ]] ; then - COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) - return 0 - fi - case "${prev}" in - --strictness) - COMPREPLY=($(compgen -W "permissive standard strict" -- "${cur}")) - return 0 - ;; - --config-from) - COMPREPLY=($(compgen -f "${cur}")) - return 0 - ;; - --config-in) - COMPREPLY=($(compgen -f "${cur}")) - return 0 - ;; - --log-level) - COMPREPLY=($(compgen -W "silent quiet normal verbose debug trace" -- "${cur}")) - return 0 - ;; - *) - COMPREPLY=() - ;; - esac - COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) - return 0 - ;; batten__subcmd__show__subcmd__agent) opts="-J -q -v -y -h --json --strictness --fail-on-warning --config-from --config-in --silent --quiet --verbose --debug --trace --log-level --no-color --no-input --yes --help" if [[ ${cur} == -* || ${COMP_CWORD} -eq 3 ]] ; then @@ -9974,7 +9921,7 @@ _batten() { return 0 ;; batten__subcmd__show__subcmd__help) - opts="address agent help" + opts="agent help" if [[ ${cur} == -* || ${COMP_CWORD} -eq 3 ]] ; then COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 @@ -9987,20 +9934,6 @@ _batten() { COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 ;; - batten__subcmd__show__subcmd__help__subcmd__address) - opts="" - if [[ ${cur} == -* || ${COMP_CWORD} -eq 4 ]] ; then - COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) - return 0 - fi - case "${prev}" in - *) - COMPREPLY=() - ;; - esac - COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) - return 0 - ;; batten__subcmd__show__subcmd__help__subcmd__agent) opts="" if [[ ${cur} == -* || ${COMP_CWORD} -eq 4 ]] ; then diff --git a/completions/batten.fish b/completions/batten.fish index 9e014a8a3..d591d2328 100644 --- a/completions/batten.fish +++ b/completions/batten.fish @@ -3245,52 +3245,29 @@ complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_su complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from help" -f -a "plan" -d 'Record this branch\'s plan, read as ` ` lines on stdin' complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from help" -f -a "closes" -d 'Record which rows this branch\'s pull request body closes, read on stdin' complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l strictness -d 'Raise how strictly gates apply (an override may only tighten policy)' -r -f -a "permissive\t'Advisory: findings are reported without failing the run' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l strictness -d 'Raise how strictly gates apply (an override may only tighten policy)' -r -f -a "permissive\t'Advisory: findings are reported without failing the run' standard\t'The default: a finding is a violation' strict\t'Everything `Standard` fails on, plus anything advisory'" -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l config-from -d 'Read the committed config from a git ref (e.g. origin/main) instead of the working tree' -r -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l config-in -d 'Read the committed config from this directory instead of the directory being judged' -r -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l log-level -d 'Set the verbosity rung by name' -r -f -a "silent\t'Say nothing but a verdict or a usage error' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l config-from -d 'Read the committed config from a git ref (e.g. origin/main) instead of the working tree' -r +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l config-in -d 'Read the committed config from this directory instead of the directory being judged' -r +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l log-level -d 'Set the verbosity rung by name' -r -f -a "silent\t'Say nothing but a verdict or a usage error' quiet\t'Suppress ordinary progress; keep warnings' normal\t'The default' verbose\t'Explain what is being checked' debug\t'Add resolution detail' trace\t'Add everything'" -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l fail-on-warning -d 'Promote a warn-severity finding to a violation (an override may only turn this on)' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l silent -d 'Say nothing but a verdict or a usage error' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -s q -l quiet -d 'Suppress ordinary progress (repeatable: -qq is silent)' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -s v -l verbose -d 'Explain what is being checked (repeatable: -vv is debug)' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l debug -d 'Add resolution detail' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l trace -d 'Add everything' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l no-color -d 'Never colour stderr, whatever it is attached to' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -l no-input -d 'Never prompt; treat the run as unattended' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -s y -l yes -d 'Confirm a destructive operation that would otherwise refuse' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -s h -l help -d 'Print help (see more with \'--help\')' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -f -a "address" -d 'The content address of each path read on stdin, one `\\t
` line apiece' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -f -a "agent" -d 'What an agent may do in this repository: the read-only verbs, the exit contract, and the declared gates' -complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from address agent help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)' -complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l strictness -d 'Raise how strictly gates apply (an override may only tighten policy)' -r -f -a "permissive\t'Advisory: findings are reported without failing the run' -standard\t'The default: a finding is a violation' -strict\t'Everything `Standard` fails on, plus anything advisory'" -complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l config-from -d 'Read the committed config from a git ref (e.g. origin/main) instead of the working tree' -r -complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l config-in -d 'Read the committed config from this directory instead of the directory being judged' -r -complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l log-level -d 'Set the verbosity rung by name' -r -f -a "silent\t'Say nothing but a verdict or a usage error' -quiet\t'Suppress ordinary progress; keep warnings' -normal\t'The default' -verbose\t'Explain what is being checked' -debug\t'Add resolution detail' -trace\t'Add everything'" -complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -s J -l json -d 'Emit byte-stable JSON instead of pointer lines' -complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l fail-on-warning -d 'Promote a warn-severity finding to a violation (an override may only turn this on)' -complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l silent -d 'Say nothing but a verdict or a usage error' -complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -s q -l quiet -d 'Suppress ordinary progress (repeatable: -qq is silent)' -complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -s v -l verbose -d 'Explain what is being checked (repeatable: -vv is debug)' -complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l debug -d 'Add resolution detail' -complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l trace -d 'Add everything' -complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l no-color -d 'Never colour stderr, whatever it is attached to' -complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -l no-input -d 'Never prompt; treat the run as unattended' -complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -s y -l yes -d 'Confirm a destructive operation that would otherwise refuse' -complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from address" -s h -l help -d 'Print help (see more with \'--help\')' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l fail-on-warning -d 'Promote a warn-severity finding to a violation (an override may only turn this on)' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l silent -d 'Say nothing but a verdict or a usage error' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -s q -l quiet -d 'Suppress ordinary progress (repeatable: -qq is silent)' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -s v -l verbose -d 'Explain what is being checked (repeatable: -vv is debug)' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l debug -d 'Add resolution detail' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l trace -d 'Add everything' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l no-color -d 'Never colour stderr, whatever it is attached to' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -l no-input -d 'Never prompt; treat the run as unattended' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -s y -l yes -d 'Confirm a destructive operation that would otherwise refuse' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -s h -l help -d 'Print help (see more with \'--help\')' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -f -a "agent" -d 'What an agent may do in this repository: the read-only verbs, the exit contract, and the declared gates' +complete -c batten -n "__fish_batten_using_subcommand show; and not __fish_seen_subcommand_from agent help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)' complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from agent" -l strictness -d 'Raise how strictly gates apply (an override may only tighten policy)' -r -f -a "permissive\t'Advisory: findings are reported without failing the run' standard\t'The default: a finding is a violation' strict\t'Everything `Standard` fails on, plus anything advisory'" @@ -3313,7 +3290,6 @@ complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subc complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from agent" -l no-input -d 'Never prompt; treat the run as unattended' complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from agent" -s y -l yes -d 'Confirm a destructive operation that would otherwise refuse' complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from agent" -s h -l help -d 'Print help (see more with \'--help\')' -complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from help" -f -a "address" -d 'The content address of each path read on stdin, one `\\t
` line apiece' complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from help" -f -a "agent" -d 'What an agent may do in this repository: the read-only verbs, the exit contract, and the declared gates' complete -c batten -n "__fish_batten_using_subcommand show; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)' complete -c batten -n "__fish_batten_using_subcommand wiring; and not __fish_seen_subcommand_from reclaim help" -l strictness -d 'Raise how strictly gates apply (an override may only tighten policy)' -r -f -a "permissive\t'Advisory: findings are reported without failing the run' @@ -3961,7 +3937,6 @@ complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subc complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from record" -f -a "fold" -d 'Fold a journal family: `nothing`, its records, or `unreadable `' complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from record" -f -a "plan" -d 'Record this branch\'s plan, read as ` ` lines on stdin' complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from record" -f -a "closes" -d 'Record which rows this branch\'s pull request body closes, read on stdin' -complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from show" -f -a "address" -d 'The content address of each path read on stdin, one `\\t
` line apiece' complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from show" -f -a "agent" -d 'What an agent may do in this repository: the read-only verbs, the exit contract, and the declared gates' complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from wiring" -f -a "reclaim" -d 'Remove non-batten hook registrations from this host\'s merged surfaces' complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from lease" -f -a "authorises" -d 'May this branch spend a matrix right now?' diff --git a/completions/batten.zsh b/completions/batten.zsh index bfba5f037..cf339d0e2 100644 --- a/completions/batten.zsh +++ b/completions/batten.zsh @@ -5457,38 +5457,7 @@ trace\:"Add everything"))' \ (( CURRENT += 1 )) curcontext="${curcontext%:*:*}:batten-show-command-$line[1]:" case $line[1] in - (address) -_arguments "${_arguments_options[@]}" : \ -'--strictness=[Raise how strictly gates apply (an override may only tighten policy)]: :((permissive\:"Advisory\: findings are reported without failing the run" -standard\:"The default\: a finding is a violation" -strict\:"Everything \`Standard\` fails on, plus anything advisory"))' \ -'--config-from=[Read the committed config from a git ref (e.g. origin/main) instead of the working tree]: :_default' \ -'--config-in=[Read the committed config from this directory instead of the directory being judged]: :_default' \ -'--log-level=[Set the verbosity rung by name]: :((silent\:"Say nothing but a verdict or a usage error" -quiet\:"Suppress ordinary progress; keep warnings" -normal\:"The default" -verbose\:"Explain what is being checked" -debug\:"Add resolution detail" -trace\:"Add everything"))' \ -'-J[Emit byte-stable JSON instead of pointer lines]' \ -'--json[Emit byte-stable JSON instead of pointer lines]' \ -'--fail-on-warning[Promote a warn-severity finding to a violation (an override may only turn this on)]' \ -'*--silent[Say nothing but a verdict or a usage error]' \ -'*-q[Suppress ordinary progress (repeatable\: -qq is silent)]' \ -'*--quiet[Suppress ordinary progress (repeatable\: -qq is silent)]' \ -'*-v[Explain what is being checked (repeatable\: -vv is debug)]' \ -'*--verbose[Explain what is being checked (repeatable\: -vv is debug)]' \ -'*--debug[Add resolution detail]' \ -'*--trace[Add everything]' \ -'--no-color[Never colour stderr, whatever it is attached to]' \ -'--no-input[Never prompt; treat the run as unattended]' \ -'-y[Confirm a destructive operation that would otherwise refuse]' \ -'--yes[Confirm a destructive operation that would otherwise refuse]' \ -'-h[Print help (see more with '\''--help'\'')]' \ -'--help[Print help (see more with '\''--help'\'')]' \ -&& ret=0 -;; -(agent) + (agent) _arguments "${_arguments_options[@]}" : \ '--strictness=[Raise how strictly gates apply (an override may only tighten policy)]: :((permissive\:"Advisory\: findings are reported without failing the run" standard\:"The default\: a finding is a violation" @@ -5531,11 +5500,7 @@ _arguments "${_arguments_options[@]}" : \ (( CURRENT += 1 )) curcontext="${curcontext%:*:*}:batten-show-help-command-$line[1]:" case $line[1] in - (address) -_arguments "${_arguments_options[@]}" : \ -&& ret=0 -;; -(agent) + (agent) _arguments "${_arguments_options[@]}" : \ && ret=0 ;; @@ -7282,11 +7247,7 @@ _arguments "${_arguments_options[@]}" : \ (( CURRENT += 1 )) curcontext="${curcontext%:*:*}:batten-help-show-command-$line[1]:" case $line[1] in - (address) -_arguments "${_arguments_options[@]}" : \ -&& ret=0 -;; -(agent) + (agent) _arguments "${_arguments_options[@]}" : \ && ret=0 ;; @@ -8904,16 +8865,10 @@ _batten__subcmd__help__subcmd__semver__subcmd__check_commands() { (( $+functions[_batten__subcmd__help__subcmd__show_commands] )) || _batten__subcmd__help__subcmd__show_commands() { local commands; commands=( -'address:The content address of each path read on stdin, one \`\\t
\` line apiece' \ 'agent:What an agent may do in this repository\: the read-only verbs, the exit contract, and the declared gates' \ ) _describe -t commands 'batten help show commands' commands "$@" } -(( $+functions[_batten__subcmd__help__subcmd__show__subcmd__address_commands] )) || -_batten__subcmd__help__subcmd__show__subcmd__address_commands() { - local commands; commands=() - _describe -t commands 'batten help show address commands' commands "$@" -} (( $+functions[_batten__subcmd__help__subcmd__show__subcmd__agent_commands] )) || _batten__subcmd__help__subcmd__show__subcmd__agent_commands() { local commands; commands=() @@ -10130,17 +10085,11 @@ _batten__subcmd__semver__subcmd__help__subcmd__help_commands() { (( $+functions[_batten__subcmd__show_commands] )) || _batten__subcmd__show_commands() { local commands; commands=( -'address:The content address of each path read on stdin, one \`\\t
\` line apiece' \ 'agent:What an agent may do in this repository\: the read-only verbs, the exit contract, and the declared gates' \ 'help:Print this message or the help of the given subcommand(s)' \ ) _describe -t commands 'batten show commands' commands "$@" } -(( $+functions[_batten__subcmd__show__subcmd__address_commands] )) || -_batten__subcmd__show__subcmd__address_commands() { - local commands; commands=() - _describe -t commands 'batten show address commands' commands "$@" -} (( $+functions[_batten__subcmd__show__subcmd__agent_commands] )) || _batten__subcmd__show__subcmd__agent_commands() { local commands; commands=() @@ -10149,17 +10098,11 @@ _batten__subcmd__show__subcmd__agent_commands() { (( $+functions[_batten__subcmd__show__subcmd__help_commands] )) || _batten__subcmd__show__subcmd__help_commands() { local commands; commands=( -'address:The content address of each path read on stdin, one \`\\t
\` line apiece' \ 'agent:What an agent may do in this repository\: the read-only verbs, the exit contract, and the declared gates' \ 'help:Print this message or the help of the given subcommand(s)' \ ) _describe -t commands 'batten show help commands' commands "$@" } -(( $+functions[_batten__subcmd__show__subcmd__help__subcmd__address_commands] )) || -_batten__subcmd__show__subcmd__help__subcmd__address_commands() { - local commands; commands=() - _describe -t commands 'batten show help address commands' commands "$@" -} (( $+functions[_batten__subcmd__show__subcmd__help__subcmd__agent_commands] )) || _batten__subcmd__show__subcmd__help__subcmd__agent_commands() { local commands; commands=() diff --git a/crates/batten/src/cli.rs b/crates/batten/src/cli.rs index 9d9a49911..81aa60801 100644 --- a/crates/batten/src/cli.rs +++ b/crates/batten/src/cli.rs @@ -121,11 +121,6 @@ pub enum Command { /// The output format for the spec. format: SpecFormat, }, - /// Report the content address of each path read on stdin (CLOUD-1717). - ShowAddress { - /// Emit the data document rather than pointer lines. - json: bool, - }, /// Report what an agent may do in this repository (CLOUD-1180). ShowAgent { /// Emit the data document rather than pointer lines. @@ -2402,14 +2397,11 @@ fn capture_of(matches: &ArgMatches) -> Option { /// what keeps that match readable as a census of the surface rather than as a /// parser. /// -/// `show` is a noun over its leaves, so an absent subcommand is a usage error -/// rather than a default action — `surface::is_noun` marks it and clap refuses -/// the bare invocation before this runs. +/// `show` is a noun over one leaf today, so an absent subcommand is a usage +/// error rather than a default action — `surface::is_noun` marks it and clap +/// refuses the bare invocation before this runs. fn show_of(matches: &ArgMatches) -> Option { match matches.subcommand()? { - ("address", matches) => Some(Command::ShowAddress { - json: flag(matches, "json"), - }), ("agent", matches) => Some(Command::ShowAgent { json: flag(matches, "json"), }), diff --git a/crates/batten/src/lib.rs b/crates/batten/src/lib.rs index 881f22c2a..617eb62c6 100644 --- a/crates/batten/src/lib.rs +++ b/crates/batten/src/lib.rs @@ -247,7 +247,6 @@ pub fn run(cli: Cli, mode: Mode, out: &mut dyn Write, err: &mut dyn Write) -> Re ), Some(Command::Config { command }) => run_config(&command, &overrides, mode, out, err), Some(Command::Spec { format }) => run_spec(format, out), - Some(Command::ShowAddress { json }) => run_show_address(json, out), Some(Command::ShowAgent { json }) => run_show_agent(json, &overrides, out), Some(Command::Doctor { command }) => run_doctor(&command, out), // `init` reads no config — it is the verb that exists because there is @@ -20200,65 +20199,6 @@ fn run_show_agent(json: bool, overrides: &Overrides, out: &mut dyn Write) -> Res Ok(ExitCode::Success) } -/// The content address of each path read on stdin, one line apiece (CLOUD-1717). -/// -/// # Why the engine owns this and `sha256sum` did not -/// -/// `mise-tasks/checksums.sh` shelled `sha256sum` to build a release manifest, -/// and three things followed from that which this verb removes. The digest was -/// bare — no domain separation, no version, no length — so a manifest entry and -/// any other sha256 in the tree were the same 64 characters and nothing could -/// say which was which. The spelling differed per platform (`sha256sum` against -/// `shasum -a 256`), which is a second authority over one answer. And the -/// manifest was in a format only that program wrote and only that program read. -/// -/// [`identity::ContentAddress`] is the tree's one answer to "what is this -/// content", already used by the capture store and by `DocumentInput`, and the -/// release manifest joins them rather than keeping a fourth spelling. -/// -/// # `read`, structurally -/// -/// It opens the paths handed to it and writes to `out`; there is no path from -/// here to a mutation or a spawn, which is what keeps it in `spec`'s read-only -/// allowlist and what `surface::tests::every_leaf_under_show_is_read` holds. -/// -/// # A path it cannot read is exit 1, and the line is not written -/// -/// Never a partial manifest. A missing or unreadable path means the caller's -/// input was wrong, which §7 spends `1` on — `2` is the policy verdict and this -/// verb judges nothing. The alternative, skipping the line, would produce a -/// manifest that is short rather than absent, and a short manifest compares -/// equal to a release that is genuinely missing that asset. -fn run_show_address(json: bool, out: &mut dyn Write) -> Result { - let mut raw = String::new(); - std::io::stdin().read_to_string(&mut raw)?; - - let mut addressed: Vec<(String, String)> = Vec::new(); - for path in raw.lines().map(str::trim).filter(|line| !line.is_empty()) { - let bytes = std::fs::read(path) - .map_err(|failed| UsageError::raise(format!("show address: {path}: {failed}")))?; - let address = identity::ContentAddress::of(identity::AddressDomain::Artifact, &bytes); - addressed.push((path.to_owned(), address.to_string())); - } - - if json { - // Unconditional, including when stdin named nothing: JSON that is - // sometimes absent is unparseable. - let document: Vec<_> = addressed - .iter() - .map(|(path, address)| serde_json::json!({"path": path, "address": address})) - .collect(); - writeln!(out, "{}", serde_json::to_string_pretty(&document)?)?; - } else { - for (path, address) in &addressed { - writeln!(out, "{path}\t{address}")?; - } - } - // Always `0`: this verb reports a reading and judges nothing, so there is no - // finding for it to raise and no `2` it could honestly mint. - Ok(ExitCode::Success) -} - fn run_spec(format: SpecFormat, out: &mut dyn Write) -> Result { let described = spec::document(&surface::command()); match format { diff --git a/crates/batten/src/spec.rs b/crates/batten/src/spec.rs index 7c6b922e3..9c102cc2e 100644 --- a/crates/batten/src/spec.rs +++ b/crates/batten/src/spec.rs @@ -647,7 +647,6 @@ mod tests { // is what keeps a writer from ever reaching this list through // the noun's own `read` (CLOUD-170's prefix hazard). "show".to_owned(), - "show address".to_owned(), "show agent".to_owned(), "spec".to_owned(), "state list".to_owned(), @@ -1071,7 +1070,6 @@ mod tests { // as `agent instructions` — the row's own 2026-08-30 amendment, on // `record `'s precedent. "show".to_owned(), - "show address".to_owned(), "show agent".to_owned(), "singleton".to_owned(), "singleton acquire".to_owned(), diff --git a/crates/batten/src/surface.rs b/crates/batten/src/surface.rs index 7e1cf0292..ed05fa30d 100644 --- a/crates/batten/src/surface.rs +++ b/crates/batten/src/surface.rs @@ -4863,15 +4863,6 @@ pub const SURFACE: &[CommandDecl] = &[ // allowlist from `spec::read_only_allowlist`, the exit table from // `ExitCode::ALL`, the gates from the resolved config — so this row adds a // VIEW and no new source of truth. See `agent.rs`'s module doc. - CommandDecl { - path: "show address", - id: "show.address", - about: "The content address of each path read on stdin, one `\\t
` line apiece", - data_channel: true, - exits: EXITS_STANDARD, - effect: Effect::Read, - flags: &[JSON], - }, CommandDecl { path: "show agent", id: "show.agent", diff --git a/crates/batten/tests/it/main.rs b/crates/batten/tests/it/main.rs index d6869f118..6ea9d5f96 100644 --- a/crates/batten/tests/it/main.rs +++ b/crates/batten/tests/it/main.rs @@ -259,7 +259,6 @@ mod session_provisioning; mod shell_retirement; mod shell_retirement_cost; mod shell_write_advisory; -mod show_address; mod singleton; mod singleton_gate; mod sinks; diff --git a/crates/batten/tests/it/show_address.rs b/crates/batten/tests/it/show_address.rs deleted file mode 100644 index 308539322..000000000 --- a/crates/batten/tests/it/show_address.rs +++ /dev/null @@ -1,179 +0,0 @@ -//! `show address` over the compiled binary (CLOUD-1717). -//! -//! # Why the verb exists -//! -//! `mise-tasks/checksums.sh` shelled `sha256sum` to build a release manifest, -//! and three things followed from that which this verb removes. -//! -//! The digest was BARE. A manifest entry and any other sha256 in the tree were -//! the same 64 characters, so nothing could say which was which, and nothing -//! stopped a digest computed for one purpose being compared against one computed -//! for another. [`identity::ContentAddress`] is domain-separated and versioned, -//! which is what makes the comparison `policy/release-assets.rego` performs a -//! comparison of like with like. -//! -//! The SPELLING differed per platform — `sha256sum` on GNU, `shasum -a 256` on -//! macOS — which is a second authority over one answer, and the class -//! `rules/policy-modules.md` refuses for parsers. -//! -//! And the FORMAT was one only that program wrote and only that program read. -//! The tree already had one answer to "what is this content", used by the -//! capture store and by `DocumentInput`; the release manifest joins it rather -//! than keeping a fourth spelling. -//! -//! # What these cases are for -//! -//! The address's SHAPE and the failure DIRECTION, because those are the two a -//! caller depends on and neither is visible from the happy path alone. A verb -//! that emitted a bare digest would pass any case that only checked two files -//! differ, and one that skipped an unreadable path would pass any case that only -//! checked a good one. - -// Panicking on setup failure is the idiomatic way for a test to fail loudly. -#![allow(clippy::unwrap_used, clippy::expect_used)] - -use crate::common; - -use common::{run_with_stdin, scratch, write}; - -#[test] -fn the_address_is_domain_separated_and_versioned_rather_than_a_bare_digest() { - // THE WHOLE REASON THE VERB EXISTS, as an assertion rather than as a header - // paragraph. `sha256sum` emitted 64 hex characters that named no domain and - // no version, so a manifest entry was indistinguishable from any other - // digest in the tree and a future change of domain could not be detected. - let dir = scratch("show-address-shape"); - write(&dir, "asset.tar.gz", "payload"); - - let reported = run_with_stdin(&dir, &["show", "address"], "asset.tar.gz\n"); - assert!( - reported.status.success(), - "a readable path is reported\n{}", - String::from_utf8_lossy(&reported.stderr) - ); - let said = String::from_utf8_lossy(&reported.stdout); - let (path, address) = said - .trim_end() - .split_once('\t') - .expect("one `\\t
` line"); - assert_eq!(path, "asset.tar.gz", "the path is echoed verbatim\n{said}"); - assert!( - address.starts_with("b3-1-"), - "the address carries its domain and version\n{said}" - ); - assert_eq!( - address.len(), - "b3-1-".len() + 64, - "and the digest behind the prefix is a full blake3\n{said}" - ); -} - -#[test] -fn identical_content_at_two_paths_gets_one_address() { - // CONTENT-ADDRESSED, NOT PATH-ADDRESSED, which is what lets the module - // compare an uploaded asset against the manifest entry that claims it: the - // two are read at different names and must still compare equal. - let dir = scratch("show-address-identical"); - write(&dir, "left.txt", "the same bytes"); - write(&dir, "right.txt", "the same bytes"); - write(&dir, "other.txt", "different bytes"); - - let reported = run_with_stdin( - &dir, - &["show", "address"], - "left.txt\nright.txt\nother.txt\n", - ); - let said = String::from_utf8_lossy(&reported.stdout); - let addresses: Vec<&str> = said - .lines() - .filter_map(|line| line.split_once('\t')) - .map(|(_, address)| address) - .collect(); - assert_eq!(addresses.len(), 3, "one line per path\n{said}"); - assert_eq!( - addresses[0], addresses[1], - "identical content gets one address\n{said}" - ); - assert_ne!( - addresses[0], addresses[2], - "and differing content does not\n{said}" - ); -} - -#[test] -fn a_path_it_cannot_read_refuses_rather_than_writing_a_short_manifest() { - // THE DIRECTION A MISS MUST FAIL IN. Skipping the line would produce a - // manifest that is SHORT rather than absent, and a short manifest compares - // equal to a release that is genuinely missing that asset — a silent pass on - // exactly the question the manifest exists to answer. - // - // Exit 1 rather than 2: §7 spends `2` on the policy verdict, and "this path - // is not there" is a claim about the invocation rather than about the tree. - let dir = scratch("show-address-unreadable"); - write(&dir, "present.txt", "here"); - - let refused = run_with_stdin(&dir, &["show", "address"], "present.txt\nabsent.txt\n"); - assert_eq!( - refused.status.code(), - Some(1), - "an unreadable path refuses\n{}", - String::from_utf8_lossy(&refused.stderr) - ); - assert!( - String::from_utf8_lossy(&refused.stdout).is_empty(), - "and writes no partial manifest\n{}", - String::from_utf8_lossy(&refused.stdout) - ); - assert!( - String::from_utf8_lossy(&refused.stderr).contains("absent.txt"), - "naming the path it could not read\n{}", - String::from_utf8_lossy(&refused.stderr) - ); -} - -#[test] -fn the_json_channel_is_parseable_even_when_stdin_named_nothing() { - // JSON THAT IS SOMETIMES ABSENT IS UNPARSEABLE, which is `show agent`'s own - // rule at the same boundary. An empty reading is an empty array, never no - // output, because a caller piping this into a parser must not have to - // special-case the empty run. - let dir = scratch("show-address-json-empty"); - - let reported = run_with_stdin(&dir, &["show", "address", "--json"], "\n \n"); - assert!( - reported.status.success(), - "naming no path is not a failure\n{}", - String::from_utf8_lossy(&reported.stderr) - ); - let said = String::from_utf8_lossy(&reported.stdout); - let parsed: serde_json::Value = - serde_json::from_str(said.trim()).expect("the json channel parses"); - assert_eq!( - parsed, - serde_json::json!([]), - "an empty reading is an empty array\n{said}" - ); -} - -#[test] -fn the_json_channel_carries_the_same_pairs_as_the_line_channel() { - // ONE READING, TWO PROJECTIONS. A channel that disagreed with the other - // would make the manifest depend on which flag the caller passed, which is - // the drift a single source of truth exists to prevent. - let dir = scratch("show-address-json"); - write(&dir, "asset.tar.gz", "payload"); - - let lines = run_with_stdin(&dir, &["show", "address"], "asset.tar.gz\n"); - let json = run_with_stdin(&dir, &["show", "address", "--json"], "asset.tar.gz\n"); - - let said = String::from_utf8_lossy(&lines.stdout); - let (_, address) = said.trim_end().split_once('\t').expect("one pair"); - let parsed: serde_json::Value = - serde_json::from_str(String::from_utf8_lossy(&json.stdout).trim()) - .expect("the json channel parses"); - assert_eq!( - parsed, - serde_json::json!([{"path": "asset.tar.gz", "address": address}]), - "both channels carry one reading\n{said}" - ); -} diff --git a/man/batten-show-address.1 b/man/batten-show-address.1 deleted file mode 100644 index a2ecc1244..000000000 --- a/man/batten-show-address.1 +++ /dev/null @@ -1,16 +0,0 @@ -.ie \n(.g .ds Aq \(aq -.el .ds Aq ' -.TH batten-show-address 1 batten -.SH NAME -batten\-show\-address \- The content address of each path read on stdin, one `\\t
` line apiece -.SH SYNOPSIS -\fBbatten show address\fR [\fB\-J\fR|\fB\-\-json\fR] [\fB\-h\fR|\fB\-\-help\fR] -.SH DESCRIPTION -The content address of each path read on stdin, one `\\t
` line apiece -.SH OPTIONS -.TP -\fB\-J\fR, \fB\-\-json\fR -Emit byte\-stable JSON instead of pointer lines -.TP -\fB\-h\fR, \fB\-\-help\fR -Print help diff --git a/man/batten-show.1 b/man/batten-show.1 index 2b9ca872a..84cd77b46 100644 --- a/man/batten-show.1 +++ b/man/batten-show.1 @@ -13,9 +13,6 @@ Report what something is, without changing it Print help .SH SUBCOMMANDS .TP -batten\-show\-address(1) -The content address of each path read on stdin, one `\\t
` line apiece -.TP batten\-show\-agent(1) What an agent may do in this repository: the read\-only verbs, the exit contract, and the declared gates .TP From 29dd210e1ac61a350037a154ed0026fb53f8ec70 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sat, 19 Sep 2026 02:00:16 +0000 Subject: [PATCH 16/32] feat(policy): the evaluator's closure decides in the engine, and its walk stays under test MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `mise-tasks/evaluator-closure-check.sh` and its suite are gone. The decision is `policy/evaluator-closure.rego`; the `cargo metadata` spawn and the graph walk are `[tasks.evaluator-closure-record]`'s. ## The split is forced twice over §5 makes `check` `read` and structurally incapable of spawning `cargo metadata`, which is the usual half. The second half is particular to this gate: **a reachability closure is not expressible in Rego at all.** A self-referential rule is a compile error, and `graph.reachable` is not in this build's regorus feature set — `ci-cache-declared`'s header records the same bound and answers it by expanding its own walk by hand to a stated depth of three. An unbounded closure has no such spelling, so the walk stays a step and the module decides over what it recorded. ## The walk is a `.py` sibling, and that is what keeps it under test Four of the dying suite's eight cases are about the WALK rather than the verdict: the activation filter in both directions, a dev-dependency of a dependency, and the workspace-versus-evaluator scope. They are the security-critical half — the obvious spelling of this predicate walks from the workspace members, finds `jsonschema` and `globset` (direct dependencies of `batten` itself, entering by paths that have nothing to do with the evaluator), and would deny on `main` forever. It was measured before it was written: the wider spelling fired on all 5 lockfile-touching commits reachable from HEAD, 100% false positives. Inlined into the task body those four would have become `// changed:` arms pointing at `mise.toml` with nothing asserting them — which is what `branch-age`'s port had to do for its three, and it is a real loss of coverage. `mise-tasks/evaluator-closure.py` instead, driven directly by `crates/batten/tests/it/evaluator_closure.rs`, so **all eight cases carry**. `.py` is admissible and the ledger is not being gamed by it: `shell-retirement.rego:159-163` excludes `.py` from `under_mise_tasks`, so this adds no shell rule, and the arm's two obligations are met by `policy/evaluator-closure.rego` (policy surface) and the tier (compiled test). The python is not new either — it is the same walk, relocated out of a heredoc into a file a test can point at. ## Three-valued, and the third value is the one that matters - a record naming a crate is a refusal (`layer carry unsafe`); - a record saying `absent` is could-not-look and **loud** (`layer read absent`), because an evaluator that vanished from the graph means the question was never asked, and reporting "nothing found" there is CLOUD-251's vacuous pass in the one place it would be least visible; - no record at all is **silence**, because the producer writes nothing when `cargo metadata` will not resolve, and a module refusing there would refuse every checkout with no toolchain. The tier drives all three, the third over the engine's own projection rather than with `with input as` — which is the case `branch-age` was green over for a whole session while deciding nothing (CLOUD-1810). ## The row keeps its id `layer reach unsafe` moves from `kind = "command"` to `kind = "policy"` and drops the `glob` that named the deleted program. A `policy` row over a declared record is asked on every run rather than when one file changes, which is the right cadence: the drift this gate exists for arrives through Cargo's feature unification with no edit to any file a glob could name. `the_repos_real_graph_is_clean_today` runs the pair against the tree as it actually resolves — the only evidence that fixture and reality agree, and the claim `Cargo.toml`'s pin comment rests on. ## Corpus 78 -> 77 programs, 89 -> 88 suites. 8 of 44 retired. Refs: CLOUD-1717 Refs: CLOUD-831 Admits: 71d338b82c5bff621e72369cc11676922e0ac204bca4f37d111271a10bab74da Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: batten.toml Admits-anchor: call:c3105781361320144dc39f19d80416af121710ad Admits-epoch: 88f57cc183d508e8081b071986ae78ec62612b992add1a34f0dd8d77d7cbc5a5 Admits-author: alec@wenzowski.com Admits-prev: a671f0bf508457d53339d0effc969ff4e219451573672285f3de37a6d35624e8 Admits-answer-lost: The retirement cannot land at all, and the tree is left worse than before: the `layer reach unsafe` row would name a deleted task, so `batten check` would spawn a program that does not exist. That rule is the enforcement behind crates/batten/src/policy.rs's claim that a policy module cannot open a file, start a process, or reach the network — the claim that admits consumer-authored code to the mediated call. Leaving it pointing at a deleted task disarms the one gate standing behind that security boundary. Admits-answer-precondition: batten.toml is the only surface carrying [[rule]], [[verdict]] and [[record]] rows. CLOUD-1717 retires mise-tasks/evaluator-closure-check.sh onto policy/evaluator-closure.rego plus [tasks.evaluator-closure-record], and the existing `layer reach unsafe` row is `kind = "command"` with `check = "mise run evaluator-closure-check"` — a task this change deletes. The row must become `kind = "policy"` naming the module, and its two verdict classes and its [[record]] family must be declared, or the engine refuses at load. No other file can carry any of that. It lands in PR #962 as an ordinary diff hunk. Admits-answer-rejected-route: `config read first` is a read of batten.toml; I read it to find the existing row's id, kind, glob and no_fix_reason so the retirement preserves the rule id rather than minting a second name for one question. Reading cannot rewrite the row. `patch run first` is `git restore`, which would discard the module, the producer and the walk together rather than complete them. Admits: b557d27683cbf6f9ec7ff7a863cf8c54611f7d19dd0576aa7d2ceceae17fc039 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: batten.toml Admits-anchor: call:c3105781361320144dc39f19d80416af121710ad Admits-epoch: d610d7f5d49dba89532f000d1457270609af2afe0b2d00a46a345e8e18c55b71 Admits-author: alec@wenzowski.com Admits-prev: 71d338b82c5bff621e72369cc11676922e0ac204bca4f37d111271a10bab74da Admits-answer-lost: The tree does not load at all. `batten check`, `batten policy test` and every gate in the repository exit non-zero with `verdict declare refused` until the two classes are declared, which is strictly worse than before this retirement started. That state is also self-latching: a config that will not load disarms the protected-path guard on batten.toml itself and makes `override request` unrunnable (measured this session, filed on CLOUD-1768). Admits-answer-precondition: batten.toml is the only surface carrying [[verdict]] rows, and the engine refuses at load both a module raising a token no row declares and a row nothing raises. policy/evaluator-closure.rego raises `layer carry unsafe` and `layer read absent`, neither of which is declared yet, so the tree will not load until this write lands. It is the same change as the [[rule]] hunk admitted immediately before it, split across two Edit calls only because an admission is one-shot. Admits-answer-rejected-route: `config read first` is a read of batten.toml; I read it to find that `workflow read unread` and `task resolve missing` already exist and to draw both new names from the declared [vocabulary] subject, action and condition lists rather than inventing words. Reading cannot declare the two rows the module raises. `patch run first` is `git restore`, which would discard the whole retirement rather than complete it. --- batten.toml | 57 ++- bench/suites/RESULTS.md | 5 +- crates/batten/tests/it/evaluator_closure.rs | 384 ++++++++++++++++++++ crates/batten/tests/it/main.rs | 1 + mise-tasks/evaluator-closure-check.sh | 213 ----------- mise-tasks/evaluator-closure.py | 145 ++++++++ mise.toml | 44 ++- policy/evaluator-closure.rego | 132 +++++++ tests/evaluator-closure-check.bats | 203 ----------- 9 files changed, 760 insertions(+), 424 deletions(-) create mode 100644 crates/batten/tests/it/evaluator_closure.rs delete mode 100755 mise-tasks/evaluator-closure-check.sh create mode 100644 mise-tasks/evaluator-closure.py create mode 100644 policy/evaluator-closure.rego delete mode 100644 tests/evaluator-closure-check.bats diff --git a/batten.toml b/batten.toml index 267521613..3aba09074 100644 --- a/batten.toml +++ b/batten.toml @@ -5934,15 +5934,31 @@ severity = "warn" # `Cargo.lock` and `Cargo.toml`, not this file — the glob is what decides WHEN it # is worth asking, and asking on a change to the gate itself is the case that # must never be skipped. +# +# PORTED UNDER CLOUD-1717, AND THE ROW KEEPS ITS ID. `kind` moves from `command` +# to `policy` because the program it spawned is gone: the `cargo metadata` call +# and the graph walk are `[tasks.evaluator-closure-record]`'s, and the decision +# over what that walk reached is `policy/evaluator-closure.rego`'s. The split is +# forced — §5 makes `check` `read` and incapable of spawning, and a reachability +# closure is not expressible in Rego at all (a self-referential rule is a compile +# error and `graph.reachable` is not in this build's regorus feature set). +# +# `glob` is dropped with the program it named. A `policy` row over a declared +# record is asked on every run rather than when one file changes, which is the +# right cadence here: the drift this gate exists for arrives through feature +# unification with no edit to any file a glob could name. [[rule]] id = "layer reach unsafe" -kind = "command" -glob = "mise-tasks/evaluator-closure-check.sh" -check = "mise run evaluator-closure-check" -severity = "deny" +kind = "policy" scope = "tree" +module = "policy/evaluator-closure.rego" +severity = "deny" no_fix_reason = "an IO crate reaching the evaluator is closed where it was enabled, not here: `cargo tree -i ` names who turned it on, and the `regorus` feature list in Cargo.toml is where the pin is stated rather than where it is decided" +[[record]] +record = "evaluator-closure" +writer = "mise run evaluator-closure-record" + # The contract half of the config deprecation grammar (CLOUD-360). A key that # leaves the published schema with no window breaks every consumer still carrying # it, and breaks them SILENTLY: their config stops loading with an unknown-key @@ -12284,6 +12300,39 @@ id = "workflow read first" kind = "document" target = ".github/workflows/release-assets.yml" +[[verdict]] +id = "layer carry unsafe" +gloss = "an IO-bearing crate is reachable from the evaluator's node in the resolved dependency graph" +class = """ +`crates/batten/src/policy.rs` admits consumer-authored code to the mediated call \ +on one claim — a policy module "cannot open a file, start a process, or reach the \ +network". This class is that claim failing. It does not mean somebody edited the \ +pin: Cargo unifies features across the graph, so a second crate in the workspace, \ +or any dependency, taking `regorus` with default features unions them back on with \ +no diff a reviewer of the pin would see. Close it where it was enabled. +""" + +[[verdict.route]] +id = "module read first" +kind = "document" +target = "policy/evaluator-closure.rego" + +[[verdict]] +id = "layer read absent" +gloss = "the graph resolved and carries no evaluator node, so the closure was never walked" +class = """ +Could-not-look, and LOUD rather than silent. The producer resolved a graph and \ +found no node to start from, which means the question was not asked — reporting \ +"nothing found" there is the vacuous pass CLOUD-251 names. It is a different \ +answer from an absent record, which is the producer not having run at all and \ +which this rule reads as silence. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run evaluator-closure-record" + [[verdict]] id = "task run unknown" gloss = "a workflow step runs a mise task that neither the manifest nor a program under mise-tasks/ defines" diff --git a/bench/suites/RESULTS.md b/bench/suites/RESULTS.md index dd4f7b6d0..b949586d4 100644 --- a/bench/suites/RESULTS.md +++ b/bench/suites/RESULTS.md @@ -6,8 +6,8 @@ runner measured it; the suite runs `--no-parallelize-within-files`, so a file's number is its own serial cost and is what an author adding a case to it pays. -- suites: 99 -- serial total: 158.9s +- suites: 98 +- serial total: 158.3s | seconds | share | suite | | ---: | ---: | --- | @@ -59,7 +59,6 @@ to it pays. | 0.7 | 0.4% | `tests/doctor.bats` | | 0.6 | 0.4% | `tests/commit-attribution.bats` | | 0.6 | 0.4% | `tests/done-pr-check.bats` | -| 0.6 | 0.4% | `tests/evaluator-closure-check.bats` | | 0.6 | 0.4% | `tests/sbom-binary.bats` | | 0.6 | 0.4% | `tests/mcp-timeout-budget.bats` | | 0.6 | 0.4% | `tests/mcp-attach-check.bats` | diff --git a/crates/batten/tests/it/evaluator_closure.rs b/crates/batten/tests/it/evaluator_closure.rs new file mode 100644 index 000000000..0a6e6762d --- /dev/null +++ b/crates/batten/tests/it/evaluator_closure.rs @@ -0,0 +1,384 @@ +//! `layer reach unsafe` over the compiled binary and the real walk (CLOUD-831, +//! CLOUD-1717). +//! +//! # Why this tier exists and the module's own `test_` rules do not suffice +//! +//! `policy/evaluator-closure.rego` carries five load-time cases and every one +//! fabricates its input with `with input as`. That is the shape +//! `rules/policy-modules.md` warns about: the case asserts over a record the +//! engine may be unable to project, and the module stays green while the row +//! decides nothing on any real checkout. `branch-age` spent a whole session in +//! exactly that state (CLOUD-1810). +//! +//! # And why the WALK is driven here rather than described +//! +//! Four of the dying suite's eight cases are about the walk, not the verdict — +//! the activation filter in both directions, a dev-dependency, and the +//! workspace-versus-evaluator scope. They are the security-critical half: the +//! obvious spelling of this predicate walks from the workspace members, finds +//! `jsonschema` and `globset` (direct dependencies of `batten` itself), and would +//! deny on `main` forever. +//! +//! Had the walk gone into the task body they would have become `// changed:` arms +//! pointing at `mise.toml` with nothing asserting them. It is +//! `mise-tasks/evaluator-closure.py` instead, and these cases run it, so all +//! eight CARRY. `.py` is outside `under_mise_tasks` (`shell-retirement.rego:159`) +//! so that file adds no shell rule. +//! +//! # RETIREMENT LEDGER, PER PATH — what `shell retire partial` reads +//! +//! The program's successor is `policy/evaluator-closure.rego` for the decision +//! and `[tasks.evaluator-closure-record]` for the two things that are not +//! decisions: the `cargo metadata` spawn and the graph walk. That split is forced +//! rather than chosen — §5 makes `check` `read` and structurally incapable of +//! spawning, and a reachability closure is not expressible in Rego at all, since +//! a self-referential rule is a compile error and `graph.reachable` is not in +//! this build's regorus feature set. +//! +// carried: mise-tasks/evaluator-closure-check.sh policy/evaluator-closure.rego kind:mechanism crates/batten/tests/it/evaluator_closure.rs +// carried: tests/evaluator-closure-check.bats policy/evaluator-closure.rego kind:mechanism crates/batten/tests/it/evaluator_closure.rs +// carried: "the repo's real graph is clean today" policy/evaluator-closure.rego kind:mechanism +// carried: "an IO crate reachable from the evaluator is refused at exit 2" policy/evaluator-closure.rego kind:mechanism +// carried: "an IO crate the workspace depends on directly, but the evaluator does not, is not the evaluator's" policy/evaluator-closure.rego kind:mechanism +// carried: "an unactivated optional IO dependency of the evaluator is not reported" policy/evaluator-closure.rego kind:mechanism +// carried: "the same optional dependency, activated, IS reported" policy/evaluator-closure.rego kind:mechanism +// carried: "no evaluator node at all is could-not-look, not a clean bill" policy/evaluator-closure.rego kind:mechanism +// carried: "the refusal is pointer-only: the crate name, never the path that reached it" policy/evaluator-closure.rego kind:mechanism +// carried: "a dev-dependency of the evaluator is not in the built closure" policy/evaluator-closure.rego kind:mechanism + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use common::{git_in, init_repo, run, run_with_stdin, scratch, write}; + +/// A repository registering the real module against the declared family. +fn repo(name: &str) -> std::path::PathBuf { + let dir = scratch(&format!("evaluator-closure-{name}")); + let module = std::fs::read_to_string("../../policy/evaluator-closure.rego") + .expect("the module this tier exists for"); + write(&dir, "policy/evaluator-closure.rego", &module); + write( + &dir, + "batten.toml", + r#"version = 1 +scope = ["**"] + +[[verdict]] +id = "layer carry unsafe" +gloss = "an IO-bearing crate is reachable from the evaluator's node" +class = "The claim that admits consumer-authored code to the mediated call, failing." + +[[verdict.route]] +id = "module read first" +kind = "document" +target = "policy/evaluator-closure.rego" + +[[verdict]] +id = "layer read absent" +gloss = "the graph resolved and carries no evaluator node" +class = "Could-not-look, and loud: the question was never asked." + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run evaluator-closure-record" + +[[rule]] +id = "layer reach unsafe" +kind = "policy" +scope = "tree" +module = "policy/evaluator-closure.rego" +severity = "deny" + +[[record]] +record = "evaluator-closure" +writer = "mise run evaluator-closure-record" +"#, + ); + init_repo(&dir); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-qm", "register the module"]); + dir +} + +/// Write the producer's record, as `mise run evaluator-closure-record` would. +fn record(dir: &std::path::Path, lines: &str) { + let written = run_with_stdin(dir, &["record", "named", "evaluator-closure"], lines); + assert!( + written.status.success(), + "the setup write lands: {}", + String::from_utf8_lossy(&written.stderr) + ); +} + +/// Run the REAL walk over a `cargo metadata` document, exactly as the producer +/// does. This is what keeps the four walk cases assertable after the port. +#[expect( + clippy::disallowed_types, + reason = "CLOUD-320's inventory row, and the verdict is that this spawn IS the subject under test: the walk moved out of the dying program into `mise-tasks/evaluator-closure.py` precisely so a compiled case could drive it, and a harness that re-implemented it in Rust would be a second authority over the reachability the producer actually runs" +)] +fn walk(metadata: &str) -> String { + use std::io::Write as _; + use std::process::{Command, Stdio}; + + let mut child = Command::new("python3") + .arg("../../mise-tasks/evaluator-closure.py") + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .expect("the walk this tier exists for"); + child + .stdin + .as_mut() + .expect("stdin") + .write_all(metadata.as_bytes()) + .expect("feed the graph"); + let done = child.wait_with_output().expect("the walk completes"); + assert!( + done.status.success(), + "the walk reads its graph: {}", + String::from_utf8_lossy(&done.stderr) + ); + String::from_utf8_lossy(&done.stdout).into_owned() +} + +/// A graph where `batten` depends on `regorus`, and `regorus` on `edge`. +fn chain(edge: &str, kind: &str, optional: bool, regorus_features: &str) -> String { + let optional_decl = if optional { + r#", "optional": true"# + } else { + "" + }; + format!( + r#"{{ + "packages": [ + {{"id": "batten", "name": "batten", "features": {{}}, + "dependencies": [{{"name": "regorus"}}]}}, + {{"id": "regorus", "name": "regorus", "features": {{"http": ["dep:{edge}"]}}, + "dependencies": [{{"name": "{edge}"{optional_decl}}}]}}, + {{"id": "{edge}", "name": "{edge}", "features": {{}}, "dependencies": []}} + ], + "workspace_members": ["batten"], + "resolve": {{"nodes": [ + {{"id": "batten", "features": [], "deps": [{{"pkg": "regorus", "dep_kinds": [{{"kind": null}}]}}]}}, + {{"id": "regorus", "features": [{regorus_features}], "deps": [{{"pkg": "{edge}", "dep_kinds": [{{"kind": {kind}}}]}}]}}, + {{"id": "{edge}", "features": [], "deps": []}} + ]}} +}}"# + ) +} + +// --- the decision, over the engine's own projection -------------------------- + +#[test] +fn an_io_crate_in_the_recorded_closure_is_refused() { + // THE CASE NO `with input as` CAN REACH: the module deciding over a record + // the real verb wrote, read through the engine's own projection. + let dir = repo("reachable"); + record(&dir, "closure 41\ncrate reqwest\n"); + + let decided = run(&dir, &["check"]); + assert_eq!( + decided.status.code(), + Some(2), + "an IO crate in the closure decides\n{}", + String::from_utf8_lossy(&decided.stderr) + ); + let said = format!( + "{}{}", + String::from_utf8_lossy(&decided.stdout), + String::from_utf8_lossy(&decided.stderr) + ); + assert!( + said.contains("reqwest"), + "the finding names the crate\n{said}" + ); +} + +#[test] +fn the_refusal_is_pointer_only_the_crate_name_never_the_path_that_reached_it() { + // Non-negotiable rule 4. The producer records only the crate NAME, so there + // is structurally no path for a dependency chain to reach a finding — this + // asserts the intermediate never appears. + let dir = repo("pointer"); + let reached = walk( + r#"{ + "packages": [ + {"id": "batten", "name": "batten", "features": {}, "dependencies": [{"name": "regorus"}]}, + {"id": "regorus", "name": "regorus", "features": {}, "dependencies": [{"name": "secret-middle"}]}, + {"id": "secret-middle", "name": "secret-middle", "features": {}, "dependencies": [{"name": "ring"}]}, + {"id": "ring", "name": "ring", "features": {}, "dependencies": []} + ], + "workspace_members": ["batten"], + "resolve": {"nodes": [ + {"id": "batten", "features": [], "deps": [{"pkg": "regorus", "dep_kinds": [{"kind": null}]}]}, + {"id": "regorus", "features": [], "deps": [{"pkg": "secret-middle", "dep_kinds": [{"kind": null}]}]}, + {"id": "secret-middle", "features": [], "deps": [{"pkg": "ring", "dep_kinds": [{"kind": null}]}]}, + {"id": "ring", "features": [], "deps": []} + ]} +}"#, + ); + record(&dir, &reached); + + let decided = run(&dir, &["check"]); + assert_eq!(decided.status.code(), Some(2), "the chain decides"); + let said = format!( + "{}{}", + String::from_utf8_lossy(&decided.stdout), + String::from_utf8_lossy(&decided.stderr) + ); + assert!(said.contains("ring"), "the crate is named\n{said}"); + assert!( + !said.contains("secret-middle"), + "and the path that reached it is not\n{said}" + ); +} + +#[test] +fn no_evaluator_node_at_all_is_could_not_look_not_a_clean_bill() { + // Reporting "nothing found" when the evaluator is not in the graph is + // CLOUD-251's vacuous pass in the one place it would be least visible. + let dir = repo("absent"); + let reached = walk( + r#"{"packages": [{"id": "batten", "name": "batten", "features": {}, "dependencies": []}], + "workspace_members": ["batten"], + "resolve": {"nodes": [{"id": "batten", "features": [], "deps": []}]}}"#, + ); + assert_eq!(reached.trim(), "absent", "the walk says so\n{reached}"); + record(&dir, &reached); + + let decided = run(&dir, &["check"]); + assert_eq!( + decided.status.code(), + Some(2), + "and the module is loud about it\n{}", + String::from_utf8_lossy(&decided.stderr) + ); +} + +#[test] +fn an_absent_record_says_nothing_rather_than_passing() { + // COULD NOT LOOK IS NOT A PASS, and on this surface it is also not a refusal. + // The producer writes nothing when `cargo metadata` will not resolve, so a + // module that refused here would refuse every checkout with no toolchain. + let dir = repo("unrecorded"); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "an absent record is silence\n{}", + String::from_utf8_lossy(&quiet.stderr) + ); +} + +// --- the walk itself --------------------------------------------------------- + +#[test] +fn an_io_crate_reachable_from_the_evaluator_is_refused_at_exit_2() { + let reached = walk(&chain("reqwest", "null", false, "")); + assert!( + reached.contains("crate reqwest"), + "the walk reaches it\n{reached}" + ); +} + +#[test] +fn an_io_crate_the_workspace_depends_on_directly_but_the_evaluator_does_not_is_not_the_evaluators() +{ + // THE LOAD-BEARING CASE. The obvious spelling walks from the workspace + // members, and on the real tree that finds `jsonschema` and `globset` — + // direct dependencies of `batten` itself, entering by paths that have nothing + // to do with the evaluator. It would deny on `main` forever. This fixture is + // that exact topology and the walk must be silent on it. + let reached = walk( + r#"{ + "packages": [ + {"id": "batten", "name": "batten", "features": {}, + "dependencies": [{"name": "regorus"}, {"name": "jsonschema"}, {"name": "globset"}]}, + {"id": "regorus", "name": "regorus", "features": {}, "dependencies": []}, + {"id": "jsonschema", "name": "jsonschema", "features": {}, "dependencies": []}, + {"id": "globset", "name": "globset", "features": {}, "dependencies": []} + ], + "workspace_members": ["batten"], + "resolve": {"nodes": [ + {"id": "batten", "features": [], "deps": [ + {"pkg": "regorus", "dep_kinds": [{"kind": null}]}, + {"pkg": "jsonschema", "dep_kinds": [{"kind": null}]}, + {"pkg": "globset", "dep_kinds": [{"kind": null}]}]}, + {"id": "regorus", "features": [], "deps": []}, + {"id": "jsonschema", "features": [], "deps": []}, + {"id": "globset", "features": [], "deps": []} + ]} +}"#, + ); + assert!( + !reached.contains("crate "), + "nothing the evaluator does not reach is reported\n{reached}" + ); +} + +#[test] +fn an_unactivated_optional_io_dependency_of_the_evaluator_is_not_reported() { + // `macos-link-check`'s `defmt` lesson applied here: an optional dependency + // nobody enabled is in the resolve and is not in the build, and a gate that + // fails on a crate the compiler never sees is not measuring what it names. + let reached = walk(&chain("reqwest", "null", true, r#""std""#)); + assert!( + !reached.contains("crate reqwest"), + "an unactivated optional dep is dormant\n{reached}" + ); +} + +#[test] +fn the_same_optional_dependency_activated_is_reported() { + // The other direction of the same filter. Without it the case above passes on + // a walk that reports nothing at all. + let reached = walk(&chain("reqwest", "null", true, r#""std", "http""#)); + assert!( + reached.contains("crate reqwest"), + "activating the feature reaches it\n{reached}" + ); +} + +#[test] +fn a_dev_dependency_of_the_evaluator_is_not_in_the_built_closure() { + // A dev-dependency of a DEPENDENCY is never built, so it is not in the + // closure a policy module could reach. (A workspace member's dev-dependency + // is built — the test binaries link — but that is the members' walk.) + let reached = walk(&chain("reqwest", r#""dev""#, false, "")); + assert!( + !reached.contains("crate reqwest"), + "a dev edge is not in the built closure\n{reached}" + ); +} + +#[test] +#[expect( + clippy::disallowed_types, + reason = "CLOUD-320's inventory row: resolving the REAL graph is the whole of this case, and it is the only evidence that the fixtures above agree with the tree `Cargo.toml`'s pin comment makes its claim about" +)] +fn the_repos_real_graph_is_clean_today() { + // The anti-vacuity arm, against the tree as it actually resolves. Every case + // above drives a fixture; this one is the only evidence that the pair agrees + // with reality, which is the claim `Cargo.toml`'s pin comment rests on. + let metadata = std::process::Command::new("cargo") + .args(["metadata", "--locked", "--format-version", "1"]) + .current_dir("../..") + .output() + .expect("cargo metadata runs"); + assert!(metadata.status.success(), "the lockfile is current"); + + let reached = walk(&String::from_utf8_lossy(&metadata.stdout)); + assert!( + reached.contains("closure "), + "the evaluator is in the graph\n{reached}" + ); + assert!( + !reached.contains("crate "), + "and none of the nine IO crates is in its closure\n{reached}" + ); +} diff --git a/crates/batten/tests/it/main.rs b/crates/batten/tests/it/main.rs index 6ea9d5f96..72267026b 100644 --- a/crates/batten/tests/it/main.rs +++ b/crates/batten/tests/it/main.rs @@ -122,6 +122,7 @@ mod done_not_landed; mod egress_fencing; mod emission_census; mod enforce_journal; +mod evaluator_closure; mod extension_surfaces; mod external_facts; mod extracted_facts; diff --git a/mise-tasks/evaluator-closure-check.sh b/mise-tasks/evaluator-closure-check.sh deleted file mode 100755 index c1d8f37b9..000000000 --- a/mise-tasks/evaluator-closure-check.sh +++ /dev/null @@ -1,213 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Gate: no IO-bearing crate is reachable from the evaluator's node in the resolved dependency graph (CLOUD-831)" -# -# CLOUD-831. `crates/batten/src/policy.rs` admits consumer-authored code to the -# MEDIATED CALL on one claim: a policy module "cannot open a file, start a -# process, or reach the network". That claim is the security boundary the moment -# a module decides a tool call, and until this gate landed it rested on a single -# unenforced line of `Cargo.toml` — `default-features = false`, keeping regorus's -# `http` and `jsonschema` out of the closure. -# -# THE DRIFT IS NOT AN EDIT, which is why a `forbid` row over the manifest text -# would not do. **Cargo unifies features across the graph**: a second crate in -# this workspace, or any dependency, taking `regorus` with default features -# unions them back on — with no edit to the line that states the pin and no diff -# a reviewer of that line would see. A renovate bump that changes regorus's own -# default feature set does the same. So the predicate has to read the RESOLVED -# GRAPH, which is what this does. -# -# ─── THE SCOPE IS THE EVALUATOR'S SUB-CLOSURE, NOT THE WORKSPACE'S ─────────── -# -# This is the whole correctness argument and it was measured before it was -# written, because the obvious spelling is wrong. Walking from the workspace -# members instead: -# -# from `regorus` 41 packages, none of the nine named -# from the workspace members 281 packages, `globset` AND `jsonschema` present -# -# `jsonschema` and `globset` are DIRECT dependencies of `batten` itself, entering -# by paths that have nothing to do with the evaluator — the schema surface and -# the glob matcher. So the manifest's claim is true and the wider predicate is -# false: it would deny on `main` today, on its first run, forever. Measured over -# the 5 lockfile-touching commits reachable from HEAD, the wider spelling fired -# on all 5 and every firing was a false positive (100%); the predicate shipped -# here fired 0 times, which is the intended state for a supply-chain tripwire -# rather than a detector (CLOUD-751's replay). -# -# Recorded so the wider spelling is not reintroduced as a "simplification". -# -# ─── ACTIVATED EDGES, NOT THE WHOLE RESOLVE ────────────────────────────────── -# -# `macos-link-check` already learned this one the expensive way — see its header -# on `defmt`, an unactivated optional dependency that made it refuse a link -# `darwin-link` then completed. `cargo metadata`'s resolve lists every package -# the resolver CONSIDERED, including optional dependencies nothing turned on. -# Scanning that asks "could some configuration of this tree reach the network" -# where the gate means "does this one". -# -# The activation reading is deliberately IDENTICAL to `macos-link-check`'s, down -# to the weak-dependency rule, because the two gates ask the same structural -# question of the same graph and a second, subtly different walk is how a pair -# like this drifts. If one is corrected, correct both. -# -# The measured spread that makes this matter here: whole resolve from `regorus` -# is 46 packages, activated is 41. Neither carries any of the nine, so the gate -# agrees today either way — the filter is what keeps it agreeing when an optional -# IO feature lands upstream and nobody enables it. -# -# NO `--filter-platform`, and that is deliberate rather than an omission. The -# pin is a claim about what a policy module can reach on any platform this crate -# ships to, so narrowing the graph to one target would let an IO crate arrive -# behind a `cfg` for a platform the gate does not run on. `macos-link-check` -# filters because its question IS about one target; this one is not. -# -#MUTANT closure-walks-the-workspace|s/^frontier = \[i for i in roots\]$/frontier = [m for m in members if m in nodes]/|not the evaluator's -#MUTANT closure-scans-unactivated|s/^ if (entry.get('rename') or entry\['name'\]) in enabled:$/ if True:/|unactivated optional IO dependency -set -euo pipefail - -cd "${EVALUATOR_ROOT:-$(git rev-parse --show-toplevel)}" - -# The evaluator's package name. Named once: it is the node the walk starts from -# AND the word the refusal uses, and two spellings of it is how a rename turns -# this gate silent instead of red. -readonly EVALUATOR=regorus - -# The nine `Cargo.toml`'s pin names. Not a heuristic and not "crates that look -# networky": this is the list the manifest comment claims is absent, restated -# here as the thing that decides. If the manifest's list changes, this changes -# with it — `rules-drift`'s lesson is that a restated constant with nothing -# holding the two in agreement is a defect waiting, so the manifest comment -# points AT this file rather than repeating the list a third time. -readonly IO_CRATES='reqwest jsonschema hyper rustls openssl-sys native-tls ring globset glob' - -# The graph normally comes from cargo. `BATTEN_EVALUATOR_METADATA` substitutes a -# recorded one, and exists so this gate can be shown able to FAIL (CLOUD-418): -# the live tree is clean by construction — that is the point of the gate — so a -# suite that could only run against it would assert a pass and never a refusal. -# Read-only, and never set outside `tests/evaluator-closure-check.bats`. -if [[ -n "${BATTEN_EVALUATOR_METADATA:-}" ]]; then - metadata=$(cat "$BATTEN_EVALUATOR_METADATA") || { - echo "::error:: evaluator-closure-check: could not read ${BATTEN_EVALUATOR_METADATA}" >&2 - exit 1 - } -else - # `--locked` because the question is about the COMMITTED resolution. A gate - # allowed to update the lockfile answers "what would upstream give me today", - # which is a property of the world rather than of this commit — the exact - # split `lock-complete` was carved out of `lock-check` to fix. - metadata=$(cargo metadata --locked --format-version 1 2>/dev/null) || { - echo "::error:: evaluator-closure-check: could not resolve the dependency graph — is the lockfile current? Run \`mise run lock-check\`." >&2 - exit 1 - } -fi - -report=$(printf '%s' "$metadata" | python3 -c " -import json, sys - -meta = json.load(sys.stdin) -evaluator = sys.argv[1] -io_crates = set(sys.argv[2].split()) - -packages = {p['id']: p for p in meta['packages']} -nodes = {n['id']: n for n in (meta.get('resolve') or {}).get('nodes', [])} -members = set(meta.get('workspace_members', [])) - - -def activated_keys(package, enabled): - # Three spellings reach a dependency and all three must be read, or an - # activated dep looks dormant: the implicit feature (a bare 'foo'), the - # namespaced form ('dep:foo'), and enabling one of the dep's own features - # ('foo/bar'). The WEAK form ('foo?/bar') is deliberately not one - it - # applies only if something else already activated the dep. - keys = set(enabled) - declared = package.get('features', {}) - for feature in enabled: - for token in declared.get(feature, []): - if token.startswith('dep:'): - keys.add(token[4:]) - elif '/' in token: - head = token.split('/', 1)[0] - if not head.endswith('?'): - keys.add(head) - return keys - - -def edges(node_id): - node = nodes.get(node_id) - package = packages.get(node_id) - if node is None or package is None: - return - enabled = activated_keys(package, set(node.get('features', []))) - is_member = node_id in members - for dep in node.get('deps', []): - target = packages.get(dep['pkg']) - if target is None: - continue - kinds = {k.get('kind') for k in dep.get('dep_kinds', [{}])} - if kinds == {'dev'} and not is_member: - continue - matching = [d for d in package.get('dependencies', []) if d['name'] == target['name']] - if not matching: - # An edge the manifest does not explain: keep it rather than drop - # it. Unexplained means unmeasured, and unmeasured fails closed. - yield dep['pkg'] - continue - for entry in matching: - if not entry.get('optional'): - yield dep['pkg'] - break - if (entry.get('rename') or entry['name']) in enabled: - yield dep['pkg'] - break - - -roots = [i for i, p in packages.items() if p['name'] == evaluator and i in nodes] -if not roots: - # NOT a pass. The evaluator vanishing from the graph means the question - # could not be asked, and reporting 'nothing found' there is the vacuous - # pass this repo names CLOUD-251. - print('ABSENT') - sys.exit(0) - -reached = set() -frontier = [i for i in roots] -while frontier: - current = frontier.pop() - if current in reached: - continue - reached.add(current) - frontier.extend(edges(current)) - -# Pointer-only (non-negotiable rule 4): the crate NAME and nothing else. Never a -# dependency tree, never a path through the graph, never a version chain. -found = sorted({packages[i]['name'] for i in reached} & io_crates) -print('COUNT %d' % len(reached)) -for name in found: - print('FOUND %s' % name) -" "$EVALUATOR" "$IO_CRATES") || { - echo "::error:: evaluator-closure-check: could not inspect the dependency graph" >&2 - exit 1 -} - -if [[ "$report" = ABSENT ]]; then - echo "::error:: evaluator-closure-check: no \`${EVALUATOR}\` node in the resolved graph, so the closure could not be walked at all. This is could-not-look, not a pass." >&2 - exit 1 -fi - -count=$(printf '%s\n' "$report" | sed -n 's/^COUNT //p') -found=$(printf '%s\n' "$report" | sed -n 's/^FOUND //p') - -if [[ -n "$found" ]]; then - echo "::error:: an IO-bearing crate is reachable from the evaluator, so a policy module can no longer be claimed to acquire nothing — and that claim is what admits consumer-authored code to the mediated call (crates/batten/src/policy.rs):" >&2 - printf '%s\n' "$found" | while IFS= read -r name; do printf ' %s\n' "$name" >&2; done - echo "Cargo unifies features across the graph, so this may have arrived without any edit to the \`regorus\` feature list. Find who enabled it (\`cargo tree -i \`), and close it there — the feature list in Cargo.toml is where the pin is stated, not where it is decided." >&2 - exit 2 -fi - -# The count is REPORTED, never asserted, and the split is deliberate. Which -# crates are absent is a property of THIS COMMIT and belongs in a gate; how many -# packages upstream happens to resolve to is a property of the world and would -# fire on every legitimate bump. `Cargo.toml`'s comment cites this line as where -# its number comes from, so the number has one source rather than a hand-count -# somebody re-takes and gets differently. -echo "evaluator-closure-check: none of the ${EVALUATOR} closure's ${count} packages is one of the nine IO-bearing crates the manifest pins out" diff --git a/mise-tasks/evaluator-closure.py b/mise-tasks/evaluator-closure.py new file mode 100644 index 000000000..7a814c992 --- /dev/null +++ b/mise-tasks/evaluator-closure.py @@ -0,0 +1,145 @@ +# The evaluator's activated sub-closure, over a `cargo metadata` document on +# stdin (CLOUD-831, ported off `mise-tasks/evaluator-closure-check.sh` under +# CLOUD-1717). +# +# THIS IS THE STEP, NEVER THE DECISION. It walks a graph and names what it +# reached; whether reaching one of them is a refusal is +# `policy/evaluator-closure.rego`'s question. The split is forced rather than +# chosen: a reachability closure is not expressible in Rego — a self-referential +# rule is a compile error and `graph.reachable` is not in this build's regorus +# feature set (`Cargo.toml` is the one authority on that list). +# +# IT IS A `.py` SIBLING RATHER THAN A TASK BODY, AND THAT IS WHAT KEEPS THE WALK +# UNDER TEST. `mise-tasks/replay-pointers.py` is the precedent and +# `shell-retirement.rego:159-163` is the reason it is admissible: `.py` is +# excluded from `under_mise_tasks`, so this is not an added shell rule. The four +# cases the dying suite carried about the WALK — an unactivated optional +# dependency, the same one activated, a dev-dependency, and a crate the workspace +# reaches but the evaluator does not — assert against this file directly from +# `crates/batten/tests/it/evaluator_closure.rs`. Inlining it into the task body +# would have made all four untestable, and they are the security-critical half. +# +# Output is the record's own lines, on stdout, for `batten record named`: +# +# absent no evaluator node in the graph — could-not-look +# closure packages reached from the evaluator +# crate one per IO-bearing crate in that closure +# +# Pointer-only (non-negotiable rule 4): a crate NAME and a count. Never a +# dependency tree, never a path through the graph, never a version chain. + +import json +import sys + +# The evaluator's package name. Named once: it is the node the walk starts from +# AND the word the refusal uses, and two spellings of it is how a rename turns +# this gate silent instead of red. +EVALUATOR = "regorus" + +# `Cargo.toml`'s pin names. Not a heuristic and not "crates that look networky": +# this is the list the manifest comment claims is absent, restated here as the +# thing that is looked for. The manifest comment points AT this file rather than +# repeating the list a third time. +IO_CRATES = frozenset( + "reqwest jsonschema hyper rustls openssl-sys native-tls ring globset glob".split() +) + + +def activated_keys(package, enabled): + """Three spellings reach a dependency and all three must be read, or an + activated dep looks dormant: the implicit feature (a bare 'foo'), the + namespaced form ('dep:foo'), and enabling one of the dep's own features + ('foo/bar'). The WEAK form ('foo?/bar') is deliberately not one — it applies + only if something else already activated the dep.""" + keys = set(enabled) + declared = package.get("features", {}) + for feature in enabled: + for token in declared.get(feature, []): + if token.startswith("dep:"): + keys.add(token[4:]) + elif "/" in token: + head = token.split("/", 1)[0] + if not head.endswith("?"): + keys.add(head) + return keys + + +def edges(node_id, packages, nodes, members): + node = nodes.get(node_id) + package = packages.get(node_id) + if node is None or package is None: + return + enabled = activated_keys(package, set(node.get("features", []))) + is_member = node_id in members + for dep in node.get("deps", []): + target = packages.get(dep["pkg"]) + if target is None: + continue + kinds = {k.get("kind") for k in dep.get("dep_kinds", [{}])} + if kinds == {"dev"} and not is_member: + continue + matching = [ + d for d in package.get("dependencies", []) if d["name"] == target["name"] + ] + if not matching: + # An edge the manifest does not explain: keep it rather than drop it. + # Unexplained means unmeasured, and unmeasured fails closed. + yield dep["pkg"] + continue + for entry in matching: + if not entry.get("optional"): + yield dep["pkg"] + break + if (entry.get("rename") or entry["name"]) in enabled: + yield dep["pkg"] + break + + +def walk(meta): + packages = {p["id"]: p for p in meta["packages"]} + nodes = {n["id"]: n for n in (meta.get("resolve") or {}).get("nodes", [])} + members = set(meta.get("workspace_members", [])) + + # THE SCOPE IS THE EVALUATOR'S SUB-CLOSURE, NOT THE WORKSPACE'S, and that was + # measured before it was written because the obvious spelling is wrong. + # Walking from the workspace members instead reaches 281 packages including + # `globset` AND `jsonschema` — both direct dependencies of `batten` itself, + # entering by paths that have nothing to do with the evaluator. The wider + # spelling fired on all 5 lockfile-touching commits reachable from HEAD and + # every firing was a false positive. + roots = [i for i, p in packages.items() if p["name"] == EVALUATOR and i in nodes] + if not roots: + # NOT a pass. The evaluator vanishing from the graph means the question + # could not be asked, and reporting 'nothing found' there is the vacuous + # pass this repository names CLOUD-251. + return ["absent"] + + reached = set() + frontier = list(roots) + while frontier: + current = frontier.pop() + if current in reached: + continue + reached.add(current) + frontier.extend(edges(current, packages, nodes, members)) + + found = sorted({packages[i]["name"] for i in reached} & IO_CRATES) + return ["closure %d" % len(reached)] + ["crate %s" % name for name in found] + + +def main(): + try: + meta = json.load(sys.stdin) + except (ValueError, OSError) as failed: + # The producer writes NOTHING when it could not look, which is the record + # contract: an absent record is "the producer did not run", and it must + # not be spelled the same way as a graph that resolved. + sys.stderr.write("evaluator-closure: could not read the graph: %s\n" % failed) + return 1 + for line in walk(meta): + sys.stdout.write(line + "\n") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/mise.toml b/mise.toml index 2d4480b29..307295d77 100644 --- a/mise.toml +++ b/mise.toml @@ -617,7 +617,7 @@ CI_FANIN_WORKFLOW = ".github/workflows/ci.yml" BATS_TEST_TIMEOUT = "300" REGORUS_OPA_COMPLIANCE = "1.2.0" REGORUS_OPA_COMPLIANCE_FOR = "0.11" -MUTANT_GATES = "mise,attestation,timeout-drift,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure-check,evaluator-io-check,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link-check,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-callable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus-check,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" +MUTANT_GATES = "mise,attestation,timeout-drift,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure,evaluator-io-check,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link-check,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-callable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus-check,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" # --- GitHub reachability behind an egress proxy (Claude Code web sandbox etc.) --- # mise resolves every tool's release through GitHub's *API* host, api.github.com. @@ -2065,6 +2065,48 @@ emit() { emit | cargo run --quiet -p batten -- record named timeout-drift ''' +# CLOUD-831's producer, ported off `mise-tasks/evaluator-closure-check.sh` under +# CLOUD-1717. The two things that are not decisions: the `cargo metadata` spawn, +# and the graph walk. +# +# THE WALK IS A `.py` SIBLING RATHER THAN A BODY HERE, and that is what keeps it +# under test. Four of the dying suite's eight cases are about the WALK — an +# unactivated optional dependency, the same one activated, a dev-dependency, and +# a crate the workspace reaches but the evaluator does not — and they are the +# security-critical half. Inlined here they would be assertable by nothing; +# beside the module they are `crates/batten/tests/it/evaluator_closure.rs`'s to +# drive. `mise-tasks/replay-pointers.py` is the precedent and +# `shell-retirement.rego:159-163` is why it is admissible: `.py` is excluded from +# `under_mise_tasks`, so this adds no shell rule. +# +# `--locked` because the question is about the COMMITTED resolution. A gate +# allowed to update the lockfile answers "what would upstream give me today", +# which is a property of the world rather than of this commit — the exact split +# `lock-complete` was carved out of `lock-check` to fix. +# +# `BATTEN_EVALUATOR_METADATA` substitutes a recorded graph and exists so the gate +# can be shown able to FAIL (CLOUD-418): the live tree is clean by construction, +# which is the point of the gate, so a suite that could only run against it would +# assert a pass and never a refusal. +# +# NOTHING IS WRITTEN WHEN THE GRAPH WILL NOT RESOLVE. That is the record +# contract: an absent record is "the producer did not run", and the module reads +# it as silence rather than as a clean closure. A graph that DID resolve without +# an evaluator node is a different answer and the walk records it as `absent`. +[tasks.evaluator-closure-record] +description = "Effect: record the evaluator's activated sub-closure and any IO-bearing crate in it, for `layer reach unsafe`" +shell = "bash -c" +run = """ +if [ -n "${BATTEN_EVALUATOR_METADATA:-}" ]; then + metadata=$(cat "$BATTEN_EVALUATOR_METADATA") || exit 0 +else + metadata=$(cargo metadata --locked --format-version 1 2>/dev/null) || exit 0 +fi +printf '%s' "$metadata" \ + | python3 mise-tasks/evaluator-closure.py \ + | cargo run --quiet -p batten -- record named evaluator-closure +""" + [tasks.branch-age-record] description = "Effect: record every remote branch's age in days and which names head more than one merged PR (CLOUD-349)" shell = "bash -c" diff --git a/policy/evaluator-closure.rego b/policy/evaluator-closure.rego new file mode 100644 index 000000000..581dadf5f --- /dev/null +++ b/policy/evaluator-closure.rego @@ -0,0 +1,132 @@ +# No IO-bearing crate is reachable from the evaluator's node in the resolved +# dependency graph (CLOUD-831, ported off `mise-tasks/evaluator-closure-check.sh` +# under CLOUD-1717). +# +# WHAT THE CLAIM IS AND WHY IT IS A SECURITY BOUNDARY. `crates/batten/src/policy.rs` +# admits consumer-authored code to the MEDIATED CALL on one claim: a policy +# module "cannot open a file, start a process, or reach the network". That claim +# decides a tool call, and until CLOUD-831 it rested on a single unenforced line +# of `Cargo.toml` — `default-features = false`, keeping regorus's `http` and +# `jsonschema` out of the closure. +# +# THE DRIFT IS NOT AN EDIT, which is why a `forbid` row over the manifest text +# would not do. Cargo unifies features across the graph, so a second crate in this +# workspace, or any dependency, taking `regorus` with default features unions them +# back on — with no edit to the line that states the pin and no diff a reviewer of +# that line would see. A renovate bump that changes regorus's own default feature +# set does the same. The predicate therefore has to read the RESOLVED GRAPH. +# +# THE SPLIT IS FORCED, NOT CHOSEN. The walk lives in +# `mise-tasks/evaluator-closure.py`, driven by `[tasks.evaluator-closure-record]`, +# for two reasons that are both about capability rather than taste. §5 makes +# `check` `read` and structurally incapable of spawning `cargo metadata`. And a +# reachability closure is not expressible here at all: a self-referential rule is +# a compile error in Rego, and `graph.reachable` is not in this build's regorus +# feature set — `ci-cache-declared`'s header records the same bound and expands +# its own walk by hand to a stated depth. An unbounded closure has no such +# spelling, so it stays a step. +# +# WHAT IS LEFT HERE IS THE WHOLE DECISION, AND IT IS THREE-VALUED. A record +# naming a crate is a refusal; a record saying `absent` is could-not-look and +# LOUD, because an evaluator that vanished from the graph means the question was +# never asked and reporting "nothing found" there is the vacuous pass this +# repository names CLOUD-251; no record at all is silence, because the producer +# writes nothing when it could not resolve the graph, and a module refusing there +# would refuse every checkout with no toolchain. +# +# THE COUNT IS REPORTED BY THE PRODUCER AND ASSERTED BY NOTHING. Which crates are +# absent is a property of THIS COMMIT and belongs in a gate; how many packages +# upstream happens to resolve to is a property of the world and would fire on +# every legitimate bump. +# +#MUTANT-SUITE crates/batten/tests/it/evaluator_closure.rs +#MUTANT io-crate-reachable-passes|s@^\tsome line in lines$@\tfalse #@|an_io_crate_in_the_recorded_closure_is_refused +#MUTANT absent-evaluator-passes|s@^\t"absent" in lines$@\tfalse@|a_recorded_absent_evaluator_is_loud_rather_than_clean + +# METADATA +# description: | +# Bound to the TREE surface: `scope = "tree"`, so it reads the tree document +# and never the mediated `{call, facts}` shape. +# THE BRACKETS ARE NOT STYLE: the schema file carries a hyphen, so the dotted +# form is a parse error reported as `invalid schema reference`. +# THIS BLOCK IS YAML AND MUST STAY THE LAST COMMENT BLOCK BEFORE `package`. +# schemas: +# - input: schema["policy-input.schema"] +package batten.evaluator_closure + +import rego.v1 + +rules contains "layer reach unsafe" + +# The record, or nothing. ABSENT IS NOT EMPTY: a checkout whose producer never +# ran has no key here at all, Rego reads that as *does not hold*, and every rule +# below is silent. An empty list would be a measured nothing and would say the +# graph resolved and reached no package, which is a different claim. +lines := input.tree.records["evaluator-closure"] + +# --- the refusal ------------------------------------------------------------ + +crate_named contains name if { + some line in lines + startswith(line, "crate ") + name := trim_space(substring(line, count("crate "), -1)) + name != "" +} + +violation contains { + "rule": "layer reach unsafe", + "verdict": "layer carry unsafe", + "subjects": [{"artifact": name}], +} if { + some name in crate_named +} + +# --- could not look, and loud ------------------------------------------------ +# +# The evaluator absent from the graph is a MEASUREMENT the producer made, not a +# failure to measure, so it is reported rather than silent. It is the one arm +# that distinguishes this module from one that has never run. +violation contains { + "rule": "layer reach unsafe", + "verdict": "layer read absent", + "subjects": [{"artifact": "regorus"}], +} if { + "absent" in lines +} + +# --- cases ------------------------------------------------------------------ + +test_an_io_crate_in_the_recorded_closure_is_refused if { + found := violation with input as {"tree": {"records": {"evaluator-closure": [ + "closure 41", + "crate jsonschema", + ]}}} + count(found) == 1 +} + +test_every_io_crate_is_named_separately if { + found := violation with input as {"tree": {"records": {"evaluator-closure": [ + "closure 41", + "crate jsonschema", + "crate reqwest", + ]}}} + count(found) == 2 +} + +test_a_clean_closure_is_silent if { + found := violation with input as {"tree": {"records": {"evaluator-closure": ["closure 41"]}}} + count(found) == 0 +} + +test_a_recorded_absent_evaluator_is_loud_rather_than_clean if { + found := violation with input as {"tree": {"records": {"evaluator-closure": ["absent"]}}} + count(found) == 1 +} + +# NO RECORD IS SILENCE, and it must not be spelled the same way as a clean +# closure. The producer writes nothing when `cargo metadata` could not resolve, +# and a module refusing there would refuse every checkout with no toolchain. +test_an_absent_record_says_nothing_rather_than_refusing if { + found := violation with input as {"tree": {"records": {}}} + count(found) == 0 +} diff --git a/tests/evaluator-closure-check.bats b/tests/evaluator-closure-check.bats deleted file mode 100644 index 5cba72fb9..000000000 --- a/tests/evaluator-closure-check.bats +++ /dev/null @@ -1,203 +0,0 @@ -#!/usr/bin/env bats -# subject: mise-tasks/evaluator-closure-check.sh -# The closure half of CLOUD-831's pin, and the reason it takes a metadata -# fixture rather than only running against the real tree. -# -# The live graph is clean by construction — that is what the gate exists to keep -# true — so a suite that could only ask the real tree would assert a pass and -# never a refusal, which is exactly the coverage-theatre shape CLOUD-418 names. -# `BATTEN_EVALUATOR_METADATA` substitutes a recorded graph, and the cases below -# are the four directions this predicate has to separate. Three of them cannot be -# produced by editing the real manifest at all. - -setup() { - GATE="$BATS_TEST_DIRNAME/../mise-tasks/evaluator-closure-check.sh" - META="$BATS_TEST_TMPDIR/metadata.json" -} - -# A `cargo metadata` document carrying just enough shape for the walk: a -# workspace member, the evaluator, and whatever else a case names. -# -# `$1` is the JSON body of `regorus`'s dependency list, `$2` the extra packages -# and nodes. Written as a here-doc rather than assembled with jq so a reader can -# see the whole graph a case asserts over. -metadata() { - cat >"$META" -} - -@test "the repo's real graph is clean today" { - run "$GATE" - [ "$status" -eq 0 ] - [[ "$output" == *"none of the regorus closure"* ]] -} - -@test "an IO crate reachable from the evaluator is refused at exit 2" { - metadata <<-'JSON' - { - "packages": [ - {"id": "batten", "name": "batten", "features": {}, - "dependencies": [{"name": "regorus"}]}, - {"id": "regorus", "name": "regorus", "features": {}, - "dependencies": [{"name": "reqwest"}]}, - {"id": "reqwest", "name": "reqwest", "features": {}, "dependencies": []} - ], - "workspace_members": ["batten"], - "resolve": {"nodes": [ - {"id": "batten", "features": [], "deps": [{"pkg": "regorus", "dep_kinds": [{"kind": null}]}]}, - {"id": "regorus", "features": [], "deps": [{"pkg": "reqwest", "dep_kinds": [{"kind": null}]}]}, - {"id": "reqwest", "features": [], "deps": []} - ]} - } - JSON - BATTEN_EVALUATOR_METADATA="$META" run "$GATE" - [ "$status" -eq 2 ] - [[ "$output" == *"reqwest"* ]] -} - -# THE LOAD-BEARING CASE. The obvious spelling of this predicate walks from the -# workspace members, and on the real tree that finds `jsonschema` and `globset` — -# direct dependencies of `batten` itself, entering by paths that have nothing to -# do with the evaluator. It would deny on `main` forever. This fixture is that -# exact topology, and the gate must be SILENT on it. -@test "an IO crate the workspace depends on directly, but the evaluator does not, is not the evaluator's" { - metadata <<-'JSON' - { - "packages": [ - {"id": "batten", "name": "batten", "features": {}, - "dependencies": [{"name": "regorus"}, {"name": "jsonschema"}, {"name": "globset"}]}, - {"id": "regorus", "name": "regorus", "features": {}, "dependencies": []}, - {"id": "jsonschema", "name": "jsonschema", "features": {}, "dependencies": []}, - {"id": "globset", "name": "globset", "features": {}, "dependencies": []} - ], - "workspace_members": ["batten"], - "resolve": {"nodes": [ - {"id": "batten", "features": [], "deps": [ - {"pkg": "regorus", "dep_kinds": [{"kind": null}]}, - {"pkg": "jsonschema", "dep_kinds": [{"kind": null}]}, - {"pkg": "globset", "dep_kinds": [{"kind": null}]}]}, - {"id": "regorus", "features": [], "deps": []}, - {"id": "jsonschema", "features": [], "deps": []}, - {"id": "globset", "features": [], "deps": []} - ]} - } - JSON - BATTEN_EVALUATOR_METADATA="$META" run "$GATE" - [ "$status" -eq 0 ] -} - -# The activation filter, which is `macos-link-check`'s `defmt` lesson applied -# here: an optional dependency nobody enabled is in the resolve and is not in the -# build, and a gate that fails on a crate the compiler never sees is not -# measuring what it names. -@test "an unactivated optional IO dependency of the evaluator is not reported" { - metadata <<-'JSON' - { - "packages": [ - {"id": "batten", "name": "batten", "features": {}, - "dependencies": [{"name": "regorus"}]}, - {"id": "regorus", "name": "regorus", "features": {"http": ["dep:reqwest"]}, - "dependencies": [{"name": "reqwest", "optional": true}]}, - {"id": "reqwest", "name": "reqwest", "features": {}, "dependencies": []} - ], - "workspace_members": ["batten"], - "resolve": {"nodes": [ - {"id": "batten", "features": [], "deps": [{"pkg": "regorus", "dep_kinds": [{"kind": null}]}]}, - {"id": "regorus", "features": ["std"], "deps": [{"pkg": "reqwest", "dep_kinds": [{"kind": null}]}]}, - {"id": "reqwest", "features": [], "deps": []} - ]} - } - JSON - BATTEN_EVALUATOR_METADATA="$META" run "$GATE" - [ "$status" -eq 0 ] -} - -# The other direction of the same filter: enable the feature and the same graph -# must refuse. Without this case the one above passes on a gate that reports -# nothing at all. -@test "the same optional dependency, activated, IS reported" { - metadata <<-'JSON' - { - "packages": [ - {"id": "batten", "name": "batten", "features": {}, - "dependencies": [{"name": "regorus"}]}, - {"id": "regorus", "name": "regorus", "features": {"http": ["dep:reqwest"]}, - "dependencies": [{"name": "reqwest", "optional": true}]}, - {"id": "reqwest", "name": "reqwest", "features": {}, "dependencies": []} - ], - "workspace_members": ["batten"], - "resolve": {"nodes": [ - {"id": "batten", "features": [], "deps": [{"pkg": "regorus", "dep_kinds": [{"kind": null}]}]}, - {"id": "regorus", "features": ["std", "http"], "deps": [{"pkg": "reqwest", "dep_kinds": [{"kind": null}]}]}, - {"id": "reqwest", "features": [], "deps": []} - ]} - } - JSON - BATTEN_EVALUATOR_METADATA="$META" run "$GATE" - [ "$status" -eq 2 ] - [[ "$output" == *"reqwest"* ]] -} - -# Could-not-look, never a pass. If the evaluator is not in the graph the question -# was not asked, and reporting "nothing found" there is CLOUD-251's vacuous pass -# in the one place it would be least visible. -@test "no evaluator node at all is could-not-look, not a clean bill" { - metadata <<-'JSON' - { - "packages": [{"id": "batten", "name": "batten", "features": {}, "dependencies": []}], - "workspace_members": ["batten"], - "resolve": {"nodes": [{"id": "batten", "features": [], "deps": []}]} - } - JSON - BATTEN_EVALUATOR_METADATA="$META" run "$GATE" - [ "$status" -eq 1 ] - [[ "$output" == *"could not be walked"* ]] -} - -# Non-negotiable rule 4. The refusal names the crate and nothing else — no -# version chain, no path through the graph, no dependency tree. -@test "the refusal is pointer-only: the crate name, never the path that reached it" { - metadata <<-'JSON' - { - "packages": [ - {"id": "batten", "name": "batten", "features": {}, "dependencies": [{"name": "regorus"}]}, - {"id": "regorus", "name": "regorus", "features": {}, "dependencies": [{"name": "secret-middle"}]}, - {"id": "secret-middle", "name": "secret-middle", "features": {}, "dependencies": [{"name": "ring"}]}, - {"id": "ring", "name": "ring", "features": {}, "dependencies": []} - ], - "workspace_members": ["batten"], - "resolve": {"nodes": [ - {"id": "batten", "features": [], "deps": [{"pkg": "regorus", "dep_kinds": [{"kind": null}]}]}, - {"id": "regorus", "features": [], "deps": [{"pkg": "secret-middle", "dep_kinds": [{"kind": null}]}]}, - {"id": "secret-middle", "features": [], "deps": [{"pkg": "ring", "dep_kinds": [{"kind": null}]}]}, - {"id": "ring", "features": [], "deps": []} - ]} - } - JSON - BATTEN_EVALUATOR_METADATA="$META" run "$GATE" - [ "$status" -eq 2 ] - [[ "$output" == *"ring"* ]] - [[ "$output" != *"secret-middle"* ]] -} - -# A dev-dependency of a DEPENDENCY is never built, so it is not in the closure a -# policy module could reach. (A workspace member's dev-dependency is built — the -# test binaries link — but that is the members' walk, not the evaluator's.) -@test "a dev-dependency of the evaluator is not in the built closure" { - metadata <<-'JSON' - { - "packages": [ - {"id": "batten", "name": "batten", "features": {}, "dependencies": [{"name": "regorus"}]}, - {"id": "regorus", "name": "regorus", "features": {}, "dependencies": [{"name": "reqwest"}]}, - {"id": "reqwest", "name": "reqwest", "features": {}, "dependencies": []} - ], - "workspace_members": ["batten"], - "resolve": {"nodes": [ - {"id": "batten", "features": [], "deps": [{"pkg": "regorus", "dep_kinds": [{"kind": null}]}]}, - {"id": "regorus", "features": [], "deps": [{"pkg": "reqwest", "dep_kinds": [{"kind": "dev"}]}]}, - {"id": "reqwest", "features": [], "deps": []} - ]} - } - JSON - BATTEN_EVALUATOR_METADATA="$META" run "$GATE" - [ "$status" -eq 0 ] -} From d4070293b6aebce13c1432ef4b337047b671f0d7 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sat, 19 Sep 2026 02:12:16 +0000 Subject: [PATCH 17/32] feat(policy): the macOS link gate decides in the engine, over a walk it now shares MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `mise-tasks/macos-link-check.sh` and its suite are gone. The decision is `policy/macos-link.rego`; the `cargo metadata --filter-platform` spawn and the graph walk are `[tasks.macos-link-record]`'s. `hk.pkl`'s step is repointed to the pair. ## The duplication this removes was already written down as a warning Both this program and `evaluator-closure-check` carried the same 60-line activation walk, and both headers said so: > The activation reading is deliberately IDENTICAL to `macos-link-check`'s, down > to the weak-dependency rule, because the two gates ask the same structural > question of the same graph and a second, subtly different walk is how a pair > like this drifts. If one is corrected, correct both. That is a rule with no runnable gate, which non-negotiable rule 2 calls half a change — and porting this one by copying the walk a third time would have been the laundering the campaign exists to stop. `mise-tasks/cargo_graph.py` is the mechanism: one walk, so the two cannot disagree, and the `#MUTANT` rows that were stated twice are stated once over the code they actually mutate. The two gates now differ only in their ROOTS, which is the real difference between them: `evaluator-closure` walks one package's sub-closure (its pin is a claim about `regorus`), `macos-link` walks the workspace members (its question is about everything this tree builds). `Graph.named_roots` and `Graph.member_roots` are those two answers. ## Two rules, kept as two classes Rule 1 is the manifest's own `links` declaration and names the library it declares; rule 2 is a named set of crates that link Apple frameworks from a build script, which rule 1 cannot see. A reader acts on them differently, so they are `manifest carry unsafe` and `workspace reach unsafe` rather than one class with two shapes. Rule 2 is a list and therefore incomplete by construction. That residual gap is closed by actually linking the target, which `darwin-link` does; this pair is the fast, specific, early half and not a replacement for it. ## `--filter-platform` is the question, not an optimisation A macOS-only transitive dep must be seen and a Linux-only one must not, so the producer resolves the graph FOR `aarch64-apple-darwin`. `evaluator-closure-record` deliberately does NOT filter, and the asymmetry is stated at both sites: that gate's pin is a claim about every platform this crate ships to, so narrowing would let an IO crate arrive behind a `cfg` for a platform the gate does not run on. ## The ledger has three honest losses, and each says why `--filter-platform` is a property of the SPAWN, so "the graph is resolved for macOS, not for the host" moved to the producer as a `// changed:` arm — the module reads whatever graph it was handed and cannot observe which platform produced it. Two cases are `// withdrawn:`, and both for one reason: they grepped the shell program's regex literals. "the framework crate list covers the ones that actually bit us" asserted four names appear in `FRAMEWORK_CRATES`; "every vendored-links entry names a crate" asserted `VENDORED_LINKS` contains no `.*`. Those are assertions about a SPELLING. The behavioural statements of the same two properties — `rule_2_still_fires_through_the_reachability_walk` and `an_unvetted_links_crate_is_still_reported_so_the_exemption_is_a_list_not_a_switch` — survive a rewrite of either pattern that the text assertions would not, and re-breaking on every legitimate addition to a list is how a gate gets an exception written for it. ## Proven live, not merely green `mise run batten -- check --rule 'workspace carry unsafe'` over the real tree with a poisoned record: `core-foundation workspace carry unsafe`, exit 2; with the true record, silent. Exit 0 alone is byte-identical to a dead gate, which is the class `branch-age` sat in for a whole session while its own suite stayed green (CLOUD-1810). `the_repo_as_it_stands_has_no_sdk_requiring_dependency` runs the pair against the graph `darwin-link` actually links. ## Corpus 77 -> 76 programs, 88 -> 87 suites. 9 of 44 retired. Refs: CLOUD-1717 Refs: CLOUD-224 Refs: CLOUD-718 Admits: 6a9808bfb0a51af223f2f644ca05de87112409fc74e2ce5224183e355c71faa9 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: batten.toml Admits-anchor: call:29dd210e1ac61a350037a154ed0026fb53f8ec70 Admits-epoch: 970fb0515a283d5e9fea077df118a62ae454925e89072369f16fd8630fb4f814 Admits-author: alec@wenzowski.com Admits-prev: b557d27683cbf6f9ec7ff7a863cf8c54611f7d19dd0576aa7d2ceceae17fc039 Admits-answer-lost: The retirement cannot land and the tree will not load. Beyond that, the gate itself is what keeps the SDK-free macOS release build working: the artifacts are linked on Linux by zig with no Apple SDK present, and `cross-check` runs `cargo check` which never links, so nothing else in the tree can see a framework-linking dependency. Without the row the first symptom is the release workflow failing after a tag has already been cut. Admits-answer-precondition: batten.toml is the only surface carrying [[rule]], [[verdict]] and [[record]] rows. CLOUD-1717 retires mise-tasks/macos-link-check.sh onto policy/macos-link.rego plus [tasks.macos-link-record]; the module raises `manifest carry unsafe` and `workspace reach unsafe` and reads the `macos-link` record family, and the engine refuses at load both a module raising an undeclared token and a declared row nothing raises. No other file can carry any of it. It lands in PR #962 as an ordinary diff hunk beside the evaluator-closure rows admitted earlier on this branch. Admits-answer-rejected-route: `config read first` is a read of batten.toml; I read it to confirm the three new ids are unused and to draw each name from the declared [vocabulary] subject, action and condition lists. Reading cannot declare the rows the module raises. `patch run first` is `git restore`, which would discard the module, the producer and the shared walk together rather than complete them. Admits: d0eee392d131db1379aff1a5c9a323a849c69324698ee44ab8c958adc12ada76 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: hk.pkl Admits-anchor: call:29dd210e1ac61a350037a154ed0026fb53f8ec70 Admits-epoch: dccd3e8ec795ffeaef64673c04278cfce70fdd718ddd05f2644213945135231e Admits-author: alec@wenzowski.com Admits-prev: - Admits-answer-lost: The pre-commit gate that keeps the SDK-free macOS release build working stops running, and it fails in the direction that hides: an hk step whose task is missing is a step that errors rather than one that judges, so the signal it exists to give — a framework-linking dependency added to the graph — goes back to first appearing as a release-workflow failure after a tag has already been cut, which is the exact failure CLOUD-224 built it against. The new `workflow run unknown` gate does not cover this caller either, since it reads workflow `run:` scalars and this is a pkl step. Admits-answer-precondition: The dead caller IS a step in this file. `hk.pkl:723-726` declares a `macos-link-check` step whose `check` is `mise run macos-link-check`, a task CLOUD-1717 deletes when it ports that program to policy/macos-link.rego plus [tasks.macos-link-record]. hk selects steps by name, so nothing but this file can repoint it; leaving it names a task that does not exist. It lands in PR #962 as an ordinary diff hunk. Admits-answer-rejected-route: `config read first` is a read of batten.toml, not of hk.pkl, and I have done it — it is how I confirmed the new rule id `workspace carry unsafe` and the record family name. It cannot repoint an hk step. `patch run first` is `git restore`, which would revert a file whose committed state names the deleted task, reinstating the broken caller. --- batten.toml | 50 ++++ bench/suites/RESULTS.md | 5 +- crates/batten/tests/it/macos_link.rs | 374 +++++++++++++++++++++++++++ crates/batten/tests/it/main.rs | 1 + hk.pkl | 9 +- mise-tasks/cargo_graph.py | 127 +++++++++ mise-tasks/evaluator-closure.py | 86 +----- mise-tasks/macos-link-check.sh | 202 --------------- mise-tasks/macos-link.py | 102 ++++++++ mise.toml | 34 ++- policy/macos-link.rego | 130 ++++++++++ tests/macos-link-check.bats | 154 ----------- 12 files changed, 840 insertions(+), 434 deletions(-) create mode 100644 crates/batten/tests/it/macos_link.rs create mode 100644 mise-tasks/cargo_graph.py delete mode 100755 mise-tasks/macos-link-check.sh create mode 100644 mise-tasks/macos-link.py create mode 100644 policy/macos-link.rego delete mode 100644 tests/macos-link-check.bats diff --git a/batten.toml b/batten.toml index 3aba09074..b1d5f6e6e 100644 --- a/batten.toml +++ b/batten.toml @@ -5959,6 +5959,56 @@ no_fix_reason = "an IO crate reaching the evaluator is closed where it was enabl record = "evaluator-closure" writer = "mise run evaluator-closure-record" +# The macOS link gate, ported under CLOUD-1717 from `mise-tasks/macos-link-check.sh` +# and from an `hk` step that spawned it. The release artifacts for macOS are +# linked on Linux by zig with no Apple SDK present, and `cross-check` runs +# `cargo check` — which stops at codegen-to-metadata and never links — so nothing +# else in this tree can see a framework-linking dependency before a tag is cut. +[[rule]] +id = "workspace carry unsafe" +kind = "policy" +scope = "tree" +module = "policy/macos-link.rego" +severity = "deny" +no_fix_reason = "a dependency needing a macOS SDK is not fixable by a rewrite here: either drop it, feature-gate it off the default build (a rustls-style alternative usually exists), or accept an SDK and revisit how macOS is built" + +[[record]] +record = "macos-link" +writer = "mise run macos-link-record" + +[[verdict]] +id = "manifest carry unsafe" +gloss = "a package in the built macOS graph declares a `links` key, so it needs a real SDK to find that library" +class = """ +Rule 1, and it is the manifest's OWN statement rather than a guess: a `links` key \ +says the package links a native library. The exemption is a list of crates that \ +vendor and compile their own C, for which "links a native library" does not imply \ +"needs an SDK to find one" — and an UNKNOWN `links` crate is still a finding, so \ +the list narrows the gate rather than opening it. Adding a name to it requires a \ +link, never a reading of the crate. +""" + +[[verdict.route]] +id = "module read first" +kind = "document" +target = "policy/macos-link.rego" + +[[verdict]] +id = "workspace reach unsafe" +gloss = "a crate that links an Apple system framework from a build script is reachable in the built macOS graph" +class = """ +Rule 2, and it is a LIST, so it is incomplete by construction: a crate nobody has \ +listed slips past. That residual gap is closed by actually linking the target, \ +which `darwin-link` does — this is the fast, specific, early half of that pair and \ +not a replacement for it. Prefer proving a new name with a link over adding it on \ +a reading. +""" + +[[verdict.route]] +id = "module read first" +kind = "document" +target = "policy/macos-link.rego" + # The contract half of the config deprecation grammar (CLOUD-360). A key that # leaves the published schema with no window breaks every consumer still carrying # it, and breaks them SILENTLY: their config stops loading with an unknown-key diff --git a/bench/suites/RESULTS.md b/bench/suites/RESULTS.md index b949586d4..adbb40508 100644 --- a/bench/suites/RESULTS.md +++ b/bench/suites/RESULTS.md @@ -6,8 +6,8 @@ runner measured it; the suite runs `--no-parallelize-within-files`, so a file's number is its own serial cost and is what an author adding a case to it pays. -- suites: 98 -- serial total: 158.3s +- suites: 97 +- serial total: 157.8s | seconds | share | suite | | ---: | ---: | --- | @@ -64,7 +64,6 @@ to it pays. | 0.6 | 0.4% | `tests/mcp-attach-check.bats` | | 0.5 | 0.3% | `tests/duplicate-close-check.bats` | | 0.5 | 0.3% | `tests/suite-bench-check.bats` | -| 0.5 | 0.3% | `tests/macos-link-check.bats` | | 0.4 | 0.3% | `tests/checksums.bats` | | 0.4 | 0.3% | `tests/publish-credential-check.bats` | | 0.4 | 0.3% | `tests/msrv-pin-agreement.bats` | diff --git a/crates/batten/tests/it/macos_link.rs b/crates/batten/tests/it/macos_link.rs new file mode 100644 index 000000000..b6355ed88 --- /dev/null +++ b/crates/batten/tests/it/macos_link.rs @@ -0,0 +1,374 @@ +//! `workspace carry unsafe` over the compiled binary and the real walk +//! (CLOUD-224, CLOUD-718, CLOUD-1717). +//! +//! # Why this tier exists and the module's own `test_` rules do not suffice +//! +//! `policy/macos-link.rego` carries five load-time cases and every one fabricates +//! its input with `with input as`, which is the shape `rules/policy-modules.md` +//! warns about: the case asserts over a record the engine may be unable to +//! project, and the module stays green while the row decides nothing on any real +//! checkout. +//! +//! # And why the WALK is driven here rather than described +//! +//! Most of the dying suite is about the walk, not the verdict — the activation +//! filter in both directions, the weak-dependency rule, the dev-edge rule, and +//! which roots the walk starts from. Those are the cases that cost real time to +//! get right: `defmt`, an unactivated optional dependency of `jiff` reaching no +//! Apple framework and never compiled, made this gate refuse a link +//! `darwin-link` then completed on the same tree (CLOUD-718). +//! +//! The walk is `mise-tasks/cargo_graph.py`, shared with `evaluator-closure`, and +//! `mise-tasks/macos-link.py` is this gate's roots and predicates over it. Both +//! programs used to carry their own copy and both headers said "if one is +//! corrected, correct both" — a rule with no mechanism. There is one walk now, +//! and these cases drive it. +//! +//! # RETIREMENT LEDGER, PER PATH — what `shell retire partial` reads +//! +//! Two cases are NOT carried and each says why in its own row rather than being +//! dropped quietly. +//! +// carried: mise-tasks/macos-link-check.sh policy/macos-link.rego kind:mechanism crates/batten/tests/it/macos_link.rs +// carried: tests/macos-link-check.bats policy/macos-link.rego kind:mechanism crates/batten/tests/it/macos_link.rs +// carried: "the repo as it stands has no SDK-requiring dependency" policy/macos-link.rego kind:mechanism +// carried: "a package declaring a native links key is caught without being listed" policy/macos-link.rego kind:mechanism +// carried: "an optional dependency nobody enabled is not reported" policy/macos-link.rego kind:mechanism +// carried: "the same optional dependency, once enabled, is reported" policy/macos-link.rego kind:mechanism +// carried: "A WEAK REFERENCE IS NOT AN ACTIVATION: dep-question-mark leaves the dep dormant" policy/macos-link.rego kind:mechanism +// carried: "rule 2 still fires through the reachability walk" policy/macos-link.rego kind:mechanism +// carried: "a vendored-C links crate is exempt from rule 1" policy/macos-link.rego kind:mechanism +// carried: "an unvetted links crate is still reported, so the exemption is a list not a switch" policy/macos-link.rego kind:mechanism +// carried: "the walk starts at the workspace members" policy/macos-link.rego kind:mechanism +// changed: "the graph is resolved for macOS, not for the host" mise.toml the `--filter-platform aarch64-apple-darwin` flag is a property of the SPAWN, so it moved to `[tasks.macos-link-record]` with the `cargo metadata` call it qualifies. The module reads whatever graph the producer recorded and cannot observe which platform it was resolved for; a case here would assert over input this surface cannot vary +// withdrawn: "the framework crate list covers the ones that actually bit us" the case grepped the shell program's FRAMEWORK_CRATES literal for four names. The list is `mise-tasks/macos-link.py`'s now and the assertion was over a spelling rather than a behaviour — `rule_2_still_fires_through_the_reachability_walk` pins what the list is FOR, and a second case re-reading its text would re-break on every legitimate addition +// withdrawn: "every vendored-links entry names a crate, so the pattern cannot be widened to a wildcard" the same shape one list over: it asserted that the VENDORED_LINKS regex contains no `.*`. `an_unvetted_links_crate_is_still_reported` is the behavioural statement of the same property — the exemption is a list rather than a switch — and it survives a rewrite of the pattern that the text assertion would not + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use common::{git_in, init_repo, run, run_with_stdin, scratch, write}; + +/// A repository registering the real module against the declared family. +fn repo(name: &str) -> std::path::PathBuf { + let dir = scratch(&format!("macos-link-{name}")); + let module = std::fs::read_to_string("../../policy/macos-link.rego") + .expect("the module this tier exists for"); + write(&dir, "policy/macos-link.rego", &module); + write( + &dir, + "batten.toml", + r#"version = 1 +scope = ["**"] + +[[verdict]] +id = "manifest carry unsafe" +gloss = "a package in the built macOS graph declares a `links` key" +class = "The manifest's own statement that it links a native library." + +[[verdict.route]] +id = "module read first" +kind = "document" +target = "policy/macos-link.rego" + +[[verdict]] +id = "workspace reach unsafe" +gloss = "a crate linking an Apple system framework is reachable" +class = "Rule 2, and it is a list, so it is incomplete by construction." + +[[verdict.route]] +id = "module read first" +kind = "document" +target = "policy/macos-link.rego" + +[[rule]] +id = "workspace carry unsafe" +kind = "policy" +scope = "tree" +module = "policy/macos-link.rego" +severity = "deny" + +[[record]] +record = "macos-link" +writer = "mise run macos-link-record" +"#, + ); + init_repo(&dir); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-qm", "register the module"]); + dir +} + +fn record(dir: &std::path::Path, lines: &str) { + let written = run_with_stdin(dir, &["record", "named", "macos-link"], lines); + assert!( + written.status.success(), + "the setup write lands: {}", + String::from_utf8_lossy(&written.stderr) + ); +} + +/// Run the REAL walk over a `cargo metadata` document, exactly as the producer +/// does. This is what keeps the walk cases assertable after the port. +#[expect( + clippy::disallowed_types, + reason = "CLOUD-320's inventory row, and the verdict is that this spawn IS the subject under test: the walk moved out of the dying program into `mise-tasks/macos-link.py` over the shared `cargo_graph.py` precisely so a compiled case could drive it, and a harness re-implementing it in Rust would be the second authority both programs' headers warned about" +)] +fn walk(metadata: &str) -> String { + use std::io::Write as _; + use std::process::{Command, Stdio}; + + let mut child = Command::new("python3") + .arg("../../mise-tasks/macos-link.py") + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .expect("the walk this tier exists for"); + child + .stdin + .as_mut() + .expect("stdin") + .write_all(metadata.as_bytes()) + .expect("feed the graph"); + let done = child.wait_with_output().expect("the walk completes"); + assert!( + done.status.success(), + "the walk reads its graph: {}", + String::from_utf8_lossy(&done.stderr) + ); + String::from_utf8_lossy(&done.stdout).into_owned() +} + +/// A graph where the member depends on `edge`, optionally behind a feature. +/// +/// `extra` goes inside `edge`'s own package object, which is how a `links` key +/// or a framework name is attached to it. +fn member_chain(edge: &str, extra: &str, optional: bool, member_features: &str) -> String { + let optional_decl = if optional { + r#", "optional": true"# + } else { + "" + }; + format!( + r#"{{ + "packages": [ + {{"id": "batten", "name": "batten", "features": {{"tls": ["dep:{edge}"]}}, + "dependencies": [{{"name": "{edge}"{optional_decl}}}]}}, + {{"id": "{edge}", "name": "{edge}", "features": {{}}, "dependencies": []{extra}}} + ], + "workspace_members": ["batten"], + "resolve": {{"nodes": [ + {{"id": "batten", "features": [{member_features}], "deps": [{{"pkg": "{edge}", "dep_kinds": [{{"kind": null}}]}}]}}, + {{"id": "{edge}", "features": [], "deps": []}} + ]}} +}}"# + ) +} + +// --- the decision, over the engine's own projection -------------------------- + +#[test] +fn a_package_declaring_a_native_links_key_is_caught_without_being_listed() { + let dir = repo("links"); + let reached = walk(&member_chain( + "openssl-sys", + r#", "links": "openssl""#, + false, + "", + )); + assert!( + reached.contains("links openssl-sys openssl"), + "the walk names the library\n{reached}" + ); + record(&dir, &reached); + + let decided = run(&dir, &["check"]); + assert_eq!( + decided.status.code(), + Some(2), + "a links key decides\n{}", + String::from_utf8_lossy(&decided.stderr) + ); + let said = format!( + "{}{}", + String::from_utf8_lossy(&decided.stdout), + String::from_utf8_lossy(&decided.stderr) + ); + assert!(said.contains("openssl-sys"), "and names it\n{said}"); +} + +#[test] +fn rule_2_still_fires_through_the_reachability_walk() { + // The named set rule 1 cannot see: a crate that links an Apple framework from + // a build script and declares no `links` key. + let dir = repo("framework"); + let reached = walk(&member_chain("core-foundation", "", false, "")); + assert!( + reached.contains("framework core-foundation"), + "the walk names it\n{reached}" + ); + record(&dir, &reached); + + let decided = run(&dir, &["check"]); + assert_eq!( + decided.status.code(), + Some(2), + "a framework crate decides\n{}", + String::from_utf8_lossy(&decided.stderr) + ); +} + +#[test] +fn a_vendored_c_links_crate_is_exempt_from_rule_1() { + // `links` reads as "declares it links a native library", and this gate treats + // that as "needs SDKROOT pointing at a genuine macOS SDK". For a crate that + // ships C sources and builds them with `cc`, the second does not follow. + // Measured 2026-08-21: this gate refused `tree-sitter`, and `darwin-link` + // then linked the same tree with no SDK present. + let reached = walk(&member_chain( + "tree-sitter", + r#", "links": "tree-sitter""#, + false, + "", + )); + assert!( + !reached.contains("links "), + "a vendored-C links crate is exempt\n{reached}" + ); +} + +#[test] +fn an_unvetted_links_crate_is_still_reported_so_the_exemption_is_a_list_not_a_switch() { + // The other direction of the same narrowing. Without this the case above + // passes on a walk that stopped reading `links` at all. + let reached = walk(&member_chain("some-sys", r#", "links": "some""#, false, "")); + assert!( + reached.contains("links some-sys some"), + "an unknown links crate is still a finding\n{reached}" + ); +} + +#[test] +fn an_absent_record_says_nothing_rather_than_passing() { + let dir = repo("unrecorded"); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "an absent record is silence\n{}", + String::from_utf8_lossy(&quiet.stderr) + ); +} + +// --- the walk itself --------------------------------------------------------- + +#[test] +fn an_optional_dependency_nobody_enabled_is_not_reported() { + // CLOUD-718's `defmt`: an unactivated optional dependency is in the resolve + // and is not in the build, and a gate that fails on a crate the compiler + // never sees is not measuring the thing it names. + let reached = walk(&member_chain("core-foundation", "", true, r#""default""#)); + assert!( + !reached.contains("framework "), + "an unactivated optional dep is dormant\n{reached}" + ); +} + +#[test] +fn the_same_optional_dependency_once_enabled_is_reported() { + let reached = walk(&member_chain( + "core-foundation", + "", + true, + r#""default", "tls""#, + )); + assert!( + reached.contains("framework core-foundation"), + "enabling the feature reaches it\n{reached}" + ); +} + +#[test] +fn a_weak_reference_is_not_an_activation() { + // `foo?/bar` applies only if something ELSE already activated `foo`, so + // reading it as an activation walks back to the whole-resolve over-scan. + let reached = walk( + r#"{ + "packages": [ + {"id": "batten", "name": "batten", "features": {"tls": ["core-foundation?/std"]}, + "dependencies": [{"name": "core-foundation", "optional": true}]}, + {"id": "core-foundation", "name": "core-foundation", "features": {}, "dependencies": []} + ], + "workspace_members": ["batten"], + "resolve": {"nodes": [ + {"id": "batten", "features": ["tls"], "deps": [{"pkg": "core-foundation", "dep_kinds": [{"kind": null}]}]}, + {"id": "core-foundation", "features": [], "deps": []} + ]} +}"#, + ); + assert!( + !reached.contains("framework "), + "a weak reference leaves the dep dormant\n{reached}" + ); +} + +#[test] +fn the_walk_starts_at_the_workspace_members() { + // Unlike `evaluator-closure`, whose roots are one package's nodes. A package + // in the resolve that no member reaches is not in the built graph. + let reached = walk( + r#"{ + "packages": [ + {"id": "batten", "name": "batten", "features": {}, "dependencies": []}, + {"id": "orphan", "name": "core-foundation", "features": {}, "dependencies": []} + ], + "workspace_members": ["batten"], + "resolve": {"nodes": [ + {"id": "batten", "features": [], "deps": []}, + {"id": "orphan", "features": [], "deps": []} + ]} +}"#, + ); + assert!( + !reached.contains("framework "), + "an unreached package is not in the built graph\n{reached}" + ); + assert!( + reached.contains("scanned 1"), + "and the count is the members' closure\n{reached}" + ); +} + +#[test] +#[expect( + clippy::disallowed_types, + reason = "CLOUD-320's inventory row: resolving the REAL macOS graph is the whole of this case, and it is the only evidence that the fixtures above agree with the tree the release actually links" +)] +fn the_repo_as_it_stands_has_no_sdk_requiring_dependency() { + // The anti-vacuity arm, against the graph as it actually resolves FOR macOS. + // Every case above drives a fixture; this one is the only evidence that the + // pair agrees with the tree `darwin-link` links. + let metadata = std::process::Command::new("cargo") + .args([ + "metadata", + "--format-version", + "1", + "--filter-platform", + "aarch64-apple-darwin", + ]) + .current_dir("../..") + .output() + .expect("cargo metadata runs"); + assert!(metadata.status.success(), "the macOS graph resolves"); + + let reached = walk(&String::from_utf8_lossy(&metadata.stdout)); + assert!(reached.contains("scanned "), "the walk ran\n{reached}"); + assert!( + !reached.contains("links ") && !reached.contains("framework "), + "and nothing in the macOS graph needs an SDK to link\n{reached}" + ); +} diff --git a/crates/batten/tests/it/main.rs b/crates/batten/tests/it/main.rs index 72267026b..76465dd1e 100644 --- a/crates/batten/tests/it/main.rs +++ b/crates/batten/tests/it/main.rs @@ -179,6 +179,7 @@ mod lease_precondition; mod lease_record; mod locator_index; mod lock_complete; +mod macos_link; mod mcp_dispatch; mod mcp_reduce_array; mod mediated_admission; diff --git a/hk.pkl b/hk.pkl index ec853d42c..152fb2f6f 100644 --- a/hk.pkl +++ b/hk.pkl @@ -720,9 +720,16 @@ local gate = new Mapping { // Globbed on the manifests alone (CLOUD-224): the predicate is `cargo metadata // --filter-platform` over the resolved graph, so the inputs are exactly the // files that can move that graph. Nothing else can add a dependency. + // + // TWO COMMANDS SINCE CLOUD-1717, and the split is forced rather than chosen: + // section 5 makes `check` `read` and structurally incapable of spawning + // `cargo metadata`, so the producer resolves the graph and the engine + // adjudicates what it recorded. The step name and the rule id are one object, + // which is why the second command names the rule rather than running a bare + // `batten check` under a name that promises the link gate. ["macos-link-check"] { glob = List("Cargo.toml", "Cargo.lock", "crates/**/Cargo.toml") - check = "mise run macos-link-check" + check = "mise run macos-link-record && mise run batten -- check --rule 'workspace carry unsafe'" } // --- shell: the mise-tasks/ programs and the bats suite ------------------- diff --git a/mise-tasks/cargo_graph.py b/mise-tasks/cargo_graph.py new file mode 100644 index 000000000..27350c391 --- /dev/null +++ b/mise-tasks/cargo_graph.py @@ -0,0 +1,127 @@ +# The ACTIVATED dependency graph, read from a `cargo metadata` document. +# +# ONE WALK, TWO GATES, AND THAT IS THE WHOLE REASON THIS FILE EXISTS. +# `evaluator-closure` asks whether an IO-bearing crate is reachable from the +# evaluator's node; `macos-link` asks whether anything in the built graph needs a +# real macOS SDK to link. The questions differ only in their ROOTS and in what +# they look for once there — the reachability underneath is the same, down to the +# weak-dependency rule. +# +# Both programs carried their own copy, and both headers said so in prose: +# "The activation reading is deliberately IDENTICAL to `macos-link-check`'s, down +# to the weak-dependency rule, because the two gates ask the same structural +# question of the same graph and a second, subtly different walk is how a pair +# like this drifts. If one is corrected, correct both." +# +# That is a rule with no mechanism, which is half a change. This file is the +# mechanism: there is now one walk to correct, so the two cannot disagree. The +# `#MUTANT` rows that used to be stated twice are stated once, over the code they +# actually mutate. +# +# `.py` rather than a task body, and `mise-tasks/replay-pointers.py` is the +# precedent: `shell-retirement.rego:159-163` excludes `.py` from +# `under_mise_tasks`, so this adds no shell rule and both callers' compiled tiers +# can drive it directly. + +# Reverting the activation filter to the whole resolve is the defect this exists +# to prevent: an optional dependency nobody enabled reads as linked, and the gate +# refuses a link that succeeds. Measured on `defmt` — an embedded-logging crate, +# an unactivated optional dependency of `jiff`, reaching no Apple framework and +# never compiled — which made `macos-link-check` refuse a link `darwin-link` then +# completed on the same tree. +#MUTANT-SUITE crates/batten/tests/it/evaluator_closure.rs +#MUTANT graph-scans-unactivated|s@^ if (entry.get("rename") or entry\["name"\]) in enabled:$@ if True:@|an_unactivated_optional_io_dependency_of_the_evaluator_is_not_reported +#MUTANT graph-weak-dep-activates|s@^ head = token.split("/", 1)\[0\]$@ head = token.split("/", 1)[0].rstrip("?")@|a_weak_reference_is_not_an_activation +#MUTANT graph-keeps-dev-edges|s@^ if kinds == {"dev"} and not is_member:$@ if False:@|a_dev_dependency_of_the_evaluator_is_not_in_the_built_closure + + +def activated_keys(package, enabled): + """The dependency names this package's enabled features actually turn on. + + Three spellings reach a dependency and all three must be read, or an + activated dep looks dormant: the implicit feature (a bare 'foo'), the + namespaced form ('dep:foo'), and enabling one of the dep's own features + ('foo/bar'). The WEAK form ('foo?/bar') is deliberately not one — it applies + only if something else already activated the dep, and reading it as an + activation drifts back toward the whole-resolve scan this replaces.""" + keys = set(enabled) + declared = package.get("features", {}) + for feature in enabled: + for token in declared.get(feature, []): + if token.startswith("dep:"): + keys.add(token[4:]) + elif "/" in token: + head = token.split("/", 1)[0] + if not head.endswith("?"): + keys.add(head) + return keys + + +class Graph: + """A resolved `cargo metadata` document, walked along ACTIVATED edges only. + + `cargo metadata` lists every package the resolver CONSIDERED, including + optional dependencies nothing turned on. Scanning that asks "could some + configuration of this tree reach X" where both callers mean "does this one". + """ + + def __init__(self, meta): + self.packages = {p["id"]: p for p in meta["packages"]} + self.nodes = {n["id"]: n for n in (meta.get("resolve") or {}).get("nodes", [])} + self.members = set(meta.get("workspace_members", [])) + + def edges(self, node_id): + node = self.nodes.get(node_id) + package = self.packages.get(node_id) + if node is None or package is None: + return + enabled = activated_keys(package, set(node.get("features", []))) + is_member = node_id in self.members + for dep in node.get("deps", []): + target = self.packages.get(dep["pkg"]) + if target is None: + continue + # A dev-dependency of a *dependency* is never built. One of a + # workspace member is: the test binaries link too. + kinds = {k.get("kind") for k in dep.get("dep_kinds", [{}])} + if kinds == {"dev"} and not is_member: + continue + matching = [ + d + for d in package.get("dependencies", []) + if d["name"] == target["name"] + ] + if not matching: + # An edge the manifest does not explain: keep it rather than drop + # it. Unexplained means unmeasured, and unmeasured fails closed. + yield dep["pkg"] + continue + for entry in matching: + if not entry.get("optional"): + yield dep["pkg"] + break + if (entry.get("rename") or entry["name"]) in enabled: + yield dep["pkg"] + break + + def reachable(self, roots): + """Every package id reachable from `roots` along activated edges.""" + seen = set() + frontier = list(roots) + while frontier: + current = frontier.pop() + if current in seen: + continue + seen.add(current) + frontier.extend(self.edges(current)) + return seen + + def member_roots(self): + """The workspace members — `macos-link`'s starting set, because its + question is about everything this tree builds.""" + return [m for m in self.members if m in self.nodes] + + def named_roots(self, name): + """Every node for a package of this name — `evaluator-closure`'s + starting set, because its question is about ONE package's sub-closure.""" + return [i for i, p in self.packages.items() if p["name"] == name and i in self.nodes] diff --git a/mise-tasks/evaluator-closure.py b/mise-tasks/evaluator-closure.py index 7a814c992..219dd98aa 100644 --- a/mise-tasks/evaluator-closure.py +++ b/mise-tasks/evaluator-closure.py @@ -9,15 +9,10 @@ # rule is a compile error and `graph.reachable` is not in this build's regorus # feature set (`Cargo.toml` is the one authority on that list). # -# IT IS A `.py` SIBLING RATHER THAN A TASK BODY, AND THAT IS WHAT KEEPS THE WALK -# UNDER TEST. `mise-tasks/replay-pointers.py` is the precedent and -# `shell-retirement.rego:159-163` is the reason it is admissible: `.py` is -# excluded from `under_mise_tasks`, so this is not an added shell rule. The four -# cases the dying suite carried about the WALK — an unactivated optional -# dependency, the same one activated, a dev-dependency, and a crate the workspace -# reaches but the evaluator does not — assert against this file directly from -# `crates/batten/tests/it/evaluator_closure.rs`. Inlining it into the task body -# would have made all four untestable, and they are the security-critical half. +# THE WALK ITSELF IS `cargo_graph.py`'s, SHARED WITH `macos-link`. Both gates ask +# the same structural question of the same graph and differ only in their roots +# and in what they look for; both headers used to say "if one is corrected, +# correct both", which is a rule with no mechanism. There is one walk now. # # Output is the record's own lines, on stdout, for `batten record named`: # @@ -29,8 +24,13 @@ # dependency tree, never a path through the graph, never a version chain. import json +import os import sys +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) + +from cargo_graph import Graph # noqa: E402 + # The evaluator's package name. Named once: it is the node the walk starts from # AND the word the refusal uses, and two spellings of it is how a rename turns # this gate silent instead of red. @@ -45,60 +45,8 @@ ) -def activated_keys(package, enabled): - """Three spellings reach a dependency and all three must be read, or an - activated dep looks dormant: the implicit feature (a bare 'foo'), the - namespaced form ('dep:foo'), and enabling one of the dep's own features - ('foo/bar'). The WEAK form ('foo?/bar') is deliberately not one — it applies - only if something else already activated the dep.""" - keys = set(enabled) - declared = package.get("features", {}) - for feature in enabled: - for token in declared.get(feature, []): - if token.startswith("dep:"): - keys.add(token[4:]) - elif "/" in token: - head = token.split("/", 1)[0] - if not head.endswith("?"): - keys.add(head) - return keys - - -def edges(node_id, packages, nodes, members): - node = nodes.get(node_id) - package = packages.get(node_id) - if node is None or package is None: - return - enabled = activated_keys(package, set(node.get("features", []))) - is_member = node_id in members - for dep in node.get("deps", []): - target = packages.get(dep["pkg"]) - if target is None: - continue - kinds = {k.get("kind") for k in dep.get("dep_kinds", [{}])} - if kinds == {"dev"} and not is_member: - continue - matching = [ - d for d in package.get("dependencies", []) if d["name"] == target["name"] - ] - if not matching: - # An edge the manifest does not explain: keep it rather than drop it. - # Unexplained means unmeasured, and unmeasured fails closed. - yield dep["pkg"] - continue - for entry in matching: - if not entry.get("optional"): - yield dep["pkg"] - break - if (entry.get("rename") or entry["name"]) in enabled: - yield dep["pkg"] - break - - def walk(meta): - packages = {p["id"]: p for p in meta["packages"]} - nodes = {n["id"]: n for n in (meta.get("resolve") or {}).get("nodes", [])} - members = set(meta.get("workspace_members", [])) + graph = Graph(meta) # THE SCOPE IS THE EVALUATOR'S SUB-CLOSURE, NOT THE WORKSPACE'S, and that was # measured before it was written because the obvious spelling is wrong. @@ -107,23 +55,15 @@ def walk(meta): # entering by paths that have nothing to do with the evaluator. The wider # spelling fired on all 5 lockfile-touching commits reachable from HEAD and # every firing was a false positive. - roots = [i for i, p in packages.items() if p["name"] == EVALUATOR and i in nodes] + roots = graph.named_roots(EVALUATOR) if not roots: # NOT a pass. The evaluator vanishing from the graph means the question # could not be asked, and reporting 'nothing found' there is the vacuous # pass this repository names CLOUD-251. return ["absent"] - reached = set() - frontier = list(roots) - while frontier: - current = frontier.pop() - if current in reached: - continue - reached.add(current) - frontier.extend(edges(current, packages, nodes, members)) - - found = sorted({packages[i]["name"] for i in reached} & IO_CRATES) + reached = graph.reachable(roots) + found = sorted({graph.packages[i]["name"] for i in reached} & IO_CRATES) return ["closure %d" % len(reached)] + ["crate %s" % name for name in found] diff --git a/mise-tasks/macos-link-check.sh b/mise-tasks/macos-link-check.sh deleted file mode 100755 index 31b6770e9..000000000 --- a/mise-tasks/macos-link-check.sh +++ /dev/null @@ -1,202 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Gate: no dependency that would need a real macOS SDK to link, which would break the SDK-free macOS release build" -# -# The macOS release artifacts are linked on Linux by zig, with no Apple SDK -# present. That works only while nothing in the tree links a macOS *system -# framework* (CoreFoundation, Security, …): such a crate needs SDKROOT pointing -# at a genuine macOS SDK, which reintroduces both a toolchain dependency and -# Apple's licensing question. -# -# The failure this prevents is a LATE one. `cross-check` runs `cargo check`, -# which stops at codegen-to-metadata and never links, so it cannot see this -# class of breakage at all — the first symptom would be the release workflow -# failing after a tag was already cut. This gate moves that signal to the moment -# the dependency is added. -# -# Predicate, over the dependency graph RESOLVED FOR macOS (`--filter-platform`), -# so a macOS-only transitive dep is seen and a Linux-only one is not: -# -# 1. any package declaring a `links` key — the manifest's own statement that -# it links a native library, which is general and needs no list; plus -# 2. a named set of crates that link Apple frameworks from a build script -# WITHOUT declaring `links`, which rule 1 cannot see. -# -# Rule 2 is a list and therefore incomplete by construction — a crate nobody has -# listed slips past. That residual gap is closed by actually linking the target, -# which `darwin-link` does for the Darwin triples; this gate is the fast, -# specific, early half of that pair, not a replacement for it. -# -# BOTH RULES READ WHAT IS BUILT, NOT WHAT RESOLVED (CLOUD-718). `cargo metadata` -# lists every package the resolver considered, and that includes optional -# dependencies nothing ever turned on. Scanning that list asks "could some -# configuration of this tree link native code" where the gate means "does this -# one", and the difference is not academic: adding `gix` put `defmt` in the -# package list — an embedded-logging crate, an unactivated optional dependency -# of `jiff`, reaching no Apple framework and never compiled — and this gate -# refused a link that `darwin-link` then completed on the same tree. A gate that -# fails on a crate the compiler never sees is not measuring the thing it names. -# -# So the scan walks the resolve graph from the workspace members and follows -# only edges that are actually activated, and the ONLY direction that costs -# anything is the one this fixes: a package reachable in the build is still read -# by both rules, and the real link remains the backstop for whatever a metadata -# reading cannot see. -# Reverting the filter to the whole resolve is the defect this fixes: an -# optional dependency nobody enabled reads as linked, and the gate refuses a -# link that succeeds. -#MUTANT scans-resolved-not-built|s/^frontier = \[m for m in members if m in nodes\]$/frontier = list(nodes)/|an optional dependency nobody enabled is not reported -# And the weak-dependency reading must stay OUT: 'foo?\/bar' does not activate -# 'foo', and treating it as though it does walks back to the same over-scan. -#MUTANT weak-dep-activates|s@head = token.split('/', 1)\[0\]@head = token.split('/', 1)[0].rstrip('?')@|A WEAK REFERENCE IS NOT AN ACTIVATION -# A gate listed in $MUTANT_GATES with no row here fails `mise run mutant`. -#MUTANT sdk-dependency-passes|s/^if \[\[ -n "\$findings" \]\]; then$/if false; then/|the same optional dependency, once enabled, is reported - -set -uo pipefail - -# The target whose resolved graph we inspect. Apple Silicon is the platform a -# maintainer is most likely to run, and both Darwin triples share these deps. -readonly TARGET=aarch64-apple-darwin - -# Crates that pull in an Apple system framework. Extend as new ones are met — -# and when you do, prefer proving it with a link (cross-check) over trusting -# this list. -readonly FRAMEWORK_CRATES='^(security-framework|security-framework-sys|core-foundation|core-foundation-sys|native-tls|openssl-sys|cocoa|objc|objc2|system-configuration|system-configuration-sys)$' - -# Crates whose `links` key names a library they VENDOR AND COMPILE THEMSELVES, -# reaching no Apple system framework — so rule 1's proxy is wrong about them. -# -# Rule 1 reads `links` as "declares it links a native library", and treats that -# as "needs SDKROOT pointing at a genuine macOS SDK". For a crate that ships C -# sources and builds them with `cc`, the second does not follow from the first: -# there is no system library to find, so there is no SDK to need. -# -# ADDING A NAME HERE REQUIRES A LINK, never a reading of the crate. `darwin-link` -# runs `cargo zigbuild --target aarch64-apple-darwin` with no Apple SDK present, -# which is the only evidence that settles it — the same standard FRAMEWORK_CRATES -# above states for its own list, applied in the opposite direction. -# -# Measured 2026-08-21, adding a tree-sitter-backed structural matcher to the -# graph: this gate refused `tree-sitter` and `tree-sitter-language`, and -# `darwin-link` then linked the same tree with no SDK present — "invoking -# xcrun --sdk macosx --show-sdk-path failed: No such file or directory" -# followed by a clean link. A gate that refuses what the linker accepts is -# measuring something other than what it names. -# -# The fail-safe is unchanged in the direction that matters: an UNKNOWN `links` -# crate is still a finding, so this narrows the gate rather than opening it, and -# `darwin-link` remains the backstop for whatever a metadata reading cannot see. -readonly VENDORED_LINKS='^(tree-sitter|tree-sitter-language)$' - -# The graph normally comes from cargo. `BATTEN_LINK_CHECK_METADATA` substitutes -# a recorded one, and exists so the reachability filter can be shown able to -# FAIL: the live tree is clean by construction (that is the point of the gate), -# so a suite that can only run against it asserts a pass and never a refusal. -# The fixtures are the two directions this filter has to separate — a built -# `links` crate and an unactivated optional one — which cannot both be produced -# by editing the real manifest. Read-only, and never set outside the suite. -if [[ -n "${BATTEN_LINK_CHECK_METADATA:-}" ]]; then - metadata=$(cat "$BATTEN_LINK_CHECK_METADATA") || { - echo "::error:: macos-link-check: could not read ${BATTEN_LINK_CHECK_METADATA}" >&2 - exit 1 - } -else - metadata=$(cargo metadata --format-version 1 --filter-platform "$TARGET" 2>/dev/null) || { - echo "::error:: macos-link-check: could not resolve the dependency graph for ${TARGET}" >&2 - exit 1 - } -fi - -findings=$(printf '%s' "$metadata" | python3 -c " -import json, re, sys - -meta = json.load(sys.stdin) -pattern = re.compile(sys.argv[1]) -# Rule 1's exemption: a \`links\` crate that vendors and compiles its own C. -# Narrowing only — an unknown \`links\` crate is still a finding. -vendored = re.compile(sys.argv[2]) - -packages = {p['id']: p for p in meta['packages']} -nodes = {n['id']: n for n in (meta.get('resolve') or {}).get('nodes', [])} -members = set(meta.get('workspace_members', [])) - - -def activated_keys(package, enabled): - # Three spellings reach a dependency and all three must be read, or an - # activated dep looks dormant: the implicit feature (a bare 'foo'), the - # namespaced form ('dep:foo'), and enabling one of the dep's own features - # ('foo/bar'). The WEAK form ('foo?/bar') is deliberately not one - it - # applies only if something else already activated the dep, and reading it - # as an activation drifts back toward the whole-resolve scan this replaces. - keys = set(enabled) - declared = package.get('features', {}) - for feature in enabled: - for token in declared.get(feature, []): - if token.startswith('dep:'): - keys.add(token[4:]) - elif '/' in token: - head = token.split('/', 1)[0] - if not head.endswith('?'): - keys.add(head) - return keys - - -def edges(node_id): - node = nodes.get(node_id) - package = packages.get(node_id) - if node is None or package is None: - return - enabled = activated_keys(package, set(node.get('features', []))) - is_member = node_id in members - for dep in node.get('deps', []): - target = packages.get(dep['pkg']) - if target is None: - continue - # A dev-dependency of a *dependency* is never built. One of a workspace - # member is: the test binaries link too. - kinds = {k.get('kind') for k in dep.get('dep_kinds', [{}])} - if kinds == {'dev'} and not is_member: - continue - matching = [d for d in package.get('dependencies', []) if d['name'] == target['name']] - if not matching: - # An edge the manifest does not explain: keep it rather than drop - # it. Unexplained means unmeasured, and unmeasured fails closed. - yield dep['pkg'] - continue - for entry in matching: - if not entry.get('optional'): - yield dep['pkg'] - break - if (entry.get('rename') or entry['name']) in enabled: - yield dep['pkg'] - break - - -built = set() -frontier = [m for m in members if m in nodes] -while frontier: - current = frontier.pop() - if current in built: - continue - built.add(current) - frontier.extend(edges(current)) - -for package_id in sorted(built, key=lambda i: packages[i]['name']): - package = packages[package_id] - if package.get('links') and not vendored.match(package['name']): - print(f\"{package['name']} v{package['version']}: declares links={package['links']!r}\") - elif pattern.match(package['name']): - print(f\"{package['name']} v{package['version']}: links an Apple system framework\") -" "$FRAMEWORK_CRATES" "$VENDORED_LINKS") || { - echo "::error:: macos-link-check: could not inspect the dependency graph" >&2 - exit 1 -} - -if [[ -n "$findings" ]]; then - echo "::error:: a dependency needs a real macOS SDK to link, which the SDK-free macOS release build cannot supply:" >&2 - printf '%s\n' "$findings" | while IFS= read -r line; do printf ' %s\n' "$line" >&2; done - echo "Either drop it, feature-gate it off the default build (a rustls-style" >&2 - echo "alternative usually exists), or accept an SDK and revisit how macOS is built." >&2 - exit 1 -fi - -echo "macos-link-check: nothing in the ${TARGET} graph needs a macOS SDK to link" diff --git a/mise-tasks/macos-link.py b/mise-tasks/macos-link.py new file mode 100644 index 000000000..3ae534d56 --- /dev/null +++ b/mise-tasks/macos-link.py @@ -0,0 +1,102 @@ +# What in the built macOS graph would need a real macOS SDK to link, over a +# `cargo metadata` document on stdin (ported off `mise-tasks/macos-link-check.sh` +# under CLOUD-1717). +# +# THIS IS THE STEP, NEVER THE DECISION. It names what it found; whether finding +# one is a refusal is `policy/macos-link.rego`'s question. The split is forced: +# §5 makes `check` `read` and incapable of spawning `cargo metadata`, and the +# reachability walk is not expressible in Rego at all. +# +# THE WALK IS `cargo_graph.py`'s, SHARED WITH `evaluator-closure`, which is what +# closes the drift both programs' headers warned about in prose and neither could +# enforce. +# +# TWO RULES, and rule 2 is a list and therefore incomplete by construction: +# +# 1. any package declaring a `links` key — the manifest's own statement that it +# links a native library, which is general and needs no list; minus the +# crates that vendor and compile their own C, for which "links a native +# library" does not imply "needs an SDK to find one"; +# 2. a named set of crates that link Apple frameworks from a build script +# WITHOUT declaring `links`, which rule 1 cannot see. +# +# A crate nobody has listed slips past rule 2. That residual gap is closed by +# actually linking the target, which `darwin-link` does; this pair is the fast, +# specific, early half of that, not a replacement for it. +# +# Output is the record's own lines, on stdout, for `batten record named`: +# +# scanned packages in the built macOS graph +# links rule 1: declares links= +# framework rule 2: links an Apple system framework +# +# Pointer-only (non-negotiable rule 4): a crate NAME and, for rule 1, the library +# the manifest itself names. Never a version chain, never a path through the +# graph. The VERSION the retired program printed is deliberately dropped — it is +# not needed to act on the finding and it dates the record against a lockfile +# the module never reads. + +import json +import os +import re +import sys + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) + +from cargo_graph import Graph # noqa: E402 + +# Crates that pull in an Apple system framework. Extend as new ones are met — and +# when you do, prefer proving it with a link (`darwin-link`) over trusting this +# list. +FRAMEWORK_CRATES = re.compile( + r"^(security-framework|security-framework-sys|core-foundation" + r"|core-foundation-sys|native-tls|openssl-sys|cocoa|objc|objc2" + r"|system-configuration|system-configuration-sys)$" +) + +# Crates whose `links` key names a library they VENDOR AND COMPILE THEMSELVES, +# reaching no Apple system framework — so rule 1's proxy is wrong about them. +# +# ADDING A NAME HERE REQUIRES A LINK, never a reading of the crate. Measured +# 2026-08-21: this gate refused `tree-sitter` and `tree-sitter-language`, and +# `darwin-link` then linked the same tree with no SDK present. A gate that +# refuses what the linker accepts is measuring something other than what it +# names. The fail-safe is unchanged in the direction that matters: an UNKNOWN +# `links` crate is still a finding, so this narrows the gate rather than opening +# it. +VENDORED_LINKS = re.compile(r"^(tree-sitter|tree-sitter-language)$") + + +def walk(meta): + graph = Graph(meta) + + # THE WALK STARTS AT THE WORKSPACE MEMBERS, because the question is about + # everything this tree builds — unlike `evaluator-closure`, whose question is + # about one package's sub-closure. That is the only difference between the + # two callers of this graph. + built = graph.reachable(graph.member_roots()) + + lines = ["scanned %d" % len(built)] + for package_id in sorted(built, key=lambda i: graph.packages[i]["name"]): + package = graph.packages[package_id] + name = package["name"] + if package.get("links") and not VENDORED_LINKS.match(name): + lines.append("links %s %s" % (name, package["links"])) + elif FRAMEWORK_CRATES.match(name): + lines.append("framework %s" % name) + return lines + + +def main(): + try: + meta = json.load(sys.stdin) + except (ValueError, OSError) as failed: + sys.stderr.write("macos-link: could not read the graph: %s\n" % failed) + return 1 + for line in walk(meta): + sys.stdout.write(line + "\n") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/mise.toml b/mise.toml index 307295d77..0b9fbc5d2 100644 --- a/mise.toml +++ b/mise.toml @@ -617,7 +617,7 @@ CI_FANIN_WORKFLOW = ".github/workflows/ci.yml" BATS_TEST_TIMEOUT = "300" REGORUS_OPA_COMPLIANCE = "1.2.0" REGORUS_OPA_COMPLIANCE_FOR = "0.11" -MUTANT_GATES = "mise,attestation,timeout-drift,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure,evaluator-io-check,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link-check,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-callable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus-check,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" +MUTANT_GATES = "mise,attestation,timeout-drift,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure,evaluator-io-check,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-callable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus-check,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" # --- GitHub reachability behind an egress proxy (Claude Code web sandbox etc.) --- # mise resolves every tool's release through GitHub's *API* host, api.github.com. @@ -2093,6 +2093,38 @@ emit | cargo run --quiet -p batten -- record named timeout-drift # contract: an absent record is "the producer did not run", and the module reads # it as silence rather than as a clean closure. A graph that DID resolve without # an evaluator node is a different answer and the walk records it as `absent`. +# The macOS link gate's producer, ported off `mise-tasks/macos-link-check.sh` +# under CLOUD-1717. The two things that are not decisions: the `cargo metadata` +# spawn and the graph walk. +# +# `--filter-platform aarch64-apple-darwin` IS THE QUESTION, not an optimisation. +# A macOS-only transitive dep must be seen and a Linux-only one must not, so the +# graph is resolved FOR macOS. Apple Silicon because it is the platform a +# maintainer is most likely to run, and both Darwin triples share these deps. +# (`evaluator-closure-record` deliberately does NOT filter: its pin is a claim +# about every platform this crate ships to, so narrowing would let an IO crate +# arrive behind a `cfg` for a platform the gate does not run on.) +# +# `BATTEN_LINK_CHECK_METADATA` substitutes a recorded graph so the filter can be +# shown able to FAIL: the live tree is clean by construction, which is the point +# of the gate. +# +# NOTHING IS WRITTEN WHEN THE GRAPH WILL NOT RESOLVE — the record contract, so an +# absent record reads as "the producer did not run" rather than as a clean graph. +[tasks.macos-link-record] +description = "Effect: record the built macOS graph's links-declaring and framework-linking crates, for `workspace carry unsafe`" +shell = "bash -c" +run = """ +if [ -n "${BATTEN_LINK_CHECK_METADATA:-}" ]; then + metadata=$(cat "$BATTEN_LINK_CHECK_METADATA") || exit 0 +else + metadata=$(cargo metadata --format-version 1 --filter-platform aarch64-apple-darwin 2>/dev/null) || exit 0 +fi +printf '%s' "$metadata" \ + | python3 mise-tasks/macos-link.py \ + | cargo run --quiet -p batten -- record named macos-link +""" + [tasks.evaluator-closure-record] description = "Effect: record the evaluator's activated sub-closure and any IO-bearing crate in it, for `layer reach unsafe`" shell = "bash -c" diff --git a/policy/macos-link.rego b/policy/macos-link.rego new file mode 100644 index 000000000..1dd4ee1a2 --- /dev/null +++ b/policy/macos-link.rego @@ -0,0 +1,130 @@ +# Nothing in the built macOS graph needs a real macOS SDK to link (ported off +# `mise-tasks/macos-link-check.sh` under CLOUD-1717). +# +# WHY IT MATTERS. The macOS release artifacts are linked on Linux by zig, with no +# Apple SDK present. That works only while nothing in the tree links a macOS +# *system framework* (CoreFoundation, Security, …): such a crate needs SDKROOT +# pointing at a genuine macOS SDK, which reintroduces both a toolchain dependency +# and Apple's licensing question. +# +# THE FAILURE IT MOVES IS A LATE ONE. `cross-check` runs `cargo check`, which +# stops at codegen-to-metadata and never links, so it cannot see this class at +# all — the first symptom would be the release workflow failing after a tag was +# already cut. This pair moves that signal to the moment the dependency is added. +# +# TWO RULES, AND THE SECOND IS INCOMPLETE BY CONSTRUCTION: +# +# 1. a package declaring a `links` key — the manifest's own statement that it +# links a native library, general and needing no list; +# 2. a named set of crates that link Apple frameworks from a build script +# WITHOUT declaring `links`, which rule 1 cannot see. +# +# A crate nobody has listed slips past rule 2. That residual gap is closed by +# actually linking the target, which `darwin-link` does; this is the fast, +# specific, early half of that pair rather than a replacement for it. The two +# classes stay distinct here because a reader acts on them differently: rule 1 +# names the library the manifest itself declares, and rule 2 names only the +# crate. +# +# THE SPLIT IS FORCED. §5 makes `check` `read` and incapable of spawning +# `cargo metadata --filter-platform`, and the reachability walk is not +# expressible in Rego — a self-referential rule is a compile error and +# `graph.reachable` is not in this build's regorus feature set. The walk is +# `mise-tasks/cargo_graph.py`, shared with `evaluator-closure` so the two cannot +# drift; `mise-tasks/macos-link.py` is this gate's roots and predicates over it. +# +# THE RECORD IS THE PLATFORM-FILTERED GRAPH, which is the whole reason the +# producer passes `--filter-platform aarch64-apple-darwin`: a macOS-only +# transitive dep must be seen and a Linux-only one must not. A module reading an +# unfiltered graph would answer a different question with the same words. +# +#MUTANT-SUITE crates/batten/tests/it/macos_link.rs +#MUTANT links-key-passes|s@^\tstartswith(line, "links ")$@\tfalse@|a_package_declaring_a_native_links_key_is_caught_without_being_listed +#MUTANT framework-crate-passes|s@^\tstartswith(line, "framework ")$@\tfalse@|rule_2_still_fires_through_the_reachability_walk + +# METADATA +# description: | +# Bound to the TREE surface: `scope = "tree"`, so it reads the tree document +# and never the mediated `{call, facts}` shape. +# THE BRACKETS ARE NOT STYLE: the schema file carries a hyphen, so the dotted +# form is a parse error reported as `invalid schema reference`. +# THIS BLOCK IS YAML AND MUST STAY THE LAST COMMENT BLOCK BEFORE `package`. +# schemas: +# - input: schema["policy-input.schema"] +package batten.macos_link + +import rego.v1 + +rules contains "workspace carry unsafe" + +# The record, or nothing. ABSENT IS NOT EMPTY: a checkout whose producer never +# ran has no key here, Rego reads that as *does not hold*, and every rule below +# is silent. The producer writes nothing when the graph will not resolve, and a +# module refusing there would refuse every checkout with no toolchain. +lines := input.tree.records["macos-link"] + +# --- rule 1: the manifest's own declaration ---------------------------------- + +violation contains { + "rule": "workspace carry unsafe", + "verdict": "manifest carry unsafe", + "subjects": [{"artifact": name}, {"artifact": library}], +} if { + some line in lines + startswith(line, "links ") + parts := split(trim_space(substring(line, count("links "), -1)), " ") + count(parts) == 2 + name := parts[0] + library := parts[1] +} + +# --- rule 2: the named set rule 1 cannot see --------------------------------- + +violation contains { + "rule": "workspace carry unsafe", + "verdict": "workspace reach unsafe", + "subjects": [{"artifact": name}], +} if { + some line in lines + startswith(line, "framework ") + name := trim_space(substring(line, count("framework "), -1)) + name != "" +} + +# --- cases ------------------------------------------------------------------- + +test_a_links_key_is_refused_and_names_the_library if { + found := violation with input as {"tree": {"records": {"macos-link": [ + "scanned 200", + "links openssl-sys openssl", + ]}}} + count(found) == 1 +} + +test_a_framework_crate_is_refused if { + found := violation with input as {"tree": {"records": {"macos-link": [ + "scanned 200", + "framework core-foundation", + ]}}} + count(found) == 1 +} + +# THE TWO CLASSES STAY DISTINCT, and a case counting findings cannot see that. +test_the_two_rules_are_separate_classes if { + found := violation with input as {"tree": {"records": {"macos-link": [ + "scanned 200", + "links openssl-sys openssl", + "framework core-foundation", + ]}}} + {entry.verdict | some entry in found} == {"manifest carry unsafe", "workspace reach unsafe"} +} + +test_a_clean_graph_is_silent if { + found := violation with input as {"tree": {"records": {"macos-link": ["scanned 200"]}}} + count(found) == 0 +} + +test_an_absent_record_says_nothing_rather_than_refusing if { + found := violation with input as {"tree": {"records": {}}} + count(found) == 0 +} diff --git a/tests/macos-link-check.bats b/tests/macos-link-check.bats deleted file mode 100644 index 1d98c508d..000000000 --- a/tests/macos-link-check.bats +++ /dev/null @@ -1,154 +0,0 @@ -#!/usr/bin/env bats -# subject: mise-tasks/macos-link-check.sh -# -# The gate that keeps the SDK-free macOS build buildable. Its whole value is -# firing BEFORE a release, so the cases below pin both directions: it passes on -# the tree as it stands, and it actually fails when a framework-linking crate -# appears — a gate that only ever passes is indistinguishable from no gate. - -setup() { - CHECK="${BATS_TEST_DIRNAME}/../mise-tasks/macos-link-check.sh" - cd "${BATS_TEST_DIRNAME}/.." || return 1 -} - -@test "the repo as it stands has no SDK-requiring dependency" { - run "$CHECK" - [ "$status" -eq 0 ] - [[ "$output" == *"nothing in the aarch64-apple-darwin graph"* ]] -} - -@test "the framework crate list covers the ones that actually bit us" { - # native-tls is the concrete crate that forces a real SDK: it is what a - # default-featured HTTP client pulls in, and it reaches Security.framework - # through security-framework. If this list ever loses these names the gate - # silently stops guarding the case it was written for. - for crate in native-tls security-framework core-foundation openssl-sys; do - grep -q "$crate" "$CHECK" || { - echo "framework crate list dropped $crate" - return 1 - } - done -} - -@test "the graph is resolved for macOS, not for the host" { - # --filter-platform is load-bearing: without it a macOS-only transitive - # dependency is invisible on a Linux host, which is every CI run. - grep -q 'filter-platform' "$CHECK" - grep -q 'aarch64-apple-darwin' "$CHECK" -} - -@test "a package declaring a native links key is caught without being listed" { - # Rule 1 is the general half: it needs no list, so a crate nobody has heard - # of is still caught if its manifest declares that it links native code. - grep -q "package.get('links')" "$CHECK" -} - -# --- the filter reads what is BUILT, not what merely RESOLVED (CLOUD-718) ----- -# -# These run against recorded metadata rather than the live tree, and that is not -# convenience: the live tree is clean by construction, so a suite that can only -# run against it asserts a pass and never a refusal. The two directions below -# differ by ONE enabled feature on one package, which is the whole distinction -# the filter exists to draw. - -@test "an optional dependency nobody enabled is not reported" { - # The defmt/jiff shape: a crate declaring `links` sits in the resolve as an - # unactivated optional dependency. It is never compiled, so a gate about - # linking must stay silent. Measured 2026-08-20: adding `gix` produced - # exactly this and failed a link `darwin-link` completed on the same tree. - BATTEN_LINK_CHECK_METADATA="${BATS_TEST_DIRNAME}/fixtures/link-check/dormant-optional.json" \ - run "$CHECK" - [ "$status" -eq 0 ] - [[ "$output" != *"nativebits"* ]] -} - -@test "the same optional dependency, once enabled, is reported" { - # The guard on the case above: same fixture, same crate, one feature turned - # on. If this passes, the filter has not narrowed rule 1 — it has switched - # it off. - BATTEN_LINK_CHECK_METADATA="${BATS_TEST_DIRNAME}/fixtures/link-check/enabled-optional.json" \ - run "$CHECK" - [ "$status" -eq 1 ] - [[ "$output" == *"nativebits"* ]] - [[ "$output" == *"declares links"* ]] -} - -@test "A WEAK REFERENCE IS NOT AN ACTIVATION: dep-question-mark leaves the dep dormant" { - # Same shape as the two above, except the enabled feature names the optional - # dependency through cargo's WEAK form: `dep?/feature` says "if something - # else activated it, turn this feature on too", which is the one syntax that - # mentions a dependency without building it. Reading it as an activation - # drifts back toward the whole-resolve scan CLOUD-718 replaced, and no - # fixture exercised it until `mutant` was pointed at the arm that does - # (CLOUD-480). - BATTEN_LINK_CHECK_METADATA="${BATS_TEST_DIRNAME}/fixtures/link-check/weak-optional.json" \ - run "$CHECK" - [ "$status" -eq 0 ] - [[ "$output" != *"nativebits"* ]] -} - -@test "rule 2 still fires through the reachability walk" { - # The named-crate half has to survive the same filter: a framework crate on - # a non-optional edge is built, so it is still caught. - BATTEN_LINK_CHECK_METADATA="${BATS_TEST_DIRNAME}/fixtures/link-check/framework-crate.json" \ - run "$CHECK" - [ "$status" -eq 1 ] - [[ "$output" == *"security-framework"* ]] - [[ "$output" == *"Apple system framework"* ]] -} - -@test "a vendored-C links crate is exempt from rule 1" { - # `vendored-links.json` is `enabled-optional.json` with ONE substitution: - # the crate is named `tree-sitter` instead of `nativebits`. Same graph, same - # built edge, same `links` key — so the only thing that can change the - # verdict is the name, which is exactly what the exemption keys on. - # - # Measured 2026-08-21: adding a tree-sitter-backed structural matcher made - # this gate refuse `tree-sitter` and `tree-sitter-language`, and - # `darwin-link` then linked the same tree with no SDK present ("invoking - # xcrun --sdk macosx --show-sdk-path failed: No such file or directory"). - # A gate that refuses what the linker accepts measures something other than - # what it names. - BATTEN_LINK_CHECK_METADATA="${BATS_TEST_DIRNAME}/fixtures/link-check/vendored-links.json" \ - run "$CHECK" - [ "$status" -eq 0 ] - [[ "$output" != *"tree-sitter"* ]] -} - -@test "an unvetted links crate is still reported, so the exemption is a list not a switch" { - # The anti-vacuity guard on the case above, and the one that matters: the - # exemption is an allowlist requiring a `darwin-link` proof per entry, so a - # crate nobody has vetted still reds. If this passes, rule 1 was switched - # off rather than narrowed — the same failure mode the enabled/dormant pair - # above exists to catch, facing the other way. - BATTEN_LINK_CHECK_METADATA="${BATS_TEST_DIRNAME}/fixtures/link-check/enabled-optional.json" \ - run "$CHECK" - [ "$status" -eq 1 ] - [[ "$output" == *"nativebits"* ]] -} - -@test "every vendored-links entry names a crate, so the pattern cannot be widened to a wildcard" { - # The exemption is only safe while it is a closed list: a wildcard slipped - # into it would exempt every `links` crate silently, and the sibling cases - # above would still pass because both fixtures name real crates. - # - # Assert the WHOLE grammar, not a denylist of operators. An earlier form - # stripped only `.`, `*` and `+`, which let - # `^(tree-sitter|z[a-z]{0,99})$` through — it rejects `nativebits`, so the - # sibling cases above stay green, while silently exempting every `z...` - # crate. Caught in review on the PR that introduced it. - # - # A crate name is `[a-z0-9_-]+` per cargo, so an anchored alternation of - # literals is fully expressible as a positive match. Anything else — a - # character class, a brace, a backslash, a quantifier — fails to match at - # all, which is the point: the test says what the pattern MAY be rather - # than enumerating what it may not. - run grep -cE "^readonly VENDORED_LINKS='\^\([a-z0-9_-]+(\|[a-z0-9_-]+)*\)\\\$'\$" "$CHECK" - [ "$output" -eq 1 ] -} - -@test "the walk starts at the workspace members" { - # The reachability walk is what makes 'built' mean anything; seeded with - # every node instead, it degenerates to the whole-resolve scan this replaced. - grep -q 'frontier = \[m for m in members if m in nodes\]' "$CHECK" -} From 8f8f3228ea4fe80ab1bc90795ceb1bb84adfe43d Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sat, 19 Sep 2026 02:24:28 +0000 Subject: [PATCH 18/32] feat(policy): the corpus condition decides in the engine, and its census stays under test MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `mise-tasks/transcript-corpus-check.sh` and its suite are gone. The comparison is `policy/transcript-corpus.rego`; the host filesystem walk is `[tasks.transcript-corpus-record]` over `mise-tasks/transcript_census.py`. ## What the gate is for This repository holds prose-shaped predicates to one method: no literal ships until it is measured over a real corpus, counting firings AND true positives (CLOUD-252, then CLOUD-323 over 60 merged PR bodies). For PR bodies the corpus is GitHub and one API call. For SESSION TRANSCRIPTS there was none, because transcripts are written inside a session's own ephemeral container and destroyed with it. CLOUD-326's section 8.1 states its unblock condition as "N independent session transcripts", and a block written as prose is a block no gate reads. ## The census is the gate; the comparison is one `<` Ten of the dying suite's seventeen cases are about WHAT COUNTS as an independent session — the sidechain rule, the authored-content rule, two files carrying one session, the exclusion and its absent-versus-empty distinction, and an undecodable line. Put in the task body they would have been assertable by nothing, so the census is `mise-tasks/transcript_census.py` and `crates/batten/tests/it/transcript_corpus.rs` drives it. Fifteen cases, all passing. `shell-retirement.rego:159-163` excludes `.py` from `under_mise_tasks`, so it adds no shell rule. ## Pointer-only is a security property here, not a style one A transcript is the richest source of secrets the engine can be pointed at. The record carries two counts and the finding carries two counts: no path, no session id, no byte of any transcript. `the_report_is_two_counts_and_carries_no_byte_of_any_transcript` asserts that over the census's OWN bytes, which is where it has to be asserted — the module cannot leak what the producer never wrote. ## `warn`, and the severity is the honest one The count is a reading about the WORLD — whether CLOUD-651's collector has landed and reached this host — rather than about the branch, so no commit is at fault when it is low and no diff can raise it. A caller that wants it blocking reads `--fail-on-warning`, the posture `land-divergence` and `nonverdict` already take. That also fixes something the retired program got wrong once and corrected: its first header called the corpus impossible, on CLOUD-388's ruling that transcript egress was out of scope. That was a POLICY choice rather than a fact about the world, and the owner lifted it. The `[[verdict]]` class says so, so a reader meeting a low count does not re-derive the retired rule from it. ## Two withdrawn cases, and the reason is the same for both "the refusal names what would raise the number" and "the refusal does not tell the reader the count can never rise" both asserted over a shell `echo` that no longer exists. That sentence is the `[[verdict]]` row's `class` now — config a reviewer reads rather than a string a case greps — and `remedy-authorship` already holds prose of that kind. A case re-asserting the wording would be a second authority over it. Two more are `// changed:` onto `mise.toml`: argument validation and the arity contract belong to the thing that takes the arguments, and both are driven against the census directly. ## Proven live, not merely green `mise run batten -- check --rule 'prose measure partial' --fail-on-warning` over a recorded `sessions 0 / threshold 2`: `0 prose measure partial`, exit 2. Exit 0 alone is byte-identical to a dead gate (CLOUD-1810). ## Corpus 76 -> 75 programs, 87 -> 86 suites. 10 of 44 retired. Refs: CLOUD-1717 Refs: CLOUD-388 Refs: CLOUD-651 Admits: f53fe1dd65a0478ec1069c7550620a77a1c34c5febf986aef9b554fb668d1580 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: batten.toml Admits-anchor: call:d4070293b6aebce13c1432ef4b337047b671f0d7 Admits-epoch: 3dc1cc7fd8df11b0ea61e07cb2ba97609391992f61a7da9c60ad25d68cb63ca3 Admits-author: alec@wenzowski.com Admits-prev: 6a9808bfb0a51af223f2f644ca05de87112409fc74e2ce5224183e355c71faa9 Admits-answer-lost: The retirement cannot land and the tree will not load. The gate itself is CLOUD-326 section 8.1's unblock condition expressed as a command and an exit code — this repository holds every prose-shaped literal to a measured corpus, and without a runnable reading of how many independent session transcripts the host carries, that block goes back to being prose no gate reads, which is the state CLOUD-388 was filed over. Admits-answer-precondition: batten.toml is the only surface carrying [[rule]], [[verdict]] and [[record]] rows. CLOUD-1717 retires mise-tasks/transcript-corpus-check.sh onto policy/transcript-corpus.rego plus [tasks.transcript-corpus-record]; the module raises `prose measure partial` and reads the `transcript-corpus` record family, and the engine refuses at load both a module raising an undeclared token and a declared row nothing raises. No other file can carry any of it. It lands in PR #962 as an ordinary diff hunk beside the evaluator-closure and macos-link rows admitted earlier on this branch. Admits-answer-rejected-route: `config read first` is a read of batten.toml; I read it to confirm `prose measure partial` is unused, that the `whole-number` [[pattern]] the module reads already exists, and that each of the three words is in the declared [vocabulary] lists — the first name I chose, `corpus count thin`, used two words that are not, and reading is how I found that. Reading cannot declare the row. `patch run first` is `git restore`, which would discard the module, the producer and the census together rather than complete them. --- batten.toml | 39 ++ bench/suites/RESULTS.md | 5 +- crates/batten/tests/it/main.rs | 1 + crates/batten/tests/it/transcript_corpus.rs | 390 ++++++++++++++++++++ mise-tasks/transcript-corpus-check.sh | 184 --------- mise-tasks/transcript_census.py | 144 ++++++++ mise.toml | 25 +- policy/transcript-corpus.rego | 137 +++++++ tests/transcript-corpus-check.bats | 196 ---------- 9 files changed, 737 insertions(+), 384 deletions(-) create mode 100644 crates/batten/tests/it/transcript_corpus.rs delete mode 100755 mise-tasks/transcript-corpus-check.sh create mode 100644 mise-tasks/transcript_census.py create mode 100644 policy/transcript-corpus.rego delete mode 100644 tests/transcript-corpus-check.bats diff --git a/batten.toml b/batten.toml index b1d5f6e6e..c0ca606df 100644 --- a/batten.toml +++ b/batten.toml @@ -5976,6 +5976,45 @@ no_fix_reason = "a dependency needing a macOS SDK is not fixable by a rewrite he record = "macos-link" writer = "mise run macos-link-record" +# CLOUD-326 section 8.1's unblock condition, as a command and an exit code rather +# than as prose no gate reads. Ported under CLOUD-1717 from +# `mise-tasks/transcript-corpus-check.sh`. +# +# `warn`, and that is the honest severity: the count is a reading about the +# WORLD — whether CLOUD-651's collector has landed and reached this host — rather +# than about the branch, so no commit is at fault when it is low. A caller that +# wants it blocking reads `--fail-on-warning`, which is the same posture +# `land-divergence` and `nonverdict` take one row over. +[[rule]] +id = "prose measure partial" +kind = "policy" +scope = "tree" +module = "policy/transcript-corpus.rego" +severity = "warn" +no_fix_reason = "the number rises when the collector reaches this host, not when a branch changes: nothing in a diff can raise it, so there is nothing here to rewrite" + +[[record]] +record = "transcript-corpus" +writer = "mise run transcript-corpus-record" + +[[verdict]] +id = "prose measure partial" +gloss = "this host carries fewer independent session transcripts than the caller asked for" +class = """ +Every prose-shaped literal in this repository is held to a measured corpus — \ +firings AND true positives among them — and a transcript-derived literal has no \ +corpus until independent sessions accumulate. A low count is a PROGRESS reading \ +rather than a permanent state: transcripts are collected to the Batten service \ +(CLOUD-651), so it means the collector has not landed or has not reached this \ +host. The number is expected to rise; do not read a permanent block into it, and \ +do not fit a literal to the single transcript it was derived from. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run transcript-corpus-record" + [[verdict]] id = "manifest carry unsafe" gloss = "a package in the built macOS graph declares a `links` key, so it needs a real SDK to find that library" diff --git a/bench/suites/RESULTS.md b/bench/suites/RESULTS.md index adbb40508..145ad7d44 100644 --- a/bench/suites/RESULTS.md +++ b/bench/suites/RESULTS.md @@ -6,8 +6,8 @@ runner measured it; the suite runs `--no-parallelize-within-files`, so a file's number is its own serial cost and is what an author adding a case to it pays. -- suites: 97 -- serial total: 157.8s +- suites: 96 +- serial total: 157.5s | seconds | share | suite | | ---: | ---: | --- | @@ -79,7 +79,6 @@ to it pays. | 0.3 | 0.2% | `tests/board-payloads.bats` | | 0.3 | 0.2% | `tests/timeout-check.bats` | | 0.3 | 0.2% | `tests/release-due.bats` | -| 0.3 | 0.2% | `tests/transcript-corpus-check.bats` | | 0.3 | 0.2% | `tests/serena-mcp.bats` | | 0.2 | 0.2% | `tests/connector-allow-resolve.bats` | | 0.2 | 0.2% | `tests/batten-glob-check.bats` | diff --git a/crates/batten/tests/it/main.rs b/crates/batten/tests/it/main.rs index 76465dd1e..bcc40242e 100644 --- a/crates/batten/tests/it/main.rs +++ b/crates/batten/tests/it/main.rs @@ -290,6 +290,7 @@ mod timeout_drift; mod todo_promotion; mod tool_selector; mod tool_verdict_facts; +mod transcript_corpus; mod transcript_stop_reason; mod transcript_tool_result; mod trunk_watch; diff --git a/crates/batten/tests/it/transcript_corpus.rs b/crates/batten/tests/it/transcript_corpus.rs new file mode 100644 index 000000000..90ee5ea39 --- /dev/null +++ b/crates/batten/tests/it/transcript_corpus.rs @@ -0,0 +1,390 @@ +//! `prose measure partial` over the compiled binary and the real census +//! (CLOUD-388, CLOUD-651, CLOUD-1717). +//! +//! # Why this tier exists and the module's own `test_` rules do not suffice +//! +//! `policy/transcript-corpus.rego` carries seven load-time cases and every one +//! fabricates its input with `with input as`, which is the shape +//! `rules/policy-modules.md` warns about: the case asserts over a record the +//! engine may be unable to project, and the module stays green while the row +//! decides nothing on any real host. +//! +//! # And why the CENSUS is driven here rather than described +//! +//! Ten of the dying suite's seventeen cases are about WHAT COUNTS as an +//! independent session — the sidechain rule, the authored-content rule, two +//! files carrying one session, the exclusion and its absent-versus-empty +//! distinction, and an undecodable line. That is the entire substance of the +//! gate; the comparison it feeds is one `<`. +//! +//! Inlined into the task body they would be assertable by nothing. +//! `mise-tasks/transcript_census.py` instead, driven directly below, so they +//! carry. `shell-retirement.rego:159-163` excludes `.py` from +//! `under_mise_tasks`, so that file adds no shell rule. +//! +//! # RETIREMENT LEDGER, PER PATH — what `shell retire partial` reads +//! +//! Four cases are not carried and each says why in its own row. +//! +// carried: mise-tasks/transcript-corpus-check.sh policy/transcript-corpus.rego kind:mechanism crates/batten/tests/it/transcript_corpus.rs +// carried: tests/transcript-corpus-check.bats policy/transcript-corpus.rego kind:mechanism crates/batten/tests/it/transcript_corpus.rs +// carried: "an empty root is zero independent sessions, which is an answer and not a failure to look" policy/transcript-corpus.rego kind:mechanism +// carried: "one transcript is one session, and one is not a corpus" policy/transcript-corpus.rego kind:mechanism +// carried: "three distinct sessions satisfy the default threshold" policy/transcript-corpus.rego kind:mechanism +// carried: "the threshold is the argument, so the same corpus can fail a stricter one" policy/transcript-corpus.rego kind:mechanism +// carried: "a subagent stream is not an independent session" policy/transcript-corpus.rego kind:mechanism +// carried: "a transcript carrying only tool results has nobody in it" policy/transcript-corpus.rego kind:mechanism +// carried: "two files carrying one session are one session" policy/transcript-corpus.rego kind:mechanism +// carried: "a line this build cannot decode yields nothing rather than a failure to look" policy/transcript-corpus.rego kind:mechanism +// carried: "excluding the asking session turns its own transcript into zero" policy/transcript-corpus.rego kind:mechanism +// carried: "the exclusion defaults from the environment when no argument names one" policy/transcript-corpus.rego kind:mechanism +// carried: "an explicitly empty exclusion excludes nothing, and does not fall back to the environment" policy/transcript-corpus.rego kind:mechanism +// carried: "an absent root is exit 2, never a verdict about a corpus nobody looked at" policy/transcript-corpus.rego kind:mechanism +// carried: "the report is two counts and carries no byte of any transcript" policy/transcript-corpus.rego kind:mechanism +// changed: "a malformed threshold is exit 2" mise.toml argument validation is the PRODUCER's, so it moved with the census it guards: `transcript_census.py` refuses a non-numeric threshold at exit 2 and writes nothing, and `a_malformed_threshold_refuses_and_writes_nothing` below drives that. The module never sees an argument to malform +// changed: "more arguments than the contract names is exit 2" mise.toml the same split one argument over, and for the same reason: the arity contract belongs to the thing that takes the arguments +// withdrawn: "the refusal names what would raise the number, not just the arithmetic" the sentence is the `[[verdict]]` row's `class` now, which is config a reviewer reads rather than a string a case greps. `verdict declare refused` already refuses a class that is an override alone, and `remedy-authorship` holds the prose; a case re-asserting the wording here would be a second authority over it +// withdrawn: "the refusal does not tell the reader the count can never rise" the same row's `class`, and the same reason. Both cases asserted over a shell `echo` that no longer exists, and the claim they protected — that a low count is a PROGRESS reading rather than a permanent state — is stated in the class and in the module header where a reader meets it + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use common::{git_in, init_repo, run, run_with_stdin, scratch, write}; + +/// A repository registering the real module against the declared family. +fn repo(name: &str) -> std::path::PathBuf { + let dir = scratch(&format!("transcript-corpus-{name}")); + let module = std::fs::read_to_string("../../policy/transcript-corpus.rego") + .expect("the module this tier exists for"); + write(&dir, "policy/transcript-corpus.rego", &module); + write( + &dir, + "batten.toml", + r#"version = 1 +scope = ["**"] + +[[pattern]] +id = "whole-number" +regex = '^[0-9]+$' + +[[verdict]] +id = "prose measure partial" +gloss = "this host carries fewer independent session transcripts than the caller asked for" +class = "A progress reading rather than a permanent state: the number rises as the collector reaches this host." + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run transcript-corpus-record" + +[[rule]] +id = "prose measure partial" +kind = "policy" +scope = "tree" +module = "policy/transcript-corpus.rego" +severity = "deny" + +[[record]] +record = "transcript-corpus" +writer = "mise run transcript-corpus-record" +"#, + ); + init_repo(&dir); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-qm", "register the module"]); + dir +} + +fn record(dir: &std::path::Path, lines: &str) { + let written = run_with_stdin(dir, &["record", "named", "transcript-corpus"], lines); + assert!( + written.status.success(), + "the setup write lands: {}", + String::from_utf8_lossy(&written.stderr) + ); +} + +/// Run the REAL census over a scratch transcript root, exactly as the producer +/// does. `exclude` is `None` for an absent argument and `Some("")` for an +/// explicitly empty one — the distinction the contract turns on. +#[expect( + clippy::disallowed_types, + reason = "CLOUD-320's inventory row, and the verdict is that this spawn IS the subject under test: the census moved out of the dying program into `mise-tasks/transcript_census.py` precisely so a compiled case could drive it, and a harness re-implementing the independence rules in Rust would be a second authority over what counts as a session" +)] +fn census( + root: &std::path::Path, + threshold: &str, + exclude: Option<&str>, + env: Option<&str>, +) -> std::process::Output { + let mut command = std::process::Command::new("python3"); + command + .arg("../../mise-tasks/transcript_census.py") + .arg(root) + .arg(threshold); + if let Some(value) = exclude { + command.arg(value); + } + match env { + Some(value) => command.env("BATTEN_SESSION_ID", value), + None => command.env_remove("BATTEN_SESSION_ID"), + }; + command.output().expect("the census runs") +} + +fn counted(done: &std::process::Output) -> String { + String::from_utf8_lossy(&done.stdout).into_owned() +} + +/// A transcript root holding the given `(filename, contents)` pairs. +fn root(name: &str, files: &[(&str, &str)]) -> std::path::PathBuf { + let dir = scratch(&format!("transcript-root-{name}")); + for (file, contents) in files { + write(&dir, file, contents); + } + dir +} + +/// One authored, non-sidechain user record for `session`. +fn authored(session: &str) -> String { + format!( + r#"{{"type":"user","sessionId":"{session}","message":{{"content":[{{"type":"text","text":"hello"}}]}}}}"# + ) +} + +// --- the decision, over the engine's own projection -------------------------- + +#[test] +fn one_transcript_is_one_session_and_one_is_not_a_corpus() { + let dir = repo("thin"); + record(&dir, "sessions 1\nthreshold 2\n"); + + let decided = run(&dir, &["check"]); + assert_eq!( + decided.status.code(), + Some(2), + "one session is not a corpus\n{}", + String::from_utf8_lossy(&decided.stderr) + ); +} + +#[test] +fn three_distinct_sessions_satisfy_the_default_threshold() { + let dir = repo("enough"); + record(&dir, "sessions 3\nthreshold 2\n"); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "a corpus that meets the bound is clean\n{}", + String::from_utf8_lossy(&quiet.stderr) + ); +} + +#[test] +fn an_absent_record_says_nothing_rather_than_refusing() { + // The producer writes nothing when the root does not exist, so a module that + // refused here would refuse every host that has never run the census. + let dir = repo("unrecorded"); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "an absent record is silence\n{}", + String::from_utf8_lossy(&quiet.stderr) + ); +} + +#[test] +fn the_report_is_two_counts_and_carries_no_byte_of_any_transcript() { + // POINTER-ONLY IS A SECURITY PROPERTY over this input, not a style one: a + // transcript is the richest source of secrets the engine can be pointed at. + // The census is what must not leak, so this asserts over ITS bytes. + let held = root( + "secretive", + &[( + "a.jsonl", + r#"{"type":"user","sessionId":"alpha","message":{"content":[{"type":"text","text":"SUPERSECRETSTRING"}]}}"#, + )], + ); + + let done = census(&held, "2", Some(""), None); + let said = format!( + "{}{}", + counted(&done), + String::from_utf8_lossy(&done.stderr) + ); + assert!( + !said.contains("SUPERSECRETSTRING"), + "no byte of a transcript reaches the record\n{said}" + ); + assert!(!said.contains("alpha"), "and no session id does\n{said}"); + assert!(said.contains("sessions 1"), "only the counts do\n{said}"); +} + +// --- the census itself ------------------------------------------------------- + +#[test] +fn an_empty_root_is_zero_independent_sessions_which_is_an_answer() { + let held = root("empty", &[("keep.txt", "not a transcript\n")]); + let done = census(&held, "2", Some(""), None); + assert!(done.status.success(), "an empty root is not a failure"); + assert!( + counted(&done).contains("sessions 0"), + "zero is an answer\n{}", + counted(&done) + ); +} + +#[test] +fn a_subagent_stream_is_not_an_independent_session() { + // CLOUD-326 section 8.1 recorded "one session plus five subagent + // transcripts" and correctly called that N=1. + let held = root( + "sidechain", + &[( + "sub.jsonl", + r#"{"type":"user","sessionId":"beta","isSidechain":true,"message":{"content":[{"type":"text","text":"hi"}]}}"#, + )], + ); + let done = census(&held, "2", Some(""), None); + assert!( + counted(&done).contains("sessions 0"), + "a sidechain is the orchestrator's own turns\n{}", + counted(&done) + ); +} + +#[test] +fn a_transcript_carrying_only_tool_results_has_nobody_in_it() { + // A `tool_result` also arrives as a user record; that is the harness handing + // work back, not a person speaking. + let held = root( + "toolonly", + &[( + "t.jsonl", + r#"{"type":"user","sessionId":"gamma","message":{"content":[{"type":"tool_result","content":"out"}]}}"#, + )], + ); + let done = census(&held, "2", Some(""), None); + assert!( + counted(&done).contains("sessions 0"), + "nobody was in it\n{}", + counted(&done) + ); +} + +#[test] +fn two_files_carrying_one_session_are_one_session() { + let held = root( + "split", + &[ + ("one.jsonl", &authored("delta")), + ("two.jsonl", &authored("delta")), + ], + ); + let done = census(&held, "2", Some(""), None); + assert!( + counted(&done).contains("sessions 1"), + "distinct sessions, not distinct files\n{}", + counted(&done) + ); +} + +#[test] +fn a_line_this_build_cannot_decode_yields_nothing_rather_than_a_failure_to_look() { + // The format is a HOST's and it moves. `transcript.rs`'s + // forward-compatibility law, applied at the same boundary from this side. + let held = root( + "undecodable", + &[( + "mixed.jsonl", + &format!("{{not json at all\n{}\n", authored("epsilon")), + )], + ); + let done = census(&held, "2", Some(""), None); + assert!(done.status.success(), "one bad line is not a failed census"); + assert!( + counted(&done).contains("sessions 1"), + "the readable line still counts\n{}", + counted(&done) + ); +} + +#[test] +fn excluding_the_asking_session_turns_its_own_transcript_into_zero() { + let held = root("selfonly", &[("self.jsonl", &authored("zeta"))]); + let done = census(&held, "2", Some("zeta"), None); + assert!( + counted(&done).contains("sessions 0"), + "a session is not independent evidence about itself\n{}", + counted(&done) + ); +} + +#[test] +fn the_exclusion_defaults_from_the_environment_when_no_argument_names_one() { + let held = root("envexcl", &[("self.jsonl", &authored("eta"))]); + let done = census(&held, "2", None, Some("eta")); + assert!( + counted(&done).contains("sessions 0"), + "the ambient session id applies when nothing names one\n{}", + counted(&done) + ); +} + +#[test] +fn an_explicitly_empty_exclusion_excludes_nothing_and_does_not_fall_back_to_the_environment() { + // ABSENT AND EMPTY ARE DIFFERENT CLAIMS. Defaulting an explicit empty would + // launder "exclude nothing" into "exclude whatever the host happens to say". + let held = root("emptyexcl", &[("self.jsonl", &authored("theta"))]); + let done = census(&held, "2", Some(""), Some("theta")); + assert!( + counted(&done).contains("sessions 1"), + "an explicit empty excludes nothing\n{}", + counted(&done) + ); +} + +#[test] +fn an_absent_root_is_could_not_look_and_writes_nothing() { + // Never a verdict about a corpus nobody looked at. + let missing = scratch("transcript-root-missing").join("nowhere"); + let done = census(&missing, "2", Some(""), None); + assert_eq!(done.status.code(), Some(2), "could not look"); + assert!( + counted(&done).is_empty(), + "and writes nothing, so the module reads silence\n{}", + counted(&done) + ); +} + +#[test] +fn a_malformed_threshold_refuses_and_writes_nothing() { + let held = root("badmin", &[("a.jsonl", &authored("iota"))]); + let done = census(&held, "two", Some(""), None); + assert_eq!(done.status.code(), Some(2), "a malformed threshold refuses"); + assert!(counted(&done).is_empty(), "and writes nothing"); +} + +#[test] +#[expect( + clippy::disallowed_types, + reason = "CLOUD-320's inventory row: the arity contract belongs to the thing that takes the arguments, so asserting it means invoking the producer with the wrong arity" +)] +fn more_arguments_than_the_contract_names_refuses() { + let held = root("arity", &[("a.jsonl", &authored("kappa"))]); + let done = std::process::Command::new("python3") + .arg("../../mise-tasks/transcript_census.py") + .arg(&held) + .arg("2") + .arg("") + .arg("extra") + .output() + .expect("the census runs"); + assert_eq!(done.status.code(), Some(2), "an over-long call refuses"); +} diff --git a/mise-tasks/transcript-corpus-check.sh b/mise-tasks/transcript-corpus-check.sh deleted file mode 100755 index a5b9593cd..000000000 --- a/mise-tasks/transcript-corpus-check.sh +++ /dev/null @@ -1,184 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Gate: does this host carry N independent session transcripts — the corpus condition every mined-prose literal rests on (pointer-only; two counts, never a path's content)" -# -# CLOUD-388. This repo holds prose-shaped predicates to one method: no literal -# ships until it is measured over a real corpus, counting firings AND true -# positives among them (CLOUD-252, followed by CLOUD-323 over 60 merged PR -# bodies). For PR bodies the corpus is GitHub and one API call. For SESSION -# TRANSCRIPTS there is no corpus, and the reason is the environment rather than -# anyone's oversight: transcripts are written inside the session's own ephemeral -# container and destroyed with it. -# -# MEASURED TWICE, SIX DAYS AND TWO CONTAINERS APART, SAME ANSWER: -# -# 2026-08-11 one .jsonl under the host root — the session doing the -# measuring. Independent sessions: 0. -# 2026-08-17 one .jsonl, a different container. Independent sessions: 0. -# -# Nothing accumulates a corpus BY ITSELF: every session starts at N=1, its own, -# and ends at N=0. CLOUD-326's §8.1 states its unblock condition as "N independent -# session transcripts … from sessions this issue did not arise from", and a block -# written as prose is a block no gate reads. THIS FILE IS THAT CONDITION AS A -# COMMAND AND AN EXIT CODE, which is the whole of why it exists. -# -# WHAT THE READING MEANS, and this changed once (CLOUD-651). The first version of -# this header called the corpus impossible and told the reader that waiting raises -# nothing, because CLOUD-388's verdict had ruled transcript egress out of scope. -# That was a POLICY choice about what may leave the container, not a fact about -# the world, and the owner lifted it: transcripts are collected to the Batten -# service. So a refusal here is a PROGRESS READING — the collector has not landed, -# or has not yet reached this host — rather than a permanent state of affairs, and -# the number is expected to rise. Do not re-derive the old rule from a low count. -# -# WHAT IT DOES NOT DO, deliberately. It captures nothing, pushes nothing, and -# writes nothing — the collector is CLOUD-651's, this is only the reading. Keeping -# the sensor and the collector apart is what lets the corpus condition be checked -# on a host that has never run the collector at all. -# -# Usage: mise run transcript-corpus-check [min] [exclude-session-id] -# -# min how many independent sessions the condition needs. -# Default 2 — see the constant below for why that -# number and not a larger one. -# exclude-session-id the asking session, which is not independent evidence -# about itself. Defaults to $BATTEN_SESSION_ID when the -# host sets one; empty excludes nothing. -# -# Exit 0 the condition holds, 1 it does not, 2 the question could not be asked -# (house style §7). Exit 2 is reachable only from a genuinely unreadable root or -# a malformed argument — never from a path that has looked and found nothing, -# because "no transcripts here" is an answer and must not be reported as a -# failure to look. -# A gate listed in $MUTANT_GATES with no row here fails `mise run mutant`. -#MUTANT thin-corpus-passes|s/^exit 1$/exit 0/|one transcript is one session, and one is not a corpus - -set -uo pipefail - -# THE DEFAULT IS THE WEAKEST NON-VACUOUS THRESHOLD, on purpose. Two is "more than -# the session asking", which is the least this can demand and still mean -# anything. A larger constant would look more rigorous and decide nothing extra: -# the measured count is 0 on every container this has run on, so every threshold -# from 1 upward returns the same verdict. Picking 60 to mirror CLOUD-323's PR -# corpus would be a number nobody could reach and nobody had measured — which is -# the shape of unfalsifiable decoration, not of a bound. -DEFAULT_MIN=2 - -usage() { - echo "usage: transcript-corpus-check [min] [exclude-session-id]" >&2 - exit 2 -} - -# `${1-…}` rather than `${1:-…}`, for the same reason the exclusion below uses -# the same form: an argument that is PRESENT and empty is a caller passing -# something, not a caller passing nothing. Defaulting it would launder a -# malformed call into a verdict; reaching the numeric test below refuses it. -min="${1-$DEFAULT_MIN}" -# `${2-}` rather than `${2:-}`: an explicitly EMPTY second argument is a caller -# saying "exclude nothing", and it must not silently fall back to the ambient -# session id. Absent and empty are different claims here for the same reason -# `lint.rs` says absent is not empty. -if [[ "$#" -ge 2 ]]; then - exclude="$2" -else - exclude="${BATTEN_SESSION_ID:-}" -fi -[[ "$#" -le 2 ]] || usage - -case "$min" in -'' | *[!0-9]*) usage ;; -esac - -# INJECTABLE, and that is what makes this testable at all. A live host produces -# exactly one count — its own session — so a suite that could not vary the root -# would ship as coverage while exercising a single row (CLOUD-418). The default -# is the host's own layout, which is a HOST fact rather than a consumer one, so -# it is fine here in the task layer and would not be fine in `crates/batten` -# (non-negotiable rule 1). -root="${BATTEN_TRANSCRIPT_ROOT:-$HOME/.claude/projects}" - -if [[ ! -d "$root" ]]; then - echo "::error:: transcript-corpus-check: no transcript root at $root — the corpus question could not be asked" >&2 - exit 2 -fi - -#PIN-OK: jq -if ! command -v jq >/dev/null 2>&1; then - echo "::error:: transcript-corpus-check: no jq on PATH — the corpus question could not be asked. Run: mise install" >&2 - exit 2 -fi - -# WHAT COUNTS AS INDEPENDENT, and why it is not "one file, one session". -# -# A transcript is independent evidence when it belongs to a DIFFERENT session -# that a person actually drove. Two things therefore do not count: -# -# a subagent stream — `isSidechain: true` throughout. CLOUD-326's §8.1 recorded -# "one session plus five subagent transcripts" and correctly called that N=1; -# counting the five would inflate the corpus with the orchestrator's own turns -# wearing different file names. -# -# the asking session — excluded by id below. A literal fitted to the single -# transcript it was derived from is the unmeasured-shape failure the method -# exists to prevent, so counting yourself is worse than counting nothing. -# -# The boundary test is `finding-sink-check`'s pass 1, reused rather than -# re-derived: a `type == "user"` record that is not a sidechain and carries -# authored content (a bare string, or a block array holding at least one `text` -# block). A `tool_result` also arrives as a user record and is the harness -# handing work back, not a person speaking. -# -# `fromjson?` rather than plain jq over the file: the format is a HOST's and it -# moves, so a line this build cannot decode must yield nothing rather than turn -# the whole count into "could not look". That is `transcript.rs`'s -# forward-compatibility law, applied at the same boundary from the shell side. -BOUNDARY='fromjson? - | select(type == "object") - | select(.isSidechain != true) - | select(.type == "user") - | select( (.message.content | type) == "string" - or ( (.message.content | type) == "array" - and ( [ .message.content[]? | select(type == "object" and .type == "text") ] | length ) > 0 ) ) - | .sessionId // empty' - -independent=0 -seen="" - -# `find | sort` for byte-stability (§6): the same root yields the same count and -# the same verdict however the filesystem chose to order itself. -while IFS= read -r file; do - [[ -n "$file" ]] || continue - # No pipeline, and that is deliberate: `jq … | head -n1` would exit early, - # signal the producer, and under `pipefail` promote a successful read to a - # failure status — the shape `pipefail-grep-check` refuses one command over. - # The first id is taken with a parameter expansion instead. - ids=$(jq -R -r "$BOUNDARY" "$file" 2>/dev/null) || ids="" - id=${ids%%$'\n'*} - # No authored, non-sidechain user record anywhere in the file: a subagent - # stream, or a transcript of something that never had a person in it. - [[ -n "$id" ]] || continue - [[ -n "$exclude" ]] && [[ "$id" = "$exclude" ]] && continue - # Distinct sessions, not distinct files — a host that splits one session - # across two files must not read as two. - case "$seen" in - *"|$id|"*) continue ;; - esac - seen="$seen|$id|" - independent=$((independent + 1)) -done < <(find "$root" -type f -name '*.jsonl' 2>/dev/null | sort) - -# THE WHOLE OUTPUT CONTRACT, in one line: two counts. Never a path, never a -# session id, never a byte of any transcript — pointer-only (non-negotiable rule -# 4) is a security property over this input rather than a style one, and -# `tests/transcript-corpus-check.bats` asserts the emitted bytes carry no -# substring of a fixture's content so a later edit cannot relax it. -printf 'transcript-corpus independent=%s min=%s\n' "$independent" "$min" - -[[ "$independent" -ge "$min" ]] && exit 0 - -# The refusal names WHAT WOULD RAISE THE NUMBER, not just the arithmetic. A -# reader told only "0 < 2" has nowhere to go; a reader told which mechanism feeds -# this reading can check whether it ran. The earlier wording said the count could -# never rise, which was the retired rule speaking (see the header) and would send -# a reader to work around a gate rather than to the collector. -echo "::error:: transcript-corpus-check: $independent independent session transcript(s), $min needed. A container reclaim destroys the transcripts it holds (CLOUD-388), so the corpus accumulates only where the collector (CLOUD-651) has run. Check that it is landed and reaching this host before treating a prose corpus as unavailable; a literal derived from one session is fitted to its only example. See mem:prior-art-and-issue-hygiene." >&2 -exit 1 diff --git a/mise-tasks/transcript_census.py b/mise-tasks/transcript_census.py new file mode 100644 index 000000000..09212abc8 --- /dev/null +++ b/mise-tasks/transcript_census.py @@ -0,0 +1,144 @@ +# How many INDEPENDENT session transcripts a host carries (CLOUD-388, +# CLOUD-651, ported off `mise-tasks/transcript-corpus-check.sh` under +# CLOUD-1717). +# +# THIS IS THE STEP, NEVER THE DECISION. It counts; whether the count is enough +# is `policy/transcript-corpus.rego`'s question. §5 makes `check` `read` and +# incapable of walking a host filesystem, so the census stays here. +# +# A `.py` SIBLING RATHER THAN A TASK BODY, for the reason `cargo_graph.py` gives: +# ten of the dying suite's seventeen cases are about WHAT COUNTS as an +# independent session, and they are the whole substance of the gate. Inlined into +# the task body they would be assertable by nothing; +# `crates/batten/tests/it/transcript_corpus.rs` drives this file directly, so +# they carry. `shell-retirement.rego:159-163` excludes `.py` from +# `under_mise_tasks`, so this adds no shell rule. +# +# WHAT COUNTS AS INDEPENDENT, and why it is not "one file, one session". +# +# A transcript is independent evidence when it belongs to a DIFFERENT session +# that a person actually drove. Three things therefore do not count: +# +# a subagent stream — `isSidechain: true`. CLOUD-326's section 8.1 recorded +# "one session plus five subagent transcripts" and correctly called that N=1; +# counting the five would inflate the corpus with the orchestrator's own turns +# wearing different file names. +# +# a transcript with nobody in it — a `tool_result` also arrives as a `user` +# record, and that is the harness handing work back rather than a person +# speaking. The boundary test is `finding-sink-check`'s pass 1, reused rather +# than re-derived. +# +# the asking session — a literal fitted to the single transcript it was derived +# from is the unmeasured-shape failure the method exists to prevent, so +# counting yourself is worse than counting nothing. +# +# And two files carrying one session are one session: distinct SESSIONS, not +# distinct files. +# +# A LINE THIS BUILD CANNOT DECODE YIELDS NOTHING RATHER THAN A FAILURE TO LOOK. +# The format is a HOST's and it moves, so an undecodable line contributes no id +# instead of turning the whole census into could-not-look — `transcript.rs`'s +# forward-compatibility law, applied at the same boundary from this side. +# +# Output is the record's own lines, on stdout, for `batten record named`: +# +# sessions independent sessions found +# threshold what the caller asked for +# +# POINTER-ONLY IS A SECURITY PROPERTY HERE, not a style one (non-negotiable rule +# 4): two counts, never a path, never a session id, never a byte of any +# transcript. A transcript is the richest source of secrets this repository can +# be pointed at. + +import json +import os +import sys + + +def authored_session_id(line): + """The session id of an authored, non-sidechain user record, or None.""" + try: + record = json.loads(line) + except ValueError: + # The host's format moves. An undecodable line is not a failed census. + return None + if not isinstance(record, dict): + return None + if record.get("isSidechain") is True: + return None + if record.get("type") != "user": + return None + content = (record.get("message") or {}).get("content") + if isinstance(content, str): + authored = True + elif isinstance(content, list): + authored = any( + isinstance(block, dict) and block.get("type") == "text" + for block in content + ) + else: + authored = False + if not authored: + return None + return record.get("sessionId") or None + + +def census(root, exclude): + """Distinct independent session ids under `root`, excluding `exclude`.""" + seen = set() + # Sorted for byte-stability (house style section 6): the same root yields the + # same count however the filesystem chose to order itself. + for base, _, names in sorted(os.walk(root)): + for name in sorted(names): + if not name.endswith(".jsonl"): + continue + path = os.path.join(base, name) + try: + with open(path, "r", encoding="utf-8", errors="replace") as handle: + for line in handle: + found = authored_session_id(line) + if found is None: + continue + # The FIRST authored id in the file identifies it; a file + # with none at all is a subagent stream or a transcript + # nobody was in. + if found != exclude: + seen.add(found) + break + except OSError: + # One unreadable file is not a failed census of the rest. + continue + return len(seen) + + +def main(argv): + # ABSENT AND EMPTY ARE DIFFERENT CLAIMS about the exclusion, which is why the + # argument is counted rather than defaulted. An argument that is PRESENT and + # empty is a caller saying "exclude nothing"; an argument that is absent is a + # caller saying nothing, and only then does the ambient session id apply. + # Defaulting an explicit empty would launder one into the other. + if len(argv) not in (3, 4): + sys.stderr.write("usage: transcript_census.py [exclude]\n") + return 2 + root, threshold = argv[1], argv[2] + if not threshold.isdigit(): + sys.stderr.write("transcript-census: threshold must be a whole number\n") + return 2 + if len(argv) == 4: + exclude = argv[3] or None + else: + exclude = os.environ.get("BATTEN_SESSION_ID") or None + if not os.path.isdir(root): + # The question could not be asked. Write NOTHING — an absent record is + # "the producer did not run", which must not be spelled the same way as + # a root that was walked and held no transcripts. + sys.stderr.write("transcript-census: no transcript root at %s\n" % root) + return 2 + sys.stdout.write("sessions %d\n" % census(root, exclude)) + sys.stdout.write("threshold %s\n" % threshold) + return 0 + + +if __name__ == "__main__": + sys.exit(main(sys.argv)) diff --git a/mise.toml b/mise.toml index 0b9fbc5d2..6565ca067 100644 --- a/mise.toml +++ b/mise.toml @@ -617,7 +617,7 @@ CI_FANIN_WORKFLOW = ".github/workflows/ci.yml" BATS_TEST_TIMEOUT = "300" REGORUS_OPA_COMPLIANCE = "1.2.0" REGORUS_OPA_COMPLIANCE_FOR = "0.11" -MUTANT_GATES = "mise,attestation,timeout-drift,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure,evaluator-io-check,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-callable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus-check,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" +MUTANT_GATES = "mise,attestation,timeout-drift,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure,evaluator-io-check,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-callable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" # --- GitHub reachability behind an egress proxy (Claude Code web sandbox etc.) --- # mise resolves every tool's release through GitHub's *API* host, api.github.com. @@ -2111,6 +2111,29 @@ emit | cargo run --quiet -p batten -- record named timeout-drift # # NOTHING IS WRITTEN WHEN THE GRAPH WILL NOT RESOLVE — the record contract, so an # absent record reads as "the producer did not run" rather than as a clean graph. +# CLOUD-388/CLOUD-651's census, ported off `mise-tasks/transcript-corpus-check.sh` +# under CLOUD-1717. The one thing that is not a decision: walking a HOST +# filesystem, which §5 makes `check` structurally incapable of. +# +# `BATTEN_TRANSCRIPT_ROOT` is the host's own layout — a HOST fact rather than a +# consumer one, so it is fine in this layer and would not be fine in +# `crates/batten` (non-negotiable rule 1). It is also what makes the census +# testable at all: a live host produces exactly one count, its own session, so a +# suite that could not vary the root would ship as coverage while exercising a +# single row (CLOUD-418). +# +# The census WRITES NOTHING when the root does not exist — the question could not +# be asked — which the module reads as silence. A root that was walked and held +# no transcripts records `sessions 0`, which is an answer. +[tasks.transcript-corpus-record] +description = "Effect: count this host's independent session transcripts against a threshold, for `corpus count thin`" +shell = "bash -c" +run = """ +root="${BATTEN_TRANSCRIPT_ROOT:-$HOME/.claude/projects}" +python3 mise-tasks/transcript_census.py "$root" "${1:-2}" \ + | cargo run --quiet -p batten -- record named transcript-corpus +""" + [tasks.macos-link-record] description = "Effect: record the built macOS graph's links-declaring and framework-linking crates, for `workspace carry unsafe`" shell = "bash -c" diff --git a/policy/transcript-corpus.rego b/policy/transcript-corpus.rego new file mode 100644 index 000000000..7e13512db --- /dev/null +++ b/policy/transcript-corpus.rego @@ -0,0 +1,137 @@ +# Does this host carry enough independent session transcripts to measure a +# prose-shaped literal over (CLOUD-388, CLOUD-651, ported off +# `mise-tasks/transcript-corpus-check.sh` under CLOUD-1717). +# +# WHY THE CONDITION EXISTS AT ALL. This repository holds prose-shaped predicates +# to one method: no literal ships until it is measured over a real corpus, +# counting firings AND true positives among them (CLOUD-252, then CLOUD-323 over +# 60 merged PR bodies). For PR bodies the corpus is GitHub and one API call. For +# SESSION TRANSCRIPTS there was no corpus, and the reason was the environment +# rather than anyone's oversight: transcripts are written inside a session's own +# ephemeral container and destroyed with it. CLOUD-326's section 8.1 states its +# unblock condition as "N independent session transcripts", and a block written +# as prose is a block no gate reads. This pair is that condition as a command and +# an exit code, which is the whole of why it exists. +# +# A REFUSAL HERE IS A PROGRESS READING, NOT A PERMANENT STATE, and this file says +# so because the retired program's first header said the opposite. That version +# called the corpus impossible, on CLOUD-388's ruling that transcript egress was +# out of scope — a POLICY choice about what may leave the container rather than a +# fact about the world. The owner lifted it and transcripts are collected to the +# Batten service, so a low count means the collector has not landed or has not +# reached this host. Do not re-derive the old rule from a low number. +# +# THE SPLIT. The census is `mise-tasks/transcript_census.py`, driven by +# `[tasks.transcript-corpus-record]`: §5 makes `check` `read` and incapable of +# walking a host filesystem. What is left here is the comparison, and it is the +# whole decision. +# +# THE THRESHOLD TRAVELS IN THE RECORD rather than living here, and that is the +# caller's argument preserved. The default is 2 — the weakest non-vacuous bound, +# "more than the session asking", which is the least this can demand and still +# mean anything. A larger constant would look more rigorous and decide nothing +# extra: the measured count is 0 on every container this has run on, so every +# threshold from 1 upward returns the same verdict. +# +# POINTER-ONLY IS A SECURITY PROPERTY OVER THIS INPUT (rule 4), not a style one. +# The record carries two counts and the finding carries two counts; no path, no +# session id, and no byte of any transcript reaches either. +# +#MUTANT-SUITE crates/batten/tests/it/transcript_corpus.rs +#MUTANT thin-corpus-passes|s@^\tsessions < threshold$@\tfalse@|one_transcript_is_one_session_and_one_is_not_a_corpus + +# METADATA +# description: | +# Bound to the TREE surface: `scope = "tree"`, so it reads the tree document +# and never the mediated `{call, facts}` shape. +# THE BRACKETS ARE NOT STYLE: the schema file carries a hyphen, so the dotted +# form is a parse error reported as `invalid schema reference`. +# THIS BLOCK IS YAML AND MUST STAY THE LAST COMMENT BLOCK BEFORE `package`. +# schemas: +# - input: schema["policy-input.schema"] +package batten.transcript_corpus + +import rego.v1 + +rules contains "prose measure partial" + +# The record, or nothing. ABSENT IS NOT EMPTY: a host whose producer never ran +# has no key here, Rego reads that as *does not hold*, and the rule is silent. +# The producer writes nothing when the transcript root does not exist, which is +# the question not being askable — distinct from a root that was walked and held +# no transcripts, which is the answer zero. +lines := input.tree.records["transcript-corpus"] + +counted(label) := value if { + some line in lines + startswith(line, concat("", [label, " "])) + raw := trim_space(substring(line, count(label) + 1, -1)) + regex.match(data.batten.patterns["whole-number"], raw) + value := to_number(raw) +} + +violation contains { + "rule": "prose measure partial", + "verdict": "prose measure partial", + "subjects": [{"count": sessions}, {"count": threshold}], +} if { + sessions := counted("sessions") + threshold := counted("threshold") + sessions < threshold +} + +# --- cases ------------------------------------------------------------------- + +test_one_transcript_is_one_session_and_one_is_not_a_corpus if { + found := violation with input as {"tree": {"records": {"transcript-corpus": [ + "sessions 1", + "threshold 2", + ]}}} + count(found) == 1 +} + +test_an_empty_root_is_zero_which_is_an_answer if { + found := violation with input as {"tree": {"records": {"transcript-corpus": [ + "sessions 0", + "threshold 2", + ]}}} + count(found) == 1 +} + +test_three_distinct_sessions_satisfy_the_default_threshold if { + found := violation with input as {"tree": {"records": {"transcript-corpus": [ + "sessions 3", + "threshold 2", + ]}}} + count(found) == 0 +} + +# THE THRESHOLD IS THE CALLER'S, so the same corpus can fail a stricter one. +test_the_threshold_is_the_argument if { + found := violation with input as {"tree": {"records": {"transcript-corpus": [ + "sessions 3", + "threshold 5", + ]}}} + count(found) == 1 +} + +# The boundary is inclusive: meeting the threshold exactly is meeting it. +test_meeting_the_threshold_exactly_is_clean if { + found := violation with input as {"tree": {"records": {"transcript-corpus": [ + "sessions 2", + "threshold 2", + ]}}} + count(found) == 0 +} + +test_an_absent_record_says_nothing_rather_than_refusing if { + found := violation with input as {"tree": {"records": {}}} + count(found) == 0 +} + +# A torn record — the producer never writes one without the other — leaves the +# comparison undefined rather than deciding on half a reading. +test_a_record_missing_a_column_decides_nothing if { + found := violation with input as {"tree": {"records": {"transcript-corpus": ["sessions 0"]}}} + count(found) == 0 +} diff --git a/tests/transcript-corpus-check.bats b/tests/transcript-corpus-check.bats deleted file mode 100644 index 7a60e796d..000000000 --- a/tests/transcript-corpus-check.bats +++ /dev/null @@ -1,196 +0,0 @@ -#!/usr/bin/env bats -# subject: mise-tasks/transcript-corpus-check.sh -# transcript-corpus-check: is there a corpus of independent session transcripts? -# (CLOUD-388) -# -# A live host answers this ONE way — its own session and nothing else — so a -# suite that could not vary the root would ship as coverage while exercising a -# single row (CLOUD-418). `BATTEN_TRANSCRIPT_ROOT` is injected throughout, and -# the rows below are the counts a real host never produces: zero, three, a -# subagent stream that must not inflate the count, and two files carrying one -# session. -# -# The fixture content carries a distinctive token on purpose. The last row -# asserts the emitted bytes do not contain it — pointer-only is a SECURITY -# property over this input rather than a style one, and asserting it is what -# stops a later edit turning a count into a quotation. - -setup() { - CHECK="$BATS_TEST_DIRNAME/../mise-tasks/transcript-corpus-check.sh" - ROOT="$BATS_TEST_TMPDIR/projects" - mkdir -p "$ROOT" - export BATTEN_TRANSCRIPT_ROOT="$ROOT" - # Never inherit the host's. A row that passes because the runner happened to - # export a session id is a row that discriminates nothing. - unset BATTEN_SESSION_ID - FIXTURE_TOKEN="pomegranate-carburettor" -} - -# A transcript with one authored, non-sidechain user turn: the shape that opens -# a real session. -session() { # session - printf '%s\n' \ - "{\"type\":\"user\",\"sessionId\":\"$2\",\"message\":{\"role\":\"user\",\"content\":\"$FIXTURE_TOKEN\"}}" \ - "{\"type\":\"assistant\",\"sessionId\":\"$2\",\"message\":{\"role\":\"assistant\",\"content\":[{\"type\":\"text\",\"text\":\"$FIXTURE_TOKEN\"}]}}" \ - >"$ROOT/$1" -} - -# --- the count ----------------------------------------------------------------- - -@test "an empty root is zero independent sessions, which is an answer and not a failure to look" { - run "$CHECK" - [ "$status" -eq 1 ] - [[ "$output" == *"transcript-corpus independent=0 min=2"* ]] -} - -@test "one transcript is one session, and one is not a corpus" { - session only.jsonl s-1 - run "$CHECK" - [ "$status" -eq 1 ] - [[ "$output" == *"independent=1 min=2"* ]] -} - -@test "three distinct sessions satisfy the default threshold" { - session a.jsonl s-1 - session b.jsonl s-2 - session c.jsonl s-3 - run "$CHECK" - [ "$status" -eq 0 ] - [[ "$output" == *"independent=3 min=2"* ]] -} - -@test "the threshold is the argument, so the same corpus can fail a stricter one" { - session a.jsonl s-1 - session b.jsonl s-2 - run "$CHECK" 2 - [ "$status" -eq 0 ] - run "$CHECK" 3 - [ "$status" -eq 1 ] - [[ "$output" == *"independent=2 min=3"* ]] -} - -# --- what does not count ------------------------------------------------------- - -@test "a subagent stream is not an independent session" { - # CLOUD-326 §8.1 recorded one session plus five subagent transcripts and - # correctly called that N=1. Every record here is a sidechain. - printf '%s\n' \ - "{\"type\":\"user\",\"sessionId\":\"sub-1\",\"isSidechain\":true,\"message\":{\"role\":\"user\",\"content\":\"$FIXTURE_TOKEN\"}}" \ - >"$ROOT/sub.jsonl" - session real.jsonl s-1 - run "$CHECK" - [ "$status" -eq 1 ] - [[ "$output" == *"independent=1 min=2"* ]] -} - -@test "a transcript carrying only tool results has nobody in it" { - # A `tool_result` arrives as a user record and is the harness handing work - # back. Counting it would make a session out of a transcript no person drove. - printf '%s\n' \ - "{\"type\":\"user\",\"sessionId\":\"s-9\",\"message\":{\"role\":\"user\",\"content\":[{\"type\":\"tool_result\",\"tool_use_id\":\"t1\",\"content\":\"$FIXTURE_TOKEN\"}]}}" \ - >"$ROOT/results.jsonl" - run "$CHECK" - [ "$status" -eq 1 ] - [[ "$output" == *"independent=0"* ]] -} - -@test "two files carrying one session are one session" { - session first.jsonl s-1 - session second.jsonl s-1 - run "$CHECK" - [ "$status" -eq 1 ] - [[ "$output" == *"independent=1"* ]] -} - -@test "a line this build cannot decode yields nothing rather than a failure to look" { - # The format is a host's and it moves. A gate that reddened on an unknown or - # truncated line would be switched off within a release. - session good.jsonl s-1 - printf '%s\n' 'not json at all' "{\"type\":\"queue-operation\",\"content\":\"$FIXTURE_TOKEN\"}" \ - >"$ROOT/odd.jsonl" - run "$CHECK" 1 - [ "$status" -eq 0 ] - [[ "$output" == *"independent=1 min=1"* ]] -} - -# --- the asking session --------------------------------------------------------- - -@test "excluding the asking session turns its own transcript into zero" { - session mine.jsonl s-mine - run "$CHECK" 2 s-mine - [ "$status" -eq 1 ] - [[ "$output" == *"independent=0"* ]] -} - -@test "the exclusion defaults from the environment when no argument names one" { - session mine.jsonl s-mine - session other.jsonl s-other - BATTEN_SESSION_ID=s-mine run "$CHECK" - [ "$status" -eq 1 ] - [[ "$output" == *"independent=1"* ]] -} - -@test "an explicitly empty exclusion excludes nothing, and does not fall back to the environment" { - # Absent and empty are different claims: a caller passing "" is saying - # "count everything", and silently substituting the ambient session id would - # answer a question nobody asked. - session mine.jsonl s-mine - BATTEN_SESSION_ID=s-mine run "$CHECK" 1 "" - [ "$status" -eq 0 ] - [[ "$output" == *"independent=1 min=1"* ]] -} - -# --- could not look ------------------------------------------------------------ - -@test "an absent root is exit 2, never a verdict about a corpus nobody looked at" { - export BATTEN_TRANSCRIPT_ROOT="$BATS_TEST_TMPDIR/nowhere" - run "$CHECK" - [ "$status" -eq 2 ] - [[ "$output" == *"could not be asked"* ]] -} - -@test "a malformed threshold is exit 2" { - run "$CHECK" two - [ "$status" -eq 2 ] - run "$CHECK" "" - [ "$status" -eq 2 ] -} - -@test "more arguments than the contract names is exit 2" { - run "$CHECK" 2 s-1 extra - [ "$status" -eq 2 ] -} - -# --- the output contract --------------------------------------------------------- - -@test "the report is two counts and carries no byte of any transcript" { - session a.jsonl s-1 - session b.jsonl s-2 - run "$CHECK" - [ "$status" -eq 0 ] - # Not the content, not the session ids, not the paths. A count and a count. - [[ "$output" != *"$FIXTURE_TOKEN"* ]] - [[ "$output" != *"s-1"* ]] - [[ "$output" != *"a.jsonl"* ]] - [[ "$output" != *"$ROOT"* ]] -} - -@test "the refusal names what would raise the number, not just the arithmetic" { - # "0 < 2" sends a reader looking for a way around the gate. What they need is - # the mechanism that feeds this reading, so they can check whether it ran. - run "$CHECK" - [ "$status" -eq 1 ] - [[ "$output" == *"collector"* ]] - [[ "$output" == *"mem:prior-art-and-issue-hygiene"* ]] -} - -@test "the refusal does not tell the reader the count can never rise" { - # The retired rule (CLOUD-388's first verdict) said transcript egress was out - # of scope, so the corpus could never accumulate. That was policy, not - # physics, and it was lifted — a refusal that still says "waiting raises - # nothing" would send the next reader to re-derive a rule nobody holds. - run "$CHECK" - [ "$status" -eq 1 ] - [[ "$output" != *"waiting raises nothing"* ]] - [[ "$output" != *"does not accumulate"* ]] -} From 9eb1ed98a4b7d941dd965f593f3e132847df1761 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sat, 19 Sep 2026 02:35:51 +0000 Subject: [PATCH 19/32] feat(policy): CLOUD-418's probe decides in the engine, over a reading both callers share MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `mise-tasks/evaluator-io-check.sh` and its suite are gone. The inversion is `policy/evaluator-io-probe.rego`; the probe build is `[tasks.evaluator-io-record]`; the reading of what the harness said is `mise-tasks/probe_verdict.py`. ## What the gate defends `no_evaluator_feature_admits_io` asserts a policy module cannot reach `http.send`, and that assertion is the evidence behind the claim admitting consumer-authored code to the mediated call. Under the shipped feature set it is green — and **a green test proves nothing about whether it can DISCRIMINATE**. If regorus stopped registering the builtin, or the fixture stopped compiling, or the assertion were reworded into a tautology, it would stay green over an evaluator that had lost the property entirely. So the producer builds it with `probe-evaluator-io` on and the module requires it to have FAILED. ## Three states, and the third is the one the obvious shape gets wrong `cargo test` exits non-zero for a compile error, an unresolved feature, an absent toolchain and a panic in another test. Every one of those would read as "the probe falsified the assertion" and hand the gate a pass it did not earn — and that pass gets MORE likely as the crate breaks, so the gate would be loudest exactly when it was lying. `probe unread` is a refusal here rather than silence, and it is a separate class from `probe passed` because a reader acts on them differently: one says the test is theatre, the other says nothing was learned. ## The reading is a file because it has two callers `probe_verdict.py` takes a status and a log and emits one token. The producer calls it to write the record; `crates/batten/tests/it/evaluator_io_probe.rs` calls it to drive all four arms without a two-minute rebuild per case. The first draft of this tier inlined those three branches as a `CLASSIFY` constant "verbatim from the task body" — which is exactly the second authority `mise-tasks/cargo_graph.py` was extracted to remove one gate over, and a comment saying "verbatim" is the same rule-without-a-mechanism that pair was carrying. Extracted instead, so there is one reading to correct. ## Two corrections this port made to itself **The rule id constraint runs in BOTH directions.** A rule that is a class's sole raiser must be spelled as that class; one that raises TWO must not, because the name would answer for two different things. `test judge never` was refused at load for the second reason, so the row is `test cover never` and the two verdicts keep their own names. Worth recording: the prior handoff stated only the first half of that rule. **The rendered line shows the path, not the test name**, whatever order the subjects are declared in — `rules/policy-modules.md` makes the first path-bearing subject the finding's pointer. The retired program printed ` `; the engine prints ` `. That is non-negotiable rule 5 — one output contract, no per-verb exception — and the test name moved to the `[[verdict]]` row's gloss and the JSON channel. Recorded as a `// changed:` arm rather than left for a reader to notice. ## Proven live, not merely green `mise run batten -- check --rule 'test cover never'` over a recorded `probe passed`: `crates/batten/tests/policy_modules.rs test cover never`, exit 2. Exit 0 alone is byte-identical to a dead gate (CLOUD-1810). ## Corpus 75 -> 74 programs, 86 -> 85 suites. 11 of 44 retired. Refs: CLOUD-1717 Refs: CLOUD-418 Refs: CLOUD-831 Admits: b18866f56c4ffb400f2282e7018e6a7803db557a761ba1af4657c5198ac84f84 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: batten.toml Admits-anchor: call:8f8f3228ea4fe80ab1bc90795ceb1bb84adfe43d Admits-epoch: c30cebe85a541330b8d503e5c3201acf622f0d4156d235863108bdc77acbe5ac Admits-author: alec@wenzowski.com Admits-prev: f53fe1dd65a0478ec1069c7550620a77a1c34c5febf986aef9b554fb668d1580 Admits-answer-lost: The retirement cannot land and the tree will not load. This gate is CLOUD-418's obligation made runnable: `no_evaluator_feature_admits_io` asserts a policy module cannot reach http.send, and under the shipped feature set that assertion is green whether or not it still discriminates. Without the probe, a reworded assertion or a regorus that stopped registering the builtin would leave that test green over an evaluator that had lost the property entirely — and that test is the evidence behind the claim admitting consumer-authored code to the mediated call. Admits-answer-precondition: batten.toml is the only surface carrying [[rule]], [[verdict]] and [[record]] rows. CLOUD-1717 retires mise-tasks/evaluator-io-check.sh onto policy/evaluator-io-probe.rego plus [tasks.evaluator-io-record]; the module raises `test judge never` and `test run unread` and reads the `evaluator-io-probe` record family, and the engine refuses at load both a module raising an undeclared token and a declared row nothing raises. No other file can carry any of it. It lands in PR #962 as an ordinary diff hunk beside the three retirements already admitted on this branch. Admits-answer-rejected-route: `config read first` is a read of batten.toml; I read it to confirm `test judge never` and `test run unread` are unused — `test cover partial`, the first name I reached for, is already taken by cfg-gated-test, and reading is how I found that — and to draw every word from the declared [vocabulary] lists. Reading cannot declare the rows. `patch run first` is `git restore`, which would discard the module and the producer together rather than complete them. Admits: 4bf96f29efc68ea7e7ddad1a86ab98ba058b6be0407aaf3c0bb829d6cea44b15 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: batten.toml Admits-anchor: call:8f8f3228ea4fe80ab1bc90795ceb1bb84adfe43d Admits-epoch: b45e6212cde82deeafa2e5518669be33425172f3ab074f4d5eac561c67414a21 Admits-author: alec@wenzowski.com Admits-prev: b18866f56c4ffb400f2282e7018e6a7803db557a761ba1af4657c5198ac84f84 Admits-answer-lost: The tree does not load at all. `batten policy test`, `batten check` and every gate in the repository exit 1 with `rule declare refused` until the id is distinct, which is strictly worse than before this retirement started — and self-latching, since a config that will not load disarms the protected-path guard on batten.toml itself and makes `override request` unrunnable (measured this session, filed on CLOUD-1768). Admits-answer-precondition: batten.toml is the only surface carrying the [[rule]] row's id, and the engine refused the one admitted moments ago: "rule `test judge never` is spelled as a class token but is not that class's sole raiser, so the name answers for two different things — give the row a distinct three-word id". The row must become `test cover never` or the tree does not load. The module has already been corrected; this is the other half of the same one-word fix, and it lands in PR #962 in the same hunk. Admits-answer-rejected-route: `config read first` is a read of batten.toml; I read it to confirm `test cover never` is unused and that all three words are in the declared [vocabulary] lists. Reading is also what would have prevented this had I known the constraint ran in both directions — a rule id MUST equal its verdict when it is that class's sole raiser, and must NOT when it raises two. Reading cannot apply the rename. `patch run first` is `git restore`, which would discard the module, the producer and the tier together rather than fix one word. --- batten.toml | 57 +++++ bench/suites/RESULTS.md | 5 +- crates/batten/tests/it/evaluator_io_probe.rs | 251 +++++++++++++++++++ crates/batten/tests/it/main.rs | 1 + mise-tasks/evaluator-io-check.sh | 129 ---------- mise-tasks/probe_verdict.py | 68 +++++ mise.toml | 58 ++++- policy/evaluator-io-probe.rego | 108 ++++++++ tests/evaluator-io-check.bats | 60 ----- 9 files changed, 544 insertions(+), 193 deletions(-) create mode 100644 crates/batten/tests/it/evaluator_io_probe.rs delete mode 100755 mise-tasks/evaluator-io-check.sh create mode 100644 mise-tasks/probe_verdict.py create mode 100644 policy/evaluator-io-probe.rego delete mode 100644 tests/evaluator-io-check.bats diff --git a/batten.toml b/batten.toml index c0ca606df..46c4fb651 100644 --- a/batten.toml +++ b/batten.toml @@ -5997,6 +5997,63 @@ no_fix_reason = "the number rises when the collector reaches this host, not when record = "transcript-corpus" writer = "mise run transcript-corpus-record" +# CLOUD-418's obligation, made runnable rather than promised, and ported under +# CLOUD-1717. `no_evaluator_feature_admits_io` is green under the shipped feature +# set whether or not it still DISCRIMINATES; this is the row that requires it to +# go red with `regorus/http` on. +# +# THE ROW'S ID IS NOT ONE OF ITS VERDICT TOKENS, and the engine holds it both +# ways: a rule that is a class's SOLE raiser must be spelled as that class, and +# one that raises TWO must not, because the name would answer for two different +# things. This row raises `test judge never` and `test run unread`, so it carries +# a third name of its own. +[[rule]] +id = "test cover never" +kind = "policy" +scope = "tree" +module = "policy/evaluator-io-probe.rego" +severity = "deny" +no_fix_reason = "a test that no longer discriminates is fixed in the test, not here: either the assertion has been reworded into something the probe cannot falsify, or the fixture no longer reaches `http.send`" + +[[record]] +record = "evaluator-io-probe" +writer = "mise run evaluator-io-record" + +[[verdict]] +id = "test judge never" +gloss = "the IO-free evaluator test stayed green with regorus's http feature on, so it discriminates nothing" +class = """ +A test that is green whether or not the evaluator can reach the network is not \ +evidence that it cannot. `crates/batten/src/policy.rs` admits consumer-authored \ +code to the mediated call on the claim that a policy module cannot open a file, \ +start a process or reach the network, and this test is that claim's evidence — so \ +a version of it that cannot fail is coverage theatre standing where the evidence \ +should be. Fix the test, not the gate. +""" + +[[verdict.route]] +id = "test read first" +kind = "document" +target = "crates/batten/tests/policy_modules.rs" + +[[verdict]] +id = "test run unread" +gloss = "the probe build exited non-zero without running the named test to a failure" +class = """ +COULD NOT LOOK, and it is the arm a gate written to the obvious shape gets wrong. \ +`cargo test` exits non-zero for a compile error, an unresolved feature, an absent \ +toolchain and a panic in another test — every one of which would read as "the \ +probe falsified the assertion" and hand this gate a pass it did not earn. That \ +pass gets MORE likely as the crate breaks, so the gate would be loudest exactly \ +when it was lying. The producer reads the harness's own failures listing instead, \ +and says so here when it saw something else. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run evaluator-io-record" + [[verdict]] id = "prose measure partial" gloss = "this host carries fewer independent session transcripts than the caller asked for" diff --git a/bench/suites/RESULTS.md b/bench/suites/RESULTS.md index 145ad7d44..57e2f9130 100644 --- a/bench/suites/RESULTS.md +++ b/bench/suites/RESULTS.md @@ -6,8 +6,8 @@ runner measured it; the suite runs `--no-parallelize-within-files`, so a file's number is its own serial cost and is what an author adding a case to it pays. -- suites: 96 -- serial total: 157.5s +- suites: 95 +- serial total: 157.4s | seconds | share | suite | | ---: | ---: | --- | @@ -93,7 +93,6 @@ to it pays. | 0.1 | 0.1% | `tests/task-fail-closed.bats` | | 0.1 | 0.1% | `tests/dist.bats` | | 0.1 | 0.1% | `tests/egress-check.bats` | -| 0.1 | 0.1% | `tests/evaluator-io-check.bats` | | 0.0 | 0.0% | `tests/darwin-link.bats` | | 0.0 | 0.0% | `tests/cross-check.bats` | | 0.0 | 0.0% | `tests/zizmor-split.bats` | diff --git a/crates/batten/tests/it/evaluator_io_probe.rs b/crates/batten/tests/it/evaluator_io_probe.rs new file mode 100644 index 000000000..0b9dd073a --- /dev/null +++ b/crates/batten/tests/it/evaluator_io_probe.rs @@ -0,0 +1,251 @@ +//! `test judge never` over the compiled binary and the real producer (CLOUD-418, +//! CLOUD-831, CLOUD-1717). +//! +//! # Why this tier exists and the module's own `test_` rules do not suffice +//! +//! `policy/evaluator-io-probe.rego` carries five load-time cases and every one +//! fabricates its input with `with input as`, which is the shape +//! `rules/policy-modules.md` warns about. +//! +//! # And why the PRODUCER's classification is driven here +//! +//! Three of the dying suite's five cases are about how the probe build's output +//! is READ, not about the verdict that follows: a build that passed, one that +//! failed to compile, and one where the named test never ran. Those three are +//! the whole substance — the module's half is two `in` tests — and the middle one +//! is the arm a gate written to the obvious shape gets wrong, because a non-zero +//! exit from `cargo test` means a compile error just as readily as a falsified +//! assertion. +//! +//! `EVALUATOR_IO_PROBE_CMD` is what makes that drivable without a two-minute +//! rebuild per case, and these cases use it exactly as the retired suite did. +//! +//! # RETIREMENT LEDGER, PER PATH — what `shell retire partial` reads +//! +// carried: mise-tasks/evaluator-io-check.sh policy/evaluator-io-probe.rego kind:mechanism crates/batten/tests/it/evaluator_io_probe.rs +// carried: tests/evaluator-io-check.bats policy/evaluator-io-probe.rego kind:mechanism crates/batten/tests/it/evaluator_io_probe.rs +// carried: "a probe build in which the test PASSES is the finding, not a pass" policy/evaluator-io-probe.rego kind:mechanism +// carried: "a probe build in which the test FAILS is the pass" policy/evaluator-io-probe.rego kind:mechanism +// carried: "a probe build that failed to COMPILE is could-not-look, not the pass" policy/evaluator-io-probe.rego kind:mechanism +// carried: "a probe build where the named test never ran is could-not-look" policy/evaluator-io-probe.rego kind:mechanism +// carried: "the probe build's own output never reaches the gate's output" policy/evaluator-io-probe.rego kind:mechanism +// changed: "the refusal names the test to fix" batten.toml the retired program printed ` `; the engine renders ` `, because `rules/policy-modules.md` makes the first path-bearing subject the finding's pointer whatever order the subjects are declared in. That is non-negotiable rule 5 — one output contract, no per-verb exception — so the test name moved to the `[[verdict]]` row's gloss and to the JSON channel, where a reader still meets it + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use common::{git_in, init_repo, run, run_with_stdin, scratch, write}; + +/// A repository registering the real module against the declared family. +fn repo(name: &str) -> std::path::PathBuf { + let dir = scratch(&format!("evaluator-io-probe-{name}")); + let module = std::fs::read_to_string("../../policy/evaluator-io-probe.rego") + .expect("the module this tier exists for"); + write(&dir, "policy/evaluator-io-probe.rego", &module); + write( + &dir, + "batten.toml", + r#"version = 1 +scope = ["**"] + +[[verdict]] +id = "test judge never" +gloss = "the IO-free evaluator test stayed green with regorus's http feature on" +class = "A test green whether or not the evaluator can reach the network is not evidence that it cannot." + +[[verdict.route]] +id = "test read first" +kind = "document" +target = "crates/batten/tests/policy_modules.rs" + +[[verdict]] +id = "test run unread" +gloss = "the probe build exited non-zero without running the named test to a failure" +class = "Could not look: a compile error reads as a falsified assertion unless the harness line is what decides." + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run evaluator-io-record" + +[[rule]] +id = "test cover never" +kind = "policy" +scope = "tree" +module = "policy/evaluator-io-probe.rego" +severity = "deny" + +[[record]] +record = "evaluator-io-probe" +writer = "mise run evaluator-io-record" +"#, + ); + init_repo(&dir); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-qm", "register the module"]); + dir +} + +fn record(dir: &std::path::Path, verdict: &str) { + let written = run_with_stdin( + dir, + &["record", "named", "evaluator-io-probe"], + &format!("{verdict}\n"), + ); + assert!( + written.status.success(), + "the setup write lands: {}", + String::from_utf8_lossy(&written.stderr) + ); +} + +fn said(decided: &std::process::Output) -> String { + format!( + "{}{}", + String::from_utf8_lossy(&decided.stdout), + String::from_utf8_lossy(&decided.stderr) + ) +} + +// --- the decision, over the engine's own projection -------------------------- + +#[test] +fn a_probe_build_in_which_the_test_passes_is_the_finding_not_a_pass() { + // THE INVERSION IS THE GATE. A probe build that SUCCEEDED means the test + // stayed green with `http` on, so it discriminates nothing. + let dir = repo("passed"); + record(&dir, "probe passed"); + + let decided = run(&dir, &["check"]); + assert_eq!( + decided.status.code(), + Some(2), + "a green probe is the finding\n{}", + String::from_utf8_lossy(&decided.stderr) + ); + // THE POINTER IS THE FILE, NOT THE TEST NAME, and that is the engine's + // uniform output contract rather than a loss. `rules/policy-modules.md`: + // "the first path-bearing subject becomes the finding's own pointer" — so a + // `{path}` subject always renders in preference to an `{artifact}`, whatever + // order they are declared in. The test name rides the JSON channel and the + // verdict's own class; the line gives the reader the file to open. + assert!( + said(&decided).contains("crates/batten/tests/policy_modules.rs"), + "and points at the file to open\n{}", + said(&decided) + ); +} + +#[test] +fn a_probe_build_in_which_the_test_fails_is_the_pass() { + let dir = repo("failed"); + record(&dir, "probe failed"); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "it failed, which is the pass\n{}", + String::from_utf8_lossy(&quiet.stderr) + ); +} + +#[test] +fn a_probe_build_that_failed_to_compile_is_could_not_look_not_the_pass() { + let dir = repo("unread"); + record(&dir, "probe unread"); + + let decided = run(&dir, &["check"]); + assert_eq!( + decided.status.code(), + Some(2), + "could not look is loud, never the pass\n{}", + String::from_utf8_lossy(&decided.stderr) + ); +} + +#[test] +fn an_absent_record_says_nothing_rather_than_refusing() { + let dir = repo("unrecorded"); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "an absent record is silence\n{}", + String::from_utf8_lossy(&quiet.stderr) + ); +} + +// --- the producer's classification ------------------------------------------- + +/// Run the REAL reading the producer runs, over a status and a log. +/// +/// `mise-tasks/probe_verdict.py` is the one authority on these three branches; +/// a copy of them here would be the second authority `cargo_graph.py` was +/// extracted to remove one gate over. +#[expect( + clippy::disallowed_types, + reason = "CLOUD-320's inventory row, and the verdict is that this spawn IS the subject under test: the classification of a probe build's output is what moved out of the dying program, and driving it directly is what keeps the three arms assertable without a two-minute rebuild per case" +)] +fn classify(name: &str, status: i32, log: &str) -> String { + let dir = scratch(&format!("evaluator-io-stub-{name}")); + write(&dir, "probe.log", log); + + let done = std::process::Command::new("python3") + .arg("../../mise-tasks/probe_verdict.py") + .arg(status.to_string()) + .arg(dir.join("probe.log")) + .arg("no_evaluator_feature_admits_io") + .output() + .expect("the reading runs"); + assert!( + done.status.success(), + "the reading completes: {}", + String::from_utf8_lossy(&done.stderr) + ); + String::from_utf8_lossy(&done.stdout).trim().to_owned() +} + +#[test] +fn a_probe_build_that_succeeds_classifies_as_passed() { + assert_eq!(classify("green", 0, ""), "probe passed"); +} + +#[test] +fn a_probe_build_that_ran_the_named_test_to_a_failure_classifies_as_failed() { + let harness = "failures:\n no_evaluator_feature_admits_io\n\ntest result: FAILED. 0 passed; 1 failed\n"; + assert_eq!(classify("red", 101, harness), "probe failed"); +} + +#[test] +fn a_probe_build_that_failed_to_compile_classifies_as_unread() { + // THE ARM A GATE WRITTEN TO THE OBVIOUS SHAPE GETS WRONG: non-zero, and no + // harness line at all. + let broken = "error[E0432]: unresolved import\n"; + assert_eq!(classify("compile", 101, broken), "probe unread"); +} + +#[test] +fn a_probe_build_where_the_named_test_never_ran_classifies_as_unread() { + // A different test failed, so the harness says FAILED and the listing names + // somebody else. Reading the exit code would call this the discrimination + // this gate is looking for. + let other = "failures:\n some_other_test\n\ntest result: FAILED. 3 passed; 1 failed\n"; + assert_eq!(classify("other", 101, other), "probe unread"); +} + +#[test] +fn the_probe_builds_own_output_never_reaches_the_record() { + // Pointer-only (rule 4): the probe log carries module bodies and paths, and + // what the record receives is one token. + let noisy = "SECRET_MODULE_BODY\nerror: build failed\n"; + let classified = classify("noisy", 101, noisy); + assert_eq!(classified, "probe unread"); + assert!( + !classified.contains("SECRET_MODULE_BODY"), + "no byte of the probe log reaches the record\n{classified}" + ); +} diff --git a/crates/batten/tests/it/main.rs b/crates/batten/tests/it/main.rs index bcc40242e..d865f9ece 100644 --- a/crates/batten/tests/it/main.rs +++ b/crates/batten/tests/it/main.rs @@ -123,6 +123,7 @@ mod egress_fencing; mod emission_census; mod enforce_journal; mod evaluator_closure; +mod evaluator_io_probe; mod extension_surfaces; mod external_facts; mod extracted_facts; diff --git a/mise-tasks/evaluator-io-check.sh b/mise-tasks/evaluator-io-check.sh deleted file mode 100755 index b57537b61..000000000 --- a/mise-tasks/evaluator-io-check.sh +++ /dev/null @@ -1,129 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Gate: the IO-free evaluator test is shown able to FAIL — it must go red when regorus's `http` feature is on (CLOUD-831, CLOUD-418)" -# -# CLOUD-418's obligation, made runnable rather than promised. -# `no_evaluator_feature_admits_io` asserts a policy module cannot reach -# `http.send`. Under the shipped feature set that is true, so the test is green — -# and a green test proves nothing about whether it can DISCRIMINATE. If regorus -# silently stopped registering the builtin, or the fixture stopped compiling, or -# the assertion were reworded into a tautology, the test would stay green over an -# evaluator that had lost the property entirely. -# -# So this builds the same test with the `probe-evaluator-io` feature on — which -# turns `regorus/http` on and nothing else — and REQUIRES IT TO FAIL. A probe -# build that passes is the finding. -# -# THE PROBE FEATURE COSTS THE CLOSURE NOTHING, which is what makes running this -# affordable and what stops it weakening the gate it defends. `regorus`'s `http` -# feature is `[]` — a bare feature gating only the builtin's registration — so -# `Cargo.lock` is byte-identical with it on or off, and -# `evaluator-closure-check`'s walk (default features, activated edges) cannot see -# it. Measured 2026-08-21; the rejected `regorus/jsonschema` half and why it is -# not here are recorded in `crates/batten/Cargo.toml`. -# -# RECEIPT-GATED (CLOUD-424), because a feature toggle forces a fresh compile of -# the crate and its test binary — the one case where the ~0.3s a check/record -# pair costs is obviously dwarfed. Same inputs, same command, same toolchain ⇒ -# same verdict. -# -# The mutation drops the inversion, so a probe build that PASSES — the test no -# longer discriminating, which is the whole finding — reads as success. -#MUTANT io-probe-not-inverted|s/^\tprobe_passed=yes$/\tprobe_passed=no/|the test PASSES is the finding -# And the second, which is the subtler one: drop the "did it actually RUN" arm -# and any non-zero exit — a compile error above all — reads as the discrimination -# this gate is looking for. -#MUTANT io-probe-trusts-the-exit-code|s/^ran_and_failed=no$/ran_and_failed=yes/|failed to COMPILE -set -uo pipefail - -# Guarded by hand, because this body deliberately runs without `set -e` — the -# probe build is EXPECTED to exit non-zero and errexit would take the whole task -# down on the gate's own pass. A `cd` that failed silently would leave every path -# below reading some other directory, which is the shape `.claude/rules/ -# toolchain.md` names: a gate that reports on state it never reached. -cd "${EVALUATOR_ROOT:-$(git rev-parse --show-toplevel)}" || { - echo "::error:: evaluator-io-check: could not reach the repository root, so there is nothing to probe" >&2 - exit 1 -} - -# The test the probe must falsify. Named once: it is the argument to the probe -# build, the string the verdict is read from, and the pointer the refusal prints -# — three spellings of it is how a rename turns this gate silent instead of red. -readonly PROBE_TEST=no_evaluator_feature_admits_io - -# The probe invocation. Overridable so `tests/evaluator-io-check.bats` can drive -# both verdicts without a two-minute rebuild per case — the decision this task -# makes is the inversion, and that is what the suite has to exercise. Read-only, -# never set outside the suite. -if [[ -n "${EVALUATOR_IO_PROBE_CMD:-}" ]]; then - # shellcheck disable=SC2206 # deliberate word-splitting: the override is an argv - PROBE_CMD=($EVALUATOR_IO_PROBE_CMD) -else - PROBE_CMD=( - cargo test --quiet -p batten - --features probe-evaluator-io - --test policy_modules - "$PROBE_TEST" - -- --include-ignored - ) -fi - -if [[ -z "${EVALUATOR_IO_PROBE_CMD:-}" ]]; then - if ./mise-tasks/step-receipt.sh check evaluator-io-check; then exit 0; fi -fi - -log=$(mktemp) -trap 'rm -f "$log"' EXIT - -# THE INVERSION IS THE GATE. A probe build that SUCCEEDS means the test stayed -# green with `http` on — so it discriminates nothing and is coverage theatre, -# which is the exact shape CLOUD-418 exists to refuse. -# -# The verdict is read into a variable rather than branched on inline, matching -# `ran_and_failed` below: these are the gate's two decisions, and having both -# spelled the same way is what makes it obvious that neither is the exit code -# taken at face value. -probe_passed=no -if "${PROBE_CMD[@]}" >"$log" 2>&1; then - probe_passed=yes -fi - -if [[ "$probe_passed" = yes ]]; then - echo "::error:: evaluator-io-check: \`no_evaluator_feature_admits_io\` PASSED with regorus's \`http\` feature enabled, so it does not discriminate — a test that is green whether or not the evaluator can reach the network is not evidence that it cannot." >&2 - echo " crates/batten/tests/policy_modules.rs no_evaluator_feature_admits_io" >&2 - echo "Either the assertion has been reworded into something the probe cannot falsify, or the fixture no longer reaches \`http.send\`. Fix the test, not this gate." >&2 - exit 2 -fi - -# A NON-ZERO EXIT IS NOT YET THE ANSWER, and this is the arm a gate written to -# the obvious shape gets wrong. `cargo test` exits non-zero for a compile error, -# an unresolved feature, an absent toolchain and a panic in another test — every -# one of which would read as "the probe falsified the assertion" and hand this -# gate a pass it did not earn. Worse, it is a pass that gets MORE likely as the -# crate breaks, so the gate would be loudest exactly when it was lying. -# -# So the verdict is the test harness's own line, not the exit code: the named -# test must have RUN and FAILED. Anything else is could-not-look and exits 1, -# which is the config-or-usage class rather than a policy verdict (§7). -# The `failures:` LISTING rather than the per-test line, because the per-test -# line is not stable across harness modes: plain prints `test ... FAILED` -# and `--quiet` prints ` --- FAILED`. The listing under `failures:` is one -# indented name in both, and anchoring on it is what stops this arm going quietly -# could-not-look the day someone adds or drops `--quiet`. -ran_and_failed=no -if grep -q '^test result: FAILED' "$log" && grep -qx "[[:space:]]\{1,\}${PROBE_TEST}" "$log"; then - ran_and_failed=yes -fi -if [[ "$ran_and_failed" != yes ]]; then - echo "::error:: evaluator-io-check: the probe build did not run \`no_evaluator_feature_admits_io\` to a failure — it exited non-zero for some other reason (a compile error, an unresolved feature, an absent toolchain). That is could-not-look, and reading it as the discrimination this gate is looking for would make the gate pass more reliably the more broken the crate got." >&2 - echo " crates/batten/tests/policy_modules.rs no_evaluator_feature_admits_io" >&2 - echo "Run it yourself to see the real error: cargo test -p batten --features probe-evaluator-io --test policy_modules no_evaluator_feature_admits_io -- --include-ignored" >&2 - exit 1 -fi - -# It failed, which is the pass. Pointer-only: the test name and the verdict, -# never the probe build's output — that log carries module bodies and paths, and -# rule 4 admits neither into a gate's stdout. -if [[ -z "${EVALUATOR_IO_PROBE_CMD:-}" ]]; then - ./mise-tasks/step-receipt.sh record evaluator-io-check || true -fi -echo "evaluator-io-check: no_evaluator_feature_admits_io goes red under --features probe-evaluator-io, so it discriminates" diff --git a/mise-tasks/probe_verdict.py b/mise-tasks/probe_verdict.py new file mode 100644 index 000000000..2dea4d166 --- /dev/null +++ b/mise-tasks/probe_verdict.py @@ -0,0 +1,68 @@ +# Which of three things a probe build did, from its exit status and its log +# (CLOUD-418, ported off `mise-tasks/evaluator-io-check.sh` under CLOUD-1717). +# +# THE VERDICT IS THE HARNESS'S OWN LINE, NEVER THE EXIT CODE ALONE, and that is +# the whole reason this reading is a file rather than three lines in a task body. +# `cargo test` exits non-zero for a compile error, an unresolved feature, an +# absent toolchain and a panic in another test — every one of which would read as +# "the probe falsified the assertion" and hand the gate a pass it did not earn. +# Worse, that pass gets MORE likely as the crate breaks, so the gate would be +# loudest exactly when it was lying. +# +# ONE READING, TWO CALLERS. `[tasks.evaluator-io-record]` calls this to write the +# record; `crates/batten/tests/it/evaluator_io_probe.rs` calls it to assert all +# four arms without a two-minute rebuild per case. A copy of these three branches +# inside the tier would be a second authority over the classification — the drift +# `cargo_graph.py` was extracted to remove one gate over. +# +# THE `failures:` LISTING RATHER THAN THE PER-TEST LINE, because the per-test line +# is not stable across harness modes: plain prints `test ... FAILED` and +# `--quiet` prints ` --- FAILED`. The listing is one indented name in both, +# and anchoring on it is what stops this going quietly could-not-look the day +# someone adds or drops `--quiet`. +# +# Emits exactly one of: +# +# probe passed the build succeeded — the test stayed green with `http` on +# probe failed the named test RAN and FAILED — the discrimination wanted +# probe unread non-zero for some other reason — could not look +# +# POINTER-ONLY (rule 4): one token. The log carries module bodies and paths and +# no byte of it is emitted. + +import re +import sys + + +def verdict(status, log, test): + if status == 0: + # A probe build that SUCCEEDED means the test stayed green with `http` + # on, so it discriminates nothing. + return "probe passed" + listed = re.compile(r"^[ \t]+%s[ \t]*$" % re.escape(test), re.MULTILINE) + if re.search(r"^test result: FAILED", log, re.MULTILINE) and listed.search(log): + return "probe failed" + return "probe unread" + + +def main(argv): + if len(argv) != 4: + sys.stderr.write("usage: probe_verdict.py \n") + return 2 + try: + status = int(argv[1]) + except ValueError: + sys.stderr.write("probe-verdict: status must be a whole number\n") + return 2 + try: + with open(argv[2], "r", encoding="utf-8", errors="replace") as handle: + log = handle.read() + except OSError as failed: + sys.stderr.write("probe-verdict: could not read the log: %s\n" % failed) + return 2 + sys.stdout.write(verdict(status, log, argv[3]) + "\n") + return 0 + + +if __name__ == "__main__": + sys.exit(main(sys.argv)) diff --git a/mise.toml b/mise.toml index 6565ca067..57568a7fe 100644 --- a/mise.toml +++ b/mise.toml @@ -617,7 +617,7 @@ CI_FANIN_WORKFLOW = ".github/workflows/ci.yml" BATS_TEST_TIMEOUT = "300" REGORUS_OPA_COMPLIANCE = "1.2.0" REGORUS_OPA_COMPLIANCE_FOR = "0.11" -MUTANT_GATES = "mise,attestation,timeout-drift,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure,evaluator-io-check,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-callable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" +MUTANT_GATES = "mise,attestation,timeout-drift,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure,evaluator-io-probe,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-callable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" # --- GitHub reachability behind an egress proxy (Claude Code web sandbox etc.) --- # mise resolves every tool's release through GitHub's *API* host, api.github.com. @@ -2125,6 +2125,62 @@ emit | cargo run --quiet -p batten -- record named timeout-drift # The census WRITES NOTHING when the root does not exist — the question could not # be asked — which the module reads as silence. A root that was walked and held # no transcripts records `sessions 0`, which is an answer. +# CLOUD-418's obligation made runnable, ported off +# `mise-tasks/evaluator-io-check.sh` under CLOUD-1717. The one thing that is not +# a decision: building the test with `probe-evaluator-io` on and reading what the +# harness said. +# +# THE VERDICT IS THE HARNESS'S OWN LINE, NEVER THE EXIT CODE. `cargo test` exits +# non-zero for a compile error, an unresolved feature, an absent toolchain and a +# panic in another test — every one of which would read as "the probe falsified +# the assertion" and hand the gate a pass it did not earn. Worse, that pass gets +# MORE likely as the crate breaks, so the gate would be loudest exactly when it +# was lying. +# +# The `failures:` LISTING rather than the per-test line, because the per-test +# line is not stable across harness modes: plain prints `test ... FAILED` +# and `--quiet` prints ` --- FAILED`. The listing is one indented name in +# both, and anchoring on it is what stops this going quietly could-not-look the +# day someone adds or drops `--quiet`. +# +# `EVALUATOR_IO_PROBE_CMD` overrides the invocation so the tier can drive all +# three verdicts without a two-minute rebuild per case. Read-only; never set +# outside `crates/batten/tests/it/evaluator_io_probe.rs`. +# +# Receipt-gated (CLOUD-424) on the real path only: a feature toggle forces a +# fresh compile of the crate and its test binary, the one case where the ~0.3s a +# check/record pair costs is obviously dwarfed. +# +# POINTER-ONLY: one token. The probe log carries module bodies and paths, and no +# byte of it reaches the record. +[tasks.evaluator-io-record] +description = "Effect: record whether the IO-free evaluator test goes red under `probe-evaluator-io`, for `test judge never`" +shell = "bash -c" +run = """ +probe_test=no_evaluator_feature_admits_io +if [ -n "${EVALUATOR_IO_PROBE_CMD:-}" ]; then + # shellcheck disable=SC2086 # deliberate word-splitting: the override is an argv + set -- $EVALUATOR_IO_PROBE_CMD +else + if ./mise-tasks/step-receipt.sh check evaluator-io-check; then exit 0; fi + set -- cargo test --quiet -p batten --features probe-evaluator-io \ + --test policy_modules "$probe_test" -- --include-ignored +fi + +log=$(mktemp) +trap 'rm -f "$log"' EXIT + +status=0 +"$@" >"$log" 2>&1 || status=$? +verdict=$(python3 mise-tasks/probe_verdict.py "$status" "$log" "$probe_test") || exit 0 + +printf '%s\\n' "$verdict" | cargo run --quiet -p batten -- record named evaluator-io-probe + +if [ -z "${EVALUATOR_IO_PROBE_CMD:-}" ] && [ "$verdict" = 'probe failed' ]; then + ./mise-tasks/step-receipt.sh record evaluator-io-check || true +fi +""" + [tasks.transcript-corpus-record] description = "Effect: count this host's independent session transcripts against a threshold, for `corpus count thin`" shell = "bash -c" diff --git a/policy/evaluator-io-probe.rego b/policy/evaluator-io-probe.rego new file mode 100644 index 000000000..f7750185d --- /dev/null +++ b/policy/evaluator-io-probe.rego @@ -0,0 +1,108 @@ +# The IO-free evaluator test is shown able to FAIL (CLOUD-831, CLOUD-418, ported +# off `mise-tasks/evaluator-io-check.sh` under CLOUD-1717). +# +# WHAT IT DEFENDS. `no_evaluator_feature_admits_io` asserts a policy module +# cannot reach `http.send`. Under the shipped feature set that is true, so the +# test is green — and A GREEN TEST PROVES NOTHING ABOUT WHETHER IT CAN +# DISCRIMINATE. If regorus silently stopped registering the builtin, or the +# fixture stopped compiling, or the assertion were reworded into a tautology, the +# test would stay green over an evaluator that had lost the property entirely. +# +# So the producer builds the same test with `probe-evaluator-io` on — which turns +# `regorus/http` on and nothing else — and this module REQUIRES IT TO HAVE +# FAILED. A probe build that passed is the finding. +# +# THE PROBE FEATURE COSTS THE CLOSURE NOTHING, which is what makes running it +# affordable and what stops it weakening the gate it defends. `regorus`'s `http` +# feature is `[]` — a bare feature gating only the builtin's registration — so +# `Cargo.lock` is byte-identical with it on or off, and +# `policy/evaluator-closure.rego`'s walk cannot see it. +# +# THREE STATES, AND THE THIRD IS THE ONE A GATE WRITTEN TO THE OBVIOUS SHAPE GETS +# WRONG. `cargo test` exits non-zero for a compile error, an unresolved feature, +# an absent toolchain and a panic in another test — every one of which would read +# as "the probe falsified the assertion" and hand this gate a pass it did not +# earn. Worse, it is a pass that gets MORE likely as the crate breaks, so the +# gate would be loudest exactly when it was lying. So the producer reads the +# harness's own `failures:` listing rather than the exit code, and records which +# of the three happened; this module refuses two of them and is silent on one. +# +# POINTER-ONLY (rule 4): the test name and the verdict, never the probe build's +# output. That log carries module bodies and paths, and the record never holds a +# byte of it. +# +#MUTANT-SUITE crates/batten/tests/it/evaluator_io_probe.rs +#MUTANT io-probe-not-inverted|s@^\t"probe passed" in lines$@\tfalse@|a_probe_build_in_which_the_test_passes_is_the_finding_not_a_pass +#MUTANT io-probe-trusts-the-exit-code|s@^\t"probe unread" in lines$@\tfalse@|a_probe_build_that_failed_to_compile_is_could_not_look_not_the_pass + +# METADATA +# description: | +# Bound to the TREE surface: `scope = "tree"`, so it reads the tree document +# and never the mediated `{call, facts}` shape. +# THE BRACKETS ARE NOT STYLE: the schema file carries a hyphen, so the dotted +# form is a parse error reported as `invalid schema reference`. +# THIS BLOCK IS YAML AND MUST STAY THE LAST COMMENT BLOCK BEFORE `package`. +# schemas: +# - input: schema["policy-input.schema"] +package batten.evaluator_io_probe + +import rego.v1 + +rules contains "test cover never" + +# The record, or nothing. An absent record is "the producer did not run" and is +# silence; it must not be spelled the same way as a probe that ran. +lines := input.tree.records["evaluator-io-probe"] + +# THE INVERSION IS THE GATE. A probe build that SUCCEEDED means the test stayed +# green with `http` on — so it discriminates nothing and is coverage theatre, +# which is the exact shape CLOUD-418 exists to refuse. +violation contains { + "rule": "test cover never", + "verdict": "test judge never", + "subjects": [{"path": "crates/batten/tests/policy_modules.rs"}, {"artifact": "no_evaluator_feature_admits_io"}], +} if { + "probe passed" in lines +} + +# A NON-ZERO EXIT IS NOT YET THE ANSWER. The producer says so explicitly rather +# than leaving the module to infer it from an absent `probe failed`, because +# inferring would make "the producer never ran" and "the build broke" one state. +violation contains { + "rule": "test cover never", + "verdict": "test run unread", + "subjects": [{"path": "crates/batten/tests/policy_modules.rs"}, {"artifact": "no_evaluator_feature_admits_io"}], +} if { + "probe unread" in lines +} + +# --- cases ------------------------------------------------------------------- + +test_a_probe_build_in_which_the_test_passes_is_the_finding_not_a_pass if { + found := violation with input as {"tree": {"records": {"evaluator-io-probe": ["probe passed"]}}} + count(found) == 1 +} + +test_a_probe_build_in_which_the_test_fails_is_the_pass if { + found := violation with input as {"tree": {"records": {"evaluator-io-probe": ["probe failed"]}}} + count(found) == 0 +} + +test_a_probe_build_that_failed_to_compile_is_could_not_look_not_the_pass if { + found := violation with input as {"tree": {"records": {"evaluator-io-probe": ["probe unread"]}}} + count(found) == 1 +} + +# The two refusals are distinct classes: one says the test is theatre, the other +# says nothing was learned. A case counting findings cannot tell them apart. +test_the_two_refusals_are_separate_classes if { + passed := violation with input as {"tree": {"records": {"evaluator-io-probe": ["probe passed"]}}} + unread := violation with input as {"tree": {"records": {"evaluator-io-probe": ["probe unread"]}}} + {entry.verdict | some entry in passed} == {"test judge never"} + {entry.verdict | some entry in unread} == {"test run unread"} +} + +test_an_absent_record_says_nothing_rather_than_refusing if { + found := violation with input as {"tree": {"records": {}}} + count(found) == 0 +} diff --git a/tests/evaluator-io-check.bats b/tests/evaluator-io-check.bats deleted file mode 100644 index 3bf30722b..000000000 --- a/tests/evaluator-io-check.bats +++ /dev/null @@ -1,60 +0,0 @@ -#!/usr/bin/env bats -# subject: mise-tasks/evaluator-io-check.sh -# The decision `evaluator-io-check` makes is ONE THING — the inversion — and this -# suite is over that decision rather than over a two-minute rebuild. -# -# `EVALUATOR_IO_PROBE_CMD` stands in for the probe build, which is what lets both -# verdicts be exercised in milliseconds. The real build is what the task runs in -# the gate; what a suite has to prove is that the task reads its result the right -# way round, and that is the half a rebuild would tell you nothing extra about. - -setup() { - GATE="$BATS_TEST_DIRNAME/../mise-tasks/evaluator-io-check.sh" -} - -# THE LOAD-BEARING CASE. A probe build in which the test PASSES means the test -# stayed green with `http` on — it discriminates nothing. Reading that as success -# is the single mistake this gate exists to not make, and it is the mistake a -# gate written without thinking about the inversion makes by default. -@test "a probe build in which the test PASSES is the finding, not a pass" { - EVALUATOR_IO_PROBE_CMD="true" run "$GATE" - [ "$status" -eq 2 ] - [[ "$output" == *"does not discriminate"* ]] - [[ "$output" == *"no_evaluator_feature_admits_io"* ]] -} - -@test "a probe build in which the test FAILS is the pass" { - EVALUATOR_IO_PROBE_CMD="$BATS_TEST_DIRNAME/fixtures/evaluator-io/failing-probe" run "$GATE" - [ "$status" -eq 0 ] - [[ "$output" == *"goes red under"* ]] -} - -# THE OTHER LOAD-BEARING CASE, and the one a gate written to the obvious shape -# gets wrong. `cargo test` exits non-zero for a compile error too — and reading -# THAT as "the probe falsified the assertion" gives the gate a pass it did not -# earn, one that gets more likely the more broken the crate is. Could-not-look -# is exit 1, never the verdict. -@test "a probe build that failed to COMPILE is could-not-look, not the pass" { - EVALUATOR_IO_PROBE_CMD="$BATS_TEST_DIRNAME/fixtures/evaluator-io/broken-build" run "$GATE" - [ "$status" -eq 1 ] - [[ "$output" == *"did not run"* ]] - [[ "$output" != *"goes red under"* ]] -} - -# The same distinction from the other side: a probe run where some OTHER test -# panicked and the named one never ran is not this gate's evidence either. -@test "a probe build where the named test never ran is could-not-look" { - EVALUATOR_IO_PROBE_CMD="$BATS_TEST_DIRNAME/fixtures/evaluator-io/other-test-failed" run "$GATE" - [ "$status" -eq 1 ] - [[ "$output" == *"did not run"* ]] -} - -# Rule 4. The probe build's output carries module bodies, file paths and a -# backtrace; none of it may reach this gate's stdout. The stub prints a line that -# would be unmistakable if it leaked. -@test "the probe build's own output never reaches the gate's output" { - EVALUATOR_IO_PROBE_CMD="$BATS_TEST_DIRNAME/fixtures/evaluator-io/noisy-probe" run "$GATE" - [ "$status" -eq 0 ] - [[ "$output" != *"deny contains"* ]] - [[ "$output" != *"POLICY-BODY-LEAKED"* ]] -} From c66cdd5e41112b7dac86d08bfe5b947f517d2669 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sat, 19 Sep 2026 02:46:46 +0000 Subject: [PATCH 20/32] feat(policy): the signing posture decides in the engine, and its write stays a write MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `mise-tasks/signing-posture.sh` and its suite are gone. The two refusals are `policy/signing-posture.rego`; the git reads and the range scan are `[tasks.signing-posture-record]`; the repair is `[tasks.signing-posture-repair]`, because a module cannot write. ## Signing is good and this is not against it A version of this gate that reads that way is wrong. Signing in CI, with a key whose public half is published, is the desired end state and CLOUD-591 owns getting there. What it refuses is the narrower thing: a signature produced by a key that cannot be verified or reproduced, which is **worse than no signature** because it looks like provenance and carries none. It is the stand-in for a blind spot the attribution gate structurally has. `Attribution` carries `identity_deny`, `trailer_deny`, `body_deny`, `trailer_allow` and `identity` with no signature field — so `gpgsig`, the one commit field that was carrying a vendor-held key, is invisible to it until CLOUD-440 lets the engine read a commit object. ## Two classes, and keeping them apart is the point `config carry unsafe` is a posture that will produce bad signatures; `commit carry unsafe` is one that already did. Config can be repaired AFTER a commit was written, so a repaired checkout still carries what it signed before the repair — and those are exactly what must not reach `main`. Collapsing them would let the cheap half stand in for the expensive one, which is this gate's declared mutation. The config class is the CONFLICT, never the mere absence of a local override: a runner has no launcher and no global setting, so an absent local value is correct there and demanding one would red every CI run. ## The signer classification is a file because three callers share it Seven of the dying suite's twenty-one cases are about exactly which configurations are unverifiable — an empty key, a directory, an unreadable file, a path that does not exist, an inline literal, a `/tmp` signer, and a healthy one. One of those was a measured defect rather than thoroughness: `-s` alone is true for a DIRECTORY, which leaves the public half just as unreadable, so there are four file tests and each names its own reason. `mise-tasks/signer_posture.py` is the one authority, read by the record task, the repair task and `crates/batten/tests/it/signing_posture.rs`. Fifteen cases, all passing. ## Six changed arms, and the reason is one sentence The `--repair` half is the WRITE. Four cases about it — that it leaves a verifiable signer alone, writes local only, is idempotent, and never touches global — are `// changed:` onto `mise.toml`, because a module cannot write and the task is where that boundary is stated. Two more are the producer's: the range is `origin/main..HEAD` and could-not-look is "outside a git repository, write nothing", which the module reads as silence. ## Proven live, not merely green `mise run batten -- check --rule 'commit grade unsafe'` over a recorded broken signer and one signed commit: `deadbeef commit grade unsafe`, exit 2. Exit 0 alone is byte-identical to a dead gate (CLOUD-1810). ## Corpus 74 -> 73 programs, 85 -> 84 suites. 12 of 44 retired. Refs: CLOUD-1717 Refs: CLOUD-669 Refs: CLOUD-591 Admits: 1782a2c3d268f8c01080d24aa4fe03011ef1f6d2fdd312c34ae863191acea65f Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: batten.toml Admits-anchor: call:9eb1ed98a4b7d941dd965f593f3e132847df1761 Admits-epoch: aa667b6a30b229a38dd697d65b4d4e58e370b7c975b0757a03ca651f8a29d836 Admits-author: alec@wenzowski.com Admits-prev: 4bf96f29efc68ea7e7ddad1a86ab98ba058b6be0407aaf3c0bb829d6cea44b15 Admits-answer-lost: The retirement cannot land and the tree will not load. This gate is the stand-in for a blind spot the attribution gate structurally has: `Attribution` carries identity_deny, trailer_deny, body_deny, trailer_allow and identity with NO signature field, so `gpgsig` — the one commit field that was carrying a vendor-held key — is invisible to it until CLOUD-440 lets the engine read a commit object. Without this row a commit signed by a key nobody can verify or reproduce reaches main with nothing to say so, which is the state CLOUD-669 was filed over and which CLOUD-591 recorded as a posture with no mechanism behind it. Admits-answer-precondition: batten.toml is the only surface carrying [[rule]], [[verdict]] and [[record]] rows. CLOUD-1717 retires mise-tasks/signing-posture.sh onto policy/signing-posture.rego plus [tasks.signing-posture-record] and [tasks.signing-posture-repair]; the module raises `config carry unsafe` and `commit carry unsafe` and reads the `signing-posture` record family, and the engine refuses at load both a module raising an undeclared token and a declared row nothing raises. No other file can carry any of it. It lands in PR #962 as an ordinary diff hunk beside the four retirements already admitted on this branch. Admits-answer-rejected-route: `config read first` is a read of batten.toml; I read it to confirm `commit grade unsafe`, `config carry unsafe` and `commit carry unsafe` are all unused, and to give the rule a third name distinct from both its verdicts — the engine refuses a rule spelled as a class token when it raises two, which cost an admission to learn one retirement ago. Reading cannot declare the rows. `patch run first` is `git restore`, which would discard the module, both tasks and the shared signer classification rather than complete them. --- batten.toml | 52 +++ bench/suites/RESULTS.md | 5 +- crates/batten/tests/it/main.rs | 1 + crates/batten/tests/it/signing_posture.rs | 365 ++++++++++++++++++++++ mise-tasks/signer_posture.py | 89 ++++++ mise-tasks/signing-posture.sh | 243 -------------- mise.toml | 98 +++++- policy/signing-posture.rego | 169 ++++++++++ tests/signing-posture.bats | 282 ----------------- 9 files changed, 775 insertions(+), 529 deletions(-) create mode 100644 crates/batten/tests/it/signing_posture.rs create mode 100644 mise-tasks/signer_posture.py delete mode 100755 mise-tasks/signing-posture.sh create mode 100644 policy/signing-posture.rego delete mode 100644 tests/signing-posture.bats diff --git a/batten.toml b/batten.toml index 46c4fb651..97aa8acc6 100644 --- a/batten.toml +++ b/batten.toml @@ -6019,6 +6019,58 @@ no_fix_reason = "a test that no longer discriminates is fixed in the test, not h record = "evaluator-io-probe" writer = "mise run evaluator-io-record" +# CLOUD-669, ported under CLOUD-1717. The stand-in for a blind spot the +# attribution gate structurally has: `Attribution` carries `identity_deny`, +# `trailer_deny`, `body_deny`, `trailer_allow` and `identity` with NO signature +# field, so `gpgsig` — the one commit field that was carrying a vendor-held key — +# is invisible to it until CLOUD-440 lets the engine read a commit object. +[[rule]] +id = "commit grade unsafe" +kind = "policy" +scope = "tree" +module = "policy/signing-posture.rego" +severity = "deny" +no_fix_reason = "a commit already signed cannot be unsigned in place: repair the checkout's config (`mise run signing-posture-repair`) and rewrite the range, or publish the key's public half so the signature becomes verifiable (CLOUD-591)" + +[[record]] +record = "signing-posture" +writer = "mise run signing-posture-record" + +[[verdict]] +id = "config carry unsafe" +gloss = "signing is on in this checkout with a signer whose key cannot be verified or reproduced" +class = """ +NOT A VERDICT ABOUT SIGNING, which is good and which CLOUD-591 is working \ +toward. It is about a signature that LOOKS like provenance and carries none: a \ +key whose public half cannot be read, or a signer under `/tmp` that the container \ +reclaims. The refusal is the CONFLICT — something outside the checkout turns \ +signing on and nothing local answers it — never the mere absence of a local \ +override, because a CI runner has no launcher and an absent local value is \ +correct there. +""" + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run signing-posture-repair" + +[[verdict]] +id = "commit carry unsafe" +gloss = "a commit in range carries a gpgsig from a key this repository cannot verify or reproduce" +class = """ +The posture already produced one. Config can be repaired AFTER a commit was \ +written, so a repaired checkout still carries what it signed before the repair — \ +and those are exactly what must not reach `main`. That is why this is a separate \ +class from the config one rather than the same finding twice: repairing the \ +config clears that arm and leaves this one firing, which is the distinction the \ +declared mutation exists to hold. +""" + +[[verdict.route]] +id = "module read first" +kind = "document" +target = "policy/signing-posture.rego" + [[verdict]] id = "test judge never" gloss = "the IO-free evaluator test stayed green with regorus's http feature on, so it discriminates nothing" diff --git a/bench/suites/RESULTS.md b/bench/suites/RESULTS.md index 57e2f9130..bbb14e73a 100644 --- a/bench/suites/RESULTS.md +++ b/bench/suites/RESULTS.md @@ -6,8 +6,8 @@ runner measured it; the suite runs `--no-parallelize-within-files`, so a file's number is its own serial cost and is what an author adding a case to it pays. -- suites: 95 -- serial total: 157.4s +- suites: 94 +- serial total: 156.0s | seconds | share | suite | | ---: | ---: | --- | @@ -37,7 +37,6 @@ to it pays. | 1.7 | 1.0% | `tests/finding-sink-check.bats` | | 1.6 | 1.0% | `tests/suite-select.bats` | | 1.5 | 0.9% | `tests/spec-ref-check.bats` | -| 1.4 | 0.9% | `tests/signing-posture.bats` | | 1.2 | 0.8% | `tests/tree-clean.bats` | | 1.2 | 0.8% | `tests/ci-slow-needed.bats` | | 1.2 | 0.7% | `tests/ready-lint-deferral.bats` | diff --git a/crates/batten/tests/it/main.rs b/crates/batten/tests/it/main.rs index d865f9ece..c338d27d0 100644 --- a/crates/batten/tests/it/main.rs +++ b/crates/batten/tests/it/main.rs @@ -262,6 +262,7 @@ mod session_provisioning; mod shell_retirement; mod shell_retirement_cost; mod shell_write_advisory; +mod signing_posture; mod singleton; mod singleton_gate; mod sinks; diff --git a/crates/batten/tests/it/signing_posture.rs b/crates/batten/tests/it/signing_posture.rs new file mode 100644 index 000000000..90888ba7d --- /dev/null +++ b/crates/batten/tests/it/signing_posture.rs @@ -0,0 +1,365 @@ +//! `commit grade unsafe` over the compiled binary and the real classification +//! (CLOUD-669, CLOUD-591, CLOUD-1717). +//! +//! # Why this tier exists and the module's own `test_` rules do not suffice +//! +//! `policy/signing-posture.rego` carries eight load-time cases and every one +//! fabricates its input with `with input as`, which is the shape +//! `rules/policy-modules.md` warns about. +//! +//! # And why the SIGNER CLASSIFICATION is driven here +//! +//! Seven of the dying suite's twenty-one cases are about exactly which +//! configurations are unverifiable: an empty key, a directory, an unreadable +//! file, a path that does not exist, an inline literal, a `/tmp` signer, and a +//! healthy one. That is the substance — the module's half is two set +//! memberships — and one of them (`-s` alone being true for a directory) was a +//! measured defect rather than a hypothetical. +//! +//! `mise-tasks/signer_posture.py` is the one authority on those branches, shared +//! with `[tasks.signing-posture-record]` and `[tasks.signing-posture-repair]`. +//! +//! # RETIREMENT LEDGER, PER PATH — what `shell retire partial` reads +//! +//! Six cases are not carried and each says why in its own row. +//! +// carried: mise-tasks/signing-posture.sh policy/signing-posture.rego kind:mechanism crates/batten/tests/it/signing_posture.rs +// carried: tests/signing-posture.bats policy/signing-posture.rego kind:mechanism crates/batten/tests/it/signing_posture.rs +// carried: "an unsigned range with the override in place passes" policy/signing-posture.rego kind:mechanism +// carried: "signing with a verifiable signer is left alone" policy/signing-posture.rego kind:mechanism +// carried: "a commit signed by a VERIFIABLE signer is left alone, header and all" policy/signing-posture.rego kind:mechanism +// carried: "an empty signing key is what makes it unverifiable" policy/signing-posture.rego kind:mechanism +// carried: "a signing key that is a directory is unverifiable" policy/signing-posture.rego kind:mechanism +// carried: "a signing key this checkout cannot read is unverifiable" policy/signing-posture.rego kind:mechanism +// carried: "a signing key naming a path that does not exist is unverifiable" policy/signing-posture.rego kind:mechanism +// carried: "an inline public key is a literal, not a path, and is verifiable" policy/signing-posture.rego kind:mechanism +// carried: "a signer under /tmp is unverifiable because the container reclaims it" policy/signing-posture.rego kind:mechanism +// carried: "a signed commit in range is refused, and named by short sha" policy/signing-posture.rego kind:mechanism +// carried: "repairing the config does not excuse a commit already signed" policy/signing-posture.rego kind:mechanism +// carried: "a missing override is refused when the environment sets signing globally" policy/signing-posture.rego kind:mechanism +// carried: "a missing override is NOT a finding when nothing sets signing globally" policy/signing-posture.rego kind:mechanism +// carried: "a local override set to true is refused when the signer is broken" policy/signing-posture.rego kind:mechanism +// carried: "the refusal echoes no part of the signature block" policy/signing-posture.rego kind:mechanism +// changed: "--repair leaves a verifiable signer alone rather than switching signing off" mise.toml the write is the WRITE, which a module cannot be, so it stayed a task — `[tasks.signing-posture-repair]` — and its guard is the same shared `signer_posture.py` reading this tier drives. `an_inline_public_key_is_a_literal_not_a_path_and_is_verifiable` pins the branch the guard turns on +// changed: "--repair writes the override, local only" mise.toml the same split: the write and its scope are the task's, and `git config --local` is the one line that states it +// changed: "--repair is idempotent" mise.toml idempotence is a property of `git config --local commit.gpgsign false`, which is the task's single write +// changed: "--repair never writes global config" mise.toml the same boundary `attribution-identity` draws, and it is stated where the write is — a developer's own unrelated repositories are not this repo's business +// changed: "history before the range is never judged" mise.toml the range is the PRODUCER's — `origin/main..HEAD` by default, the range `commit-attribution` and `commit-lint` already share. The module reads whatever the producer recorded and cannot observe which commits were outside it +// changed: "outside a git repository it is exit 2, never a silent pass" mise.toml could-not-look is the producer's: it writes NOTHING outside a git repository, and `an_absent_record_says_nothing_rather_than_refusing` is the module's half of that contract + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use common::{git_in, init_repo, run, run_with_stdin, scratch, write}; + +/// A repository registering the real module against the declared family. +fn repo(name: &str) -> std::path::PathBuf { + let dir = scratch(&format!("signing-posture-{name}")); + let module = std::fs::read_to_string("../../policy/signing-posture.rego") + .expect("the module this tier exists for"); + write(&dir, "policy/signing-posture.rego", &module); + write( + &dir, + "batten.toml", + r#"version = 1 +scope = ["**"] + +[[verdict]] +id = "config carry unsafe" +gloss = "signing is on with a signer whose key cannot be verified or reproduced" +class = "A signature that looks like provenance and carries none." + +[[verdict.route]] +id = "task run first" +kind = "command" +target = "mise run signing-posture-repair" + +[[verdict]] +id = "commit carry unsafe" +gloss = "a commit in range carries a gpgsig from a key this repository cannot verify" +class = "The posture already produced one, and repairing the config does not unsign it." + +[[verdict.route]] +id = "module read first" +kind = "document" +target = "policy/signing-posture.rego" + +[[rule]] +id = "commit grade unsafe" +kind = "policy" +scope = "tree" +module = "policy/signing-posture.rego" +severity = "deny" + +[[record]] +record = "signing-posture" +writer = "mise run signing-posture-record" +"#, + ); + init_repo(&dir); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-qm", "register the module"]); + dir +} + +fn record(dir: &std::path::Path, lines: &str) { + let written = run_with_stdin(dir, &["record", "named", "signing-posture"], lines); + assert!( + written.status.success(), + "the setup write lands: {}", + String::from_utf8_lossy(&written.stderr) + ); +} + +fn said(decided: &std::process::Output) -> String { + format!( + "{}{}", + String::from_utf8_lossy(&decided.stdout), + String::from_utf8_lossy(&decided.stderr) + ) +} + +const BROKEN: &str = "signer broken user.signingkey names an empty file, so the public half cannot be read or published"; + +// --- the decision, over the engine's own projection -------------------------- + +#[test] +fn a_signed_commit_in_range_is_refused_and_named_by_short_sha() { + let dir = repo("signed"); + record(&dir, &format!("{BROKEN}\nsigned 1a2b3c4d\n")); + + let decided = run(&dir, &["check"]); + assert_eq!( + decided.status.code(), + Some(2), + "a signed commit decides\n{}", + String::from_utf8_lossy(&decided.stderr) + ); + assert!( + said(&decided).contains("1a2b3c4d"), + "and is named by short sha\n{}", + said(&decided) + ); +} + +#[test] +fn signing_with_a_verifiable_signer_is_left_alone() { + // THE END STATE CLOUD-591 IS WORKING TOWARD, and this gate must not block it. + let dir = repo("verifiable"); + record( + &dir, + "signer verifiable\nconfig conflict\nsigned 1a2b3c4d\n", + ); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "a verifiable signer may sign freely\n{}", + String::from_utf8_lossy(&quiet.stderr) + ); +} + +#[test] +fn a_missing_override_is_refused_when_the_environment_sets_signing_globally() { + let dir = repo("conflict"); + record(&dir, &format!("{BROKEN}\nconfig conflict\n")); + + let decided = run(&dir, &["check"]); + assert_eq!(decided.status.code(), Some(2), "the conflict decides"); +} + +#[test] +fn a_missing_override_is_not_a_finding_when_nothing_sets_signing_globally() { + // A runner has no launcher and no global setting, so an absent local value is + // the correct state there. Demanding the override unconditionally would red + // every CI run. + let dir = repo("runner"); + record(&dir, &format!("{BROKEN}\n")); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "nothing to override is not a finding\n{}", + String::from_utf8_lossy(&quiet.stderr) + ); +} + +#[test] +fn repairing_the_config_does_not_excuse_a_commit_already_signed() { + // The whole reason the two classes are separate: `--repair` clears the config + // arm and leaves this one firing on whatever was already written. + let dir = repo("repaired"); + record(&dir, &format!("{BROKEN}\nsigned 1a2b3c4d\n")); + + let decided = run(&dir, &["check"]); + assert_eq!(decided.status.code(), Some(2), "the commit still decides"); + assert!( + said(&decided).contains("1a2b3c4d"), + "and it is the commit that is named\n{}", + said(&decided) + ); +} + +#[test] +fn the_refusal_echoes_no_part_of_the_signature_block() { + // POINTER-ONLY (rule 4): a short SHA and a setting name. A signature block is + // a credential artefact this repository does not control, and the record + // never carries one — which is where that has to be true. + let dir = repo("quiet"); + record(&dir, &format!("{BROKEN}\nsigned 1a2b3c4d\n")); + + let decided = run(&dir, &["check"]); + let reported = said(&decided); + assert!( + !reported.contains("BEGIN SSH SIGNATURE"), + "no signature block reaches the finding\n{reported}" + ); + assert!( + !reported.contains("gpgsig "), + "and no header line does\n{reported}" + ); +} + +#[test] +fn an_unsigned_range_with_the_override_in_place_passes() { + let dir = repo("clean"); + record(&dir, &format!("{BROKEN}\n")); + + let quiet = run(&dir, &["check"]); + assert_eq!(quiet.status.code(), Some(0), "nothing to report"); +} + +#[test] +fn an_absent_record_says_nothing_rather_than_refusing() { + // The producer writes nothing outside a git repository, so a module refusing + // here would report a posture in force over a tree it never looked at. + let dir = repo("unrecorded"); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "an absent record is silence\n{}", + String::from_utf8_lossy(&quiet.stderr) + ); +} + +// --- the signer classification ----------------------------------------------- + +/// Run the REAL classification both tasks run. +#[expect( + clippy::disallowed_types, + reason = "CLOUD-320's inventory row, and the verdict is that this spawn IS the subject under test: which configurations are unverifiable is what moved out of the dying program into `mise-tasks/signer_posture.py`, and a harness re-implementing the four file tests in Rust would be a second authority over the one question both the record and the repair turn on" +)] +fn posture(signingkey: &str, program: &str) -> String { + let done = std::process::Command::new("python3") + .arg("../../mise-tasks/signer_posture.py") + .arg(signingkey) + .arg(program) + .output() + .expect("the classification runs"); + assert!( + done.status.success(), + "it completes: {}", + String::from_utf8_lossy(&done.stderr) + ); + String::from_utf8_lossy(&done.stdout).trim().to_owned() +} + +#[test] +fn an_empty_signing_key_is_what_makes_it_unverifiable() { + let dir = scratch("signer-empty"); + write(&dir, "key.pub", ""); + let said = posture(dir.join("key.pub").to_string_lossy().as_ref(), ""); + assert!(said.starts_with("broken"), "an empty key is broken\n{said}"); + assert!(said.contains("empty file"), "and says which test\n{said}"); +} + +#[test] +fn a_signing_key_that_is_a_directory_is_unverifiable() { + // `-s` ALONE WAS THE TEST HERE AND IT IS TRUE FOR A DIRECTORY, which leaves + // the public half unreadable — the condition being named. + let dir = scratch("signer-dir"); + write(&dir, "keydir/placeholder", ""); + let said = posture(dir.join("keydir").to_string_lossy().as_ref(), ""); + assert!( + said.contains("not a regular file"), + "a directory is broken, and says so\n{said}" + ); +} + +#[test] +fn a_signing_key_this_checkout_cannot_read_is_unverifiable() { + let dir = scratch("signer-unreadable"); + write(&dir, "key.pub", "ssh-ed25519 AAAA"); + let path = dir.join("key.pub"); + let mut perms = std::fs::metadata(&path).expect("it exists").permissions(); + { + use std::os::unix::fs::PermissionsExt as _; + perms.set_mode(0o000); + } + std::fs::set_permissions(&path, perms).expect("it is unreadable"); + + let said = posture(path.to_string_lossy().as_ref(), ""); + // Running as root defeats the permission bit, so this asserts the branch it + // can reach rather than claiming one it cannot. + assert!( + said.contains("cannot read") || said == "verifiable", + "an unreadable key is broken where the OS enforces it\n{said}" + ); +} + +#[test] +fn a_signing_key_naming_a_path_that_does_not_exist_is_unverifiable() { + let said = posture("/nowhere/at/all/key.pub", ""); + assert!( + said.contains("does not exist"), + "an absent path is broken, and says so\n{said}" + ); +} + +#[test] +fn an_inline_public_key_is_a_literal_not_a_path_and_is_verifiable() { + // A LITERAL IS THE MOST PUBLISHABLE FORM THERE IS — it is already the public + // half — so testing it as a file would report the healthiest possible + // configuration as broken. + assert_eq!( + posture("ssh-ed25519 AAAAC3NzaC1lZDI1NTE5", ""), + "verifiable" + ); + assert_eq!( + posture("key::ssh-ed25519 AAAAC3NzaC1lZDI1NTE5", ""), + "verifiable" + ); + assert_eq!(posture("sk-ssh-ed25519@openssh.com AAAA", ""), "verifiable"); +} + +#[test] +fn a_signer_under_tmp_is_unverifiable_because_the_container_reclaims_it() { + // And it outranks the key test: a reproducible key behind an irreproducible + // signer is still not re-verifiable later. + let said = posture("ssh-ed25519 AAAAC3NzaC1lZDI1NTE5", "/tmp/code-sign"); + assert!( + said.contains("/tmp"), + "a /tmp signer is broken whatever the key is\n{said}" + ); +} + +#[test] +fn a_signer_failing_neither_test_is_left_alone() { + // The anti-vacuity arm: without it every case above passes on a + // classification that returned `broken` unconditionally. + let dir = scratch("signer-healthy"); + write(&dir, "key.pub", "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5\n"); + assert_eq!( + posture( + dir.join("key.pub").to_string_lossy().as_ref(), + "/usr/bin/ssh-keygen" + ), + "verifiable" + ); +} diff --git a/mise-tasks/signer_posture.py b/mise-tasks/signer_posture.py new file mode 100644 index 000000000..9a546df37 --- /dev/null +++ b/mise-tasks/signer_posture.py @@ -0,0 +1,89 @@ +# Whether a git signing configuration names a key anyone can verify (CLOUD-669, +# ported off `mise-tasks/signing-posture.sh` under CLOUD-1717). +# +# SIGNING IS GOOD, and this is not against it. A version of this that reads that +# way is wrong: signing in CI, with a key whose public half is published, is the +# desired end state and CLOUD-591 owns getting there. What it names is the +# narrower thing — a signature produced by a key that cannot be verified or +# reproduced, which is WORSE than no signature because it looks like provenance +# and carries none. +# +# TWO INDEPENDENT CONDITIONS, either of which makes a signature unverifiable by +# anyone, us included. Both were measured rather than assumed (2026-08-18): +# +# * `gpg.ssh.program` resolves inside `/tmp`. The container reclaims it, so the +# signer and whatever key it holds are not reproducible across sessions. A +# signature nobody can re-verify later is provenance theatre. +# * `user.signingkey` names a file that is empty or unreadable. The public half +# cannot be read, so no `allowed_signers` entry can be derived from it and +# nothing downstream can check the signature. +# +# A signer failing NEITHER test is left alone and signing stays on. +# +# A LITERAL KEY IS NOT A PATH. With `gpg.format ssh`, git accepts the public key +# inline (`ssh-ed25519 AAAA…`) or via a `key::` prefix as well as a filename. A +# literal is the MOST publishable form there is — it is already the public half — +# so testing it as a file would report the healthiest possible configuration as +# broken. +# +# FOUR FILE TESTS, NOT ONE, and that was a real defect rather than thoroughness. +# `-s` alone is true for anything non-empty that `stat` can size, including an +# unreadable file and a DIRECTORY — both of which leave the public half +# unreadable, which is the condition being named. Each test carries its own +# reason so the refusal says which one. +# +# ONE READING, TWO CALLERS: `[tasks.signing-posture-record]` writes the record +# from it, and `crates/batten/tests/it/signing_posture.rs` drives all seven arms. +# A copy of these branches in the tier would be the second authority +# `cargo_graph.py` was extracted to remove one gate over. +# +# Emits one line: `verifiable`, or `broken `. + +import os +import sys + +# The spellings git accepts for a key given INLINE rather than as a path. +LITERAL_PREFIXES = ("ssh-", "key::", "sk-ssh-", "sk-ecdsa-") + + +def posture(signingkey, program): + if program.startswith("/tmp/"): + return ( + "broken the signer resolves inside /tmp, which the container " + "reclaims, so the key is not reproducible" + ) + if signingkey == "" or signingkey.startswith(LITERAL_PREFIXES): + return "verifiable" + if not os.path.exists(signingkey): + return ( + "broken user.signingkey names a path that does not exist, so the " + "public half cannot be read or published" + ) + if not os.path.isfile(signingkey): + return ( + "broken user.signingkey names something that is not a regular file, " + "so the public half cannot be read or published" + ) + if not os.access(signingkey, os.R_OK): + return ( + "broken user.signingkey names a file this checkout cannot read, so " + "the public half cannot be read or published" + ) + if os.path.getsize(signingkey) == 0: + return ( + "broken user.signingkey names an empty file, so the public half " + "cannot be read or published" + ) + return "verifiable" + + +def main(argv): + if len(argv) != 3: + sys.stderr.write("usage: signer_posture.py \n") + return 2 + sys.stdout.write(posture(argv[1], argv[2]) + "\n") + return 0 + + +if __name__ == "__main__": + sys.exit(main(sys.argv)) diff --git a/mise-tasks/signing-posture.sh b/mise-tasks/signing-posture.sh deleted file mode 100755 index 956cdd2cd..000000000 --- a/mise-tasks/signing-posture.sh +++ /dev/null @@ -1,243 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Gate (and, with --repair, the write): no commit is signed by a key that cannot be verified or reproduced (CLOUD-669)" -# -# SIGNING IS GOOD. This gate is not against it, and a version of it that reads -# that way is wrong — signing in CI, with a key whose public half is published, -# is the desired end state and CLOUD-591 owns getting there. What this refuses is -# the narrower thing: a signature produced by a key that cannot be verified or -# reproduced, which is worse than no signature because it LOOKS like provenance -# and carries none. -# -# CLOUD-591 recorded the interim posture and shipped no mechanism, so nothing was -# in force. Non-negotiable rule 2: a rule without a runnable gate is half a -# change. -# -# WHAT WAS ACTUALLY HAPPENING. The launcher writes the signing configuration -# `--global` every session, and nothing repo-local overrode it. Measured -# 2026-08-18, all four global and none local: -# -# commit.gpgsign true -# gpg.format ssh -# user.signingkey /home/claude/.ssh/commit_signing_key.pub <- 0 bytes -# gpg.ssh.program /tmp/code-sign -> /opt/env-runner/environment-manager -# -# Signatures were produced regardless of the empty key file, because -# `gpg.ssh.program` substitutes the harness's own signer for `ssh-keygen`. So the -# key never passes through the configured path, and it is the ENVIRONMENT's: this -# repository does not hold it, cannot publish it, and it need not survive a -# container. GitHub answers `verified: false, reason: unknown_key`. -# -# WHY THAT IS AN ATTRIBUTION DEFECT AND NOT A PREFERENCE. Every commit read: -# `author` and `committer` the accountable human — correct, and gated by -# `identity_deny` — and `gpgsig` a vendor-held key. CLOUD-268's position is that -# no vendor identity rides on the commit, and `Attribution` carries -# `identity_deny`, `trailer_deny`, `body_deny`, `trailer_allow` and `identity` -# with NO signature field. So the one commit field the attribution gate -# structurally cannot see is the one carrying a vendor identity. This gate is -# that blind spot's stand-in until CLOUD-440 lets the engine see a commit object. -# -# TWO MODES, ONE DEFINITION. The posture is a single fact, so the write and the -# check live in one file rather than drifting apart the way a paired task can. -# `--repair` is the write, self-declared per house style §5, scoped to this -# checkout's `.git/config` and NEVER `--global` — a developer's own unrelated -# repositories are not this repo's business, the same boundary -# `attribution-identity` draws. -# -# RANGE, NEVER HISTORY. The check judges `BASE_SHA..HEAD_SHA`, the range -# `commit-attribution` and `commit-lint` already share. Every commit on `main` -# predating this gate is signed by that environment key; judging history would -# make the gate permanently red for commits nobody can now unsign, which is how a -# gate gets switched off. -# -# Pointer-only per non-negotiable rule 4: a short SHA and a setting name. Never a -# signature block — it is a credential artefact this repo does not control. -# -# Exit 0 posture in force / 1 a signed commit in range, or the override missing / -# 2 could not look. -# -# The mutation drops the commit scan and keeps only the config check. Config is -# the cheap half and the one a session can satisfy after the damage: a checkout -# repaired late still carries the signed commits written before the repair, and -# those are exactly what must not reach `main`. -#MUTANT config-check-is-not-a-commit-check|s/for sha in \$range; do/for sha in ; do/|a signed commit in range is refused, and named by short sha -set -uo pipefail - -repair=0 -base="${BASE_SHA:-}" -head="${HEAD_SHA:-}" -while [[ $# -gt 0 ]]; do - case "$1" in - --repair) repair=1 ;; - --base) - base="${2:-}" - shift - ;; - --head) - head="${2:-}" - shift - ;; - *) - echo "usage: signing-posture [--repair] [--base ] [--head ]" >&2 - exit 2 - ;; - esac - shift -done - -if ! git rev-parse --git-dir >/dev/null 2>&1; then - echo "::error:: signing-posture: not a git repository — a gate that cannot look must not report the posture in force" >&2 - exit 2 -fi - -# WHAT "BROKEN" MEANS, measured rather than assumed. Two independent conditions, -# either of which makes a signature unverifiable by anyone, us included: -# -# * `user.signingkey` names a file that is empty or unreadable. The public half -# cannot be read, so no `allowed_signers` entry can be derived from it and -# nothing downstream can check the signature. Measured here: 0 bytes. -# * `gpg.ssh.program` resolves inside `/tmp`. The container reclaims it, so the -# signer and whatever key it holds are not reproducible across sessions. A -# signature nobody can re-verify later is provenance theatre. -# -# A signer failing NEITHER test is left alone and signing stays on. -broken_reason() { - key=$(git config --get user.signingkey 2>/dev/null || true) - prog=$(git config --get gpg.ssh.program 2>/dev/null || true) - case "$prog" in - /tmp/*) - printf 'the signer resolves inside /tmp, which the container reclaims, so the key is not reproducible' - return - ;; - esac - # A LITERAL KEY IS NOT A PATH. With `gpg.format ssh`, git accepts the public - # key inline (`ssh-ed25519 AAAA...`) or via a `key::` prefix as well as a - # filename. A literal is the MOST publishable form there is — it is already - # the public half — so testing it as a file would report the healthiest - # possible configuration as broken. - case "$key" in - '' | ssh-* | key::* | sk-ssh-* | sk-ecdsa-*) ;; - *) - # `-s` alone was the test here, and it is true for anything non-empty - # that `stat` can size — an unreadable file, or a DIRECTORY. Both leave - # the public half unreadable, which is the condition this is trying to - # name, so all three tests are needed and each names its own reason. - if [[ ! -e "$key" ]]; then - printf 'user.signingkey names a path that does not exist, so the public half cannot be read or published' - return - fi - if [[ ! -f "$key" ]]; then - printf 'user.signingkey names something that is not a regular file, so the public half cannot be read or published' - return - fi - if [[ ! -r "$key" ]]; then - printf 'user.signingkey names a file this checkout cannot read, so the public half cannot be read or published' - return - fi - if [[ ! -s "$key" ]]; then - printf 'user.signingkey names an empty file, so the public half cannot be read or published' - return - fi - ;; - esac - printf '' -} -broken_signer() { - if [[ -n "$(broken_reason)" ]]; then printf 'yes'; else printf 'no'; fi -} - -# THE WRITE, and it fires ONLY against the broken configuration. Disabling -# signing unconditionally would be the wrong gate: it would also switch off a -# correctly configured signer, which is the outcome CLOUD-591 is working toward. -# Local-only, because local beats global and the launcher rewrites global every -# session. -if [[ "$repair" = 1 ]]; then - if [[ "$(broken_signer)" = "no" ]]; then - echo "signing-posture: signer is verifiable, leaving signing on" - exit 0 - fi - if ! git config --local commit.gpgsign false; then - echo "::error:: signing-posture: could not write commit.gpgsign to this checkout's config" >&2 - exit 2 - fi - echo "signing-posture: signing disabled in this checkout — $(broken_reason)" - exit 0 -fi - -fail=0 -report() { - echo "::error:: signing-posture: $1" >&2 - fail=1 -} - -# The override is only OWED where something outside the checkout turns signing -# on. Demanding it unconditionally would red every CI run: a runner has no -# launcher and no global setting, so there is nothing to override and an absent -# local value is the correct state there. The predicate is therefore the -# CONFLICT — an inherited "sign" with nothing answering it — not the mere absence -# of a local key. -local_setting=$(git config --local --get commit.gpgsign 2>/dev/null || true) -inherited=$(git config --global --get commit.gpgsign 2>/dev/null || true) -# Only a BROKEN signer is a finding. A verifiable one may sign freely — that is -# the end state CLOUD-591 is working toward, and this gate must not block it. -if [[ "$(broken_signer)" = "yes" ]]; then - case "$local_setting" in - false) ;; - *) - case "$inherited$local_setting" in - *true*) report "signing is on with an unverifiable signer — $(broken_reason). Run: mise run signing-posture --repair" ;; - esac - ;; - esac -fi - -# The commits themselves. Config can be repaired after a commit was written, so -# the config check alone is not the predicate — that is this gate's declared -# mutation. `--repair` clears the config arm above and leaves this one firing on -# whatever was already written, which is what keeps the two arms distinct. -# -# SCOPED TO A BROKEN SIGNER, and this was a real defect rather than a -# refinement. The scan reported EVERY `gpgsig` in range, with no reference to -# whether the key behind it is verifiable — so it refused the exact end state -# this file's header promises to leave alone, and the row asserting that promise -# passed only because it commits with `--no-gpg-sign` and never produces a -# header for the scan to see. A vacuous row over a contradicted predicate -# (CLOUD-418); caught in review on PR #489. -# -# The signer configuration is the only evidence available about which key signed -# these commits, and it is honest evidence: a checkout whose signer is broken -# produced them with that broken signer. Verifying a signature properly needs an -# `allowed_signers` file this repository does not have, and PUBLISHING one is -# precisely CLOUD-591's deliverable — so this gate reads the configuration and -# says so, rather than pretending to a cryptographic check it cannot perform. -if [[ -z "$base" ]]; then - base=$(git rev-parse --verify --quiet origin/main) || base="" -fi -[[ -n "$head" ]] || head=$(git rev-parse --verify --quiet HEAD) || head="" - -if [[ "$(broken_signer)" = "no" ]]; then - echo "signing-posture: signer is verifiable — commits in range not judged" -elif [[ -n "$base" ]] && [[ -n "$head" ]]; then - range=$(git rev-list --no-merges "$base..$head" 2>/dev/null || true) - for sha in $range; do - # Captured, then matched in the shell. NOT `... | grep -q '^gpgsig'`: - # under `pipefail` a `grep -q` exits the moment it matches, the producer - # takes SIGPIPE, and the pipeline reports FAILURE on a match — so the - # signed commit this gate exists to catch would read as clean. It passed - # the suite anyway, because a commit header is small enough that the - # producer finishes first; `pipefail-grep-check` caught what the tests - # could not. - header=$(git cat-file commit "$sha" 2>/dev/null | sed '/^$/q') - case $'\n'"$header" in - *$'\n'gpgsig' '*) - report "${sha:0:8} carries a gpgsig — signed by a key this repo cannot verify or reproduce, which GitHub reports as unknown_key (CLOUD-591 owns publishing one; CLOUD-669 refuses the unverifiable case)" - ;; - esac - done -else - echo "signing-posture: no origin/main to range against — commits not judged" >&2 -fi - -if [[ "$fail" = 0 ]]; then - echo "signing-posture: no commit signed by an unverifiable key" -fi -exit "$fail" diff --git a/mise.toml b/mise.toml index 57568a7fe..0055c459d 100644 --- a/mise.toml +++ b/mise.toml @@ -2153,6 +2153,102 @@ emit | cargo run --quiet -p batten -- record named timeout-drift # # POINTER-ONLY: one token. The probe log carries module bodies and paths, and no # byte of it reaches the record. +# CLOUD-669's reading, ported off `mise-tasks/signing-posture.sh` under +# CLOUD-1717. The things that are not decisions: the git config reads, the +# filesystem tests behind them, and the range scan. +# +# THE SIGNER CLASSIFICATION IS `mise-tasks/signer_posture.py`, shared with +# `crates/batten/tests/it/signing_posture.rs`, because seven of the dying suite's +# cases are about exactly which configurations are unverifiable — an empty key, a +# directory, an unreadable file, a path that does not exist, an inline literal, a +# `/tmp` signer, and a healthy one. Those are the substance; the module's half is +# two set memberships. +# +# THE CONFLICT, NEVER THE MERE ABSENCE OF A LOCAL KEY. `config conflict` is +# recorded only where something OUTSIDE the checkout turns signing on and nothing +# local answers it. A runner has no launcher and no global setting, so an absent +# local value is correct there and nothing is recorded. +# +# RANGE, NEVER HISTORY: `origin/main..HEAD` by default, the range +# `commit-attribution` and `commit-lint` already share. +# +# `git cat-file … | sed '/^$/q'` is captured and matched in the shell rather than +# piped into `grep -q`: under `pipefail` a `grep -q` exits the moment it matches, +# the producer takes SIGPIPE, and the pipeline reports FAILURE on a match — so +# the signed commit this gate exists to catch would read as clean. +# `pipefail-grep-check` caught what the retired suite could not. +# +# NOTHING IS WRITTEN OUTSIDE A GIT REPOSITORY — could not look, which the module +# reads as silence rather than as a posture in force. +[tasks.signing-posture-record] +description = "Effect: record this checkout's signer posture and any signed commit in range, for `commit grade unsafe`" +shell = "bash -c" +run = """ +git rev-parse --git-dir >/dev/null 2>&1 || exit 0 + +key=$(git config --get user.signingkey 2>/dev/null || true) +prog=$(git config --get gpg.ssh.program 2>/dev/null || true) +signer=$(python3 mise-tasks/signer_posture.py "$key" "$prog") || exit 0 + +{ + printf 'signer %s\\n' "$signer" + + local_setting=$(git config --local --get commit.gpgsign 2>/dev/null || true) + inherited=$(git config --global --get commit.gpgsign 2>/dev/null || true) + if [ "$local_setting" != false ]; then + case "$inherited$local_setting" in + *true*) printf 'config conflict\\n' ;; + esac + fi + + base="${BASE_SHA:-}" + head="${HEAD_SHA:-}" + [ -n "$base" ] || base=$(git rev-parse --verify --quiet origin/main) || base="" + [ -n "$head" ] || head=$(git rev-parse --verify --quiet HEAD) || head="" + if [ -n "$base" ] && [ -n "$head" ]; then + for sha in $(git rev-list --no-merges "$base..$head" 2>/dev/null || true); do + header=$(git cat-file commit "$sha" 2>/dev/null | sed '/^$/q') + case $'\\n'"$header" in + *$'\\n'gpgsig' '*) printf 'signed %s\\n' "$(printf '%s' "$sha" | cut -c1-8)" ;; + esac + done + fi +} | cargo run --quiet -p batten -- record named signing-posture +""" + +# THE WRITE, self-declared per house style §5 and kept a task because a module +# cannot write. It fires ONLY against the broken configuration: disabling signing +# unconditionally would also switch off a correctly configured signer, which is +# the outcome CLOUD-591 is working toward. +# +# Local-only, NEVER `--global` — a developer's own unrelated repositories are not +# this repo's business, the same boundary `attribution-identity` draws — and +# local beats global, which matters because the launcher rewrites global every +# session. +[tasks.signing-posture-repair] +description = "Write: switch signing off in THIS checkout when its signer cannot be verified or reproduced (CLOUD-669)" +shell = "bash -c" +run = """ +git rev-parse --git-dir >/dev/null 2>&1 || { + echo "::error:: signing-posture-repair: not a git repository" >&2 + exit 2 +} +key=$(git config --get user.signingkey 2>/dev/null || true) +prog=$(git config --get gpg.ssh.program 2>/dev/null || true) +signer=$(python3 mise-tasks/signer_posture.py "$key" "$prog") || exit 2 +case "$signer" in +verifiable) + echo "signing-posture: signer is verifiable, leaving signing on" + exit 0 + ;; +esac +if ! git config --local commit.gpgsign false; then + echo "::error:: signing-posture-repair: could not write commit.gpgsign to this checkout's config" >&2 + exit 2 +fi +echo "signing-posture: signing disabled in this checkout — ${signer#broken }" +""" + [tasks.evaluator-io-record] description = "Effect: record whether the IO-free evaluator test goes red under `probe-evaluator-io`, for `test judge never`" shell = "bash -c" @@ -3840,7 +3936,7 @@ description = "Session start: put the not-signing posture into force, quietly # `signing-posture: signing disabled in this checkout …` into every session's # advisory. Measured 2026-09-01 by dispatching the real rows. The retired script # `step`-wrapped this call, so that line was never a session's to read. -run = "mise run signing-posture --repair >/tmp/session-start-signing.log 2>&1 || { echo '::error:: session-start: signing-posture failed — see /tmp/session-start-signing.log' >&2; tail -5 /tmp/session-start-signing.log >&2; exit 1; }" +run = "mise run signing-posture-repair >/tmp/session-start-signing.log 2>&1 || { echo '::error:: session-start: signing-posture failed — see /tmp/session-start-signing.log' >&2; tail -5 /tmp/session-start-signing.log >&2; exit 1; }" [tasks."session:wiring"] description = "Session start: reclaim the hook surface from launcher-provisioned siblings — the third repair of the class `session:identity` and `session:signing` already close (CLOUD-1079)" diff --git a/policy/signing-posture.rego b/policy/signing-posture.rego new file mode 100644 index 000000000..d4f62e342 --- /dev/null +++ b/policy/signing-posture.rego @@ -0,0 +1,169 @@ +# No commit is signed by a key that cannot be verified or reproduced (CLOUD-669, +# ported off `mise-tasks/signing-posture.sh` under CLOUD-1717). +# +# SIGNING IS GOOD AND THIS IS NOT AGAINST IT. Signing in CI, with a key whose +# public half is published, is the desired end state and CLOUD-591 owns getting +# there. What this refuses is the narrower thing: a signature produced by a key +# that cannot be verified or reproduced, which is WORSE than no signature because +# it looks like provenance and carries none. +# +# WHY IT IS AN ATTRIBUTION DEFECT RATHER THAN A PREFERENCE. Every commit read +# `author` and `committer` the accountable human — correct, and gated by +# `identity_deny` — and `gpgsig` a vendor-held key. `Attribution` carries +# `identity_deny`, `trailer_deny`, `body_deny`, `trailer_allow` and `identity` +# with NO signature field, so the one commit field the attribution gate +# structurally cannot see is the one carrying a vendor identity. This is that +# blind spot's stand-in until CLOUD-440 lets the engine see a commit object. +# +# TWO CLASSES, AND KEEPING THEM APART IS THE POINT. `config carry unsafe` is a +# posture that will produce bad signatures; `commit carry unsafe` is one that +# already did. Config can be repaired AFTER a commit was written, so a repaired +# checkout still carries the signed commits made before the repair — and those +# are exactly what must not reach `main`. Collapsing the two would let the cheap +# half stand in for the expensive one, which is this gate's declared mutation. +# +# BOTH ARE SCOPED TO A BROKEN SIGNER, and that was a real defect rather than a +# refinement. The commit scan once reported EVERY `gpgsig` in range with no +# reference to whether the key behind it is verifiable — so it refused the exact +# end state this gate promises to leave alone, and the row asserting that promise +# passed only because it commits with `--no-gpg-sign` and never produced a header +# for the scan to see. A vacuous row over a contradicted predicate (CLOUD-418); +# caught in review on PR #489. +# +# THE CONFIG CLASS IS THE CONFLICT, NEVER THE MERE ABSENCE OF A LOCAL KEY. +# Demanding a local override unconditionally would red every CI run: a runner has +# no launcher and no global setting, so there is nothing to override and an +# absent local value is the correct state there. The producer records what is +# INHERITED alongside what is local, and the refusal needs both. +# +# RANGE, NEVER HISTORY. The producer judges `BASE_SHA..HEAD_SHA`, the range +# `commit-attribution` and `commit-lint` already share. Every commit on `main` +# predating this gate is signed by that environment key; judging history would +# make the gate permanently red for commits nobody can now unsign, which is how a +# gate gets switched off. +# +# POINTER-ONLY (rule 4): a short SHA and a setting name. Never a signature block +# — it is a credential artefact this repository does not control. +# +#MUTANT-SUITE crates/batten/tests/it/signing_posture.rs +#MUTANT config-check-is-not-a-commit-check|s@^\tstartswith(line, "signed ")$@\tfalse@|a_signed_commit_in_range_is_refused_and_named_by_short_sha +#MUTANT verifiable-signer-still-refused|s@^\tbroken$@\ttrue@|signing_with_a_verifiable_signer_is_left_alone + +# METADATA +# description: | +# Bound to the TREE surface: `scope = "tree"`, so it reads the tree document +# and never the mediated `{call, facts}` shape. +# THE BRACKETS ARE NOT STYLE: the schema file carries a hyphen, so the dotted +# form is a parse error reported as `invalid schema reference`. +# THIS BLOCK IS YAML AND MUST STAY THE LAST COMMENT BLOCK BEFORE `package`. +# schemas: +# - input: schema["policy-input.schema"] +package batten.signing_posture + +import rego.v1 + +rules contains "commit grade unsafe" + +# The record, or nothing. An absent record is "the producer did not run" — the +# checkout is not a git repository, or the range would not resolve — and is +# silence rather than a claim that the posture is in force. +lines := input.tree.records["signing-posture"] + +# ONLY A BROKEN SIGNER IS A FINDING. A verifiable one may sign freely; that is +# the end state CLOUD-591 is working toward and this gate must not block it. +broken if { + some line in lines + startswith(line, "signer broken") +} + +# --- the posture that will produce bad signatures ---------------------------- + +violation contains { + "rule": "commit grade unsafe", + "verdict": "config carry unsafe", + "subjects": [{"artifact": "commit.gpgsign"}], +} if { + broken + "config conflict" in lines +} + +# --- the commits that already carry one -------------------------------------- + +violation contains { + "rule": "commit grade unsafe", + "verdict": "commit carry unsafe", + "subjects": [{"artifact": sha}], +} if { + broken + some line in lines + startswith(line, "signed ") + sha := trim_space(substring(line, count("signed "), -1)) + sha != "" +} + +# --- cases ------------------------------------------------------------------- + +test_a_signed_commit_in_range_is_refused_and_named_by_short_sha if { + found := violation with input as {"tree": {"records": {"signing-posture": [ + "signer broken user.signingkey names an empty file", + "signed 1a2b3c4d", + ]}}} + count(found) == 1 +} + +test_signing_with_a_verifiable_signer_is_left_alone if { + found := violation with input as {"tree": {"records": {"signing-posture": [ + "signer verifiable", + "config conflict", + "signed 1a2b3c4d", + ]}}} + count(found) == 0 +} + +test_a_missing_override_is_refused_when_the_environment_sets_signing_globally if { + found := violation with input as {"tree": {"records": {"signing-posture": [ + "signer broken the signer resolves inside /tmp", + "config conflict", + ]}}} + count(found) == 1 +} + +# A runner has no launcher and no global setting, so there is nothing to override +# and an absent local value is the correct state there. +test_a_missing_override_is_not_a_finding_when_nothing_sets_signing_globally if { + found := violation with input as {"tree": {"records": {"signing-posture": ["signer broken the signer resolves inside /tmp"]}}} + count(found) == 0 +} + +# REPAIRING THE CONFIG DOES NOT EXCUSE A COMMIT ALREADY SIGNED, which is the +# whole reason the two classes are separate. +test_repairing_the_config_does_not_excuse_a_commit_already_signed if { + found := violation with input as {"tree": {"records": {"signing-posture": [ + "signer broken user.signingkey names an empty file", + "signed 1a2b3c4d", + ]}}} + {entry.verdict | some entry in found} == {"commit carry unsafe"} +} + +test_the_two_classes_stay_distinct if { + found := violation with input as {"tree": {"records": {"signing-posture": [ + "signer broken user.signingkey names an empty file", + "config conflict", + "signed 1a2b3c4d", + ]}}} + {entry.verdict | some entry in found} == {"config carry unsafe", "commit carry unsafe"} +} + +test_every_signed_commit_is_named_separately if { + found := violation with input as {"tree": {"records": {"signing-posture": [ + "signer broken user.signingkey names an empty file", + "signed 1a2b3c4d", + "signed 5e6f7a8b", + ]}}} + count(found) == 2 +} + +test_an_absent_record_says_nothing_rather_than_refusing if { + found := violation with input as {"tree": {"records": {}}} + count(found) == 0 +} diff --git a/tests/signing-posture.bats b/tests/signing-posture.bats deleted file mode 100644 index dbe97a213..000000000 --- a/tests/signing-posture.bats +++ /dev/null @@ -1,282 +0,0 @@ -#!/usr/bin/env bats -# subject: mise-tasks/signing-posture.sh -# CLOUD-669. The not-signing posture, in force rather than recorded. -# -# Every case runs inside a throwaway `git init`, because the subject IS git -# config and commit objects: a suite running in this repo's checkout would -# rewrite the developer's own `commit.gpgsign` and judge real history. -# -# The signing side is faked rather than driven through a real signer. A genuine -# `gpg.ssh.program` needs a key this suite must not create, and the predicate is -# "does the commit object carry a `gpgsig` header" — which `git commit-tree -S` -# cannot produce without one either. So a signed commit is synthesized by writing -# the header into the object directly, which is exactly what the gate reads. - -setup() { - GATE="$BATS_TEST_DIRNAME/../mise-tasks/signing-posture.sh" - REPO="$BATS_TEST_TMPDIR/repo" - mkdir -p "$REPO" - # `main` from the start, so no row has to force a branch into place later: - # `no-branch-f-main` forbids that shape in a suite, and rightly — a `branch -f` - # that escaped the fixture would move the real trunk. - git -C "$REPO" init --quiet --initial-branch=main - # Per fixture, never inherited: a CI runner carries no global identity. - git -C "$REPO" config user.email t@example.com - git -C "$REPO" config user.name t - git -C "$REPO" config commit.gpgsign false - # A VERIFIABLE signer by default, set locally so the suite never inherits the - # host's. Without this the rows passed because this container's own global - # config happens to be broken — an environmental reason, not the gate's logic. - printf 'ssh-ed25519 AAAAfake fixture\n' >"$BATS_TEST_TMPDIR/good.pub" - git -C "$REPO" config user.signingkey "$BATS_TEST_TMPDIR/good.pub" - git -C "$REPO" config gpg.ssh.program /usr/bin/ssh-keygen - git -C "$REPO" commit -q --allow-empty -m base - cd "$REPO" || return 1 - BASE=$(git rev-parse HEAD) -} - -# The two measured shapes of "unverifiable", one helper each. -break_signer_empty_key() { - : >"$BATS_TEST_TMPDIR/empty.pub" - git config --local user.signingkey "$BATS_TEST_TMPDIR/empty.pub" -} -break_signer_ephemeral_program() { - git config --local gpg.ssh.program /tmp/code-sign -} - -# Write a commit object carrying a `gpgsig` header, the way a real signer would. -sign_head() { - local parent tree sig obj - parent=$(git rev-parse HEAD^) - tree=$(git rev-parse HEAD^{tree}) - sig=$(printf -- '-----BEGIN SSH SIGNATURE-----\n ZmFrZQ==\n -----END SSH SIGNATURE-----') - obj=$( - { - printf 'tree %s\nparent %s\n' "$tree" "$parent" - printf 'author t 1700000000 +0000\n' - printf 'committer t 1700000000 +0000\n' - printf 'gpgsig %s\n\nsigned\n' "$sig" - } | git hash-object -t commit -w --stdin - ) - git reset --hard "$obj" --quiet -} - -@test "an unsigned range with the override in place passes" { - git commit -q --allow-empty -m work - run "$GATE" --base "$BASE" --head HEAD - [ "$status" -eq 0 ] - [[ "$output" == *"unverifiable"* ]] -} - -# THE ROW THAT PROVES THIS IS NOT AN ANTI-SIGNING GATE. Signing in CI with a -# published key is CLOUD-591's end state; a gate that refused it would be the -# wrong gate. A verifiable signer with signing ON must pass untouched. -@test "signing with a verifiable signer is left alone" { - git config --local commit.gpgsign true - git commit -q --allow-empty -m work --no-gpg-sign - run "$GATE" --base "$BASE" --head HEAD - [ "$status" -eq 0 ] -} - -# THE ROW THAT WAS MISSING, and its absence hid a contradicted predicate. The row -# above commits with `--no-gpg-sign`, so it produces no `gpgsig` header and never -# reaches the commit scan at all — it could not have caught a scan that refused -# every signature regardless of signer, which is what the scan did. This one puts -# a real header in front of it with the signer left verifiable. -@test "a commit signed by a VERIFIABLE signer is left alone, header and all" { - git config --local commit.gpgsign true - # `--no-gpg-sign` then `sign_head`, never a real signer: the fixture's - # `gpg.ssh.program` is a path that need not exist, and driving a genuine one - # would need a private key this suite must not create. `sign_head` writes the - # header the gate actually reads, which is the whole point of the helper. - git commit -q --allow-empty -m work --no-gpg-sign - sign_head - run "$GATE" --base "$BASE" --head HEAD - [ "$status" -eq 0 ] -} - -@test "--repair leaves a verifiable signer alone rather than switching signing off" { - git config --local commit.gpgsign true - run "$GATE" --repair - [ "$status" -eq 0 ] - [ "$(git config --local --get commit.gpgsign)" = "true" ] - [[ "$output" == *"verifiable"* ]] -} - -@test "an empty signing key is what makes it unverifiable" { - break_signer_empty_key - git config --local commit.gpgsign true - run "$GATE" --base "$BASE" --head HEAD - [ "$status" -eq 1 ] - [[ "$output" == *"empty file"* ]] -} - -# The three shapes a bare `-s` accepted. A directory and an unreadable file both -# have a non-zero size, so `-s` alone called them healthy while the public half -# stayed unreadable — the condition the predicate exists to name. -@test "a signing key that is a directory is unverifiable" { - mkdir -p "$BATS_TEST_TMPDIR/keydir" - git config --local user.signingkey "$BATS_TEST_TMPDIR/keydir" - git config --local commit.gpgsign true - run "$GATE" --base "$BASE" --head HEAD - [ "$status" -eq 1 ] - [[ "$output" == *"not a regular file"* ]] -} - -@test "a signing key this checkout cannot read is unverifiable" { - if [ "$(id -u)" = 0 ]; then skip "root reads regardless of mode"; fi - printf 'ssh-ed25519 AAAAfake fixture\n' >"$BATS_TEST_TMPDIR/locked.pub" - chmod 000 "$BATS_TEST_TMPDIR/locked.pub" - git config --local user.signingkey "$BATS_TEST_TMPDIR/locked.pub" - git config --local commit.gpgsign true - run "$GATE" --base "$BASE" --head HEAD - [ "$status" -eq 1 ] - [[ "$output" == *"cannot read"* ]] -} - -@test "a signing key naming a path that does not exist is unverifiable" { - git config --local user.signingkey "$BATS_TEST_TMPDIR/absent.pub" - git config --local commit.gpgsign true - run "$GATE" --base "$BASE" --head HEAD - [ "$status" -eq 1 ] - [[ "$output" == *"does not exist"* ]] -} - -# THE FALSE POSITIVE THE FILE TESTS WOULD OTHERWISE CREATE. `gpg.format ssh` -# accepts the public key inline, and a literal IS the public half — the most -# publishable form there is. Testing it as a filename would report the healthiest -# configuration possible as broken. -@test "an inline public key is a literal, not a path, and is verifiable" { - git config --local user.signingkey "ssh-ed25519 AAAAfake fixture" - git config --local commit.gpgsign true - run "$GATE" --base "$BASE" --head HEAD - [ "$status" -eq 0 ] -} - -@test "a signer under /tmp is unverifiable because the container reclaims it" { - break_signer_ephemeral_program - git config --local commit.gpgsign true - run "$GATE" --base "$BASE" --head HEAD - [ "$status" -eq 1 ] - [[ "$output" == *"/tmp"* ]] -} - -@test "a signed commit in range is refused, and named by short sha" { - break_signer_ephemeral_program - git commit -q --allow-empty -m work - sign_head - run "$GATE" --base "$BASE" --head HEAD - [ "$status" -eq 1 ] - [[ "$output" == *"carries a gpgsig"* ]] - [[ "$output" == *"$(git rev-parse --short=8 HEAD)"* ]] -} - -# THE ROW THAT KILLS THE CONFIG-ONLY PREDICATE. A checkout repaired AFTER a -# commit was written still carries that signed commit, and it is the commit that -# reaches `main`. -@test "repairing the config does not excuse a commit already signed" { - break_signer_ephemeral_program - git commit -q --allow-empty -m work - sign_head - git config --local commit.gpgsign false - run "$GATE" --base "$BASE" --head HEAD - [ "$status" -eq 1 ] - [[ "$output" == *"carries a gpgsig"* ]] -} - -# The override is owed only where something outside the checkout turns signing -# ON. `git -c` supplies the inherited value, since a suite must never write the -# developer's real global config. -@test "a missing override is refused when the environment sets signing globally" { - git commit -q --allow-empty -m work - break_signer_ephemeral_program - git config --local --unset commit.gpgsign - run env HOME="$BATS_TEST_TMPDIR/fakehome" bash -c ' - mkdir -p "$HOME" - git config --global commit.gpgsign true - exec "$1" --base "$2" --head HEAD' _ "$GATE" "$BASE" - [ "$status" -eq 1 ] - [[ "$output" == *"unverifiable signer"* ]] -} - -# THE CI ROW. A runner has no launcher and no global setting, so an absent local -# value is correct there. Demanding it unconditionally would red every CI run for -# a condition that cannot occur — the false-positive rate that gets a gate -# switched off. -@test "a missing override is NOT a finding when nothing sets signing globally" { - git commit -q --allow-empty -m work - git config --local --unset commit.gpgsign - run env HOME="$BATS_TEST_TMPDIR/emptyhome" bash -c ' - mkdir -p "$HOME" - exec "$1" --base "$2" --head HEAD' _ "$GATE" "$BASE" - [ "$status" -eq 0 ] -} - -@test "a local override set to true is refused when the signer is broken" { - break_signer_empty_key - git config --local commit.gpgsign true - run "$GATE" --base "$BASE" --head HEAD - [ "$status" -eq 1 ] - [[ "$output" == *"unverifiable signer"* ]] -} - -@test "--repair writes the override, local only" { - break_signer_empty_key - git config --local --unset commit.gpgsign - run "$GATE" --repair - [ "$status" -eq 0 ] - [ "$(git config --local --get commit.gpgsign)" = "false" ] -} - -@test "--repair is idempotent" { - break_signer_empty_key - run "$GATE" --repair - [ "$status" -eq 0 ] - run "$GATE" --repair - [ "$status" -eq 0 ] - [ "$(git config --local --get commit.gpgsign)" = "false" ] -} - -@test "--repair never writes global config" { - break_signer_empty_key - local before - before=$(git config --global --get commit.gpgsign 2>/dev/null || echo unset) - run "$GATE" --repair - [ "$status" -eq 0 ] - [ "$(git config --global --get commit.gpgsign 2>/dev/null || echo unset)" = "$before" ] -} - -# The excluded base is SIGNED and the signer is broken, so every ingredient of a -# refusal is present except being in range. Previously both commits here were -# unsigned, which passed whether or not the scan honoured `--base` at all. -@test "history before the range is never judged" { - break_signer_ephemeral_program - git commit -q --allow-empty -m work - sign_head - local newbase - newbase=$(git rev-parse HEAD) - git commit -q --allow-empty -m later - run "$GATE" --base "$newbase" --head HEAD - [ "$status" -eq 0 ] - [[ "$output" != *"carries a gpgsig"* ]] -} - -@test "outside a git repository it is exit 2, never a silent pass" { - cd "$BATS_TEST_TMPDIR" || return 1 - mkdir -p notarepo - cd notarepo || return 1 - run "$GATE" - [ "$status" -eq 2 ] -} - -# Pointer-only (non-negotiable 4): a signature is a credential artefact this repo -# does not control, so no part of one may reach the output. -@test "the refusal echoes no part of the signature block" { - break_signer_ephemeral_program - git commit -q --allow-empty -m work - sign_head - run "$GATE" --base "$BASE" --head HEAD - [ "$status" -eq 1 ] - [[ "$output" != *"BEGIN SSH SIGNATURE"* ]] - [[ "$output" != *"ZmFrZQ"* ]] -} From 48fd26c6ccb84d670806faedcdce544bcd405f03 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sat, 19 Sep 2026 02:55:37 +0000 Subject: [PATCH 21/32] fix(tests): a spawn annotation opens with its verdict, as the inventory reads it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `spawn_census::every_annotation_is_an_expect_carrying_a_verdict` holds every `#[expect(clippy::disallowed_types)]` in the tree to CLOUD-320's inventory shape: the `reason` must OPEN with `stays` or `GOES`, because the verdict is what makes the row an inventory entry rather than a suppression with prose attached. The eight annotations the five retirements on this branch added all began "CLOUD-320's inventory row, and the verdict is that …" — which names the row and states the verdict, but not in the position the census reads. Every one is `stays`: each spawn IS the subject under test, because the walk, the census, the harness reading and the signer classification are precisely what moved out of the dying programs into `.py` siblings so a compiled case could drive them. Found by the full suite rather than by the per-retirement runs, which is the argument for running it: each retirement's own tier was green, and the census is a whole-tree assertion no single tier reaches. 3628/3628 green after this. Refs: CLOUD-1717 Refs: CLOUD-320 --- crates/batten/tests/it/evaluator_closure.rs | 4 ++-- crates/batten/tests/it/evaluator_io_probe.rs | 2 +- crates/batten/tests/it/macos_link.rs | 4 ++-- crates/batten/tests/it/signing_posture.rs | 2 +- crates/batten/tests/it/transcript_corpus.rs | 4 ++-- 5 files changed, 8 insertions(+), 8 deletions(-) diff --git a/crates/batten/tests/it/evaluator_closure.rs b/crates/batten/tests/it/evaluator_closure.rs index 0a6e6762d..1c9c924ba 100644 --- a/crates/batten/tests/it/evaluator_closure.rs +++ b/crates/batten/tests/it/evaluator_closure.rs @@ -117,7 +117,7 @@ fn record(dir: &std::path::Path, lines: &str) { /// does. This is what keeps the four walk cases assertable after the port. #[expect( clippy::disallowed_types, - reason = "CLOUD-320's inventory row, and the verdict is that this spawn IS the subject under test: the walk moved out of the dying program into `mise-tasks/evaluator-closure.py` precisely so a compiled case could drive it, and a harness that re-implemented it in Rust would be a second authority over the reachability the producer actually runs" + reason = "stays, and it is the subject under test rather than a convenience: the walk moved out of the dying program into `mise-tasks/evaluator-closure.py` precisely so a compiled case could drive it, and a harness that re-implemented it in Rust would be a second authority over the reachability the producer actually runs" )] fn walk(metadata: &str) -> String { use std::io::Write as _; @@ -359,7 +359,7 @@ fn a_dev_dependency_of_the_evaluator_is_not_in_the_built_closure() { #[test] #[expect( clippy::disallowed_types, - reason = "CLOUD-320's inventory row: resolving the REAL graph is the whole of this case, and it is the only evidence that the fixtures above agree with the tree `Cargo.toml`'s pin comment makes its claim about" + reason = "stays: resolving the REAL graph is the whole of this case, and it is the only evidence that the fixtures above agree with the tree `Cargo.toml`'s pin comment makes its claim about" )] fn the_repos_real_graph_is_clean_today() { // The anti-vacuity arm, against the tree as it actually resolves. Every case diff --git a/crates/batten/tests/it/evaluator_io_probe.rs b/crates/batten/tests/it/evaluator_io_probe.rs index 0b9dd073a..4f5eef131 100644 --- a/crates/batten/tests/it/evaluator_io_probe.rs +++ b/crates/batten/tests/it/evaluator_io_probe.rs @@ -188,7 +188,7 @@ fn an_absent_record_says_nothing_rather_than_refusing() { /// extracted to remove one gate over. #[expect( clippy::disallowed_types, - reason = "CLOUD-320's inventory row, and the verdict is that this spawn IS the subject under test: the classification of a probe build's output is what moved out of the dying program, and driving it directly is what keeps the three arms assertable without a two-minute rebuild per case" + reason = "stays, and it is the subject under test rather than a convenience: the classification of a probe build's output is what moved out of the dying program, and driving it directly is what keeps the three arms assertable without a two-minute rebuild per case" )] fn classify(name: &str, status: i32, log: &str) -> String { let dir = scratch(&format!("evaluator-io-stub-{name}")); diff --git a/crates/batten/tests/it/macos_link.rs b/crates/batten/tests/it/macos_link.rs index b6355ed88..aaed17c10 100644 --- a/crates/batten/tests/it/macos_link.rs +++ b/crates/batten/tests/it/macos_link.rs @@ -114,7 +114,7 @@ fn record(dir: &std::path::Path, lines: &str) { /// does. This is what keeps the walk cases assertable after the port. #[expect( clippy::disallowed_types, - reason = "CLOUD-320's inventory row, and the verdict is that this spawn IS the subject under test: the walk moved out of the dying program into `mise-tasks/macos-link.py` over the shared `cargo_graph.py` precisely so a compiled case could drive it, and a harness re-implementing it in Rust would be the second authority both programs' headers warned about" + reason = "stays, and it is the subject under test rather than a convenience: the walk moved out of the dying program into `mise-tasks/macos-link.py` over the shared `cargo_graph.py` precisely so a compiled case could drive it, and a harness re-implementing it in Rust would be the second authority both programs' headers warned about" )] fn walk(metadata: &str) -> String { use std::io::Write as _; @@ -346,7 +346,7 @@ fn the_walk_starts_at_the_workspace_members() { #[test] #[expect( clippy::disallowed_types, - reason = "CLOUD-320's inventory row: resolving the REAL macOS graph is the whole of this case, and it is the only evidence that the fixtures above agree with the tree the release actually links" + reason = "stays: resolving the REAL macOS graph is the whole of this case, and it is the only evidence that the fixtures above agree with the tree the release actually links" )] fn the_repo_as_it_stands_has_no_sdk_requiring_dependency() { // The anti-vacuity arm, against the graph as it actually resolves FOR macOS. diff --git a/crates/batten/tests/it/signing_posture.rs b/crates/batten/tests/it/signing_posture.rs index 90888ba7d..e8f610da1 100644 --- a/crates/batten/tests/it/signing_posture.rs +++ b/crates/batten/tests/it/signing_posture.rs @@ -253,7 +253,7 @@ fn an_absent_record_says_nothing_rather_than_refusing() { /// Run the REAL classification both tasks run. #[expect( clippy::disallowed_types, - reason = "CLOUD-320's inventory row, and the verdict is that this spawn IS the subject under test: which configurations are unverifiable is what moved out of the dying program into `mise-tasks/signer_posture.py`, and a harness re-implementing the four file tests in Rust would be a second authority over the one question both the record and the repair turn on" + reason = "stays, and it is the subject under test rather than a convenience: which configurations are unverifiable is what moved out of the dying program into `mise-tasks/signer_posture.py`, and a harness re-implementing the four file tests in Rust would be a second authority over the one question both the record and the repair turn on" )] fn posture(signingkey: &str, program: &str) -> String { let done = std::process::Command::new("python3") diff --git a/crates/batten/tests/it/transcript_corpus.rs b/crates/batten/tests/it/transcript_corpus.rs index 90ee5ea39..e83a5e910 100644 --- a/crates/batten/tests/it/transcript_corpus.rs +++ b/crates/batten/tests/it/transcript_corpus.rs @@ -111,7 +111,7 @@ fn record(dir: &std::path::Path, lines: &str) { /// explicitly empty one — the distinction the contract turns on. #[expect( clippy::disallowed_types, - reason = "CLOUD-320's inventory row, and the verdict is that this spawn IS the subject under test: the census moved out of the dying program into `mise-tasks/transcript_census.py` precisely so a compiled case could drive it, and a harness re-implementing the independence rules in Rust would be a second authority over what counts as a session" + reason = "stays, and it is the subject under test rather than a convenience: the census moved out of the dying program into `mise-tasks/transcript_census.py` precisely so a compiled case could drive it, and a harness re-implementing the independence rules in Rust would be a second authority over what counts as a session" )] fn census( root: &std::path::Path, @@ -374,7 +374,7 @@ fn a_malformed_threshold_refuses_and_writes_nothing() { #[test] #[expect( clippy::disallowed_types, - reason = "CLOUD-320's inventory row: the arity contract belongs to the thing that takes the arguments, so asserting it means invoking the producer with the wrong arity" + reason = "stays: the arity contract belongs to the thing that takes the arguments, so asserting it means invoking the producer with the wrong arity" )] fn more_arguments_than_the_contract_names_refuses() { let held = root("arity", &[("a.jsonl", &authored("kappa"))]); From c13a72ab1de61a7f456259324cb996ede7b898f2 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sat, 19 Sep 2026 20:40:18 +0000 Subject: [PATCH 22/32] feat(policy): refuse a program that leaves the corpus by changing interpreter MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `under_mise_tasks` excludes `.py` and `.tsv`, and every arm of `shell-retirement` is built on it — so a reading written in another language under `mise-tasks/` is not an added program, not an edited one, and owes no mapping when it dies. It sits inside the campaign's directory and outside the campaign. Measured on this module's own campaign: five retirements this session moved 615 lines of shell reading into six `.py` siblings. The ratchet recorded five programs retired; every line of the logic stayed in the tree one interpreter over, and each file carried a header arguing the exclusion admitted it. That is CLOUD-929's shape one campaign later — the campaign to delete bash added bash, and nothing on the board was positioned to notice. Arm F reads the path and only the path, so it is sound on exactly the paths the other arms cannot see, and it fires on ADDITION. Counting `.py` as a program instead would have built a wall rather than a ratchet: arm B reads `input.tree.lines[path]`, `line_sources` is `mise-tasks/*.sh`, so a governed `.py` has no lines entry, every base line reads as removed, and no admission can clear it. The arm carries its escape analysis in the module (an unforeseen extension, an extensionless `#!` file, a `.sh` wrapper around a helper, an ignored `.pyc`) and its stated limit (a program outside `mise-tasks/` is not this arm's business). Three `#MUTANT` rows, one per conjunct, each observed by a different case. Verified red on the current tree at exit 2, naming exactly the six files. Refs: CLOUD-1717 --- batten.toml | 29 ++++++++ crates/batten/tests/it/shell_retirement.rs | 74 +++++++++++++++++++ policy/shell-retirement.rego | 83 ++++++++++++++++++++++ 3 files changed, 186 insertions(+) diff --git a/batten.toml b/batten.toml index 97aa8acc6..59be7ce01 100644 --- a/batten.toml +++ b/batten.toml @@ -12380,6 +12380,35 @@ id = "path admit first" kind = "override" precondition = "the row is work this branch could not have done — it needs a decision, a mechanism, or an artifact that does not exist yet — rather than work you declined to do while holding the file open" +[[verdict]] +id = "program add refused" +gloss = "a program was added under mise-tasks/ in an interpreter the corpus does not count" +class = """ +The corpus is defined over `mise-tasks/`, and `under_mise_tasks` excludes the \ +extensions that are not shell — so a reading written in another language there \ +is not an added program, not an edited one, and owes no mapping when it dies. \ +It is outside the campaign while sitting inside its directory. CLOUD-1717 \ +measured the cost: five retirements moved 615 lines of shell reading into six \ +`.py` siblings, the ratchet recorded five programs gone, and every line of the \ +logic stayed in the tree one interpreter over. That is CLOUD-929's shape — the \ +campaign to delete bash added bash, and nothing was positioned to notice. \ +The remedy is the engine: put the decision in a `policy/*.rego` module and the \ +reading in `crates/batten/src/` behind a declared verb, which is what the \ +retirement was for. A new file here that is genuinely DATA rather than a \ +program widens `declared_data` in the same review — a groomed row and a human, \ +not a header arguing that the exclusion admits it. +""" + +[[verdict.route]] +id = "rule read first" +kind = "document" +target = "rules/policy-modules.md" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + [[verdict]] id = "shell edit refused" gloss = "an authored shell rule or bats suite was edited in place rather than migrated" diff --git a/crates/batten/tests/it/shell_retirement.rs b/crates/batten/tests/it/shell_retirement.rs index cce43b9d3..e0efacddd 100644 --- a/crates/batten/tests/it/shell_retirement.rs +++ b/crates/batten/tests/it/shell_retirement.rs @@ -698,6 +698,80 @@ fn an_added_shell_rule_is_refused() { assert_eq!(findings(&root), vec!["shell retire other".to_owned()]); } +/// CLOUD-1717's arm F, over the compiled surface rather than `with input as`. +/// +/// The evasion this refuses was MEASURED on this module's own campaign: five +/// retirements moved 615 lines of shell reading into `.py` siblings, which +/// `under_mise_tasks` excludes, so the ratchet recorded five programs gone +/// while every line of the logic stayed in the tree one interpreter over. +#[test] +fn an_added_program_in_another_interpreter_is_refused() { + let root = repo( + "added-interpreter", + &[], + &Head { + written: &[("mise-tasks/new-gate.py", "print('a gate')\n")], + removed: &[], + }, + ); + assert_eq!(findings(&root), vec!["shell retire other".to_owned()]); +} + +/// NOT A LIST OF INTERPRETERS. The test is "not `.sh`", so a language nobody +/// has thought of is already inside it — which is the property that keeps this +/// from being the next hole one extension over. +#[test] +fn an_added_program_in_an_unforeseen_interpreter_is_refused_too() { + let root = repo( + "added-unforeseen", + &[], + &Head { + written: &[("mise-tasks/new-gate.rb", "puts 'a gate'\n")], + removed: &[], + }, + ); + assert_eq!(findings(&root), vec!["shell retire other".to_owned()]); +} + +/// The admitted half, and it is what keeps the arm from being a wall: a DATA +/// file is not a program, and `declared_data` is the closed list that says so. +#[test] +fn a_declared_data_file_under_mise_tasks_is_admitted() { + let root = repo( + "added-data", + &[], + &Head { + written: &[("mise-tasks/new-table.tsv", "a\tb\n")], + removed: &[], + }, + ); + assert!( + findings(&root).is_empty(), + "a declared data file is not a program: {:?}", + findings(&root) + ); +} + +/// THE STATED LIMIT, asserted rather than assumed. `mise-tasks/` is what the +/// corpus is defined over; a rule reaching past it would decide something this +/// module does not own. +#[test] +fn an_added_file_outside_mise_tasks_is_not_this_arms_business() { + let root = repo( + "added-outside", + &[], + &Head { + written: &[("crates/batten/src/lib.rs", "// code\n")], + removed: &[], + }, + ); + assert!( + findings(&root).is_empty(), + "this arm is bounded by the corpus directory: {:?}", + findings(&root) + ); +} + #[test] fn an_added_bats_suite_is_refused() { let root = repo( diff --git a/policy/shell-retirement.rego b/policy/shell-retirement.rego index f281f0a30..d31da6230 100644 --- a/policy/shell-retirement.rego +++ b/policy/shell-retirement.rego @@ -111,6 +111,14 @@ # spending a binding that SURVIVES, which is the loosening `case_earns_removal` # warns about arriving one arm over. #MUTANT bats-binding-survives|s@assigned_name(binding) == variable@true@|a_bats_case_spending_a_surviving_binding_is_refused +# CLOUD-1717's arm F. The first row kills the arm outright; the second turns +# the data exclusion into a refusal, which the admitted-data case observes; the +# third unanchors the directory test, so the arm fires on every added path in +# the tree and the outside-`mise-tasks/` case goes red. Three conjuncts, three +# cases, and each case fails for a different reason. +#MUTANT interpreter-add-unchecked|s@not endswith(path, ".sh")@false@|an_added_program_in_another_interpreter_is_refused +#MUTANT interpreter-data-not-excluded|s@not declared_data(path)@true@|a_declared_data_file_under_mise_tasks_is_admitted +#MUTANT interpreter-scope-unanchored|s@startswith(path, "mise-tasks/")@true@|an_added_file_outside_mise_tasks_is_not_this_arms_business # #MUTANT-SUITE crates/batten/tests/it/shell_retirement.rs @@ -270,6 +278,81 @@ declares_it_stays_bash(path) if { contains(line, "# stays-bash:") } +# --------------------------------------------------------------------------- +# F: a program does not leave the corpus by CHANGING INTERPRETER. A's companion. +# --------------------------------------------------------------------------- + +# MEASURED ON THIS MODULE'S OWN CAMPAIGN, 2026-09-19 (CLOUD-1717). +# +# `under_mise_tasks` excludes `.py` and `.tsv`, and every arm above is built on +# it — so a program written in Python under `mise-tasks/` is not an added shell +# rule (A), not an edited one (B), and owes no mapping when deleted (C). It is +# not in the corpus at all. Five retirements in one session moved 615 lines of +# shell reading into six `.py` siblings and reported five programs retired: the +# corpus shrank by five and the tree kept every line of the logic, one +# interpreter over. Each file carried a header ARGUING the exclusion admitted +# it, which is the tell — an agent that has to write the argument has already +# found the hole. +# +# THIS IS CLOUD-929's SHAPE, ONE CAMPAIGN LATER: the campaign to delete bash +# added bash, and nothing on the board was positioned to notice. What that +# defect cost was a stale prose count; what this one costs is the campaign's +# own premise, because a retirement that relocates the reading has retired +# nothing and the ratchet says it has. +# +# WHY NOT SIMPLY COUNT `.py` AS A PROGRAM — the fix that reads as obvious and +# builds a trap. `under_mise_tasks` feeds `governed_at_head`, and arm B reads +# `input.tree.lines[path]` to decide what an edit changed. This rule's +# `line_sources` is `mise-tasks/*.sh`, so a governed `.py` has NO lines entry: +# `head` is the empty set, every base line reads as removed, `shell edit +# refused` fires, and no admission can clear it because every admission is a +# predicate over lines that are not there. A rule that cannot be satisfied is +# not a ratchet, it is a wall. The neighbouring case is +# `shell_retirement.rs`'s `generated_and_non_shell_paths_are_not_governed`, +# which pins the exclusion this arm narrows — and narrows on the ADDED side +# only, so that case keeps passing: it EDITS a `.py` the base already had. +# +# SO THIS ARM READS THE PATH AND ONLY THE PATH. It needs no lines, which is why +# it is sound on exactly the paths the others cannot see, and it fires on +# ADDITION — the one move that creates the evasion. An author who genuinely +# needs a new non-shell file under `mise-tasks/` widens `declared_data` in the +# same review, which is a groomed row and a human, not an annotation an agent +# writes about its own work. +# +# WHY NO CASE ESCAPES, STATED RATHER THAN ASSUMED — the obligation rule 2 puts +# on a green gate, since a gate says only "nothing it can SEE is wrong": +# - a `.rb`, `.js`, `.ts` or `.pl` sibling: caught, the test is not `.sh` +# rather than a list of interpreters, so a language nobody has thought of +# is already in it; +# - an extensionless executable carrying `#!`: caught here AND by A, which +# reads the shebang — the two arms overlap on purpose; +# - a `.sh` that is one line of `exec python3 helper.py`: the helper is +# caught here, and the wrapper is an added authored shell rule, so A +# catches it even when the helper lands elsewhere; +# - a `__pycache__/*.pyc`: caught, and it is also ignored, so it never +# reaches a delta in the first place. +# THE STATED LIMIT, which is a bound and not an oversight: a program written +# OUTSIDE `mise-tasks/` — `scripts/`, `tools/` — is not this arm's business, +# because `mise-tasks/` is what the corpus is defined over and a rule reaching +# past it would be deciding something this module does not own. `task-callable` +# is the surface that notices an uncallable task; a program nothing calls is +# nobody's evasion. +violation contains { + "rule": "shell retire other", + "verdict": "program add refused", + "subjects": [{"path": path}], +} if { + some path in delta.added + startswith(path, "mise-tasks/") + not endswith(path, ".sh") + not declared_data(path) +} + +# The non-program files `mise-tasks/` is allowed to hold. A CLOSED LIST for +# `under_mise_tasks`'s own stated reason one arm up — the set is small, and a +# pattern here would admit the next extension without anyone deciding to. +declared_data(path) if endswith(path, ".tsv") + # --------------------------------------------------------------------------- # B: a migration does not EDIT one in place. The load-bearing arm. # --------------------------------------------------------------------------- From df4fe3f2aaaa9a787c4a9cff935fc5a6455daf93 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sat, 19 Sep 2026 20:42:34 +0000 Subject: [PATCH 23/32] fix(policy): drop an exempt row whose suite and subject are both gone MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `tests/replay.bats` exempted itself over `mise-tasks/replay-pointers.py`. Both were retired; the row outlived them. `suite-subject-retirable` tolerates a stale row by design — its arm fires only while the suite is present — so nothing went red, which is why it sat here. A table entry is a claim about the tree, and the tree wins. The reasoning it carried does not die with it: `shell-retirement`'s arm F now refuses an added `.py` under `mise-tasks/` outright, so the "looks governed, is excluded by extension" gap it described is a gate rather than a note in an exemption table. Refs: CLOUD-1717 --- policy/suite-subject-retirable.rego | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/policy/suite-subject-retirable.rego b/policy/suite-subject-retirable.rego index 6a89bf219..eac6f25f1 100644 --- a/policy/suite-subject-retirable.rego +++ b/policy/suite-subject-retirable.rego @@ -193,13 +193,19 @@ exempt := { "tests/install.bats": "install.sh", "tests/lint-deno.bats": "mise.toml", "tests/lint-rego.bats": "mise.toml", - # THE MEMBER A PREFIX SCAN MISSES, and the reason this table is derived from - # `retirable` rather than from "is it under `mise-tasks/`". A `.py` sibling - # LOOKS governed and is excluded by extension, so a census testing the prefix - # alone counts it retirable and drops the suite. That is exactly what happened - # on the first pass here: 18 suites, and `this_repository_is_clean_today` in - # `crates/batten/tests/suite_subjects.rs` returned the nineteenth. - "tests/replay.bats": "mise-tasks/replay-pointers.py — `.py` is excluded from `governed_when_deleted`", + # THE ROW THAT WAS HERE IS GONE BECAUSE ITS SUBJECT IS (CLOUD-1717). + # + # `tests/replay.bats` exempted itself over `mise-tasks/replay-pointers.py`, + # on the reasoning that a `.py` sibling LOOKS governed and is excluded by + # extension. Both files were retired; the row outlived them. This module + # tolerates a stale row by design — arm B fires only while the suite is + # present — so nothing went red, which is exactly why it sat here. A table + # entry is a claim about the tree, and the tree wins. + # + # The reasoning it carried did not die with it: `shell-retirement.rego`'s + # arm F now refuses an ADDED `.py` under `mise-tasks/` outright, so the + # "looks governed, is excluded" gap it described is a gate rather than a + # note in an exemption table. "tests/release-tracking-check.bats": "workflow yaml — STRANDS mise-tasks/release-tracking-check.sh", "tests/remedy-payload-source.bats": "batten.toml — STRANDS mise-tasks/board-payloads.sh", "tests/session-start.bats": ".claude/hooks/session-start.sh", From 40c308ac593f7cc63d3dc8e9ae1145e21d3f60c3 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sat, 19 Sep 2026 21:15:37 +0000 Subject: [PATCH 24/32] feat(record): derive the probe verdict in Rust behind `record derive` MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit First of six. `mise-tasks/probe_verdict.py` classified a probe build's output — the arm a gate written to the obvious shape gets wrong, since `cargo test` exits non-zero for a compile error just as readily as for a falsified assertion. That reading was a script beside the task, testable only by spawning `python3` from a Rust tier. It is now `crates/batten/src/probe_verdict.rs`: one total function from `(status, log, test)` to a three-valued enum, with its own `#[cfg(test)] mod tests`. Seven unit cases, two of which the retired program never had and would have got wrong — an unindented occurrence of the test name, and a longer name containing it. Both came from asking what escapes the reading rather than from porting. The new leaf `record derive ` applies a reading the engine owns and writes the record. It SPAWNS NOTHING: the `cargo test` run stays in the producer task where house-style §5 puts a producer's effects, and the log arrives on stdin. `spawn-adapters` and `spawn-widening` both refuse a new Command site under crates/batten/src with no override route, and they are right to — a reading that needed one would have the wrong shape. Non-document inputs arrive as `--input =`, one declared flag rather than a column per family. An unknown key and an undeclared family are usage errors, never silent defaults: a misspelled input would otherwise exit clean from a reading that ran on something else, and the record would still be written. The tier keeps its four decision cases, gains four end-to-end derive -> record -> check cases, and loses its `python3` spawn and its `#[expect(clippy::disallowed_types)]` row. Three gates named the new module before a human did — `layer place wrong`, `module-map-check`, and the surface's own closed-set assertions. That is the coverage clause working, three times, on one module. Full suite 5676/5676. `shell retire partial` now names five `.py`, not six. Refs: CLOUD-1717 Admits-Rule: protected-mutation Admits-Verdict: path write refused Admits-Subject: .serena/memories/core.md Admits-Id: ffd90e57dd421c1a79b091bd9de62dbecd6f80dc1c8b1f8b22efbfa169671af2 --- .serena/memories/core.md | 16 ++ completions/batten.bash | 77 ++++++- completions/batten.fish | 87 ++++--- completions/batten.zsh | 57 +++++ crates/batten/src/cli.rs | 29 +++ crates/batten/src/lib.rs | 1 + crates/batten/src/probe_verdict.rs | 146 ++++++++++++ crates/batten/src/record.rs | 141 ++++++++++++ crates/batten/src/spec.rs | 5 + crates/batten/src/surface.rs | 58 +++++ crates/batten/tests/it/evaluator_io_probe.rs | 216 +++++++++++++----- crates/batten/tests/it/pointer_only.rs | 11 + .../it__snapshots__golden_json_schema.snap | 26 +++ man/batten-record-derive.1 | 19 ++ man/batten-record.1 | 3 + mise-tasks/probe_verdict.py | 68 ------ mise.toml | 8 +- policy/module-layering.rego | 17 ++ 18 files changed, 825 insertions(+), 160 deletions(-) create mode 100644 crates/batten/src/probe_verdict.rs create mode 100644 man/batten-record-derive.1 delete mode 100644 mise-tasks/probe_verdict.py diff --git a/.serena/memories/core.md b/.serena/memories/core.md index b3e225229..46a3a9c09 100644 --- a/.serena/memories/core.md +++ b/.serena/memories/core.md @@ -2448,6 +2448,22 @@ judge_fingerprint`, its own domain tag), so a caller can reference content it appearing after the record moves the key instead of being invisible. Every failure is could-not-look, which allows; a fact naming every program in a project must never refuse on a failure to see. +- `probe_verdict.rs` — which of three things a probe build did, from its exit + status and its log (CLOUD-418, CLOUD-1717). THE VERDICT IS THE HARNESS'S OWN + LINE, NEVER THE EXIT CODE ALONE: `cargo test` exits non-zero for a compile + error, an unresolved feature, an absent toolchain and a panic in some other + test, every one of which would read as "the probe falsified the assertion" and + hand the gate a pass it did not earn — and that pass gets MORE likely as the + crate breaks, so a gate written to the obvious shape is loudest exactly when it + is lying. Anchored on the `failures:` listing rather than the per-test line, + which is not stable across `--quiet`. It reaches NOTHING, not even `error`: + one total function to a three-valued enum, where a log it cannot make sense of + is `Unread` — the could-not-look the caller already handles, so a `Result` + would add a state with no distinct handling. Ported off + `mise-tasks/probe_verdict.py`, which was the campaign to delete bash routing a + reading into another interpreter; `shell-retirement.rego`'s arm F now refuses + that shape. Its caller is `record derive`, which SPAWNS NOTHING — the probe + build stays in the producer task (§5) and the log arrives on stdin. - `prune.rs` — the build tree's reclaim and its disk floor (CLOUD-766/861/1030), retired out of `mise-tasks/target-prune.sh` under CLOUD-1059. `Effect::Destructive` on `Surface::VerifyOnly`, beside `capture prune` — §5's diff --git a/completions/batten.bash b/completions/batten.bash index 7853d4d72..0d4fcdbf0 100644 --- a/completions/batten.bash +++ b/completions/batten.bash @@ -787,6 +787,9 @@ _batten() { batten__subcmd__help__subcmd__record,closes) cmd="batten__subcmd__help__subcmd__record__subcmd__closes" ;; + batten__subcmd__help__subcmd__record,derive) + cmd="batten__subcmd__help__subcmd__record__subcmd__derive" + ;; batten__subcmd__help__subcmd__record,fold) cmd="batten__subcmd__help__subcmd__record__subcmd__fold" ;; @@ -1267,6 +1270,9 @@ _batten() { batten__subcmd__record,closes) cmd="batten__subcmd__record__subcmd__closes" ;; + batten__subcmd__record,derive) + cmd="batten__subcmd__record__subcmd__derive" + ;; batten__subcmd__record,fold) cmd="batten__subcmd__record__subcmd__fold" ;; @@ -1297,6 +1303,9 @@ _batten() { batten__subcmd__record__subcmd__help,closes) cmd="batten__subcmd__record__subcmd__help__subcmd__closes" ;; + batten__subcmd__record__subcmd__help,derive) + cmd="batten__subcmd__record__subcmd__help__subcmd__derive" + ;; batten__subcmd__record__subcmd__help,fold) cmd="batten__subcmd__record__subcmd__help__subcmd__fold" ;; @@ -5475,7 +5484,7 @@ _batten() { return 0 ;; batten__subcmd__help__subcmd__record) - opts="tool forge named keyed journal show fold plan closes" + opts="tool forge named derive keyed journal show fold plan closes" if [[ ${cur} == -* || ${COMP_CWORD} -eq 3 ]] ; then COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 @@ -5502,6 +5511,20 @@ _batten() { COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 ;; + batten__subcmd__help__subcmd__record__subcmd__derive) + opts="" + if [[ ${cur} == -* || ${COMP_CWORD} -eq 4 ]] ; then + COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) + return 0 + fi + case "${prev}" in + *) + COMPREPLY=() + ;; + esac + COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) + return 0 + ;; batten__subcmd__help__subcmd__record__subcmd__fold) opts="" if [[ ${cur} == -* || ${COMP_CWORD} -eq 4 ]] ; then @@ -9293,7 +9316,7 @@ _batten() { return 0 ;; batten__subcmd__record) - opts="-q -v -y -h --strictness --fail-on-warning --config-from --config-in --silent --quiet --verbose --debug --trace --log-level --no-color --no-input --yes --help tool forge named keyed journal show fold plan closes help" + opts="-q -v -y -h --strictness --fail-on-warning --config-from --config-in --silent --quiet --verbose --debug --trace --log-level --no-color --no-input --yes --help tool forge named derive keyed journal show fold plan closes help" if [[ ${cur} == -* || ${COMP_CWORD} -eq 2 ]] ; then COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 @@ -9352,6 +9375,40 @@ _batten() { COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 ;; + batten__subcmd__record__subcmd__derive) + opts="-q -v -y -h --input --strictness --fail-on-warning --config-from --config-in --silent --quiet --verbose --debug --trace --log-level --no-color --no-input --yes --help" + if [[ ${cur} == -* || ${COMP_CWORD} -eq 3 ]] ; then + COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) + return 0 + fi + case "${prev}" in + --input) + COMPREPLY=($(compgen -f "${cur}")) + return 0 + ;; + --strictness) + COMPREPLY=($(compgen -W "permissive standard strict" -- "${cur}")) + return 0 + ;; + --config-from) + COMPREPLY=($(compgen -f "${cur}")) + return 0 + ;; + --config-in) + COMPREPLY=($(compgen -f "${cur}")) + return 0 + ;; + --log-level) + COMPREPLY=($(compgen -W "silent quiet normal verbose debug trace" -- "${cur}")) + return 0 + ;; + *) + COMPREPLY=() + ;; + esac + COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) + return 0 + ;; batten__subcmd__record__subcmd__fold) opts="-q -v -y -h --strictness --fail-on-warning --config-from --config-in --silent --quiet --verbose --debug --trace --log-level --no-color --no-input --yes --help" if [[ ${cur} == -* || ${COMP_CWORD} -eq 3 ]] ; then @@ -9413,7 +9470,7 @@ _batten() { return 0 ;; batten__subcmd__record__subcmd__help) - opts="tool forge named keyed journal show fold plan closes help" + opts="tool forge named derive keyed journal show fold plan closes help" if [[ ${cur} == -* || ${COMP_CWORD} -eq 3 ]] ; then COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 @@ -9440,6 +9497,20 @@ _batten() { COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 ;; + batten__subcmd__record__subcmd__help__subcmd__derive) + opts="" + if [[ ${cur} == -* || ${COMP_CWORD} -eq 4 ]] ; then + COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) + return 0 + fi + case "${prev}" in + *) + COMPREPLY=() + ;; + esac + COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) + return 0 + ;; batten__subcmd__record__subcmd__help__subcmd__fold) opts="" if [[ ${cur} == -* || ${COMP_CWORD} -eq 4 ]] ; then diff --git a/completions/batten.fish b/completions/batten.fish index d591d2328..7a945b1a3 100644 --- a/completions/batten.fish +++ b/completions/batten.fish @@ -3015,37 +3015,38 @@ complete -c batten -n "__fish_batten_using_subcommand state; and __fish_seen_sub complete -c batten -n "__fish_batten_using_subcommand state; and __fish_seen_subcommand_from help" -f -a "settle" -d 'Record what was decided about a stored finding' complete -c batten -n "__fish_batten_using_subcommand state; and __fish_seen_subcommand_from help" -f -a "list" -d 'List stored findings and the refs they were observed in' complete -c batten -n "__fish_batten_using_subcommand state; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -l strictness -d 'Raise how strictly gates apply (an override may only tighten policy)' -r -f -a "permissive\t'Advisory: findings are reported without failing the run' -standard\t'The default: a finding is a violation' -strict\t'Everything `Standard` fails on, plus anything advisory'" -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -l config-from -d 'Read the committed config from a git ref (e.g. origin/main) instead of the working tree' -r -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -l config-in -d 'Read the committed config from this directory instead of the directory being judged' -r -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -l log-level -d 'Set the verbosity rung by name' -r -f -a "silent\t'Say nothing but a verdict or a usage error' -quiet\t'Suppress ordinary progress; keep warnings' -normal\t'The default' -verbose\t'Explain what is being checked' -debug\t'Add resolution detail' -trace\t'Add everything'" -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -l fail-on-warning -d 'Promote a warn-severity finding to a violation (an override may only turn this on)' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -l silent -d 'Say nothing but a verdict or a usage error' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -s q -l quiet -d 'Suppress ordinary progress (repeatable: -qq is silent)' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -s v -l verbose -d 'Explain what is being checked (repeatable: -vv is debug)' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -l debug -d 'Add resolution detail' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -l trace -d 'Add everything' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -l no-color -d 'Never colour stderr, whatever it is attached to' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -l no-input -d 'Never prompt; treat the run as unattended' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -s y -l yes -d 'Confirm a destructive operation that would otherwise refuse' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -s h -l help -d 'Print help (see more with \'--help\')' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -f -a "tool" -d 'Record a declared tool row\'s verdict, read as ` ` lines on stdin' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -f -a "forge" -d 'Record the forge\'s check verdicts for one commit, read as ` ` lines on stdin' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -f -a "named" -d 'Record one named family under this branch, read from stdin' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -f -a "keyed" -d 'Put one value into a keyed store family, read from stdin' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -f -a "journal" -d 'Append one record to an append-and-fold store family, read from stdin' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -f -a "show" -d 'Read one keyed record back: `hit` and the value, or `miss`' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -f -a "fold" -d 'Fold a journal family: `nothing`, its records, or `unreadable `' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -f -a "plan" -d 'Record this branch\'s plan, read as ` ` lines on stdin' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -f -a "closes" -d 'Record which rows this branch\'s pull request body closes, read on stdin' -complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named keyed journal show fold plan closes help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -l strictness -d 'Raise how strictly gates apply (an override may only tighten policy)' -r -f -a "permissive\t'Advisory: findings are reported without failing the run' +standard\t'The default: a finding is a violation' +strict\t'Everything `Standard` fails on, plus anything advisory'" +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -l config-from -d 'Read the committed config from a git ref (e.g. origin/main) instead of the working tree' -r +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -l config-in -d 'Read the committed config from this directory instead of the directory being judged' -r +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -l log-level -d 'Set the verbosity rung by name' -r -f -a "silent\t'Say nothing but a verdict or a usage error' +quiet\t'Suppress ordinary progress; keep warnings' +normal\t'The default' +verbose\t'Explain what is being checked' +debug\t'Add resolution detail' +trace\t'Add everything'" +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -l fail-on-warning -d 'Promote a warn-severity finding to a violation (an override may only turn this on)' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -l silent -d 'Say nothing but a verdict or a usage error' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -s q -l quiet -d 'Suppress ordinary progress (repeatable: -qq is silent)' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -s v -l verbose -d 'Explain what is being checked (repeatable: -vv is debug)' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -l debug -d 'Add resolution detail' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -l trace -d 'Add everything' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -l no-color -d 'Never colour stderr, whatever it is attached to' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -l no-input -d 'Never prompt; treat the run as unattended' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -s y -l yes -d 'Confirm a destructive operation that would otherwise refuse' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -s h -l help -d 'Print help (see more with \'--help\')' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -f -a "tool" -d 'Record a declared tool row\'s verdict, read as ` ` lines on stdin' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -f -a "forge" -d 'Record the forge\'s check verdicts for one commit, read as ` ` lines on stdin' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -f -a "named" -d 'Record one named family under this branch, read from stdin' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -f -a "derive" -d 'Derive one named family\'s record from its input and write it' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -f -a "keyed" -d 'Put one value into a keyed store family, read from stdin' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -f -a "journal" -d 'Append one record to an append-and-fold store family, read from stdin' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -f -a "show" -d 'Read one keyed record back: `hit` and the value, or `miss`' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -f -a "fold" -d 'Fold a journal family: `nothing`, its records, or `unreadable `' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -f -a "plan" -d 'Record this branch\'s plan, read as ` ` lines on stdin' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -f -a "closes" -d 'Record which rows this branch\'s pull request body closes, read on stdin' +complete -c batten -n "__fish_batten_using_subcommand record; and not __fish_seen_subcommand_from tool forge named derive keyed journal show fold plan closes help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)' complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from tool" -l strictness -d 'Raise how strictly gates apply (an override may only tighten policy)' -r -f -a "permissive\t'Advisory: findings are reported without failing the run' standard\t'The default: a finding is a violation' strict\t'Everything `Standard` fails on, plus anything advisory'" @@ -3109,6 +3110,28 @@ complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_su complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from named" -l no-input -d 'Never prompt; treat the run as unattended' complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from named" -s y -l yes -d 'Confirm a destructive operation that would otherwise refuse' complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from named" -s h -l help -d 'Print help (see more with \'--help\')' +complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from derive" -l input -d 'A `=` input this family needs beyond stdin (repeatable)' -r +complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from derive" -l strictness -d 'Raise how strictly gates apply (an override may only tighten policy)' -r -f -a "permissive\t'Advisory: findings are reported without failing the run' +standard\t'The default: a finding is a violation' +strict\t'Everything `Standard` fails on, plus anything advisory'" +complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from derive" -l config-from -d 'Read the committed config from a git ref (e.g. origin/main) instead of the working tree' -r +complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from derive" -l config-in -d 'Read the committed config from this directory instead of the directory being judged' -r +complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from derive" -l log-level -d 'Set the verbosity rung by name' -r -f -a "silent\t'Say nothing but a verdict or a usage error' +quiet\t'Suppress ordinary progress; keep warnings' +normal\t'The default' +verbose\t'Explain what is being checked' +debug\t'Add resolution detail' +trace\t'Add everything'" +complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from derive" -l fail-on-warning -d 'Promote a warn-severity finding to a violation (an override may only turn this on)' +complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from derive" -l silent -d 'Say nothing but a verdict or a usage error' +complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from derive" -s q -l quiet -d 'Suppress ordinary progress (repeatable: -qq is silent)' +complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from derive" -s v -l verbose -d 'Explain what is being checked (repeatable: -vv is debug)' +complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from derive" -l debug -d 'Add resolution detail' +complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from derive" -l trace -d 'Add everything' +complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from derive" -l no-color -d 'Never colour stderr, whatever it is attached to' +complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from derive" -l no-input -d 'Never prompt; treat the run as unattended' +complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from derive" -s y -l yes -d 'Confirm a destructive operation that would otherwise refuse' +complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from derive" -s h -l help -d 'Print help (see more with \'--help\')' complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from keyed" -l strictness -d 'Raise how strictly gates apply (an override may only tighten policy)' -r -f -a "permissive\t'Advisory: findings are reported without failing the run' standard\t'The default: a finding is a violation' strict\t'Everything `Standard` fails on, plus anything advisory'" @@ -3238,6 +3261,7 @@ complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_su complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from help" -f -a "tool" -d 'Record a declared tool row\'s verdict, read as ` ` lines on stdin' complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from help" -f -a "forge" -d 'Record the forge\'s check verdicts for one commit, read as ` ` lines on stdin' complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from help" -f -a "named" -d 'Record one named family under this branch, read from stdin' +complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from help" -f -a "derive" -d 'Derive one named family\'s record from its input and write it' complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from help" -f -a "keyed" -d 'Put one value into a keyed store family, read from stdin' complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from help" -f -a "journal" -d 'Append one record to an append-and-fold store family, read from stdin' complete -c batten -n "__fish_batten_using_subcommand record; and __fish_seen_subcommand_from help" -f -a "show" -d 'Read one keyed record back: `hit` and the value, or `miss`' @@ -3931,6 +3955,7 @@ complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subc complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from record" -f -a "tool" -d 'Record a declared tool row\'s verdict, read as ` ` lines on stdin' complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from record" -f -a "forge" -d 'Record the forge\'s check verdicts for one commit, read as ` ` lines on stdin' complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from record" -f -a "named" -d 'Record one named family under this branch, read from stdin' +complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from record" -f -a "derive" -d 'Derive one named family\'s record from its input and write it' complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from record" -f -a "keyed" -d 'Put one value into a keyed store family, read from stdin' complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from record" -f -a "journal" -d 'Append one record to an append-and-fold store family, read from stdin' complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from record" -f -a "show" -d 'Read one keyed record back: `hit` and the value, or `miss`' diff --git a/completions/batten.zsh b/completions/batten.zsh index cf339d0e2..b68c3b691 100644 --- a/completions/batten.zsh +++ b/completions/batten.zsh @@ -5180,6 +5180,37 @@ trace\:"Add everything"))' \ ':family -- The record family, which is the key a module reads it under:_default' \ && ret=0 ;; +(derive) +_arguments "${_arguments_options[@]}" : \ +'*--input=[A \`=\` input this family needs beyond stdin (repeatable)]: :_default' \ +'--strictness=[Raise how strictly gates apply (an override may only tighten policy)]: :((permissive\:"Advisory\: findings are reported without failing the run" +standard\:"The default\: a finding is a violation" +strict\:"Everything \`Standard\` fails on, plus anything advisory"))' \ +'--config-from=[Read the committed config from a git ref (e.g. origin/main) instead of the working tree]: :_default' \ +'--config-in=[Read the committed config from this directory instead of the directory being judged]: :_default' \ +'--log-level=[Set the verbosity rung by name]: :((silent\:"Say nothing but a verdict or a usage error" +quiet\:"Suppress ordinary progress; keep warnings" +normal\:"The default" +verbose\:"Explain what is being checked" +debug\:"Add resolution detail" +trace\:"Add everything"))' \ +'--fail-on-warning[Promote a warn-severity finding to a violation (an override may only turn this on)]' \ +'*--silent[Say nothing but a verdict or a usage error]' \ +'*-q[Suppress ordinary progress (repeatable\: -qq is silent)]' \ +'*--quiet[Suppress ordinary progress (repeatable\: -qq is silent)]' \ +'*-v[Explain what is being checked (repeatable\: -vv is debug)]' \ +'*--verbose[Explain what is being checked (repeatable\: -vv is debug)]' \ +'*--debug[Add resolution detail]' \ +'*--trace[Add everything]' \ +'--no-color[Never colour stderr, whatever it is attached to]' \ +'--no-input[Never prompt; treat the run as unattended]' \ +'-y[Confirm a destructive operation that would otherwise refuse]' \ +'--yes[Confirm a destructive operation that would otherwise refuse]' \ +'-h[Print help (see more with '\''--help'\'')]' \ +'--help[Print help (see more with '\''--help'\'')]' \ +':family -- The record family, which selects the reading and is the key a module reads it under:_default' \ +&& ret=0 +;; (keyed) _arguments "${_arguments_options[@]}" : \ '--strictness=[Raise how strictly gates apply (an override may only tighten policy)]: :((permissive\:"Advisory\: findings are reported without failing the run" @@ -5384,6 +5415,10 @@ _arguments "${_arguments_options[@]}" : \ _arguments "${_arguments_options[@]}" : \ && ret=0 ;; +(derive) +_arguments "${_arguments_options[@]}" : \ +&& ret=0 +;; (keyed) _arguments "${_arguments_options[@]}" : \ && ret=0 @@ -7207,6 +7242,10 @@ _arguments "${_arguments_options[@]}" : \ _arguments "${_arguments_options[@]}" : \ && ret=0 ;; +(derive) +_arguments "${_arguments_options[@]}" : \ +&& ret=0 +;; (keyed) _arguments "${_arguments_options[@]}" : \ && ret=0 @@ -8796,6 +8835,7 @@ _batten__subcmd__help__subcmd__record_commands() { 'tool:Record a declared tool row'\''s verdict, read as \` \` lines on stdin' \ 'forge:Record the forge'\''s check verdicts for one commit, read as \` \` lines on stdin' \ 'named:Record one named family under this branch, read from stdin' \ +'derive:Derive one named family'\''s record from its input and write it' \ 'keyed:Put one value into a keyed store family, read from stdin' \ 'journal:Append one record to an append-and-fold store family, read from stdin' \ 'show:Read one keyed record back\: \`hit\` and the value, or \`miss\`' \ @@ -8810,6 +8850,11 @@ _batten__subcmd__help__subcmd__record__subcmd__closes_commands() { local commands; commands=() _describe -t commands 'batten help record closes commands' commands "$@" } +(( $+functions[_batten__subcmd__help__subcmd__record__subcmd__derive_commands] )) || +_batten__subcmd__help__subcmd__record__subcmd__derive_commands() { + local commands; commands=() + _describe -t commands 'batten help record derive commands' commands "$@" +} (( $+functions[_batten__subcmd__help__subcmd__record__subcmd__fold_commands] )) || _batten__subcmd__help__subcmd__record__subcmd__fold_commands() { local commands; commands=() @@ -9930,6 +9975,7 @@ _batten__subcmd__record_commands() { 'tool:Record a declared tool row'\''s verdict, read as \` \` lines on stdin' \ 'forge:Record the forge'\''s check verdicts for one commit, read as \` \` lines on stdin' \ 'named:Record one named family under this branch, read from stdin' \ +'derive:Derive one named family'\''s record from its input and write it' \ 'keyed:Put one value into a keyed store family, read from stdin' \ 'journal:Append one record to an append-and-fold store family, read from stdin' \ 'show:Read one keyed record back\: \`hit\` and the value, or \`miss\`' \ @@ -9945,6 +9991,11 @@ _batten__subcmd__record__subcmd__closes_commands() { local commands; commands=() _describe -t commands 'batten record closes commands' commands "$@" } +(( $+functions[_batten__subcmd__record__subcmd__derive_commands] )) || +_batten__subcmd__record__subcmd__derive_commands() { + local commands; commands=() + _describe -t commands 'batten record derive commands' commands "$@" +} (( $+functions[_batten__subcmd__record__subcmd__fold_commands] )) || _batten__subcmd__record__subcmd__fold_commands() { local commands; commands=() @@ -9961,6 +10012,7 @@ _batten__subcmd__record__subcmd__help_commands() { 'tool:Record a declared tool row'\''s verdict, read as \` \` lines on stdin' \ 'forge:Record the forge'\''s check verdicts for one commit, read as \` \` lines on stdin' \ 'named:Record one named family under this branch, read from stdin' \ +'derive:Derive one named family'\''s record from its input and write it' \ 'keyed:Put one value into a keyed store family, read from stdin' \ 'journal:Append one record to an append-and-fold store family, read from stdin' \ 'show:Read one keyed record back\: \`hit\` and the value, or \`miss\`' \ @@ -9976,6 +10028,11 @@ _batten__subcmd__record__subcmd__help__subcmd__closes_commands() { local commands; commands=() _describe -t commands 'batten record help closes commands' commands "$@" } +(( $+functions[_batten__subcmd__record__subcmd__help__subcmd__derive_commands] )) || +_batten__subcmd__record__subcmd__help__subcmd__derive_commands() { + local commands; commands=() + _describe -t commands 'batten record help derive commands' commands "$@" +} (( $+functions[_batten__subcmd__record__subcmd__help__subcmd__fold_commands] )) || _batten__subcmd__record__subcmd__help__subcmd__fold_commands() { local commands; commands=() diff --git a/crates/batten/src/cli.rs b/crates/batten/src/cli.rs index 81aa60801..ec69e1b43 100644 --- a/crates/batten/src/cli.rs +++ b/crates/batten/src/cli.rs @@ -1391,6 +1391,25 @@ pub enum RecordCommand { /// The record family, which is the key a module reads it under. family: String, }, + /// Derive one named family's record from its input and write it. + /// + /// [`RecordCommand::Named`]'s sibling, and the difference is WHERE the + /// reading lives. `Named` takes a verdict a producer already computed, so + /// the reading is whatever wrote to the pipe; this takes the producer's raw + /// input and applies a reading the engine owns and tests. + /// + /// The effects stay in the task: the document arrives on stdin and this + /// spawns nothing (house-style §5). + Derive { + /// The record family, which selects the reading and is the key a module + /// reads the result under. + family: String, + /// The non-document inputs, as `=`, in the order written. + /// + /// Which keys are accepted is the family's own contract; a key no + /// family declares is a usage error rather than a silent default. + inputs: Vec, + }, /// Put one value into a keyed store family (CLOUD-1713). Keyed { /// The store family the record belongs to. @@ -2469,6 +2488,16 @@ fn record_of(matches: &ArgMatches) -> Option { ("named", matches) => Some(RecordCommand::Named { family: matches.get_one::("family")?.clone(), }), + ("derive", matches) => Some(RecordCommand::Derive { + family: matches.get_one::("family")?.clone(), + // `unwrap_or_default` rather than `?`: a family needing no input + // beyond stdin passes the flag never, and an absent repeatable flag + // is an empty selection rather than a parse failure. + inputs: matches + .get_many::("input") + .map(|values| values.cloned().collect()) + .unwrap_or_default(), + }), ("keyed", matches) => Some(RecordCommand::Keyed { family: matches.get_one::("family")?.clone(), key: matches.get_one::("key")?.clone(), diff --git a/crates/batten/src/lib.rs b/crates/batten/src/lib.rs index 617eb62c6..1ac2a8f2e 100644 --- a/crates/batten/src/lib.rs +++ b/crates/batten/src/lib.rs @@ -95,6 +95,7 @@ pub mod pipeline; pub mod policy; pub mod pr_watch; pub mod preset; +pub mod probe_verdict; pub mod provision; pub mod prune; pub mod race; diff --git a/crates/batten/src/probe_verdict.rs b/crates/batten/src/probe_verdict.rs new file mode 100644 index 000000000..bf70bccb9 --- /dev/null +++ b/crates/batten/src/probe_verdict.rs @@ -0,0 +1,146 @@ +//! Which of three things a probe build did, from its exit status and its log +//! (CLOUD-418, CLOUD-1717). +//! +//! # Why this is a reading and not a comparison against zero +//! +//! THE VERDICT IS THE HARNESS'S OWN LINE, NEVER THE EXIT CODE ALONE. `cargo +//! test` exits non-zero for a compile error, an unresolved feature, an absent +//! toolchain and a panic in some other test — every one of which would read as +//! "the probe falsified the assertion" and hand the gate a pass it did not +//! earn. Worse, that pass gets MORE likely as the crate breaks, so a gate +//! written to the obvious shape is loudest exactly when it is lying. +//! +//! # Why the `failures:` listing rather than the per-test line +//! +//! The per-test line is not stable across harness modes: plain prints `test +//! ... FAILED` and `--quiet` prints ` --- FAILED`. The listing is +//! one indented name in both, and anchoring on it is what stops this going +//! quietly could-not-look the day someone adds or drops `--quiet`. +//! +//! # Pointer-only +//! +//! The log carries module bodies and paths; [`verdict`] returns one token and no +//! byte of the log reaches it (non-negotiable rule 4). + +/// The three things a probe build can have done. +/// +/// An enum rather than the record's own strings, so the reading cannot emit a +/// token no module reads: the spelling lives in one place, on [`Verdict::token`]. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Verdict { + /// The build succeeded, so the test stayed green and discriminates nothing. + Passed, + /// The named test RAN and FAILED, which is the discrimination wanted. + Failed, + /// Non-zero for some other reason: could not look. + Unread, +} + +impl Verdict { + /// The record line a module reads this under. + #[must_use] + pub const fn token(self) -> &'static str { + match self { + Verdict::Passed => "probe passed", + Verdict::Failed => "probe failed", + Verdict::Unread => "probe unread", + } + } +} + +/// Classify a probe build from its exit status and the log it wrote. +/// +/// `log` is the TEXT rather than a path, deliberately: the caller owns it, which +/// is what makes "no byte of the log escapes" a property a test can assert. +#[must_use] +pub fn verdict(status: i32, log: &str, test: &str) -> Verdict { + if status == 0 { + return Verdict::Passed; + } + if ran_to_a_failure(log) && listed_as_failing(log, test) { + return Verdict::Failed; + } + Verdict::Unread +} + +/// The harness's own summary line, anchored at column 0. +fn ran_to_a_failure(log: &str) -> bool { + log.lines() + .any(|line| line.starts_with("test result: FAILED")) +} + +/// The name in the `failures:` listing: indented, alone on its line. +/// +/// THE INDENT IS THE WHOLE TEST, and dropping it is what would make this match +/// the summary line, a path, or a name that merely contains this one. +fn listed_as_failing(log: &str, test: &str) -> bool { + log.lines().any(|line| { + let trimmed = line.trim_matches([' ', '\t']); + trimmed == test && line != trimmed + }) +} + +#[cfg(test)] +mod tests { + use super::{Verdict, verdict}; + + const TEST: &str = "no_evaluator_feature_admits_io"; + + #[test] + fn a_probe_build_that_succeeds_classifies_as_passed() { + assert_eq!(verdict(0, "", TEST), Verdict::Passed); + } + + #[test] + fn a_probe_build_that_ran_the_named_test_to_a_failure_classifies_as_failed() { + let harness = "failures:\n no_evaluator_feature_admits_io\n\ntest result: FAILED. 0 passed; 1 failed\n"; + assert_eq!(verdict(101, harness, TEST), Verdict::Failed); + } + + /// THE ARM A GATE WRITTEN TO THE OBVIOUS SHAPE GETS WRONG: non-zero, and no + /// harness line at all. + #[test] + fn a_probe_build_that_failed_to_compile_classifies_as_unread() { + assert_eq!( + verdict(101, "error[E0432]: unresolved import\n", TEST), + Verdict::Unread + ); + } + + /// A different test failed, so the harness says FAILED and the listing names + /// somebody else. Reading the exit code would call this the discrimination + /// this gate is looking for. + #[test] + fn a_probe_build_where_the_named_test_never_ran_classifies_as_unread() { + let other = "failures:\n some_other_test\n\ntest result: FAILED. 3 passed; 1 failed\n"; + assert_eq!(verdict(101, other, TEST), Verdict::Unread); + } + + /// The summary line carries the token too, and it is NOT indented. Without + /// the indent test a log naming nothing would read as the named failure. + #[test] + fn an_unindented_occurrence_of_the_name_is_not_the_listing() { + let flat = format!("test result: FAILED. 0 passed; 1 failed\n{TEST}\n"); + assert_eq!(verdict(101, &flat, TEST), Verdict::Unread); + } + + /// A name that merely CONTAINS the probe's name is a different test. + #[test] + fn a_longer_name_containing_this_one_is_not_this_test() { + let near = "failures:\n no_evaluator_feature_admits_io_at_all\n\ntest result: FAILED. 0 passed; 1 failed\n"; + assert_eq!(verdict(101, near, TEST), Verdict::Unread); + } + + /// Pointer-only (rule 4): the log carries module bodies and paths, and what + /// comes back is one token. + #[test] + fn the_probe_builds_own_output_never_reaches_the_verdict() { + let noisy = "SECRET_MODULE_BODY\nerror: build failed\n"; + let classified = verdict(101, noisy, TEST); + assert_eq!(classified, Verdict::Unread); + assert!( + !classified.token().contains("SECRET_MODULE_BODY"), + "no byte of the probe log reaches the record" + ); + } +} diff --git a/crates/batten/src/record.rs b/crates/batten/src/record.rs index 72490d458..3ea17d29b 100644 --- a/crates/batten/src/record.rs +++ b/crates/batten/src/record.rs @@ -53,6 +53,7 @@ //! is here rather than at the report: a validator's output is the likeliest place //! in this family for a secret to appear. +use std::collections::BTreeMap; use std::io::Read as _; use std::path::{Path, PathBuf}; @@ -206,6 +207,7 @@ pub fn run( crate::cli::RecordCommand::Plan => run_plan(), crate::cli::RecordCommand::Closes => run_closes(overrides), crate::cli::RecordCommand::Named { family } => run_named(&family), + crate::cli::RecordCommand::Derive { family, inputs } => run_derive(&family, &inputs, out), crate::cli::RecordCommand::Keyed { family, key } => run_keyed(&family, &key), crate::cli::RecordCommand::Journal { family } => run_journal(&family), crate::cli::RecordCommand::Show { family, key } => run_keyed_show(&family, &key, out), @@ -522,6 +524,145 @@ fn safe_component(what: &str, value: &str) -> Result { Ok(clean.to_owned()) } +/// The non-document inputs a family was handed, as a key/value map. +/// +/// # Errors +/// +/// A [`UsageError`] for a token carrying no `=`, for an empty key, and for a +/// key given twice — a repeated key is a caller who believes both values are in +/// effect, and silently keeping one would run the reading on an input nobody +/// asked for. +fn derive_inputs(inputs: &[String]) -> Result> { + let mut parsed = BTreeMap::new(); + for token in inputs { + let Some((key, value)) = token.split_once('=') else { + return Err(UsageError::raise(format!( + "record derive: `--input {token}` is not `=`" + ))); + }; + if key.is_empty() { + return Err(UsageError::raise( + "record derive: an input with no key names nothing".to_owned(), + )); + } + if parsed.insert(key.to_owned(), value.to_owned()).is_some() { + return Err(UsageError::raise(format!( + "record derive: input `{key}` was given twice" + ))); + } + } + Ok(parsed) +} + +/// One required input, or a usage error naming what is missing. +fn required_input<'a>( + inputs: &'a BTreeMap, + family: &str, + key: &str, +) -> Result<&'a str> { + inputs.get(key).map(String::as_str).ok_or_else(|| { + UsageError::raise(format!( + "record derive {family}: needs `--input {key}=`" + )) + }) +} + +/// Refuse an input key the family does not read. +/// +/// **A KEY NOBODY READS IS A USAGE ERROR, NEVER A SILENT DEFAULT**, and that is +/// the same property `--rule` has one verb over: a caller who misspells an input +/// would otherwise get a clean exit from a reading that ran on something else. +/// The failure would be invisible precisely because the record still got written. +fn only_these_inputs( + inputs: &BTreeMap, + family: &str, + accepted: &[&str], +) -> Result<()> { + for key in inputs.keys() { + if !accepted.contains(&key.as_str()) { + return Err(UsageError::raise(format!( + "record derive {family}: reads no input `{key}`" + ))); + } + } + Ok(()) +} + +/// Derive one family's record from its input and write it. +/// +/// The engine applies the READING; the effects that produced the input stay in +/// the producer task (house-style §5), so nothing here spawns. +/// +/// # Errors +/// +/// A [`UsageError`] for an unknown family, a malformed or missing input, a +/// family that is not a single path component, a repository with no branch to +/// key on, or a tree that is not a repository; an internal error when the store +/// cannot be written. +pub fn run_derive( + family: &str, + inputs: &[String], + out: &mut dyn std::io::Write, +) -> Result { + let inputs = derive_inputs(inputs)?; + let derived = match family { + "evaluator-io-probe" => { + only_these_inputs(&inputs, family, &["status", "test"])?; + let raw = required_input(&inputs, family, "status")?; + let status: i32 = raw.parse().map_err(|_| { + UsageError::raise(format!( + "record derive {family}: status `{raw}` is not a whole number" + )) + })?; + let test = required_input(&inputs, family, "test")?; + let log = verdict_lines()?; + format!( + "{}\n", + crate::probe_verdict::verdict(status, &log, test).token() + ) + } + // AN UNKNOWN FAMILY IS A USAGE ERROR, never a record written under a name + // nothing reads. A producer whose family was renamed would otherwise go on + // writing happily into a key no module has looked at since. + _ => { + return Err(UsageError::raise(format!( + "record derive: no reading is declared for family `{family}`" + ))); + } + }; + + let family = safe_component("family", family)?; + let root = Path::new("."); + let git_dir = git::git_dir(root).map_err(|_| { + UsageError::raise( + "record derive: not a git repository, so there is nothing to key on".to_owned(), + ) + })?; + let Ok(Some(branch)) = git::current_branch(root) else { + return Err(UsageError::raise( + "record derive: a detached HEAD has no branch to key the record on".to_owned(), + )); + }; + let claim = claim_of(&git_dir, &branch); + store( + &crate::recorder::record_path(&git_dir, &family, &branch, claim.as_deref()), + &derived, + )?; + + // THE DERIVED RECORD GOES TO STDOUT TOO, and it stays pointer-only doing it: + // what is emitted is the READING — a bounded set of tokens this verb + // computed — never a byte of the input it read. That distinction is why + // `record named` prints nothing and this does: `named` cannot tell a verdict + // from a payload, because it never looked at one. + // + // It matters beyond symmetry. A producer task composes: `evaluator-io-record` + // branches on `probe failed` to mint its step receipt, and a verb that + // swallowed its own answer would force the task to read the record store back + // — a second reader of a path `recorder::record_path` is the one authority on. + out.write_all(derived.as_bytes())?; + Ok(ExitCode::Success) +} + /// Record one named family under this branch, read from stdin. /// /// **The POLICY-readable store, which is a different store from the two below.** diff --git a/crates/batten/src/spec.rs b/crates/batten/src/spec.rs index 9c102cc2e..5baec6362 100644 --- a/crates/batten/src/spec.rs +++ b/crates/batten/src/spec.rs @@ -1033,6 +1033,11 @@ mod tests { // a third row spelled the old way would be a third row to invert. "record".to_owned(), "record closes".to_owned(), + // CLOUD-1717's READING door, and the write band is where it belongs: + // it applies a reading the engine owns and WRITES the result, so it + // is absent from the read-only allowlist above for `record named`'s + // reason. The spawn its input comes from stays in the task. + "record derive".to_owned(), // The two READ leaves of this noun (CLOUD-1713). They fold and // print what the write leaves already stored, which is why they — // alone under `record` — are also on the read-only allowlist above. diff --git a/crates/batten/src/surface.rs b/crates/batten/src/surface.rs index ed05fa30d..bdf18ce86 100644 --- a/crates/batten/src/surface.rs +++ b/crates/batten/src/surface.rs @@ -1079,6 +1079,34 @@ const VERDICT_TOKEN: FlagDecl = FlagDecl { /// filter that matched nothing and exited 0 is the vacuous pass in its purest /// form: the caller would read "the gate passed" from a gate that was never /// selected, and a renamed row would silently stop being enforced. +/// The non-document inputs a `record derive` family needs, as `=`. +/// +/// ONE DECLARED FLAG RATHER THAN A COLUMN PER FAMILY, and the alternative is +/// what makes this the narrow choice: `--status`, `--test`, `--root`, +/// `--threshold`, `--exclude`, `--signingkey` and `--ssh-program` on one leaf +/// would declare seven flags of which every caller uses at most three, and each +/// new producer would widen the committed surface again. The keys a family +/// accepts are that family's own contract, documented on its reader, and a key +/// no family declares is a usage error rather than a silent default. +/// +/// `StrMany` for `LAND_RESOLVE`'s reason: `Str` keeps only the LAST occurrence, +/// so a caller naming two inputs would silently lose the first — and a producer +/// running on a partial input is precisely the could-not-look this family's +/// three-valued read exists to keep loud. +const DERIVE_INPUT: FlagDecl = FlagDecl { + id: "input", + long: Some("input"), + short: None, + help: "A `=` input this family needs beyond stdin (repeatable)", + env: EnvDecl::None, + global: false, + positional: false, + required: false, + hidden: false, + rung: Rung::None, + value: ValueDecl::StrMany, +}; + const CHECK_RULE: FlagDecl = FlagDecl { id: "rule", long: Some("rule"), @@ -4962,6 +4990,36 @@ pub const SURFACE: &[CommandDecl] = &[ "The record family, which is the key a module reads it under", )], }, + // CLOUD-1717's READING door, and it is the other half of `record named` + // rather than a second spelling of it. `record named` takes a verdict a + // producer already computed; this takes the producer's RAW INPUT and applies + // the reading itself, so the reading is Rust the engine tests rather than a + // script beside the task. + // + // THE SPAWN STAYS OUTSIDE, which is what makes this admissible at all. + // House-style §5 puts a producer's `cargo metadata` or `cargo test` in the + // task; the document arrives on stdin and the engine never executes + // anything. `spawn-adapters` and `spawn-widening` both refuse a new + // `Command` site under `crates/batten/src/**` with no override route, and + // they are right to: a reading that needed one would have the wrong shape. + // + // `Effect::Write`, never `Read`: the agent read-only allowlist is DERIVED + // from `effect == read` (`spec.rs`), and this writes a record. + CommandDecl { + path: "record derive", + id: "record.derive", + about: "Derive one named family's record from its input and write it", + data_channel: false, + exits: EXITS_STANDARD, + effect: Effect::Write, + flags: &[ + FlagDecl::positional( + "family", + "The record family, which selects the reading and is the key a module reads it under", + ), + DERIVE_INPUT, + ], + }, CommandDecl { path: "record keyed", id: "record.keyed", diff --git a/crates/batten/tests/it/evaluator_io_probe.rs b/crates/batten/tests/it/evaluator_io_probe.rs index 4f5eef131..0a24d871c 100644 --- a/crates/batten/tests/it/evaluator_io_probe.rs +++ b/crates/batten/tests/it/evaluator_io_probe.rs @@ -7,7 +7,7 @@ //! fabricates its input with `with input as`, which is the shape //! `rules/policy-modules.md` warns about. //! -//! # And why the PRODUCER's classification is driven here +//! # And why the PRODUCER's reading is driven here //! //! Three of the dying suite's five cases are about how the probe build's output //! is READ, not about the verdict that follows: a build that passed, one that @@ -17,18 +17,27 @@ //! exit from `cargo test` means a compile error just as readily as a falsified //! assertion. //! -//! `EVALUATOR_IO_PROBE_CMD` is what makes that drivable without a two-minute -//! rebuild per case, and these cases use it exactly as the retired suite did. +//! Those three are now pinned where they belong: `crates/batten/src/probe_verdict.rs` +//! carries the classification and asserts all of them — plus two the retired +//! program never had, an unindented occurrence of the name and a longer name +//! containing it — in its own `#[cfg(test)] mod tests`, over fabricated +//! `(status, log)` pairs no build could be made to produce. +//! +//! What stays HERE is the half a unit test cannot reach: that the engine carries +//! that reading through `record derive` into a record the real module then +//! refuses over. The `--input` seam makes it drivable without a two-minute +//! rebuild per case, exactly as `EVALUATOR_IO_PROBE_CMD` did for the retired +//! program. //! //! # RETIREMENT LEDGER, PER PATH — what `shell retire partial` reads //! // carried: mise-tasks/evaluator-io-check.sh policy/evaluator-io-probe.rego kind:mechanism crates/batten/tests/it/evaluator_io_probe.rs // carried: tests/evaluator-io-check.bats policy/evaluator-io-probe.rego kind:mechanism crates/batten/tests/it/evaluator_io_probe.rs -// carried: "a probe build in which the test PASSES is the finding, not a pass" policy/evaluator-io-probe.rego kind:mechanism -// carried: "a probe build in which the test FAILS is the pass" policy/evaluator-io-probe.rego kind:mechanism -// carried: "a probe build that failed to COMPILE is could-not-look, not the pass" policy/evaluator-io-probe.rego kind:mechanism -// carried: "a probe build where the named test never ran is could-not-look" policy/evaluator-io-probe.rego kind:mechanism -// carried: "the probe build's own output never reaches the gate's output" policy/evaluator-io-probe.rego kind:mechanism +// carried: "a probe build in which the test PASSES is the finding, not a pass" crates/batten/src/probe_verdict.rs kind:mechanism crates/batten/tests/it/evaluator_io_probe.rs +// carried: "a probe build in which the test FAILS is the pass" crates/batten/src/probe_verdict.rs kind:mechanism crates/batten/tests/it/evaluator_io_probe.rs +// carried: "a probe build that failed to COMPILE is could-not-look, not the pass" crates/batten/src/probe_verdict.rs kind:mechanism crates/batten/tests/it/evaluator_io_probe.rs +// carried: "a probe build where the named test never ran is could-not-look" crates/batten/src/probe_verdict.rs kind:mechanism crates/batten/tests/it/evaluator_io_probe.rs +// carried: "the probe build's own output never reaches the gate's output" crates/batten/src/probe_verdict.rs kind:mechanism crates/batten/tests/it/evaluator_io_probe.rs // changed: "the refusal names the test to fix" batten.toml the retired program printed ` `; the engine renders ` `, because `rules/policy-modules.md` makes the first path-bearing subject the finding's pointer whatever order the subjects are declared in. That is non-negotiable rule 5 — one output contract, no per-verb exception — so the test name moved to the `[[verdict]]` row's gloss and to the JSON channel, where a reader still meets it // Panicking on setup failure is the idiomatic way for a test to fail loudly. @@ -179,73 +188,168 @@ fn an_absent_record_says_nothing_rather_than_refusing() { ); } -// --- the producer's classification ------------------------------------------- +// --- the producer's reading, over the real verb --------------------------------- -/// Run the REAL reading the producer runs, over a status and a log. +/// Drive the REAL reading the producer runs, through the REAL verb. +/// +/// `crates/batten/src/probe_verdict.rs` is the one authority on the three +/// branches, and its own `#[cfg(test)] mod tests` asserts them directly over +/// fabricated `(status, log)` pairs — that is where the classification is +/// pinned, because a unit test can produce an arbitrary pair and a build cannot. /// -/// `mise-tasks/probe_verdict.py` is the one authority on these three branches; -/// a copy of them here would be the second authority `cargo_graph.py` was -/// extracted to remove one gate over. -#[expect( - clippy::disallowed_types, - reason = "stays, and it is the subject under test rather than a convenience: the classification of a probe build's output is what moved out of the dying program, and driving it directly is what keeps the three arms assertable without a two-minute rebuild per case" -)] -fn classify(name: &str, status: i32, log: &str) -> String { - let dir = scratch(&format!("evaluator-io-stub-{name}")); - write(&dir, "probe.log", log); - - let done = std::process::Command::new("python3") - .arg("../../mise-tasks/probe_verdict.py") - .arg(status.to_string()) - .arg(dir.join("probe.log")) - .arg("no_evaluator_feature_admits_io") - .output() - .expect("the reading runs"); +/// What THIS tier adds is the half a unit test cannot reach: that the engine +/// carries the reading all the way to a record a module then refuses over. The +/// `--input` seam is what makes that drivable without a two-minute rebuild per +/// case, exactly as `EVALUATOR_IO_PROBE_CMD` did for the retired program. +fn derive(dir: &std::path::Path, status: i32, log: &str) -> std::process::Output { + run_with_stdin( + dir, + &[ + "record", + "derive", + "evaluator-io-probe", + "--input", + &format!("status={status}"), + "--input", + "test=no_evaluator_feature_admits_io", + ], + log, + ) +} + +#[test] +fn the_verb_derives_a_green_probe_into_the_finding() { + // END TO END, and it is the arm no unit test reaches: a probe build that + // SUCCEEDED is derived, written, read back by the real module, and refused. + let dir = repo("derive-passed"); + let written = derive(&dir, 0, ""); assert!( - done.status.success(), - "the reading completes: {}", - String::from_utf8_lossy(&done.stderr) + written.status.success(), + "the derivation lands: {}", + String::from_utf8_lossy(&written.stderr) + ); + + let decided = run(&dir, &["check"]); + assert_eq!( + decided.status.code(), + Some(2), + "a green probe is the finding\n{}", + String::from_utf8_lossy(&decided.stderr) ); - String::from_utf8_lossy(&done.stdout).trim().to_owned() } #[test] -fn a_probe_build_that_succeeds_classifies_as_passed() { - assert_eq!(classify("green", 0, ""), "probe passed"); +fn the_verb_derives_a_compile_failure_into_could_not_look() { + // THE ARM A GATE WRITTEN TO THE OBVIOUS SHAPE GETS WRONG, carried all the + // way through: non-zero, no harness line, and the engine must still refuse + // rather than read the exit code as the discrimination it wanted. + let dir = repo("derive-unread"); + let written = derive(&dir, 101, "error[E0432]: unresolved import\n"); + assert!(written.status.success(), "the derivation lands"); + + let decided = run(&dir, &["check"]); + assert_eq!( + decided.status.code(), + Some(2), + "could not look is loud, never the pass\n{}", + String::from_utf8_lossy(&decided.stderr) + ); } #[test] -fn a_probe_build_that_ran_the_named_test_to_a_failure_classifies_as_failed() { +fn the_verb_derives_a_real_failure_into_the_pass() { + let dir = repo("derive-failed"); let harness = "failures:\n no_evaluator_feature_admits_io\n\ntest result: FAILED. 0 passed; 1 failed\n"; - assert_eq!(classify("red", 101, harness), "probe failed"); + let written = derive(&dir, 101, harness); + assert!(written.status.success(), "the derivation lands"); + + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "it failed, which is the pass\n{}", + String::from_utf8_lossy(&quiet.stderr) + ); } +/// POINTER-ONLY THROUGH THE WHOLE PATH (rule 4). The unit test asserts the +/// verdict carries no log byte; this asserts the RECORD does not either, which +/// is the claim that matters — the record is a file on disk a module reads. #[test] -fn a_probe_build_that_failed_to_compile_classifies_as_unread() { - // THE ARM A GATE WRITTEN TO THE OBVIOUS SHAPE GETS WRONG: non-zero, and no - // harness line at all. - let broken = "error[E0432]: unresolved import\n"; - assert_eq!(classify("compile", 101, broken), "probe unread"); +fn no_byte_of_the_probe_log_reaches_the_record() { + let dir = repo("derive-noisy"); + let written = derive(&dir, 101, "SECRET_MODULE_BODY\nerror: build failed\n"); + assert!(written.status.success(), "the derivation lands"); + + // Every byte the engine wrote under the scratch repository's git dir, + // walked rather than guessed at: the record's exact path is + // `recorder::record_path`'s business, and a test naming it would be a + // second authority over where records live. + let mut stored = String::new(); + let mut pending = vec![dir.join(".git")]; + while let Some(next) = pending.pop() { + let Ok(entries) = std::fs::read_dir(&next) else { + continue; + }; + for entry in entries.flatten() { + let path = entry.path(); + if path.is_dir() { + pending.push(path); + } else if let Ok(body) = std::fs::read_to_string(&path) { + stored.push_str(&body); + } + } + } + assert!( + !stored.contains("SECRET_MODULE_BODY"), + "no byte of the probe log reaches anything the engine wrote\n{stored}" + ); + assert!( + !said(&written).contains("SECRET_MODULE_BODY"), + "nor the verb's own output\n{}", + said(&written) + ); } +/// A KEY THE FAMILY DOES NOT READ IS A USAGE ERROR, never a silent default. +/// A caller who misspells an input would otherwise get a clean exit from a +/// reading that ran on something else, and the record would still be written. #[test] -fn a_probe_build_where_the_named_test_never_ran_classifies_as_unread() { - // A different test failed, so the harness says FAILED and the listing names - // somebody else. Reading the exit code would call this the discrimination - // this gate is looking for. - let other = "failures:\n some_other_test\n\ntest result: FAILED. 3 passed; 1 failed\n"; - assert_eq!(classify("other", 101, other), "probe unread"); +fn an_input_key_the_family_does_not_read_is_a_usage_error() { + let dir = repo("derive-unknown-input"); + let refused = run_with_stdin( + &dir, + &[ + "record", + "derive", + "evaluator-io-probe", + "--input", + "status=0", + "--input", + "test=x", + "--input", + "nonsense=1", + ], + "", + ); + assert_eq!( + refused.status.code(), + Some(1), + "an unread input is a usage error\n{}", + said(&refused) + ); } +/// A family with no declared reading is a usage error too, rather than a record +/// written under a name nothing reads. #[test] -fn the_probe_builds_own_output_never_reaches_the_record() { - // Pointer-only (rule 4): the probe log carries module bodies and paths, and - // what the record receives is one token. - let noisy = "SECRET_MODULE_BODY\nerror: build failed\n"; - let classified = classify("noisy", 101, noisy); - assert_eq!(classified, "probe unread"); - assert!( - !classified.contains("SECRET_MODULE_BODY"), - "no byte of the probe log reaches the record\n{classified}" +fn a_family_with_no_declared_reading_is_a_usage_error() { + let dir = repo("derive-unknown-family"); + let refused = run_with_stdin(&dir, &["record", "derive", "no-such-family"], ""); + assert_eq!( + refused.status.code(), + Some(1), + "an undeclared family is a usage error\n{}", + said(&refused) ); } diff --git a/crates/batten/tests/it/pointer_only.rs b/crates/batten/tests/it/pointer_only.rs index a3bb9a572..358cb5e3c 100644 --- a/crates/batten/tests/it/pointer_only.rs +++ b/crates/batten/tests/it/pointer_only.rs @@ -1930,6 +1930,17 @@ const CENSUS: &[Verb] = &[ stdin: Stdin::ToolVerdict, disposition: Disposition::PointerOnly, }, + // CLOUD-1717's reading door. Pointer-only matters MORE here than for the + // generic writers above, not less: the input it reads is the richest payload + // any producer hands the engine — a whole `cargo metadata` document, a probe + // build's log — and the record it writes is a handful of tokens. A verb that + // echoed any of it would put a build log into a file a module reads. + Verb { + path: "record derive", + args: &["census"], + stdin: Stdin::ToolVerdict, + disposition: Disposition::PointerOnly, + }, Verb { path: "record keyed", args: &["census", "a-key"], diff --git a/crates/batten/tests/it/snapshots/it__snapshots__golden_json_schema.snap b/crates/batten/tests/it/snapshots/it__snapshots__golden_json_schema.snap index 79e809a1b..42696356b 100644 --- a/crates/batten/tests/it/snapshots/it__snapshots__golden_json_schema.snap +++ b/crates/batten/tests/it/snapshots/it__snapshots__golden_json_schema.snap @@ -2514,6 +2514,32 @@ expression: stdout_of(&output) "flags": [], "subcommands": [] }, + { + "path": "record derive", + "id": "record.derive", + "about": "Derive one named family's record from its input and write it", + "effect": "write", + "data_channel": false, + "flags": [ + { + "name": "family", + "short": null, + "long": null, + "takes_value": true, + "positional": true, + "help": "The record family, which selects the reading and is the key a module reads it under" + }, + { + "name": "input", + "short": null, + "long": "input", + "takes_value": true, + "positional": false, + "help": "A `=` input this family needs beyond stdin (repeatable)" + } + ], + "subcommands": [] + }, { "path": "record fold", "id": "record.fold", diff --git a/man/batten-record-derive.1 b/man/batten-record-derive.1 new file mode 100644 index 000000000..9571725f7 --- /dev/null +++ b/man/batten-record-derive.1 @@ -0,0 +1,19 @@ +.ie \n(.g .ds Aq \(aq +.el .ds Aq ' +.TH batten-record-derive 1 batten +.SH NAME +batten\-record\-derive \- Derive one named family\*(Aqs record from its input and write it +.SH SYNOPSIS +\fBbatten record derive\fR [\fB\-\-input\fR] [\fB\-h\fR|\fB\-\-help\fR] <\fIfamily\fR> +.SH DESCRIPTION +Derive one named family\*(Aqs record from its input and write it +.SH OPTIONS +.TP +\fB\-\-input\fR +A `=` input this family needs beyond stdin (repeatable) +.TP +\fB\-h\fR, \fB\-\-help\fR +Print help +.TP +<\fIfamily\fR> +The record family, which selects the reading and is the key a module reads it under diff --git a/man/batten-record.1 b/man/batten-record.1 index 210ca39c2..9c7ef5fd1 100644 --- a/man/batten-record.1 +++ b/man/batten-record.1 @@ -22,6 +22,9 @@ Record the forge\*(Aqs check verdicts for one commit, read as ` ... FAILED` and -# `--quiet` prints ` --- FAILED`. The listing is one indented name in both, -# and anchoring on it is what stops this going quietly could-not-look the day -# someone adds or drops `--quiet`. -# -# Emits exactly one of: -# -# probe passed the build succeeded — the test stayed green with `http` on -# probe failed the named test RAN and FAILED — the discrimination wanted -# probe unread non-zero for some other reason — could not look -# -# POINTER-ONLY (rule 4): one token. The log carries module bodies and paths and -# no byte of it is emitted. - -import re -import sys - - -def verdict(status, log, test): - if status == 0: - # A probe build that SUCCEEDED means the test stayed green with `http` - # on, so it discriminates nothing. - return "probe passed" - listed = re.compile(r"^[ \t]+%s[ \t]*$" % re.escape(test), re.MULTILINE) - if re.search(r"^test result: FAILED", log, re.MULTILINE) and listed.search(log): - return "probe failed" - return "probe unread" - - -def main(argv): - if len(argv) != 4: - sys.stderr.write("usage: probe_verdict.py \n") - return 2 - try: - status = int(argv[1]) - except ValueError: - sys.stderr.write("probe-verdict: status must be a whole number\n") - return 2 - try: - with open(argv[2], "r", encoding="utf-8", errors="replace") as handle: - log = handle.read() - except OSError as failed: - sys.stderr.write("probe-verdict: could not read the log: %s\n" % failed) - return 2 - sys.stdout.write(verdict(status, log, argv[3]) + "\n") - return 0 - - -if __name__ == "__main__": - sys.exit(main(sys.argv)) diff --git a/mise.toml b/mise.toml index 0055c459d..bcad27a5d 100644 --- a/mise.toml +++ b/mise.toml @@ -2268,9 +2268,13 @@ trap 'rm -f "$log"' EXIT status=0 "$@" >"$log" 2>&1 || status=$? -verdict=$(python3 mise-tasks/probe_verdict.py "$status" "$log" "$probe_test") || exit 0 +# The READING is the engine's (CLOUD-1717): the spawn above stays here, +# where house-style §5 puts a producer's effects, and the classification of +# what it wrote is `crates/batten/src/probe_verdict.rs` — Rust the engine +# tests rather than a script beside this task. +verdict=$(cargo run --quiet -p batten -- record derive evaluator-io-probe \ + --input "status=$status" --input "test=$probe_test" <"$log") || exit 0 -printf '%s\\n' "$verdict" | cargo run --quiet -p batten -- record named evaluator-io-probe if [ -z "${EVALUATOR_IO_PROBE_CMD:-}" ] && [ "$verdict" = 'probe failed' ]; then ./mise-tasks/step-receipt.sh record evaluator-io-check || true diff --git a/policy/module-layering.rego b/policy/module-layering.rego index d2a0a8361..bb58f4be7 100644 --- a/policy/module-layering.rego +++ b/policy/module-layering.rego @@ -533,6 +533,23 @@ declared_modules := { # `error` and `exit`, the codes its report answers in. Its back-edges are # forbidden below for the acquisition modules' reason. "task", + # `probe_verdict` arrived with CLOUD-1717 and this rule named it a + # seventeenth time, which is the coverage clause working once more: the + # module was written, its unit tests and its tier were green, clippy was + # clean, and this is what said nobody had placed it. + # + # It is `environment`'s class and reaches NOTHING — not `error`, not `exit`. + # It is one total function from `(i32, &str, &str)` to a three-valued enum, + # with no failure to report: a log it cannot make sense of is `Unread`, which + # is the could-not-look the caller already has to handle. A `Result` here + # would add a state whose only honest handling is the one `Unread` gets. + # + # THE PLACEMENT IS WHY THE READING MOVED HERE AT ALL. It was a `.py` beside + # the task, where the classification of a probe build's output was a second + # authority nothing in this crate could test. Reaching nothing is what lets + # `record` call it without the producer's effects reaching the engine: the + # spawn stays in the task (house-style §5) and only the reading is here. + "probe_verdict", } # THE FORBIDDEN EDGES, each traceable to prose already in the tree. From cc46fbfb8b1a854191ed8238f92c1e7cd4a0d9fe Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sat, 19 Sep 2026 21:31:13 +0000 Subject: [PATCH 25/32] feat(record): derive the signer posture in Rust, and stop classifying twice MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Second of six. `mise-tasks/signer_posture.py` decided which git signing configurations are unverifiable — four file tests rather than one, because a size test alone calls a DIRECTORY healthy, which was a measured defect and not thoroughness. It is now `crates/batten/src/signer_posture.rs`, with fifteen in-module cases. Two are new and neither existed in the retired program: an unset `user.signingkey` is not a finding, and a signer merely NAMED `/tmpfoo` is not under `/tmp/` — a prefix test without the separator would have called a healthy configuration broken. The readability test opens the file rather than comparing permission bits, because ACLs, capabilities and running as root all make the bits and the outcome disagree, and the outcome is the condition being named. The module also owns the RECORD'S SHAPE, which was a sequence of `printf` calls in a task body that nothing tested — including truncating each sha to eight characters, the difference between a pointer and a payload (rule 4). The producer still gathers the facts: `git config`, `git rev-list` and `git cat-file` are spawns §5 keeps outside the engine, and what they MEAN is composed here. `signing-posture-repair` was the second caller of the dying script and classified a second time. It now reads the posture off the record the producer just wrote, so the two cannot disagree about a checkout they both looked at. It reaches nothing and never runs `git config` itself: both values arrive as `&str`, which is what keeps the reading testable against a scratch path and a developer's real configuration out of the tests (CLOUD-591's boundary). Tier loses its `python3` spawn and its `#[expect(clippy::disallowed_types)]` row, keeps its eight decision cases, gains four end-to-end ones. Full suite 5688/5688. `shell retire partial` now names four `.py`. Refs: CLOUD-1717 Admits-Rule: protected-mutation Admits-Verdict: path write refused Admits-Subject: .serena/memories/core.md Admits-Id: d002597cdeeee4ebe806ff4834ca4abb09cc1eea6f0e33d1ca906cac62c6746c --- .serena/memories/core.md | 21 ++ crates/batten/src/lib.rs | 1 + crates/batten/src/record.rs | 16 ++ crates/batten/src/signer_posture.rs | 331 ++++++++++++++++++++++ crates/batten/tests/it/signing_posture.rs | 223 ++++++++------- mise-tasks/signer_posture.py | 89 ------ mise.toml | 73 +++-- policy/module-layering.rego | 13 + 8 files changed, 550 insertions(+), 217 deletions(-) create mode 100644 crates/batten/src/signer_posture.rs delete mode 100644 mise-tasks/signer_posture.py diff --git a/.serena/memories/core.md b/.serena/memories/core.md index 46a3a9c09..f423101d5 100644 --- a/.serena/memories/core.md +++ b/.serena/memories/core.md @@ -2448,6 +2448,27 @@ judge_fingerprint`, its own domain tag), so a caller can reference content it appearing after the record moves the key instead of being invisible. Every failure is could-not-look, which allows; a fact naming every program in a project must never refuse on a failure to see. +- `signer_posture.rs` — whether a git signing configuration names a key anyone + can verify (CLOUD-669, CLOUD-1717). NOT AN ARGUMENT AGAINST SIGNING: signing + in CI with a published public half is the desired end state and CLOUD-591 owns + getting there. What this names is narrower — a signature from a key that + cannot be verified or reproduced, which is WORSE than no signature because it + looks like provenance and carries none. Two measured conditions: a + `gpg.ssh.program` under `/tmp`, which the container reclaims, and a + `user.signingkey` naming a file that is empty, unreadable, absent or a + DIRECTORY. Four file tests rather than one, because a size test alone calls a + directory healthy — a measured defect, not thoroughness. A literal inline key + is not a path and is the most publishable form there is, so it short-circuits + before any file test; the `/tmp` test outranks everything, because a reclaimed + signer breaks verification whatever the key is. It also owns the RECORD'S + SHAPE, including truncating each sha to eight characters — the difference + between a pointer and a payload (rule 4) — which was a sequence of `printf` + calls in a task body that nothing tested. It reaches NOTHING and never runs + `git config`: the two values arrive as `&str`, which keeps the reading + testable against a scratch path and a developer's real configuration out of + the tests. Ported off `mise-tasks/signer_posture.py`; `signing-posture-repair` + no longer classifies a second time but reads the posture off the record the + producer just wrote. - `probe_verdict.rs` — which of three things a probe build did, from its exit status and its log (CLOUD-418, CLOUD-1717). THE VERDICT IS THE HARNESS'S OWN LINE, NEVER THE EXIT CODE ALONE: `cargo test` exits non-zero for a compile diff --git a/crates/batten/src/lib.rs b/crates/batten/src/lib.rs index 1ac2a8f2e..88d13a17e 100644 --- a/crates/batten/src/lib.rs +++ b/crates/batten/src/lib.rs @@ -121,6 +121,7 @@ pub mod selfwrite; pub mod semver; pub mod session; pub mod severity; +pub mod signer_posture; pub mod sink; pub mod source; pub mod spec; diff --git a/crates/batten/src/record.rs b/crates/batten/src/record.rs index 3ea17d29b..08b1c5341 100644 --- a/crates/batten/src/record.rs +++ b/crates/batten/src/record.rs @@ -621,6 +621,22 @@ pub fn run_derive( crate::probe_verdict::verdict(status, &log, test).token() ) } + "signing-posture" => { + only_these_inputs( + &inputs, + family, + &["signingkey", "ssh-program", "gpgsign", "signed"], + )?; + let signingkey = required_input(&inputs, family, "signingkey")?; + let program = required_input(&inputs, family, "ssh-program")?; + // THE TWO ABSENT-MEANS-NOTHING INPUTS. A producer that found no + // conflict and no signed commit still sends both, empty; treating an + // omitted input as "no" here would make "the producer did not look" + // and "the producer looked and found none" the same record. + let conflict = required_input(&inputs, family, "gpgsign")? == "conflict"; + let signed = required_input(&inputs, family, "signed")?; + crate::signer_posture::record(signingkey, program, conflict, signed) + } // AN UNKNOWN FAMILY IS A USAGE ERROR, never a record written under a name // nothing reads. A producer whose family was renamed would otherwise go on // writing happily into a key no module has looked at since. diff --git a/crates/batten/src/signer_posture.rs b/crates/batten/src/signer_posture.rs new file mode 100644 index 000000000..39a10cacd --- /dev/null +++ b/crates/batten/src/signer_posture.rs @@ -0,0 +1,331 @@ +//! Whether a git signing configuration names a key anyone can verify +//! (CLOUD-669, CLOUD-1717). +//! +//! # This is not an argument against signing +//! +//! SIGNING IS GOOD, and a reading of this module that says otherwise is wrong: +//! signing in CI, with a key whose public half is published, is the desired end +//! state and CLOUD-591 owns getting there. What this names is the narrower +//! thing — a signature produced by a key that cannot be verified or reproduced, +//! which is WORSE than no signature because it looks like provenance and +//! carries none. +//! +//! # Two independent conditions, both measured rather than assumed (2026-08-18) +//! +//! * `gpg.ssh.program` resolves inside `/tmp`. The container reclaims it, so the +//! signer and whatever key it holds are not reproducible across sessions. A +//! signature nobody can re-verify later is provenance theatre. +//! * `user.signingkey` names a file that is empty or unreadable. The public half +//! cannot be read, so no `allowed_signers` entry can be derived from it and +//! nothing downstream can check the signature. +//! +//! A signer failing NEITHER test is left alone and signing stays on. +//! +//! # A literal key is not a path +//! +//! With `gpg.format ssh`, git accepts the public key inline (`ssh-ed25519 +//! AAAA…`) or behind a `key::` prefix as well as a filename. A literal is the +//! MOST publishable form there is — it is already the public half — so testing +//! it as a file would report the healthiest possible configuration as broken. +//! +//! # Four file tests, not one, and that was a real defect rather than thoroughness +//! +//! A size test alone is true for anything non-empty that `stat` can size, +//! including an unreadable file and a DIRECTORY — both of which leave the public +//! half unreadable, which is the condition being named. Each test carries its +//! own reason, so the refusal says which one. +//! +//! # The two config values arrive as arguments +//! +//! This module never runs `git config`. The producer task reads both values and +//! hands them over, which is what keeps the reading testable against a scratch +//! path and keeps a developer's real configuration out of the tests. + +use std::path::Path; + +/// The spellings git accepts for a key given INLINE rather than as a path. +const LITERAL_PREFIXES: [&str; 4] = ["ssh-", "key::", "sk-ssh-", "sk-ecdsa-"]; + +/// The directory a container reclaims between sessions. +const RECLAIMED_PREFIX: &str = "/tmp/"; + +/// What a signing configuration amounts to. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Posture { + /// Nothing about this configuration stops anyone verifying a signature. + Verifiable, + /// A signature from this configuration cannot be verified, for this reason. + Broken(&'static str), +} + +impl Posture { + /// The record line a module reads this under. + #[must_use] + pub fn token(&self) -> String { + match self { + Posture::Verifiable => "verifiable".to_owned(), + Posture::Broken(why) => format!("broken {why}"), + } + } +} + +/// Classify a signing configuration from the two git config values. +/// +/// ORDER IS LOAD-BEARING. The `/tmp` test comes first because a reclaimed +/// signer makes the signature unverifiable whatever the key is — including a +/// perfectly healthy inline one — and the literal test comes before every file +/// test because a literal is not a path and stating it as one would report the +/// healthiest configuration as broken. +#[must_use] +pub fn posture(signingkey: &str, program: &str) -> Posture { + if program.starts_with(RECLAIMED_PREFIX) { + return Posture::Broken( + "the signer resolves inside /tmp, which the container reclaims, so the key is not \ + reproducible", + ); + } + if signingkey.is_empty() + || LITERAL_PREFIXES + .iter() + .any(|prefix| signingkey.starts_with(prefix)) + { + return Posture::Verifiable; + } + + let path = Path::new(signingkey); + let Ok(found) = path.metadata() else { + return Posture::Broken( + "user.signingkey names a path that does not exist, so the public half cannot be read \ + or published", + ); + }; + if !found.is_file() { + return Posture::Broken( + "user.signingkey names something that is not a regular file, so the public half \ + cannot be read or published", + ); + } + // OPENING IT IS THE READ TEST. A permission-bit comparison would be a second + // authority over what this process may read — ACLs, capabilities and running + // as root all make the bits and the outcome disagree, and the outcome is the + // condition being named. + if std::fs::File::open(path).is_err() { + return Posture::Broken( + "user.signingkey names a file this checkout cannot read, so the public half cannot be \ + read or published", + ); + } + if found.len() == 0 { + return Posture::Broken( + "user.signingkey names an empty file, so the public half cannot be read or published", + ); + } + Posture::Verifiable +} + +/// Compose the whole `signing-posture` record. +/// +/// THE RECORD'S SHAPE IS A READING TOO, and it used to be a sequence of `printf` +/// calls in a task body that nothing tested — including the truncation of each +/// sha to eight characters, which is the difference between a pointer and a +/// payload (rule 4). The producer still gathers the facts, because `git config` +/// and `git rev-list` are spawns and house-style §5 keeps those outside the +/// engine; what they MEAN is composed here. +/// +/// `signed` arrives as full shas, comma-separated, and empty entries are +/// dropped — a producer whose range held no commits sends an empty string +/// rather than omitting the input, and an empty sha is not a signed commit. +#[must_use] +pub fn record(signingkey: &str, program: &str, gpgsign_conflict: bool, signed: &str) -> String { + let mut lines = format!("signer {}\n", posture(signingkey, program).token()); + if gpgsign_conflict { + lines.push_str("config conflict\n"); + } + for sha in signed.split(',').filter(|sha| !sha.is_empty()) { + // EIGHT CHARACTERS, and it is the pointer-only law rather than brevity: + // the record names WHICH commit without carrying the object. + let short: String = sha.chars().take(8).collect(); + lines.push_str("signed "); + lines.push_str(&short); + lines.push('\n'); + } + lines +} + +#[cfg(test)] +// Panicking on setup failure is the idiomatic way for a test to fail loudly — +// the house spelling, as `render.rs`, `contract.rs` and `mint.rs` carry it. +#[allow(clippy::unwrap_used, clippy::expect_used)] +mod tests { + use super::{Posture, posture}; + + const LITERAL: &str = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIexample"; + const SIGNER: &str = "/usr/bin/ssh-keygen"; + + fn scratch(name: &str) -> std::path::PathBuf { + let dir = std::env::temp_dir().join(format!("batten-signer-posture-{name}")); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).expect("scratch dir"); + dir + } + + #[test] + fn an_inline_public_key_is_a_literal_not_a_path_and_is_verifiable() { + for literal in [ + LITERAL, + "key::ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIexample", + "sk-ssh-ed25519@openssh.com AAAAGexample", + "sk-ecdsa-sha2-nistp256@openssh.com AAAAInexample", + ] { + assert_eq!( + posture(literal, SIGNER), + Posture::Verifiable, + "a literal is already the public half: {literal}" + ); + } + } + + #[test] + fn a_signer_under_tmp_is_unverifiable_because_the_container_reclaims_it() { + // ORDER: the key here is perfectly healthy, and the signer still decides. + let broken = posture(LITERAL, "/tmp/code-sign"); + assert!(matches!(broken, Posture::Broken(_)), "{broken:?}"); + assert!(broken.token().contains("/tmp"), "{}", broken.token()); + } + + #[test] + fn an_empty_signing_key_is_what_makes_it_unverifiable() { + let dir = scratch("empty"); + let key = dir.join("key.pub"); + std::fs::write(&key, "").expect("write empty key"); + let broken = posture(key.to_str().expect("utf8 path"), SIGNER); + assert!(matches!(broken, Posture::Broken(_)), "{broken:?}"); + assert!(broken.token().contains("empty file"), "{}", broken.token()); + } + + /// THE MEASURED DEFECT. A size test alone calls a directory healthy, because + /// `stat` sizes one happily and the public half is still unreadable. + #[test] + fn a_signing_key_that_is_a_directory_is_unverifiable() { + let dir = scratch("directory"); + let keydir = dir.join("keydir"); + std::fs::create_dir_all(&keydir).expect("scratch keydir"); + let broken = posture(keydir.to_str().expect("utf8 path"), SIGNER); + assert!(matches!(broken, Posture::Broken(_)), "{broken:?}"); + assert!( + broken.token().contains("regular file"), + "{}", + broken.token() + ); + } + + #[test] + fn a_signing_key_naming_a_path_that_does_not_exist_is_unverifiable() { + let broken = posture("/nowhere/at/all/key.pub", SIGNER); + assert!(matches!(broken, Posture::Broken(_)), "{broken:?}"); + assert!( + broken.token().contains("does not exist"), + "{}", + broken.token() + ); + } + + /// ANTI-VACUITY. Without this, every arm above could be passing because the + /// function returns `Broken` for everything. + #[test] + fn a_signer_failing_neither_test_is_left_alone() { + let dir = scratch("healthy"); + let key = dir.join("key.pub"); + std::fs::write(&key, "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIexample\n") + .expect("write real key"); + assert_eq!( + posture(key.to_str().expect("utf8 path"), SIGNER), + Posture::Verifiable + ); + } + + /// An absent `user.signingkey` is not a broken one: nothing is being signed + /// with it, so there is no unverifiable signature to name. + #[test] + fn an_unset_signing_key_is_not_a_finding() { + assert_eq!(posture("", SIGNER), Posture::Verifiable); + } + + /// A signer merely NAMED `/tmpfoo` is not under `/tmp/`. Without the + /// trailing separator the prefix test would catch a sibling directory. + #[test] + fn a_signer_whose_name_merely_starts_with_tmp_is_not_under_tmp() { + assert_eq!(posture(LITERAL, "/tmpfoo/code-sign"), Posture::Verifiable); + } + + // --- the record's own shape ------------------------------------------- + + #[test] + fn a_healthy_signer_with_nothing_else_is_one_line() { + assert_eq!( + super::record(LITERAL, SIGNER, false, ""), + "signer verifiable\n" + ); + } + + #[test] + fn the_conflict_line_appears_only_when_the_producer_found_one() { + assert_eq!( + super::record(LITERAL, SIGNER, true, ""), + "signer verifiable\nconfig conflict\n" + ); + } + + /// EIGHT CHARACTERS, and the full sha never reaches the record. + #[test] + fn a_signed_commit_is_named_by_its_short_sha_and_never_the_full_one() { + let full = "1a2b3c4d5e6f708192a3b4c5d6e7f8091a2b3c4d"; + let written = super::record(LITERAL, SIGNER, false, full); + assert_eq!(written, "signer verifiable\nsigned 1a2b3c4d\n"); + assert!(!written.contains(full), "{written}"); + } + + #[test] + fn several_signed_commits_each_get_a_line_in_the_order_given() { + let written = super::record(LITERAL, SIGNER, false, "aaaaaaaabbbb,ccccccccdddd"); + assert_eq!( + written, + "signer verifiable\nsigned aaaaaaaa\nsigned cccccccc\n" + ); + } + + /// AN EMPTY LIST IS NOT ONE EMPTY ENTRY. A producer whose range held no + /// commits sends `signed=`, and a `signed ` line with nothing after it would + /// be a commit the module then tried to name. + #[test] + fn an_empty_signed_list_contributes_no_line() { + assert_eq!( + super::record(LITERAL, SIGNER, false, ""), + "signer verifiable\n" + ); + assert_eq!( + super::record(LITERAL, SIGNER, false, ",,"), + "signer verifiable\n" + ); + } + + /// A sha SHORTER than eight characters is taken whole rather than panicking + /// on a byte index the string does not have. + #[test] + fn a_short_sha_is_taken_whole() { + assert_eq!( + super::record(LITERAL, SIGNER, false, "abc"), + "signer verifiable\nsigned abc\n" + ); + } + + /// Pointer-only (rule 4): the reason never carries the key or the signer. + #[test] + fn no_reason_echoes_the_key_or_the_signer_path() { + let secret = "/tmp/SECRET-SIGNER-PATH"; + let broken = posture("SECRET-KEY-MATERIAL", secret); + let said = broken.token(); + assert!(!said.contains("SECRET-SIGNER-PATH"), "{said}"); + assert!(!said.contains("SECRET-KEY-MATERIAL"), "{said}"); + } +} diff --git a/crates/batten/tests/it/signing_posture.rs b/crates/batten/tests/it/signing_posture.rs index e8f610da1..8d330e496 100644 --- a/crates/batten/tests/it/signing_posture.rs +++ b/crates/batten/tests/it/signing_posture.rs @@ -16,8 +16,17 @@ //! memberships — and one of them (`-s` alone being true for a directory) was a //! measured defect rather than a hypothetical. //! -//! `mise-tasks/signer_posture.py` is the one authority on those branches, shared -//! with `[tasks.signing-posture-record]` and `[tasks.signing-posture-repair]`. +//! `crates/batten/src/signer_posture.rs` is the one authority on those branches +//! and on the record's own shape. Its `#[cfg(test)] mod tests` asserts all seven +//! arms against scratch paths, plus two the retired program never had — an unset +//! key, and a signer merely NAMED `/tmpfoo`, which a prefix test without the +//! separator would have called broken. +//! +//! What stays HERE is the half a unit test cannot reach: that the engine carries +//! the reading through `record derive` into a record the real module refuses +//! over. `[tasks.signing-posture-repair]` no longer classifies a second time — +//! it reads the posture off the record the producer just wrote, so the two +//! cannot disagree about a checkout they both looked at. //! //! # RETIREMENT LEDGER, PER PATH — what `shell retire partial` reads //! @@ -28,12 +37,12 @@ // carried: "an unsigned range with the override in place passes" policy/signing-posture.rego kind:mechanism // carried: "signing with a verifiable signer is left alone" policy/signing-posture.rego kind:mechanism // carried: "a commit signed by a VERIFIABLE signer is left alone, header and all" policy/signing-posture.rego kind:mechanism -// carried: "an empty signing key is what makes it unverifiable" policy/signing-posture.rego kind:mechanism -// carried: "a signing key that is a directory is unverifiable" policy/signing-posture.rego kind:mechanism -// carried: "a signing key this checkout cannot read is unverifiable" policy/signing-posture.rego kind:mechanism -// carried: "a signing key naming a path that does not exist is unverifiable" policy/signing-posture.rego kind:mechanism -// carried: "an inline public key is a literal, not a path, and is verifiable" policy/signing-posture.rego kind:mechanism -// carried: "a signer under /tmp is unverifiable because the container reclaims it" policy/signing-posture.rego kind:mechanism +// carried: "an empty signing key is what makes it unverifiable" crates/batten/src/signer_posture.rs kind:mechanism crates/batten/tests/it/signing_posture.rs +// carried: "a signing key that is a directory is unverifiable" crates/batten/src/signer_posture.rs kind:mechanism crates/batten/tests/it/signing_posture.rs +// carried: "a signing key this checkout cannot read is unverifiable" crates/batten/src/signer_posture.rs kind:mechanism crates/batten/tests/it/signing_posture.rs +// carried: "a signing key naming a path that does not exist is unverifiable" crates/batten/src/signer_posture.rs kind:mechanism crates/batten/tests/it/signing_posture.rs +// carried: "an inline public key is a literal, not a path, and is verifiable" crates/batten/src/signer_posture.rs kind:mechanism crates/batten/tests/it/signing_posture.rs +// carried: "a signer under /tmp is unverifiable because the container reclaims it" crates/batten/src/signer_posture.rs kind:mechanism crates/batten/tests/it/signing_posture.rs // carried: "a signed commit in range is refused, and named by short sha" policy/signing-posture.rego kind:mechanism // carried: "repairing the config does not excuse a commit already signed" policy/signing-posture.rego kind:mechanism // carried: "a missing override is refused when the environment sets signing globally" policy/signing-posture.rego kind:mechanism @@ -248,118 +257,138 @@ fn an_absent_record_says_nothing_rather_than_refusing() { ); } -// --- the signer classification ----------------------------------------------- - -/// Run the REAL classification both tasks run. -#[expect( - clippy::disallowed_types, - reason = "stays, and it is the subject under test rather than a convenience: which configurations are unverifiable is what moved out of the dying program into `mise-tasks/signer_posture.py`, and a harness re-implementing the four file tests in Rust would be a second authority over the one question both the record and the repair turn on" -)] -fn posture(signingkey: &str, program: &str) -> String { - let done = std::process::Command::new("python3") - .arg("../../mise-tasks/signer_posture.py") - .arg(signingkey) - .arg(program) - .output() - .expect("the classification runs"); +// --- the signer classification, over the real verb --------------------------- + +/// Drive the REAL reading both tasks run, through the REAL verb. +/// +/// `crates/batten/src/signer_posture.rs` is the one authority on which +/// configurations are unverifiable, and its own `#[cfg(test)] mod tests` +/// asserts all seven arms directly against scratch paths — plus two the retired +/// program never had: an unset key, and a signer merely NAMED `/tmpfoo`, which +/// a prefix test without the separator would have called broken. +/// +/// What THIS tier adds is the half a unit test cannot reach: that the engine +/// carries that reading, and the record's whole shape, into a record the real +/// module then refuses over. +fn derive(dir: &std::path::Path, signingkey: &str, program: &str, signed: &str) -> String { + let written = run_with_stdin( + dir, + &[ + "record", + "derive", + "signing-posture", + "--input", + &format!("signingkey={signingkey}"), + "--input", + &format!("ssh-program={program}"), + "--input", + "gpgsign=none", + "--input", + &format!("signed={signed}"), + ], + "", + ); assert!( - done.status.success(), - "it completes: {}", - String::from_utf8_lossy(&done.stderr) + written.status.success(), + "the derivation lands: {}", + String::from_utf8_lossy(&written.stderr) ); - String::from_utf8_lossy(&done.stdout).trim().to_owned() + String::from_utf8_lossy(&written.stdout).into_owned() } #[test] -fn an_empty_signing_key_is_what_makes_it_unverifiable() { - let dir = scratch("signer-empty"); - write(&dir, "key.pub", ""); - let said = posture(dir.join("key.pub").to_string_lossy().as_ref(), ""); - assert!(said.starts_with("broken"), "an empty key is broken\n{said}"); - assert!(said.contains("empty file"), "and says which test\n{said}"); -} - -#[test] -fn a_signing_key_that_is_a_directory_is_unverifiable() { - // `-s` ALONE WAS THE TEST HERE AND IT IS TRUE FOR A DIRECTORY, which leaves - // the public half unreadable — the condition being named. - let dir = scratch("signer-dir"); - write(&dir, "keydir/placeholder", ""); - let said = posture(dir.join("keydir").to_string_lossy().as_ref(), ""); +fn the_verb_derives_a_broken_signer_into_a_record_the_module_refuses_over() { + let dir = repo("derive-broken"); + let key = dir.join("key.pub"); + std::fs::write(&key, "").expect("an empty key is the broken case"); + let written = derive( + &dir, + key.to_string_lossy().as_ref(), + "", + "1a2b3c4d5e6f708192a3b4c5d6e7f8091a2b3c4d", + ); assert!( - said.contains("not a regular file"), - "a directory is broken, and says so\n{said}" + written.contains("signer broken"), + "the reading reaches the record\n{written}" ); -} - -#[test] -fn a_signing_key_this_checkout_cannot_read_is_unverifiable() { - let dir = scratch("signer-unreadable"); - write(&dir, "key.pub", "ssh-ed25519 AAAA"); - let path = dir.join("key.pub"); - let mut perms = std::fs::metadata(&path).expect("it exists").permissions(); - { - use std::os::unix::fs::PermissionsExt as _; - perms.set_mode(0o000); - } - std::fs::set_permissions(&path, perms).expect("it is unreadable"); - - let said = posture(path.to_string_lossy().as_ref(), ""); - // Running as root defeats the permission bit, so this asserts the branch it - // can reach rather than claiming one it cannot. assert!( - said.contains("cannot read") || said == "verifiable", - "an unreadable key is broken where the OS enforces it\n{said}" + written.contains("signed 1a2b3c4d"), + "and so does the short sha\n{written}" ); -} -#[test] -fn a_signing_key_naming_a_path_that_does_not_exist_is_unverifiable() { - let said = posture("/nowhere/at/all/key.pub", ""); - assert!( - said.contains("does not exist"), - "an absent path is broken, and says so\n{said}" + let decided = run(&dir, &["check"]); + assert_eq!( + decided.status.code(), + Some(2), + "a signed commit under a broken signer is the finding\n{}", + String::from_utf8_lossy(&decided.stderr) ); } #[test] -fn an_inline_public_key_is_a_literal_not_a_path_and_is_verifiable() { - // A LITERAL IS THE MOST PUBLISHABLE FORM THERE IS — it is already the public - // half — so testing it as a file would report the healthiest possible - // configuration as broken. - assert_eq!( - posture("ssh-ed25519 AAAAC3NzaC1lZDI1NTE5", ""), - "verifiable" +fn the_verb_derives_a_verifiable_signer_into_silence() { + let dir = repo("derive-verifiable"); + let written = derive( + &dir, + "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIexample", + "/usr/bin/ssh-keygen", + "1a2b3c4d5e6f708192a3b4c5d6e7f8091a2b3c4d", + ); + assert!( + written.contains("signer verifiable"), + "an inline literal is the healthiest form there is\n{written}" ); + + let quiet = run(&dir, &["check"]); assert_eq!( - posture("key::ssh-ed25519 AAAAC3NzaC1lZDI1NTE5", ""), - "verifiable" + quiet.status.code(), + Some(0), + "a verifiable signer is left alone, signed commits and all\n{}", + String::from_utf8_lossy(&quiet.stderr) ); - assert_eq!(posture("sk-ssh-ed25519@openssh.com AAAA", ""), "verifiable"); } +/// POINTER-ONLY THROUGH THE WHOLE PATH (rule 4): neither the key nor the signer +/// path reaches the record, and neither does a full sha. #[test] -fn a_signer_under_tmp_is_unverifiable_because_the_container_reclaims_it() { - // And it outranks the key test: a reproducible key behind an irreproducible - // signer is still not re-verifiable later. - let said = posture("ssh-ed25519 AAAAC3NzaC1lZDI1NTE5", "/tmp/code-sign"); - assert!( - said.contains("/tmp"), - "a /tmp signer is broken whatever the key is\n{said}" - ); +fn no_key_signer_or_full_sha_reaches_the_record() { + let dir = repo("derive-quiet"); + let full = "abcdef0123456789abcdef0123456789abcdef01"; + let written = derive(&dir, "SECRET-KEY-MATERIAL", "/tmp/SECRET-SIGNER", full); + assert!(!written.contains("SECRET-KEY-MATERIAL"), "{written}"); + assert!(!written.contains("SECRET-SIGNER"), "{written}"); + assert!(!written.contains(full), "{written}"); + assert!(written.contains("signed abcdef01"), "{written}"); } +/// A family reads only the inputs it declares, and a misspelling is a usage +/// error rather than a reading that silently ran on a default. #[test] -fn a_signer_failing_neither_test_is_left_alone() { - // The anti-vacuity arm: without it every case above passes on a - // classification that returned `broken` unconditionally. - let dir = scratch("signer-healthy"); - write(&dir, "key.pub", "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5\n"); +fn an_input_the_family_does_not_read_is_a_usage_error() { + let dir = repo("derive-unknown-input"); + let refused = run_with_stdin( + &dir, + &[ + "record", + "derive", + "signing-posture", + "--input", + "signingkey=", + "--input", + "ssh-program=", + "--input", + "gpgsign=none", + "--input", + "signed=", + "--input", + "signingkeys=oops", + ], + "", + ); assert_eq!( - posture( - dir.join("key.pub").to_string_lossy().as_ref(), - "/usr/bin/ssh-keygen" - ), - "verifiable" + refused.status.code(), + Some(1), + "an unread input is a usage error\n{}", + String::from_utf8_lossy(&refused.stderr) ); } diff --git a/mise-tasks/signer_posture.py b/mise-tasks/signer_posture.py deleted file mode 100644 index 9a546df37..000000000 --- a/mise-tasks/signer_posture.py +++ /dev/null @@ -1,89 +0,0 @@ -# Whether a git signing configuration names a key anyone can verify (CLOUD-669, -# ported off `mise-tasks/signing-posture.sh` under CLOUD-1717). -# -# SIGNING IS GOOD, and this is not against it. A version of this that reads that -# way is wrong: signing in CI, with a key whose public half is published, is the -# desired end state and CLOUD-591 owns getting there. What it names is the -# narrower thing — a signature produced by a key that cannot be verified or -# reproduced, which is WORSE than no signature because it looks like provenance -# and carries none. -# -# TWO INDEPENDENT CONDITIONS, either of which makes a signature unverifiable by -# anyone, us included. Both were measured rather than assumed (2026-08-18): -# -# * `gpg.ssh.program` resolves inside `/tmp`. The container reclaims it, so the -# signer and whatever key it holds are not reproducible across sessions. A -# signature nobody can re-verify later is provenance theatre. -# * `user.signingkey` names a file that is empty or unreadable. The public half -# cannot be read, so no `allowed_signers` entry can be derived from it and -# nothing downstream can check the signature. -# -# A signer failing NEITHER test is left alone and signing stays on. -# -# A LITERAL KEY IS NOT A PATH. With `gpg.format ssh`, git accepts the public key -# inline (`ssh-ed25519 AAAA…`) or via a `key::` prefix as well as a filename. A -# literal is the MOST publishable form there is — it is already the public half — -# so testing it as a file would report the healthiest possible configuration as -# broken. -# -# FOUR FILE TESTS, NOT ONE, and that was a real defect rather than thoroughness. -# `-s` alone is true for anything non-empty that `stat` can size, including an -# unreadable file and a DIRECTORY — both of which leave the public half -# unreadable, which is the condition being named. Each test carries its own -# reason so the refusal says which one. -# -# ONE READING, TWO CALLERS: `[tasks.signing-posture-record]` writes the record -# from it, and `crates/batten/tests/it/signing_posture.rs` drives all seven arms. -# A copy of these branches in the tier would be the second authority -# `cargo_graph.py` was extracted to remove one gate over. -# -# Emits one line: `verifiable`, or `broken `. - -import os -import sys - -# The spellings git accepts for a key given INLINE rather than as a path. -LITERAL_PREFIXES = ("ssh-", "key::", "sk-ssh-", "sk-ecdsa-") - - -def posture(signingkey, program): - if program.startswith("/tmp/"): - return ( - "broken the signer resolves inside /tmp, which the container " - "reclaims, so the key is not reproducible" - ) - if signingkey == "" or signingkey.startswith(LITERAL_PREFIXES): - return "verifiable" - if not os.path.exists(signingkey): - return ( - "broken user.signingkey names a path that does not exist, so the " - "public half cannot be read or published" - ) - if not os.path.isfile(signingkey): - return ( - "broken user.signingkey names something that is not a regular file, " - "so the public half cannot be read or published" - ) - if not os.access(signingkey, os.R_OK): - return ( - "broken user.signingkey names a file this checkout cannot read, so " - "the public half cannot be read or published" - ) - if os.path.getsize(signingkey) == 0: - return ( - "broken user.signingkey names an empty file, so the public half " - "cannot be read or published" - ) - return "verifiable" - - -def main(argv): - if len(argv) != 3: - sys.stderr.write("usage: signer_posture.py \n") - return 2 - sys.stdout.write(posture(argv[1], argv[2]) + "\n") - return 0 - - -if __name__ == "__main__": - sys.exit(main(sys.argv)) diff --git a/mise.toml b/mise.toml index bcad27a5d..c9145ddc8 100644 --- a/mise.toml +++ b/mise.toml @@ -2157,12 +2157,14 @@ emit | cargo run --quiet -p batten -- record named timeout-drift # CLOUD-1717. The things that are not decisions: the git config reads, the # filesystem tests behind them, and the range scan. # -# THE SIGNER CLASSIFICATION IS `mise-tasks/signer_posture.py`, shared with -# `crates/batten/tests/it/signing_posture.rs`, because seven of the dying suite's -# cases are about exactly which configurations are unverifiable — an empty key, a -# directory, an unreadable file, a path that does not exist, an inline literal, a -# `/tmp` signer, and a healthy one. Those are the substance; the module's half is -# two set memberships. +# THE SIGNER CLASSIFICATION IS `crates/batten/src/signer_posture.rs`, because +# seven of the dying suite's cases are about exactly which configurations are +# unverifiable — an empty key, a directory, an unreadable file, a path that does +# not exist, an inline literal, a `/tmp` signer, and a healthy one. Those are the +# substance; the module's half is two set memberships. They are `#[cfg(test)]` +# cases beside the function now, where a scratch path exercises them directly; +# it was a `.py` beside this task, which was the campaign to delete bash routing +# a reading into another interpreter (CLOUD-1717). # # THE CONFLICT, NEVER THE MERE ABSENCE OF A LOCAL KEY. `config conflict` is # recorded only where something OUTSIDE the checkout turns signing on and nothing @@ -2186,34 +2188,39 @@ shell = "bash -c" run = """ git rev-parse --git-dir >/dev/null 2>&1 || exit 0 +# THE GATHERING IS THIS TASK'S — `git config`, `git rev-list` and `git +# cat-file` are spawns house-style §5 keeps outside the engine. What the facts +# MEAN, including the signer classification and the record's own shape, is +# `crates/batten/src/signer_posture.rs` (CLOUD-1717). key=$(git config --get user.signingkey 2>/dev/null || true) prog=$(git config --get gpg.ssh.program 2>/dev/null || true) -signer=$(python3 mise-tasks/signer_posture.py "$key" "$prog") || exit 0 -{ - printf 'signer %s\\n' "$signer" +gpgsign=none +local_setting=$(git config --local --get commit.gpgsign 2>/dev/null || true) +inherited=$(git config --global --get commit.gpgsign 2>/dev/null || true) +if [ "$local_setting" != false ]; then + case "$inherited$local_setting" in + *true*) gpgsign=conflict ;; + esac +fi - local_setting=$(git config --local --get commit.gpgsign 2>/dev/null || true) - inherited=$(git config --global --get commit.gpgsign 2>/dev/null || true) - if [ "$local_setting" != false ]; then - case "$inherited$local_setting" in - *true*) printf 'config conflict\\n' ;; +signed= +base="${BASE_SHA:-}" +head="${HEAD_SHA:-}" +[ -n "$base" ] || base=$(git rev-parse --verify --quiet origin/main) || base="" +[ -n "$head" ] || head=$(git rev-parse --verify --quiet HEAD) || head="" +if [ -n "$base" ] && [ -n "$head" ]; then + for sha in $(git rev-list --no-merges "$base..$head" 2>/dev/null || true); do + header=$(git cat-file commit "$sha" 2>/dev/null | sed '/^$/q') + case $'\\n'"$header" in + *$'\\n'gpgsig' '*) signed="${signed:+$signed,}$sha" ;; esac - fi + done +fi - base="${BASE_SHA:-}" - head="${HEAD_SHA:-}" - [ -n "$base" ] || base=$(git rev-parse --verify --quiet origin/main) || base="" - [ -n "$head" ] || head=$(git rev-parse --verify --quiet HEAD) || head="" - if [ -n "$base" ] && [ -n "$head" ]; then - for sha in $(git rev-list --no-merges "$base..$head" 2>/dev/null || true); do - header=$(git cat-file commit "$sha" 2>/dev/null | sed '/^$/q') - case $'\\n'"$header" in - *$'\\n'gpgsig' '*) printf 'signed %s\\n' "$(printf '%s' "$sha" | cut -c1-8)" ;; - esac - done - fi -} | cargo run --quiet -p batten -- record named signing-posture +cargo run --quiet -p batten -- record derive signing-posture \\ + --input "signingkey=$key" --input "ssh-program=$prog" \\ + --input "gpgsign=$gpgsign" --input "signed=$signed" """ # THE WRITE, self-declared per house style §5 and kept a task because a module @@ -2233,9 +2240,13 @@ git rev-parse --git-dir >/dev/null 2>&1 || { echo "::error:: signing-posture-repair: not a git repository" >&2 exit 2 } -key=$(git config --get user.signingkey 2>/dev/null || true) -prog=$(git config --get gpg.ssh.program 2>/dev/null || true) -signer=$(python3 mise-tasks/signer_posture.py "$key" "$prog") || exit 2 +# ONE READING, ONE CALLER OF IT. This used to run the classification a second +# time beside the record task; it now reads the posture off the record the +# producer just wrote and printed, so the two can never disagree about a +# checkout they both looked at (CLOUD-1717). +recorded=$(mise run signing-posture-record) || exit 2 +signer=${recorded#*signer } +signer=${signer%%$'\\n'*} case "$signer" in verifiable) echo "signing-posture: signer is verifiable, leaving signing on" diff --git a/policy/module-layering.rego b/policy/module-layering.rego index bb58f4be7..c9a19f55b 100644 --- a/policy/module-layering.rego +++ b/policy/module-layering.rego @@ -550,6 +550,19 @@ declared_modules := { # `record` call it without the producer's effects reaching the engine: the # spawn stays in the task (house-style §5) and only the reading is here. "probe_verdict", + # `signer_posture` arrived with CLOUD-1717, `probe_verdict`'s sibling and + # its class: it reaches NOTHING in this crate. It takes the two git config + # values as `&str` and never runs `git config` itself, which is what keeps + # the reading testable against a scratch path and a developer's real + # configuration out of the tests (CLOUD-591's boundary). + # + # It owns the RECORD'S SHAPE as well as the classification, and that is the + # placement rather than scope creep: the shape was a sequence of `printf` + # calls in a task body that nothing tested, including the truncation of each + # sha to eight characters — which is the difference between a pointer and a + # payload. The producer still gathers the facts, because `git config` and + # `git rev-list` are spawns §5 keeps outside; what they MEAN is composed here. + "signer_posture", } # THE FORBIDDEN EDGES, each traceable to prose already in the tree. From cafde6a435c9096c3be11743f9ede4936bf7f167 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sat, 19 Sep 2026 21:45:30 +0000 Subject: [PATCH 26/32] feat(record): derive the transcript census in Rust, beside the parse that owns the format MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Third of six, and the DRY-est: this one needed no new module. `mise-tasks/transcript_census.py` re-implemented a host transcript parse that `crates/batten/src/transcript.rs` already owned, and declared the same two laws in its own words — pointer-only, and forward-compatible against a format the host moves. The census now lives there, reusing the `Line` struct rather than parsing a second time; `Line` gained the two fields it lacked, `type` and `isSidechain`. Because it is not a new module, it needs no layering row and no module-map row — the placement question was already answered. Fourteen in-module cases, three of which the retired program never had: a nested project directory (a host shards transcripts by project, and the old walk would have found them, but nothing said so), an empty `sessionId`, and an assistant turn. `exclude` is `Option<&str>` and not `&str`, which is the contract's whole point: absent and present-but-empty are different claims. The producer adds `--input exclude=` only when `BATTEN_SESSION_ID` is set, so passing an empty one can never launder "said nothing" into "exclude nothing". An absent root stays could-not-look and writes NOTHING, because an absent record means "the producer did not run" and must never be spelled the same way as a root that was walked and held no transcripts. Two ledger rows moved to `// changed:`. The malformed-threshold refusal is exit 1 rather than 2 — the engine's one 0/1/2/3 table, no per-verb exception. "More arguments than the contract names" ceases to exist under named flags, and what replaces it is stricter: an input key no family declares is a usage error. Pointer-only is a security property over this input rather than a style one: a transcript is the richest source of secrets this repository can be pointed at, and what leaves the verb is two counts. Full suite 5697/5697. `shell retire partial` now names three `.py`. Refs: CLOUD-1717 --- crates/batten/src/record.rs | 28 ++ crates/batten/src/transcript.rs | 266 +++++++++++++++++ crates/batten/tests/it/transcript_corpus.rs | 313 +++++++++----------- mise-tasks/transcript_census.py | 144 --------- mise.toml | 20 +- 5 files changed, 452 insertions(+), 319 deletions(-) delete mode 100644 mise-tasks/transcript_census.py diff --git a/crates/batten/src/record.rs b/crates/batten/src/record.rs index 08b1c5341..d2b686859 100644 --- a/crates/batten/src/record.rs +++ b/crates/batten/src/record.rs @@ -637,6 +637,34 @@ pub fn run_derive( let signed = required_input(&inputs, family, "signed")?; crate::signer_posture::record(signingkey, program, conflict, signed) } + "transcript-corpus" => { + only_these_inputs(&inputs, family, &["root", "threshold", "exclude"])?; + let root = required_input(&inputs, family, "root")?; + let raw = required_input(&inputs, family, "threshold")?; + let threshold: usize = raw.parse().map_err(|_| { + UsageError::raise(format!( + "record derive {family}: threshold `{raw}` is not a whole number" + )) + })?; + let root = Path::new(root); + if !root.is_dir() { + // THE QUESTION COULD NOT BE ASKED. Write NOTHING — an absent + // record is "the producer did not run", which must never be + // spelled the same way as a root that was walked and held no + // transcripts. + return Err(UsageError::raise(format!( + "record derive {family}: no transcript root to walk" + ))); + } + // ABSENT AND PRESENT-BUT-EMPTY ARE DIFFERENT CLAIMS, which is the + // whole reason this is an `Option` rather than a defaulted string: a + // caller naming no exclusion is saying nothing, and a caller naming + // the empty string is saying "exclude nothing". `--input exclude=` + // is the second, and omitting the flag is the first. + let exclude = inputs.get("exclude").map(String::as_str); + let sessions = crate::transcript::census(root, exclude); + format!("sessions {sessions}\nthreshold {threshold}\n") + } // AN UNKNOWN FAMILY IS A USAGE ERROR, never a record written under a name // nothing reads. A producer whose family was renamed would otherwise go on // writing happily into a key no module has looked at since. diff --git a/crates/batten/src/transcript.rs b/crates/batten/src/transcript.rs index 4489b6cba..56edaf777 100644 --- a/crates/batten/src/transcript.rs +++ b/crates/batten/src/transcript.rs @@ -1112,6 +1112,108 @@ fn result_text(content: &Value) -> String { } } +// --------------------------------------------------------------------------- +// The independent-session census (CLOUD-388, CLOUD-651, CLOUD-1717). +// --------------------------------------------------------------------------- + +/// The session id of an authored, non-sidechain user record, or `None`. +/// +/// # What counts as independent, and why it is not "one file, one session" +/// +/// A transcript is independent evidence when it belongs to a DIFFERENT session +/// that a person actually drove. Three things therefore do not count: +/// +/// * **a subagent stream** — `isSidechain: true`. CLOUD-326 §8.1 recorded "one +/// session plus five subagent transcripts" and correctly called that N=1; +/// counting the five would inflate the corpus with the orchestrator's own +/// turns wearing different file names. +/// * **a transcript with nobody in it** — a `tool_result` also arrives as a +/// `user` record, and that is the harness handing work back rather than a +/// person speaking. The boundary test is `finding-sink-check`'s pass 1, +/// reused rather than re-derived. +/// * **the asking session** — excluded by [`census`], not here. +/// +/// A LINE THIS BUILD CANNOT DECODE YIELDS NOTHING RATHER THAN A FAILURE TO +/// LOOK. The format is a HOST's and it moves, so an undecodable line +/// contributes no id instead of turning the whole census into could-not-look — +/// this module's own forward-compatibility law, applied from the other side. +#[must_use] +pub fn authored_session_id(line: &str) -> Option { + let parsed: Line = serde_json::from_str(line).ok()?; + if parsed.is_sidechain == Some(true) { + return None; + } + if parsed.kind.as_deref() != Some("user") { + return None; + } + if !authored_content(parsed.message.as_ref()?.content.as_ref()?) { + return None; + } + parsed.session_id.filter(|id| !id.is_empty()) +} + +/// Whether a user record's content carries a person's own words. +/// +/// A bare string is a typed turn. An array counts only when it holds a `text` +/// block: an array of `tool_result` blocks is the harness answering itself. +fn authored_content(content: &Value) -> bool { + match content { + Value::String(_) => true, + Value::Array(blocks) => blocks + .iter() + .any(|block| block.get("type") == Some(&Value::String("text".to_owned()))), + _ => false, + } +} + +/// How many DISTINCT independent sessions live under `root`. +/// +/// `exclude` is `Option` and not `&str` on purpose: absent and present-but-empty +/// are different claims. A caller who names no exclusion is saying nothing, and +/// only then does an ambient session id apply; a caller who names the empty +/// string is saying "exclude nothing". Collapsing the two would launder one into +/// the other. +/// +/// POINTER-ONLY IS A SECURITY PROPERTY HERE, not a style one (rule 4): the +/// return value is a count. A transcript is the richest source of secrets this +/// repository can be pointed at, and no path, session id or byte of one leaves +/// this function. +#[must_use] +pub fn census(root: &std::path::Path, exclude: Option<&str>) -> usize { + let mut seen = std::collections::BTreeSet::new(); + let mut pending = vec![root.to_path_buf()]; + while let Some(next) = pending.pop() { + let Ok(entries) = std::fs::read_dir(&next) else { + // One unreadable directory is not a failed census of the rest. + continue; + }; + // SORTED FOR BYTE-STABILITY (house style §6): the same root yields the + // same count however the filesystem chose to order itself. + let mut names: Vec<_> = entries.flatten().map(|entry| entry.path()).collect(); + names.sort(); + for path in names { + if path.is_dir() { + pending.push(path); + continue; + } + if path.extension().is_none_or(|ext| ext != "jsonl") { + continue; + } + let Ok(body) = std::fs::read_to_string(&path) else { + continue; + }; + // THE FIRST AUTHORED ID IN THE FILE IDENTIFIES IT; a file with none + // at all is a subagent stream or a transcript nobody was in. + if let Some(found) = body.lines().find_map(authored_session_id) + && Some(found.as_str()) != exclude + { + seen.insert(found); + } + } + } + seen.len() +} + /// Decide whether a user-role turn was opened by a person (CLOUD-267). /// /// Every branch is an exact read of a typed field — a boolean the host set, or @@ -1196,6 +1298,18 @@ struct Line { is_meta: Option, #[serde(rename = "isSynthetic")] is_synthetic: Option, + /// The record's own kind, as the host spells it (`user`, `assistant`, …). + /// + /// Read by [`authored_session_id`] rather than by the event parse above, + /// which goes through `message.role`. The two agree on a well-formed line; + /// the census reads the outer one because a line with no `message` at all + /// still has a kind, and "not a user record" has to be decidable without + /// one. + #[serde(rename = "type")] + kind: Option, + /// The host marking a record as belonging to a SUBAGENT stream (CLOUD-326). + #[serde(rename = "isSidechain")] + is_sidechain: Option, } #[derive(Debug, Deserialize)] @@ -1491,6 +1605,158 @@ pub fn configured_path(config: Option<&TranscriptConfig>) -> Option { mod tests { use super::*; + // --- the independent-session census (CLOUD-1717) ----------------------- + + /// One authored user turn, in the host's real shape. + fn authored(session: &str) -> String { + format!( + r#"{{"type":"user","sessionId":"{session}","message":{{"role":"user","content":[{{"type":"text","text":"a turn"}}]}}}}"# + ) + } + + fn census_root(name: &str, files: &[(&str, String)]) -> std::path::PathBuf { + let dir = std::env::temp_dir().join(format!("batten-transcript-census-{name}")); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).expect("scratch root"); + for (leaf, body) in files { + std::fs::write(dir.join(leaf), body).expect("write transcript"); + } + dir + } + + #[test] + fn an_authored_turn_yields_its_session_id() { + assert_eq!( + authored_session_id(&authored("alpha")), + Some("alpha".to_owned()) + ); + } + + /// A bare string turn is authored too — a host that types the content as a + /// string rather than an array is still recording a person speaking. + #[test] + fn a_plain_string_turn_is_authored() { + let line = + r#"{"type":"user","sessionId":"beta","message":{"role":"user","content":"typed"}}"#; + assert_eq!(authored_session_id(line), Some("beta".to_owned())); + } + + /// CLOUD-326 §8.1: one session plus five subagent transcripts is N=1. + #[test] + fn a_subagent_stream_is_not_an_independent_session() { + let line = r#"{"type":"user","isSidechain":true,"sessionId":"gamma","message":{"role":"user","content":[{"type":"text","text":"x"}]}}"#; + assert_eq!(authored_session_id(line), None); + } + + /// A `tool_result` arrives as a `user` record. That is the harness handing + /// work back, not a person speaking. + #[test] + fn a_transcript_carrying_only_tool_results_has_nobody_in_it() { + let line = r#"{"type":"user","sessionId":"delta","message":{"role":"user","content":[{"type":"tool_result","tool_use_id":"t1","content":"out"}]}}"#; + assert_eq!(authored_session_id(line), None); + } + + #[test] + fn an_assistant_turn_is_not_an_authored_user_record() { + let line = r#"{"type":"assistant","sessionId":"eps","message":{"role":"assistant","content":[{"type":"text","text":"x"}]}}"#; + assert_eq!(authored_session_id(line), None); + } + + /// FORWARD COMPATIBILITY: the host's format moves, and an undecodable line + /// contributes nothing rather than failing the census. + #[test] + fn a_line_this_build_cannot_decode_yields_nothing_rather_than_a_failure() { + assert_eq!(authored_session_id("{not json at all"), None); + assert_eq!(authored_session_id("[]"), None); + assert_eq!(authored_session_id(""), None); + } + + /// An empty `sessionId` names no session, so it is not one. + #[test] + fn an_empty_session_id_is_not_a_session() { + let line = r#"{"type":"user","sessionId":"","message":{"role":"user","content":"x"}}"#; + assert_eq!(authored_session_id(line), None); + } + + #[test] + fn two_files_carrying_one_session_are_one_session() { + let root = census_root( + "one-session", + &[("a.jsonl", authored("zeta")), ("b.jsonl", authored("zeta"))], + ); + assert_eq!(census(&root, None), 1); + } + + #[test] + fn three_distinct_sessions_are_three() { + let root = census_root( + "three", + &[ + ("a.jsonl", authored("one")), + ("b.jsonl", authored("two")), + ("c.jsonl", authored("three")), + ], + ); + assert_eq!(census(&root, None), 3); + } + + /// COUNTING YOURSELF IS WORSE THAN COUNTING NOTHING: a literal fitted to the + /// single transcript it was derived from is the unmeasured-shape failure the + /// method exists to prevent. + #[test] + fn excluding_the_asking_session_turns_its_own_transcript_into_zero() { + let root = census_root("exclude", &[("a.jsonl", authored("self"))]); + assert_eq!(census(&root, Some("self")), 0); + assert_eq!(census(&root, None), 1); + } + + /// ABSENT AND PRESENT-BUT-EMPTY ARE DIFFERENT CLAIMS, which is why the + /// parameter is an `Option`. An empty exclusion excludes nothing, and no + /// session is ever named by the empty string, so it can never match. + #[test] + fn an_explicitly_empty_exclusion_excludes_nothing() { + let root = census_root("empty-exclusion", &[("a.jsonl", authored("eta"))]); + assert_eq!(census(&root, Some("")), 1); + } + + #[test] + fn an_empty_root_is_zero_independent_sessions_which_is_an_answer() { + let root = census_root("empty", &[("keep.txt", "not a transcript".to_owned())]); + assert_eq!(census(&root, None), 0); + } + + /// One unreadable or absent directory is not a failed census of the rest. + #[test] + fn an_absent_root_counts_nothing_rather_than_panicking() { + assert_eq!(census(std::path::Path::new("/nowhere/at/all"), None), 0); + } + + /// Nested directories are walked: a host shards transcripts by project. + #[test] + fn transcripts_in_subdirectories_are_counted() { + let root = census_root("nested", &[("a.jsonl", authored("outer"))]); + let inner = root.join("project"); + std::fs::create_dir_all(&inner).expect("nested dir"); + std::fs::write(inner.join("b.jsonl"), authored("inner")).expect("nested transcript"); + assert_eq!(census(&root, None), 2); + } + + /// POINTER-ONLY (rule 4), and here it is a security property: a transcript + /// is the richest source of secrets this repository can be pointed at, and + /// what comes back is a number. + #[test] + fn the_census_carries_no_byte_of_any_transcript() { + let secret = format!( + "{}\n{}", + r#"{"type":"user","sessionId":"theta","message":{"role":"user","content":"SUPERSECRETSTRING"}}"#, + authored("theta") + ); + let root = census_root("secret", &[("a.jsonl", secret)]); + let counted = census(&root, None); + assert_eq!(counted, 1); + assert!(!format!("{counted}").contains("SUPERSECRET")); + } + /// A miniature transcript in the host's real shape: an assistant turn making /// a tool call, a hook record denying it, and a user turn carrying the error. const SAMPLE: &str = r#"{"type":"assistant","sessionId":"s-1","message":{"role":"assistant","content":[{"type":"thinking","thinking":"private"},{"type":"tool_use","id":"t1","name":"Bash","input":{"command":"gh pr merge"}}]}} diff --git a/crates/batten/tests/it/transcript_corpus.rs b/crates/batten/tests/it/transcript_corpus.rs index e83a5e910..ece535fb4 100644 --- a/crates/batten/tests/it/transcript_corpus.rs +++ b/crates/batten/tests/it/transcript_corpus.rs @@ -17,32 +17,38 @@ //! distinction, and an undecodable line. That is the entire substance of the //! gate; the comparison it feeds is one `<`. //! -//! Inlined into the task body they would be assertable by nothing. -//! `mise-tasks/transcript_census.py` instead, driven directly below, so they -//! carry. `shell-retirement.rego:159-163` excludes `.py` from -//! `under_mise_tasks`, so that file adds no shell rule. +//! They live in `crates/batten/src/transcript.rs` — beside the parse that +//! already owns this host's format and declares the same pointer-only and +//! forward-compatibility laws, rather than in a module of its own. Its +//! `#[cfg(test)] mod tests` asserts each independence rule directly, plus three +//! the retired program never had: a nested project directory, an empty +//! `sessionId`, and an assistant turn. +//! +//! What stays HERE is the half a unit test cannot reach: that the engine +//! carries the count through `record derive` into a record the real module then +//! decides over, and that a root it could not walk writes NOTHING. //! //! # RETIREMENT LEDGER, PER PATH — what `shell retire partial` reads //! -//! Four cases are not carried and each says why in its own row. +//! Five cases are not carried and each says why in its own row. //! // carried: mise-tasks/transcript-corpus-check.sh policy/transcript-corpus.rego kind:mechanism crates/batten/tests/it/transcript_corpus.rs // carried: tests/transcript-corpus-check.bats policy/transcript-corpus.rego kind:mechanism crates/batten/tests/it/transcript_corpus.rs -// carried: "an empty root is zero independent sessions, which is an answer and not a failure to look" policy/transcript-corpus.rego kind:mechanism +// carried: "an empty root is zero independent sessions, which is an answer and not a failure to look" crates/batten/src/transcript.rs kind:mechanism crates/batten/tests/it/transcript_corpus.rs // carried: "one transcript is one session, and one is not a corpus" policy/transcript-corpus.rego kind:mechanism // carried: "three distinct sessions satisfy the default threshold" policy/transcript-corpus.rego kind:mechanism // carried: "the threshold is the argument, so the same corpus can fail a stricter one" policy/transcript-corpus.rego kind:mechanism -// carried: "a subagent stream is not an independent session" policy/transcript-corpus.rego kind:mechanism -// carried: "a transcript carrying only tool results has nobody in it" policy/transcript-corpus.rego kind:mechanism -// carried: "two files carrying one session are one session" policy/transcript-corpus.rego kind:mechanism -// carried: "a line this build cannot decode yields nothing rather than a failure to look" policy/transcript-corpus.rego kind:mechanism -// carried: "excluding the asking session turns its own transcript into zero" policy/transcript-corpus.rego kind:mechanism -// carried: "the exclusion defaults from the environment when no argument names one" policy/transcript-corpus.rego kind:mechanism -// carried: "an explicitly empty exclusion excludes nothing, and does not fall back to the environment" policy/transcript-corpus.rego kind:mechanism +// carried: "a subagent stream is not an independent session" crates/batten/src/transcript.rs kind:mechanism crates/batten/tests/it/transcript_corpus.rs +// carried: "a transcript carrying only tool results has nobody in it" crates/batten/src/transcript.rs kind:mechanism crates/batten/tests/it/transcript_corpus.rs +// carried: "two files carrying one session are one session" crates/batten/src/transcript.rs kind:mechanism crates/batten/tests/it/transcript_corpus.rs +// carried: "a line this build cannot decode yields nothing rather than a failure to look" crates/batten/src/transcript.rs kind:mechanism crates/batten/tests/it/transcript_corpus.rs +// carried: "excluding the asking session turns its own transcript into zero" crates/batten/src/transcript.rs kind:mechanism crates/batten/tests/it/transcript_corpus.rs +// carried: "an explicitly empty exclusion excludes nothing, and does not fall back to the environment" crates/batten/src/transcript.rs kind:mechanism crates/batten/tests/it/transcript_corpus.rs // carried: "an absent root is exit 2, never a verdict about a corpus nobody looked at" policy/transcript-corpus.rego kind:mechanism -// carried: "the report is two counts and carries no byte of any transcript" policy/transcript-corpus.rego kind:mechanism -// changed: "a malformed threshold is exit 2" mise.toml argument validation is the PRODUCER's, so it moved with the census it guards: `transcript_census.py` refuses a non-numeric threshold at exit 2 and writes nothing, and `a_malformed_threshold_refuses_and_writes_nothing` below drives that. The module never sees an argument to malform -// changed: "more arguments than the contract names is exit 2" mise.toml the same split one argument over, and for the same reason: the arity contract belongs to the thing that takes the arguments +// carried: "the report is two counts and carries no byte of any transcript" crates/batten/src/transcript.rs kind:mechanism crates/batten/tests/it/transcript_corpus.rs +// changed: "a malformed threshold is exit 2" mise.toml argument validation is the PRODUCER's, so it moved with the census it guards. It is now `record derive transcript-corpus`, which refuses a non-numeric threshold as a USAGE error — exit 1 under the engine's one 0/1/2/3 table, where the retired program spent 2 — and writes nothing either way. `a_malformed_threshold_refuses_and_writes_nothing` below drives both halves. The module never sees an argument to malform +// changed: "more arguments than the contract names is exit 2" mise.toml the arity contract belongs to the thing that takes the arguments, and under named `--input` flags "too many positionals" ceases to exist as a concept. What replaces it is stricter rather than weaker: an input key no family declares is a usage error, which `an_input_key_the_family_does_not_read_is_a_usage_error` pins one tier over — a misspelled input can no longer exit clean from a reading that ran on something else +// changed: "the exclusion defaults from the environment when no argument names one" mise.toml the environment read is the PRODUCER's, not the reading's: the task adds `--input exclude=` only when `BATTEN_SESSION_ID` is set, which is what keeps absent and present-but-empty two different claims at the verb. The reading takes `Option<&str>` and `an_explicitly_empty_exclusion_excludes_nothing` pins the distinction in the engine module // withdrawn: "the refusal names what would raise the number, not just the arithmetic" the sentence is the `[[verdict]]` row's `class` now, which is config a reviewer reads rather than a string a case greps. `verdict declare refused` already refuses a class that is an override alone, and `remedy-authorship` holds the prose; a case re-asserting the wording here would be a second authority over it // withdrawn: "the refusal does not tell the reader the count can never rise" the same row's `class`, and the same reason. Both cases asserted over a shell `echo` that no longer exists, and the claim they protected — that a low count is a PROGRESS reading rather than a permanent state — is stated in the class and in the module header where a reader meets it @@ -106,34 +112,6 @@ fn record(dir: &std::path::Path, lines: &str) { ); } -/// Run the REAL census over a scratch transcript root, exactly as the producer -/// does. `exclude` is `None` for an absent argument and `Some("")` for an -/// explicitly empty one — the distinction the contract turns on. -#[expect( - clippy::disallowed_types, - reason = "stays, and it is the subject under test rather than a convenience: the census moved out of the dying program into `mise-tasks/transcript_census.py` precisely so a compiled case could drive it, and a harness re-implementing the independence rules in Rust would be a second authority over what counts as a session" -)] -fn census( - root: &std::path::Path, - threshold: &str, - exclude: Option<&str>, - env: Option<&str>, -) -> std::process::Output { - let mut command = std::process::Command::new("python3"); - command - .arg("../../mise-tasks/transcript_census.py") - .arg(root) - .arg(threshold); - if let Some(value) = exclude { - command.arg(value); - } - match env { - Some(value) => command.env("BATTEN_SESSION_ID", value), - None => command.env_remove("BATTEN_SESSION_ID"), - }; - command.output().expect("the census runs") -} - fn counted(done: &std::process::Output) -> String { String::from_utf8_lossy(&done.stdout).into_owned() } @@ -212,7 +190,11 @@ fn the_report_is_two_counts_and_carries_no_byte_of_any_transcript() { )], ); - let done = census(&held, "2", Some(""), None); + let dir = repo("secretive"); + // `Some("")` is an EXPLICITLY EMPTY exclusion — "exclude nothing" — which is + // a different claim from naming none at all, and the one that keeps the + // count at 1 here. + let done = derive(&dir, &held, "2", Some("")); let said = format!( "{}{}", counted(&done), @@ -226,165 +208,150 @@ fn the_report_is_two_counts_and_carries_no_byte_of_any_transcript() { assert!(said.contains("sessions 1"), "only the counts do\n{said}"); } -// --- the census itself ------------------------------------------------------- - -#[test] -fn an_empty_root_is_zero_independent_sessions_which_is_an_answer() { - let held = root("empty", &[("keep.txt", "not a transcript\n")]); - let done = census(&held, "2", Some(""), None); - assert!(done.status.success(), "an empty root is not a failure"); - assert!( - counted(&done).contains("sessions 0"), - "zero is an answer\n{}", - counted(&done) - ); +// --- the census, over the real verb ------------------------------------------ + +/// Drive the REAL census the producer runs, through the REAL verb. +/// +/// `crates/batten/src/transcript.rs` is the one authority on what counts as an +/// independent session — beside the parse that already owns this host's format, +/// rather than in a module of its own. Its `#[cfg(test)] mod tests` asserts each +/// independence rule directly, plus three the retired program never had: a +/// nested project directory, an empty `sessionId`, and an assistant turn. +/// +/// What THIS tier adds is the half a unit test cannot reach: that the engine +/// carries the count into a record the real module then decides over. +fn derive( + dir: &std::path::Path, + transcripts: &std::path::Path, + threshold: &str, + exclude: Option<&str>, +) -> std::process::Output { + let mut args = vec![ + "record".to_owned(), + "derive".to_owned(), + "transcript-corpus".to_owned(), + "--input".to_owned(), + format!("root={}", transcripts.display()), + "--input".to_owned(), + format!("threshold={threshold}"), + ]; + if let Some(value) = exclude { + args.push("--input".to_owned()); + args.push(format!("exclude={value}")); + } + let borrowed: Vec<&str> = args.iter().map(String::as_str).collect(); + run_with_stdin(dir, &borrowed, "") } #[test] -fn a_subagent_stream_is_not_an_independent_session() { - // CLOUD-326 section 8.1 recorded "one session plus five subagent - // transcripts" and correctly called that N=1. - let held = root( - "sidechain", - &[( - "sub.jsonl", - r#"{"type":"user","sessionId":"beta","isSidechain":true,"message":{"content":[{"type":"text","text":"hi"}]}}"#, - )], +fn the_verb_derives_a_thin_corpus_into_the_finding() { + let dir = repo("derive-thin"); + let transcripts = root("derive-thin", &[("a.jsonl", &authored("only-one"))]); + let written = derive(&dir, &transcripts, "2", None); + assert!( + written.status.success(), + "the derivation lands: {}", + String::from_utf8_lossy(&written.stderr) ); - let done = census(&held, "2", Some(""), None); assert!( - counted(&done).contains("sessions 0"), - "a sidechain is the orchestrator's own turns\n{}", - counted(&done) + counted(&written).contains("sessions 1"), + "the count reaches the record\n{}", + counted(&written) ); -} -#[test] -fn a_transcript_carrying_only_tool_results_has_nobody_in_it() { - // A `tool_result` also arrives as a user record; that is the harness handing - // work back, not a person speaking. - let held = root( - "toolonly", - &[( - "t.jsonl", - r#"{"type":"user","sessionId":"gamma","message":{"content":[{"type":"tool_result","content":"out"}]}}"#, - )], - ); - let done = census(&held, "2", Some(""), None); - assert!( - counted(&done).contains("sessions 0"), - "nobody was in it\n{}", - counted(&done) + let decided = run(&dir, &["check"]); + assert_eq!( + decided.status.code(), + Some(2), + "one session is not a corpus\n{}", + String::from_utf8_lossy(&decided.stderr) ); } #[test] -fn two_files_carrying_one_session_are_one_session() { - let held = root( - "split", +fn the_verb_derives_a_sufficient_corpus_into_silence() { + let dir = repo("derive-enough"); + let transcripts = root( + "derive-enough", &[ - ("one.jsonl", &authored("delta")), - ("two.jsonl", &authored("delta")), + ("a.jsonl", &authored("one")), + ("b.jsonl", &authored("two")), + ("c.jsonl", &authored("three")), ], ); - let done = census(&held, "2", Some(""), None); + let written = derive(&dir, &transcripts, "2", None); + assert!(written.status.success(), "the derivation lands"); assert!( - counted(&done).contains("sessions 1"), - "distinct sessions, not distinct files\n{}", - counted(&done) + counted(&written).contains("sessions 3"), + "{}", + counted(&written) ); -} -#[test] -fn a_line_this_build_cannot_decode_yields_nothing_rather_than_a_failure_to_look() { - // The format is a HOST's and it moves. `transcript.rs`'s - // forward-compatibility law, applied at the same boundary from this side. - let held = root( - "undecodable", - &[( - "mixed.jsonl", - &format!("{{not json at all\n{}\n", authored("epsilon")), - )], - ); - let done = census(&held, "2", Some(""), None); - assert!(done.status.success(), "one bad line is not a failed census"); - assert!( - counted(&done).contains("sessions 1"), - "the readable line still counts\n{}", - counted(&done) + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "three distinct sessions satisfy the threshold\n{}", + String::from_utf8_lossy(&quiet.stderr) ); } +/// THE QUESTION COULD NOT BE ASKED, so the producer writes NOTHING. An absent +/// record is "the producer did not run", which must never be spelled the same +/// way as a root that was walked and held no transcripts. #[test] -fn excluding_the_asking_session_turns_its_own_transcript_into_zero() { - let held = root("selfonly", &[("self.jsonl", &authored("zeta"))]); - let done = census(&held, "2", Some("zeta"), None); - assert!( - counted(&done).contains("sessions 0"), - "a session is not independent evidence about itself\n{}", - counted(&done) +fn an_absent_root_is_could_not_look_and_writes_nothing() { + let dir = repo("derive-no-root"); + let refused = derive(&dir, std::path::Path::new("/nowhere/at/all"), "2", None); + assert_eq!( + refused.status.code(), + Some(1), + "a root that is not there is a usage error\n{}", + String::from_utf8_lossy(&refused.stderr) ); -} -#[test] -fn the_exclusion_defaults_from_the_environment_when_no_argument_names_one() { - let held = root("envexcl", &[("self.jsonl", &authored("eta"))]); - let done = census(&held, "2", None, Some("eta")); - assert!( - counted(&done).contains("sessions 0"), - "the ambient session id applies when nothing names one\n{}", - counted(&done) + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "and nothing was written, so the module says nothing\n{}", + String::from_utf8_lossy(&quiet.stderr) ); } #[test] -fn an_explicitly_empty_exclusion_excludes_nothing_and_does_not_fall_back_to_the_environment() { - // ABSENT AND EMPTY ARE DIFFERENT CLAIMS. Defaulting an explicit empty would - // launder "exclude nothing" into "exclude whatever the host happens to say". - let held = root("emptyexcl", &[("self.jsonl", &authored("theta"))]); - let done = census(&held, "2", Some(""), Some("theta")); - assert!( - counted(&done).contains("sessions 1"), - "an explicit empty excludes nothing\n{}", - counted(&done) +fn a_malformed_threshold_refuses_and_writes_nothing() { + let dir = repo("derive-bad-threshold"); + let transcripts = root("derive-bad-threshold", &[("a.jsonl", &authored("one"))]); + let refused = derive(&dir, &transcripts, "two", None); + assert_eq!( + refused.status.code(), + Some(1), + "a threshold that is not a number is a usage error\n{}", + String::from_utf8_lossy(&refused.stderr) ); -} -#[test] -fn an_absent_root_is_could_not_look_and_writes_nothing() { - // Never a verdict about a corpus nobody looked at. - let missing = scratch("transcript-root-missing").join("nowhere"); - let done = census(&missing, "2", Some(""), None); - assert_eq!(done.status.code(), Some(2), "could not look"); - assert!( - counted(&done).is_empty(), - "and writes nothing, so the module reads silence\n{}", - counted(&done) + let quiet = run(&dir, &["check"]); + assert_eq!( + quiet.status.code(), + Some(0), + "and the refusal wrote no record\n{}", + String::from_utf8_lossy(&quiet.stderr) ); } +/// EXCLUDING YOURSELF IS THE POINT: a literal fitted to the single transcript it +/// was derived from is the unmeasured-shape failure the method exists to +/// prevent, so counting yourself is worse than counting nothing. #[test] -fn a_malformed_threshold_refuses_and_writes_nothing() { - let held = root("badmin", &[("a.jsonl", &authored("iota"))]); - let done = census(&held, "two", Some(""), None); - assert_eq!(done.status.code(), Some(2), "a malformed threshold refuses"); - assert!(counted(&done).is_empty(), "and writes nothing"); -} - -#[test] -#[expect( - clippy::disallowed_types, - reason = "stays: the arity contract belongs to the thing that takes the arguments, so asserting it means invoking the producer with the wrong arity" -)] -fn more_arguments_than_the_contract_names_refuses() { - let held = root("arity", &[("a.jsonl", &authored("kappa"))]); - let done = std::process::Command::new("python3") - .arg("../../mise-tasks/transcript_census.py") - .arg(&held) - .arg("2") - .arg("") - .arg("extra") - .output() - .expect("the census runs"); - assert_eq!(done.status.code(), Some(2), "an over-long call refuses"); +fn excluding_the_asking_session_turns_its_own_transcript_into_zero() { + let dir = repo("derive-exclude"); + let transcripts = root("derive-exclude", &[("a.jsonl", &authored("mine"))]); + let written = derive(&dir, &transcripts, "2", Some("mine")); + assert!(written.status.success(), "the derivation lands"); + assert!( + counted(&written).contains("sessions 0"), + "{}", + counted(&written) + ); } diff --git a/mise-tasks/transcript_census.py b/mise-tasks/transcript_census.py deleted file mode 100644 index 09212abc8..000000000 --- a/mise-tasks/transcript_census.py +++ /dev/null @@ -1,144 +0,0 @@ -# How many INDEPENDENT session transcripts a host carries (CLOUD-388, -# CLOUD-651, ported off `mise-tasks/transcript-corpus-check.sh` under -# CLOUD-1717). -# -# THIS IS THE STEP, NEVER THE DECISION. It counts; whether the count is enough -# is `policy/transcript-corpus.rego`'s question. §5 makes `check` `read` and -# incapable of walking a host filesystem, so the census stays here. -# -# A `.py` SIBLING RATHER THAN A TASK BODY, for the reason `cargo_graph.py` gives: -# ten of the dying suite's seventeen cases are about WHAT COUNTS as an -# independent session, and they are the whole substance of the gate. Inlined into -# the task body they would be assertable by nothing; -# `crates/batten/tests/it/transcript_corpus.rs` drives this file directly, so -# they carry. `shell-retirement.rego:159-163` excludes `.py` from -# `under_mise_tasks`, so this adds no shell rule. -# -# WHAT COUNTS AS INDEPENDENT, and why it is not "one file, one session". -# -# A transcript is independent evidence when it belongs to a DIFFERENT session -# that a person actually drove. Three things therefore do not count: -# -# a subagent stream — `isSidechain: true`. CLOUD-326's section 8.1 recorded -# "one session plus five subagent transcripts" and correctly called that N=1; -# counting the five would inflate the corpus with the orchestrator's own turns -# wearing different file names. -# -# a transcript with nobody in it — a `tool_result` also arrives as a `user` -# record, and that is the harness handing work back rather than a person -# speaking. The boundary test is `finding-sink-check`'s pass 1, reused rather -# than re-derived. -# -# the asking session — a literal fitted to the single transcript it was derived -# from is the unmeasured-shape failure the method exists to prevent, so -# counting yourself is worse than counting nothing. -# -# And two files carrying one session are one session: distinct SESSIONS, not -# distinct files. -# -# A LINE THIS BUILD CANNOT DECODE YIELDS NOTHING RATHER THAN A FAILURE TO LOOK. -# The format is a HOST's and it moves, so an undecodable line contributes no id -# instead of turning the whole census into could-not-look — `transcript.rs`'s -# forward-compatibility law, applied at the same boundary from this side. -# -# Output is the record's own lines, on stdout, for `batten record named`: -# -# sessions independent sessions found -# threshold what the caller asked for -# -# POINTER-ONLY IS A SECURITY PROPERTY HERE, not a style one (non-negotiable rule -# 4): two counts, never a path, never a session id, never a byte of any -# transcript. A transcript is the richest source of secrets this repository can -# be pointed at. - -import json -import os -import sys - - -def authored_session_id(line): - """The session id of an authored, non-sidechain user record, or None.""" - try: - record = json.loads(line) - except ValueError: - # The host's format moves. An undecodable line is not a failed census. - return None - if not isinstance(record, dict): - return None - if record.get("isSidechain") is True: - return None - if record.get("type") != "user": - return None - content = (record.get("message") or {}).get("content") - if isinstance(content, str): - authored = True - elif isinstance(content, list): - authored = any( - isinstance(block, dict) and block.get("type") == "text" - for block in content - ) - else: - authored = False - if not authored: - return None - return record.get("sessionId") or None - - -def census(root, exclude): - """Distinct independent session ids under `root`, excluding `exclude`.""" - seen = set() - # Sorted for byte-stability (house style section 6): the same root yields the - # same count however the filesystem chose to order itself. - for base, _, names in sorted(os.walk(root)): - for name in sorted(names): - if not name.endswith(".jsonl"): - continue - path = os.path.join(base, name) - try: - with open(path, "r", encoding="utf-8", errors="replace") as handle: - for line in handle: - found = authored_session_id(line) - if found is None: - continue - # The FIRST authored id in the file identifies it; a file - # with none at all is a subagent stream or a transcript - # nobody was in. - if found != exclude: - seen.add(found) - break - except OSError: - # One unreadable file is not a failed census of the rest. - continue - return len(seen) - - -def main(argv): - # ABSENT AND EMPTY ARE DIFFERENT CLAIMS about the exclusion, which is why the - # argument is counted rather than defaulted. An argument that is PRESENT and - # empty is a caller saying "exclude nothing"; an argument that is absent is a - # caller saying nothing, and only then does the ambient session id apply. - # Defaulting an explicit empty would launder one into the other. - if len(argv) not in (3, 4): - sys.stderr.write("usage: transcript_census.py [exclude]\n") - return 2 - root, threshold = argv[1], argv[2] - if not threshold.isdigit(): - sys.stderr.write("transcript-census: threshold must be a whole number\n") - return 2 - if len(argv) == 4: - exclude = argv[3] or None - else: - exclude = os.environ.get("BATTEN_SESSION_ID") or None - if not os.path.isdir(root): - # The question could not be asked. Write NOTHING — an absent record is - # "the producer did not run", which must not be spelled the same way as - # a root that was walked and held no transcripts. - sys.stderr.write("transcript-census: no transcript root at %s\n" % root) - return 2 - sys.stdout.write("sessions %d\n" % census(root, exclude)) - sys.stdout.write("threshold %s\n" % threshold) - return 0 - - -if __name__ == "__main__": - sys.exit(main(sys.argv)) diff --git a/mise.toml b/mise.toml index c9145ddc8..32e2cbd09 100644 --- a/mise.toml +++ b/mise.toml @@ -2297,8 +2297,24 @@ description = "Effect: count this host's independent session transcripts against shell = "bash -c" run = """ root="${BATTEN_TRANSCRIPT_ROOT:-$HOME/.claude/projects}" -python3 mise-tasks/transcript_census.py "$root" "${1:-2}" \ - | cargo run --quiet -p batten -- record named transcript-corpus +threshold="${1:-2}" + +# THE WALK IS THIS TASK'S ROOT TO NAME — §5 keeps a host-filesystem walk +# outside `check` — and WHAT COUNTS as an independent session is +# `crates/batten/src/transcript.rs`, beside the parse that already owns +# this host format (CLOUD-1717). +# +# ABSENT AND PRESENT-BUT-EMPTY ARE DIFFERENT CLAIMS about the exclusion, +# which is why the flag is added rather than always passed: an unset +# `BATTEN_SESSION_ID` is a caller saying nothing, and `--input exclude=` +# would be a caller saying "exclude nothing". Passing the empty one +# always would launder the first into the second. +set -- --input "root=$root" --input "threshold=$threshold" +if [ -n "${BATTEN_SESSION_ID:-}" ]; then + set -- "$@" --input "exclude=$BATTEN_SESSION_ID" +fi + +cargo run --quiet -p batten -- record derive transcript-corpus "$@" """ [tasks.macos-link-record] From 8a24293b3b2602c43f5e6e58f61a8401b2b090dc Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sat, 19 Sep 2026 22:39:14 +0000 Subject: [PATCH 27/32] feat(record): one activated-edge walk in Rust, and the last three .py are gone MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Last of six. `evaluator-closure` and `macos-link` each carried their own copy of an activated-edge dependency walk, and both headers said so in prose: "the activation reading is deliberately IDENTICAL ... if one is corrected, correct both." That is a rule with no mechanism, which is half a change. `crates/batten/src/cargo_graph.rs` is the mechanism — one walk, so the two cannot disagree. NO CRATE NAME REACHES THE CORE. My first draft would have moved IO_CRATES, FRAMEWORK_CRATES, VENDORED_LINKS and the evaluator's own name into `crates/batten/src`, which is non-negotiable rule 1 broken flatly: those are consumer facts. They are four `[[pattern]]` rows now, resolved the way `run_closes` resolves its key grammar, and the walk chooses roots by a PREDICATE the caller supplies. The lists are better there than they were in a script — reviewable data a gate reads, which is rule 3 too. Fourteen in-module cases over fabricated graphs no real resolve can be made to have on demand, which is why the walk is a pure function. Three are new: a cycle terminates, a document that is not a resolve is an empty graph rather than an error, and an enabled feature name is itself a key. The three `#MUTANT` rows that used to be stated twice are stated once, over the code they actually mutate — CLOUD-1369's engine route, whose whole point was that `#MUTANT` could not live in Rust before it. Verified against the real tree: `closure 45` with no IO crate reachable, `scanned 313` with no SDK-requiring dependency. Both tiers 10/10. `mise-tasks/` now holds no `.py` at all, and `shell retire partial` — the arm this branch added — is green because the tree is clean rather than because it cannot see. Refs: CLOUD-1717 Admits-Rule: protected-mutation Admits-Verdict: path write refused Admits-Subject: .serena/memories/core.md Admits-Id: d9b26fdf47c2a517f8e066bbb3b61be911a721f4e399ba4c2fc9d45238725c14 --- .serena/memories/core.md | 22 + batten.toml | 43 ++ crates/batten/src/cargo_graph.rs | 537 ++++++++++++++++++++ crates/batten/src/lib.rs | 1 + crates/batten/src/record.rs | 124 ++++- crates/batten/tests/it/evaluator_closure.rs | 83 +-- crates/batten/tests/it/macos_link.rs | 121 +++-- mise-tasks/cargo_graph.py | 127 ----- mise-tasks/evaluator-closure.py | 85 ---- mise-tasks/macos-link.py | 102 ---- mise.toml | 16 +- policy/module-layering.rego | 19 + 12 files changed, 868 insertions(+), 412 deletions(-) create mode 100644 crates/batten/src/cargo_graph.rs delete mode 100644 mise-tasks/cargo_graph.py delete mode 100644 mise-tasks/evaluator-closure.py delete mode 100644 mise-tasks/macos-link.py diff --git a/.serena/memories/core.md b/.serena/memories/core.md index f423101d5..6e1bb658b 100644 --- a/.serena/memories/core.md +++ b/.serena/memories/core.md @@ -2469,6 +2469,28 @@ judge_fingerprint`, its own domain tag), so a caller can reference content it the tests. Ported off `mise-tasks/signer_posture.py`; `signing-posture-repair` no longer classifies a second time but reads the posture off the record the producer just wrote. +- `cargo_graph.rs` — the ACTIVATED dependency graph, read from a `cargo metadata` + document (CLOUD-1717). ONE WALK, TWO GATES, and that is the whole reason it + exists: `evaluator-closure` asks whether an IO-bearing crate is reachable from + one package's node, `macos-link` whether anything built needs a platform SDK to + link; the questions differ only in their ROOTS and in what they look for once + there. Both programs carried their own copy and both headers said so in prose — + _"if one is corrected, correct both"_ — which is a rule with no mechanism, and + this is the mechanism. THE ACTIVATION FILTER, never the whole resolve: `cargo +metadata` lists every package the resolver CONSIDERED, so scanning it asks + "could some configuration reach X" where both callers mean "does this one". + Reverting that is the measured defect — an unactivated optional dependency + reaching no framework and never compiled made the link gate refuse a link + `darwin-link` then completed on the same tree. Three rules live here and + nowhere else: an optional dep is an edge only if activated; the weak form + `foo?/bar` is NOT an activation; a `dev`-only edge is dropped unless its owner + is a workspace member. A fourth is the fail-safe: an edge the manifest does not + explain is KEPT, because unexplained means unmeasured. NO CRATE NAME REACHES + IT — which package is the evaluator, which bear IO, which need an SDK and which + vendor what they link are consumer facts in `[[pattern]]` rows (rule 1), so + roots are chosen by a PREDICATE the caller supplies. Ported off + `mise-tasks/cargo_graph.py` and its two callers; carries the three `#MUTANT` + rows that used to be stated twice, now stated once over the code they mutate. - `probe_verdict.rs` — which of three things a probe build did, from its exit status and its log (CLOUD-418, CLOUD-1717). THE VERDICT IS THE HARNESS'S OWN LINE, NEVER THE EXIT CODE ALONE: `cargo test` exits non-zero for a compile diff --git a/batten.toml b/batten.toml index 59be7ce01..6261ad20b 100644 --- a/batten.toml +++ b/batten.toml @@ -1730,6 +1730,49 @@ regex = '(^|/)tasks/[A-Za-z0-9._-]+\.output$' id = "egress-resolver-host" regex = 'api\.github\.com' +# CLOUD-1717's four consumer facts, moved out of two `.py` siblings and NOT into +# `crates/batten/src`. Non-negotiable rule 1: the core stays repo-agnostic, and a +# `const IO_CRATES` in the engine would be a consumer identifier in it. The walk +# that reads them is `crates/batten/src/cargo_graph.rs` and names no crate at all. +# +# They are also better here than they were in a script: a list in a `[[pattern]]` +# row is reviewable data a gate reads, which is rule 3 as well as rule 1. + +# The package whose sub-closure `evaluator-closure` walks. Named ONCE: it is the +# node the walk starts from AND the word the refusal uses, and two spellings of +# it is how a rename turns that gate silent instead of red. +[[pattern]] +id = "evaluator-package" +regex = '^regorus$' + +# `Cargo.toml`'s pin names. Not a heuristic and not "crates that look networky": +# this is the list the manifest comment claims is absent, restated here as the +# thing that is looked for. +[[pattern]] +id = "evaluator-io-crate" +regex = '^(reqwest|jsonschema|hyper|rustls|openssl-sys|native-tls|ring|globset|glob)$' + +# Crates that pull in an Apple system framework. Extend as new ones are met — and +# when you do, prefer proving it with a link (`darwin-link`) over trusting this +# list. +[[pattern]] +id = "sdk-framework-crate" +regex = '^(security-framework|security-framework-sys|core-foundation|core-foundation-sys|native-tls|openssl-sys|cocoa|objc|objc2|system-configuration|system-configuration-sys)$' + +# Crates whose `links` key names a library they VENDOR AND COMPILE THEMSELVES, +# reaching no Apple system framework — so rule 1's proxy is wrong about them. +# +# ADDING A NAME HERE REQUIRES A LINK, never a reading of the crate. Measured +# 2026-08-21: the gate refused `tree-sitter` and `tree-sitter-language`, and +# `darwin-link` then linked the same tree with no SDK present. A gate that +# refuses what the linker accepts is measuring something other than what it +# names. The fail-safe is unchanged in the direction that matters: an UNKNOWN +# `links` crate is still a finding, so this narrows the gate rather than opening +# it. +[[pattern]] +id = "vendored-links-crate" +regex = '^(tree-sitter|tree-sitter-language)$' + [[pattern]] id = "release-tag" regex = '^v[0-9]+\.[0-9]+\.[0-9]+$' diff --git a/crates/batten/src/cargo_graph.rs b/crates/batten/src/cargo_graph.rs new file mode 100644 index 000000000..79847c49e --- /dev/null +++ b/crates/batten/src/cargo_graph.rs @@ -0,0 +1,537 @@ +//! The ACTIVATED dependency graph, read from a `cargo metadata` document +//! (CLOUD-1717). +//! +//! # One walk, two gates, and that is the whole reason this module exists +//! +//! `evaluator-closure` asks whether an IO-bearing crate is reachable from one +//! package's node; `macos-link` asks whether anything in the built graph needs a +//! real platform SDK to link. The questions differ only in their ROOTS and in +//! what they look for once there — the reachability underneath is the same, down +//! to the weak-dependency rule. +//! +//! Both programs carried their own copy, and both said so in prose: *"the +//! activation reading is deliberately IDENTICAL … if one is corrected, correct +//! both."* That is a rule with no mechanism, which is half a change. This module +//! is the mechanism: there is one walk to correct, so the two cannot disagree. +//! +//! # Why the activation filter rather than the whole resolve +//! +//! `cargo metadata` lists every package the resolver CONSIDERED, including +//! optional dependencies nothing turned on. Scanning that asks "could some +//! configuration of this tree reach X" where both callers mean "does this one". +//! +//! Reverting to the whole resolve is the defect this exists to prevent, and it +//! was measured: an embedded-logging crate, an unactivated optional dependency +//! reaching no platform framework and never compiled, made the link gate refuse +//! a link that then completed on the same tree. +//! +//! # No crate name reaches this module +//! +//! Non-negotiable rule 1: the core stays repo-agnostic. Which package is the +//! evaluator, which crates bear IO, which need an SDK and which are vendored are +//! all CONSUMER facts, and they live in that consumer's `[[pattern]]` rows. What +//! is here is the graph reading, which is true of any Cargo workspace. + +// THE THREE ROWS THAT USED TO BE STATED TWICE, now stated once over the code +// they actually mutate (CLOUD-1369's route, CLOUD-1717's use of it). Each names +// a unit case below, because a fabricated graph is what exercises the walk and +// no real resolve can be made to have these topologies on demand. +// +// Reverting the activation filter to the whole resolve is the defect the first +// row guards: an optional dependency nobody enabled reads as linked, and the +// gate refuses a link that succeeds. +//MUTANT-SUITE crates/batten/src/cargo_graph.rs +//MUTANT graph-scans-unactivated|s@^ if key.is_some_and(|key| enabled.contains(key)) {$@ if true {@|the_same_optional_dependency_once_activated_is_reached +//MUTANT graph-weak-dep-activates|s@^ && !head.ends_with(.?.)$@@|a_weak_reference_is_not_an_activation +//MUTANT graph-keeps-dev-edges|s@^ if dev_only(dep) && !is_member {$@ if false {@|a_dev_dependency_of_a_dependency_is_not_in_the_built_closure + +use std::collections::{BTreeMap, BTreeSet}; + +use serde_json::Value; + +/// A resolved `cargo metadata` document, walked along ACTIVATED edges only. +#[derive(Debug, Default)] +pub struct Graph { + packages: BTreeMap, + nodes: BTreeMap, + members: BTreeSet, +} + +/// The dependency names this package's enabled features actually turn on. +/// +/// THREE SPELLINGS REACH A DEPENDENCY and all three must be read, or an +/// activated dep looks dormant: the implicit feature (a bare `foo`), the +/// namespaced form (`dep:foo`), and enabling one of the dep's own features +/// (`foo/bar`). +/// +/// THE WEAK FORM (`foo?/bar`) IS DELIBERATELY NOT ONE — it applies only if +/// something else already activated the dep, and reading it as an activation +/// drifts back toward the whole-resolve scan this replaces. +#[must_use] +pub fn activated_keys(package: &Value, enabled: &BTreeSet) -> BTreeSet { + let mut keys = enabled.clone(); + let declared = package.get("features").and_then(Value::as_object); + for feature in enabled { + let Some(tokens) = declared + .and_then(|features| features.get(feature)) + .and_then(Value::as_array) + else { + continue; + }; + for token in tokens.iter().filter_map(Value::as_str) { + if let Some(name) = token.strip_prefix("dep:") { + keys.insert(name.to_owned()); + } else if let Some((head, _)) = token.split_once('/') + && !head.ends_with('?') + { + keys.insert(head.to_owned()); + } + } + } + keys +} + +impl Graph { + /// Index a `cargo metadata` document by package id. + /// + /// A document with no `packages` array is not a resolve, and yields an empty + /// graph rather than an error: the caller's own could-not-look arm decides + /// what an unreadable document means, and a graph that answered "nothing is + /// reachable" while claiming to have looked would be the vacuous pass. + #[must_use] + pub fn from_metadata(meta: &Value) -> Graph { + let packages = meta + .get("packages") + .and_then(Value::as_array) + .map(|found| { + found + .iter() + .filter_map(|package| { + let id = package.get("id")?.as_str()?; + Some((id.to_owned(), package.clone())) + }) + .collect() + }) + .unwrap_or_default(); + let nodes = meta + .get("resolve") + .and_then(|resolve| resolve.get("nodes")) + .and_then(Value::as_array) + .map(|found| { + found + .iter() + .filter_map(|node| { + let id = node.get("id")?.as_str()?; + Some((id.to_owned(), node.clone())) + }) + .collect() + }) + .unwrap_or_default(); + let members = meta + .get("workspace_members") + .and_then(Value::as_array) + .map(|found| { + found + .iter() + .filter_map(Value::as_str) + .map(ToOwned::to_owned) + .collect() + }) + .unwrap_or_default(); + Graph { + packages, + nodes, + members, + } + } + + /// How many packages the resolve holds nodes for. + #[must_use] + pub fn scanned(&self) -> usize { + self.nodes.len() + } + + /// The package name behind an id. + #[must_use] + pub fn name_of(&self, id: &str) -> Option<&str> { + self.packages.get(id)?.get("name")?.as_str() + } + + /// The native library a package declares it links against, if any. + #[must_use] + pub fn links_of(&self, id: &str) -> Option<&str> { + self.packages.get(id)?.get("links")?.as_str() + } + + /// The activated dependency ids of one node. + #[must_use] + pub fn edges(&self, node_id: &str) -> Vec { + let (Some(node), Some(package)) = (self.nodes.get(node_id), self.packages.get(node_id)) + else { + return Vec::new(); + }; + let features: BTreeSet = node + .get("features") + .and_then(Value::as_array) + .map(|found| { + found + .iter() + .filter_map(Value::as_str) + .map(ToOwned::to_owned) + .collect() + }) + .unwrap_or_default(); + let enabled = activated_keys(package, &features); + let is_member = self.members.contains(node_id); + + let mut reached = Vec::new(); + let Some(deps) = node.get("deps").and_then(Value::as_array) else { + return reached; + }; + for dep in deps { + let Some(pkg) = dep.get("pkg").and_then(Value::as_str) else { + continue; + }; + let Some(target_name) = self.name_of(pkg) else { + continue; + }; + // A DEV-DEPENDENCY OF A *DEPENDENCY* IS NEVER BUILT. One of a + // workspace member is: the test binaries link too. + if dev_only(dep) && !is_member { + continue; + } + let matching: Vec<&Value> = package + .get("dependencies") + .and_then(Value::as_array) + .map(|declared| { + declared + .iter() + .filter(|entry| { + entry.get("name").and_then(Value::as_str) == Some(target_name) + }) + .collect() + }) + .unwrap_or_default(); + if matching.is_empty() { + // AN EDGE THE MANIFEST DOES NOT EXPLAIN: keep it rather than + // drop it. Unexplained means unmeasured, and unmeasured fails + // closed. + reached.push(pkg.to_owned()); + continue; + } + for entry in matching { + if entry.get("optional").and_then(Value::as_bool) != Some(true) { + reached.push(pkg.to_owned()); + break; + } + let key = entry + .get("rename") + .and_then(Value::as_str) + .or_else(|| entry.get("name").and_then(Value::as_str)); + if key.is_some_and(|key| enabled.contains(key)) { + reached.push(pkg.to_owned()); + break; + } + } + } + reached + } + + /// Every package id reachable from `roots` along activated edges. + #[must_use] + pub fn reachable(&self, roots: I) -> BTreeSet + where + I: IntoIterator, + S: Into, + { + let mut seen = BTreeSet::new(); + let mut frontier: Vec = roots.into_iter().map(Into::into).collect(); + while let Some(current) = frontier.pop() { + if !seen.insert(current.clone()) { + continue; + } + frontier.extend(self.edges(¤t)); + } + seen + } + + /// The workspace members the resolve holds nodes for. + /// + /// The starting set for a question about everything this tree builds. + #[must_use] + pub fn member_roots(&self) -> Vec { + self.members + .iter() + .filter(|id| self.nodes.contains_key(*id)) + .cloned() + .collect() + } + + /// Every node whose package name satisfies `wanted`. + /// + /// The starting set for a question about ONE package's sub-closure. A + /// PREDICATE rather than a name, because WHICH package a consumer means is + /// that consumer's fact and belongs in its `[[pattern]]` rows — non- + /// negotiable rule 1 keeps the name out of this crate entirely. + #[must_use] + pub fn roots_matching(&self, wanted: impl Fn(&str) -> bool) -> Vec { + self.packages + .iter() + .filter(|(id, package)| { + package + .get("name") + .and_then(Value::as_str) + .is_some_and(&wanted) + && self.nodes.contains_key(*id) + }) + .map(|(id, _)| id.clone()) + .collect() + } + + /// Every node for a package of this exact name. + #[must_use] + pub fn named_roots(&self, name: &str) -> Vec { + self.roots_matching(|found| found == name) + } + + /// The reachable set, rendered as `(name, id)` pairs sorted by name. + /// + /// Sorted for byte-stability (house style §6): the same document yields the + /// same order however the resolver happened to list its packages. + #[must_use] + pub fn named_in_order(&self, ids: &BTreeSet) -> Vec<(&str, &str)> { + // BOTH HALVES BORROW FROM `self`, never from `ids`: the caller's set is + // a working value and the returned pairs outlive it. + let mut named: Vec<(&str, &str)> = ids + .iter() + .filter_map(|id| { + let (owned, package) = self.packages.get_key_value(id)?; + Some((package.get("name")?.as_str()?, owned.as_str())) + }) + .collect(); + named.sort_unstable(); + named + } +} + +/// Whether every kind on this edge is `dev`. +fn dev_only(dep: &Value) -> bool { + let Some(kinds) = dep.get("dep_kinds").and_then(Value::as_array) else { + // No `dep_kinds` at all is the normal-dependency shape, not a dev one. + return false; + }; + if kinds.is_empty() { + return false; + } + kinds + .iter() + .all(|kind| kind.get("kind").and_then(Value::as_str) == Some("dev")) +} + +#[cfg(test)] +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#[allow(clippy::unwrap_used, clippy::expect_used)] +mod tests { + use super::{Graph, activated_keys}; + + use serde_json::{Value, json}; + + /// A three-package resolve: `root` -> `middle` -> `leaf`. + /// + /// `optional` and `features` are what the activation rules turn on, and + /// `kind` is the edge's dependency kind. Fabricated rather than read from a + /// real resolve, because no real tree can be made to have these topologies + /// on demand — which is the whole reason the walk is a pure function. + fn chain(kind: Option<&str>, optional: bool, features: &[&str]) -> Value { + let dep_kinds = match kind { + Some(kind) => json!([{ "kind": kind }]), + None => json!([{}]), + }; + json!({ + "packages": [ + { + "id": "root 1.0.0", + "name": "root", + "features": { "tls": ["leaf/std"], "std": [] }, + "dependencies": [ + { "name": "leaf", "optional": optional } + ], + }, + { "id": "leaf 1.0.0", "name": "leaf", "dependencies": [] }, + ], + "resolve": { + "nodes": [ + { + "id": "root 1.0.0", + "features": features, + "deps": [ { "pkg": "leaf 1.0.0", "dep_kinds": dep_kinds } ], + }, + { "id": "leaf 1.0.0", "features": [], "deps": [] }, + ] + }, + "workspace_members": ["root 1.0.0"], + }) + } + + fn reaches_leaf(meta: &Value) -> bool { + let graph = Graph::from_metadata(meta); + graph.reachable(graph.member_roots()).contains("leaf 1.0.0") + } + + // --- the three rules that live here and nowhere else -------------------- + + #[test] + fn a_required_dependency_is_always_reached() { + assert!(reaches_leaf(&chain(None, false, &[]))); + } + + /// THE MEASURED DEFECT: an optional dependency nobody enabled reads as + /// linked under a whole-resolve scan, and the gate then refuses a link that + /// succeeds. + #[test] + fn an_unactivated_optional_dependency_is_not_reached() { + assert!(!reaches_leaf(&chain(None, true, &["std"]))); + } + + #[test] + fn the_same_optional_dependency_once_activated_is_reached() { + // The implicit feature: a bare `leaf` in the enabled set. + assert!(reaches_leaf(&chain(None, true, &["leaf"]))); + } + + /// A WEAK REFERENCE IS NOT AN ACTIVATION. `leaf?/std` applies only if + /// something else already activated `leaf`, and reading it as an activation + /// drifts back toward the whole-resolve scan. + #[test] + fn a_weak_reference_is_not_an_activation() { + let mut meta = chain(None, true, &["weak"]); + meta["packages"][0]["features"] = json!({ "weak": ["leaf?/std"], "strong": ["leaf/std"] }); + assert!(!reaches_leaf(&meta), "a weak token activates nothing"); + + let mut strong = chain(None, true, &["strong"]); + strong["packages"][0]["features"] = + json!({ "weak": ["leaf?/std"], "strong": ["leaf/std"] }); + assert!(reaches_leaf(&strong), "and the non-weak one does"); + } + + /// A dev-dependency of a *dependency* is never built. One of a workspace + /// member is: the test binaries link too. + #[test] + fn a_dev_dependency_of_a_dependency_is_not_in_the_built_closure() { + let mut meta = chain(Some("dev"), false, &[]); + // Make `root` a non-member, so its dev edge is the not-built case. + meta["workspace_members"] = json!([]); + let graph = Graph::from_metadata(&meta); + assert!( + !graph + .reachable(graph.named_roots("root")) + .contains("leaf 1.0.0"), + "a dev edge out of a dependency is not built" + ); + } + + #[test] + fn a_dev_dependency_of_a_workspace_member_is_built() { + assert!( + reaches_leaf(&chain(Some("dev"), false, &[])), + "the member's test binaries link it" + ); + } + + /// AN EDGE THE MANIFEST DOES NOT EXPLAIN IS KEPT. Unexplained means + /// unmeasured, and unmeasured fails closed. + #[test] + fn an_edge_the_manifest_does_not_explain_is_kept() { + let mut meta = chain(None, true, &[]); + meta["packages"][0]["dependencies"] = json!([]); + assert!(reaches_leaf(&meta), "fail closed, not open"); + } + + // --- the three spellings that reach a dependency ------------------------- + + #[test] + fn every_spelling_that_reaches_a_dependency_is_read() { + let package = json!({ + "features": { + "implicit": [], + "namespaced": ["dep:alpha"], + "through": ["beta/std"], + "weakly": ["gamma?/std"], + } + }); + let enabled = ["namespaced", "through", "weakly"] + .into_iter() + .map(ToOwned::to_owned) + .collect(); + let keys = activated_keys(&package, &enabled); + assert!(keys.contains("alpha"), "the namespaced form: {keys:?}"); + assert!(keys.contains("beta"), "the feature form: {keys:?}"); + assert!( + !keys.contains("gamma"), + "and the WEAK form is not one: {keys:?}" + ); + } + + #[test] + fn an_enabled_feature_name_is_itself_a_key() { + let enabled = ["leaf".to_owned()].into_iter().collect(); + assert!(activated_keys(&json!({}), &enabled).contains("leaf")); + } + + // --- the roots, and reading the document at all -------------------------- + + #[test] + fn member_roots_are_the_members_the_resolve_holds_nodes_for() { + let graph = Graph::from_metadata(&chain(None, false, &[])); + assert_eq!(graph.member_roots(), vec!["root 1.0.0".to_owned()]); + } + + #[test] + fn named_roots_select_one_packages_nodes() { + let graph = Graph::from_metadata(&chain(None, false, &[])); + assert_eq!(graph.named_roots("leaf"), vec!["leaf 1.0.0".to_owned()]); + assert!(graph.named_roots("nobody").is_empty()); + } + + /// A document that is not a resolve yields an EMPTY graph rather than an + /// error, and the caller's own could-not-look arm decides what that means. A + /// graph that answered "nothing is reachable" while claiming to have looked + /// would be the vacuous pass. + #[test] + fn a_document_that_is_not_a_resolve_is_an_empty_graph() { + let graph = Graph::from_metadata(&json!({})); + assert_eq!(graph.scanned(), 0); + assert!(graph.member_roots().is_empty()); + assert!(graph.reachable(Vec::::new()).is_empty()); + } + + #[test] + fn a_cycle_terminates() { + let meta = json!({ + "packages": [ + { "id": "a 1.0.0", "name": "a", "dependencies": [{ "name": "b" }] }, + { "id": "b 1.0.0", "name": "b", "dependencies": [{ "name": "a" }] }, + ], + "resolve": { "nodes": [ + { "id": "a 1.0.0", "features": [], "deps": [{ "pkg": "b 1.0.0" }] }, + { "id": "b 1.0.0", "features": [], "deps": [{ "pkg": "a 1.0.0" }] }, + ]}, + "workspace_members": ["a 1.0.0"], + }); + let graph = Graph::from_metadata(&meta); + assert_eq!(graph.reachable(graph.member_roots()).len(), 2); + } + + #[test] + fn the_links_key_is_read_without_being_walked_for() { + let meta = json!({ + "packages": [{ "id": "s 1.0.0", "name": "s", "links": "ssl" }], + "resolve": { "nodes": [{ "id": "s 1.0.0", "features": [], "deps": [] }] }, + "workspace_members": [], + }); + let graph = Graph::from_metadata(&meta); + assert_eq!(graph.links_of("s 1.0.0"), Some("ssl")); + assert_eq!(graph.links_of("nobody"), None); + assert_eq!(graph.name_of("s 1.0.0"), Some("s")); + } +} diff --git a/crates/batten/src/lib.rs b/crates/batten/src/lib.rs index 88d13a17e..e5934a662 100644 --- a/crates/batten/src/lib.rs +++ b/crates/batten/src/lib.rs @@ -24,6 +24,7 @@ pub mod bypass; pub mod capture; /// Declared reductions over responses the agent already captured. pub mod captured; +pub mod cargo_graph; pub mod carry; pub mod checks_green; pub mod ci; diff --git a/crates/batten/src/record.rs b/crates/batten/src/record.rs index d2b686859..624daf3db 100644 --- a/crates/batten/src/record.rs +++ b/crates/batten/src/record.rs @@ -207,7 +207,9 @@ pub fn run( crate::cli::RecordCommand::Plan => run_plan(), crate::cli::RecordCommand::Closes => run_closes(overrides), crate::cli::RecordCommand::Named { family } => run_named(&family), - crate::cli::RecordCommand::Derive { family, inputs } => run_derive(&family, &inputs, out), + crate::cli::RecordCommand::Derive { family, inputs } => { + run_derive(&family, &inputs, overrides, out) + } crate::cli::RecordCommand::Keyed { family, key } => run_keyed(&family, &key), crate::cli::RecordCommand::Journal { family } => run_journal(&family), crate::cli::RecordCommand::Show { family, key } => run_keyed_show(&family, &key, out), @@ -588,6 +590,49 @@ fn only_these_inputs( Ok(()) } +/// One declared `[[pattern]]` row, compiled. +/// +/// NON-NEGOTIABLE RULE 1 IS WHY THIS EXISTS. Which package is the evaluator, +/// which crates bear IO, which need a platform SDK and which vendor what they +/// link are all CONSUMER facts, and a `const` here would put a consumer +/// identifier in the repo-agnostic core. They live in the one committed +/// authority instead, exactly as `run_closes` resolves its key grammar — with +/// the side benefit that the lists become reviewable data rather than constants +/// compiled into a binary (rule 3). +fn declared_pattern( + patterns: &[crate::pattern::NamedPattern], + family: &str, + id: &str, +) -> Result { + let row = patterns.iter().find(|row| row.id == id).ok_or_else(|| { + UsageError::raise(format!( + "record derive {family}: no `[[pattern]]` row declares `{id}`" + )) + })?; + regex::Regex::new(&row.regex).map_err(|_| { + UsageError::raise(format!( + "record derive {family}: `[[pattern]]` row `{id}` will not compile" + )) + .into() + }) +} + +/// The `cargo metadata` document a graph-reading family takes on stdin. +/// +/// COULD-NOT-LOOK IS A REFUSAL, NEVER AN EMPTY GRAPH. The producer writes +/// nothing when the document will not parse, because an absent record means +/// "the producer did not run" and must not be spelled the same way as a graph +/// that resolved and found nothing. +fn graph_on_stdin(family: &str) -> Result { + let raw = verdict_lines()?; + let meta: serde_json::Value = serde_json::from_str(&raw).map_err(|_| { + UsageError::raise(format!( + "record derive {family}: stdin is not a `cargo metadata` document" + )) + })?; + Ok(crate::cargo_graph::Graph::from_metadata(&meta)) +} + /// Derive one family's record from its input and write it. /// /// The engine applies the READING; the effects that produced the input stay in @@ -602,6 +647,7 @@ fn only_these_inputs( pub fn run_derive( family: &str, inputs: &[String], + overrides: &Overrides, out: &mut dyn std::io::Write, ) -> Result { let inputs = derive_inputs(inputs)?; @@ -665,6 +711,82 @@ pub fn run_derive( let sessions = crate::transcript::census(root, exclude); format!("sessions {sessions}\nthreshold {threshold}\n") } + "evaluator-closure" => { + only_these_inputs(&inputs, family, &[])?; + let config = resolve::resolve(Path::new("."), overrides)?; + let evaluator = declared_pattern(&config.patterns, family, "evaluator-package")?; + let bears_io = declared_pattern(&config.patterns, family, "evaluator-io-crate")?; + let graph = graph_on_stdin(family)?; + + // THE SCOPE IS THE EVALUATOR'S SUB-CLOSURE, NOT THE WORKSPACE'S, and + // that was measured before it was written because the obvious + // spelling is wrong: walking from the workspace members instead + // reached 281 packages, including two direct dependencies of the + // consumer itself entering by paths with nothing to do with the + // evaluator. The wider spelling fired on all five lockfile-touching + // commits reachable from HEAD and every firing was a false positive. + let roots = graph.roots_matching(|name| evaluator.is_match(name)); + if roots.is_empty() { + // NOT A PASS. The evaluator vanishing from the graph means the + // question could not be asked, and reporting "nothing found" + // there is the vacuous pass this repository names CLOUD-251. + "absent\n".to_owned() + } else { + let reached = graph.reachable(roots); + let mut lines = format!("closure {}\n", reached.len()); + let mut found: Vec<&str> = graph + .named_in_order(&reached) + .into_iter() + .map(|(name, _)| name) + .filter(|name| bears_io.is_match(name)) + .collect(); + found.dedup(); + for name in found { + lines.push_str("crate "); + lines.push_str(name); + lines.push('\n'); + } + lines + } + } + "macos-link" => { + only_these_inputs(&inputs, family, &[])?; + let config = resolve::resolve(Path::new("."), overrides)?; + let framework = declared_pattern(&config.patterns, family, "sdk-framework-crate")?; + let vendored = declared_pattern(&config.patterns, family, "vendored-links-crate")?; + let graph = graph_on_stdin(family)?; + + // THE WALK STARTS AT THE WORKSPACE MEMBERS, because the question is + // about everything this tree builds — unlike `evaluator-closure`, + // whose question is about one package's sub-closure. That is the + // only difference between the two callers of this graph. + let built = graph.reachable(graph.member_roots()); + let mut lines = format!("scanned {}\n", built.len()); + for (name, id) in graph.named_in_order(&built) { + match graph.links_of(id) { + // RULE 1's PROXY, minus the crates it is wrong about. A crate + // that VENDORS AND COMPILES the library it names reaches no + // platform framework — measured, after this gate refused a + // tree the linker then built with no SDK present. An UNKNOWN + // `links` crate is still a finding, so this narrows the gate + // rather than opening it. + Some(library) if !vendored.is_match(name) => { + lines.push_str("links "); + lines.push_str(name); + lines.push(' '); + lines.push_str(library); + lines.push('\n'); + } + _ if framework.is_match(name) => { + lines.push_str("framework "); + lines.push_str(name); + lines.push('\n'); + } + _ => {} + } + } + lines + } // AN UNKNOWN FAMILY IS A USAGE ERROR, never a record written under a name // nothing reads. A producer whose family was renamed would otherwise go on // writing happily into a key no module has looked at since. diff --git a/crates/batten/tests/it/evaluator_closure.rs b/crates/batten/tests/it/evaluator_closure.rs index 1c9c924ba..a709e6adf 100644 --- a/crates/batten/tests/it/evaluator_closure.rs +++ b/crates/batten/tests/it/evaluator_closure.rs @@ -21,7 +21,7 @@ //! //! Had the walk gone into the task body they would have become `// changed:` arms //! pointing at `mise.toml` with nothing asserting them. It is -//! `mise-tasks/evaluator-closure.py` instead, and these cases run it, so all +//! `crates/batten/src/cargo_graph.rs` instead, and these cases run it, so all //! eight CARRY. `.py` is outside `under_mise_tasks` (`shell-retirement.rego:159`) //! so that file adds no shell rule. //! @@ -40,11 +40,11 @@ // carried: "the repo's real graph is clean today" policy/evaluator-closure.rego kind:mechanism // carried: "an IO crate reachable from the evaluator is refused at exit 2" policy/evaluator-closure.rego kind:mechanism // carried: "an IO crate the workspace depends on directly, but the evaluator does not, is not the evaluator's" policy/evaluator-closure.rego kind:mechanism -// carried: "an unactivated optional IO dependency of the evaluator is not reported" policy/evaluator-closure.rego kind:mechanism -// carried: "the same optional dependency, activated, IS reported" policy/evaluator-closure.rego kind:mechanism +// carried: "an unactivated optional IO dependency of the evaluator is not reported" crates/batten/src/cargo_graph.rs kind:mechanism crates/batten/tests/it/evaluator_closure.rs +// carried: "the same optional dependency, activated, IS reported" crates/batten/src/cargo_graph.rs kind:mechanism crates/batten/tests/it/evaluator_closure.rs // carried: "no evaluator node at all is could-not-look, not a clean bill" policy/evaluator-closure.rego kind:mechanism // carried: "the refusal is pointer-only: the crate name, never the path that reached it" policy/evaluator-closure.rego kind:mechanism -// carried: "a dev-dependency of the evaluator is not in the built closure" policy/evaluator-closure.rego kind:mechanism +// carried: "a dev-dependency of the evaluator is not in the built closure" crates/batten/src/cargo_graph.rs kind:mechanism crates/batten/tests/it/evaluator_closure.rs // Panicking on setup failure is the idiomatic way for a test to fail loudly. #![allow(clippy::unwrap_used, clippy::expect_used)] @@ -95,6 +95,18 @@ severity = "deny" [[record]] record = "evaluator-closure" writer = "mise run evaluator-closure-record" + +# THE TWO CONSUMER FACTS THE VERB RESOLVES, declared here rather than compiled +# into the engine (non-negotiable rule 1). A fixture that omitted them would +# make `record derive` refuse — which is itself asserted below, because a gate +# whose consumer facts are undeclared must say so rather than answer over none. +[[pattern]] +id = "evaluator-package" +regex = '^regorus$' + +[[pattern]] +id = "evaluator-io-crate" +regex = '^(reqwest|jsonschema|hyper|rustls|openssl-sys|native-tls|ring|globset|glob)$' "#, ); init_repo(&dir); @@ -113,36 +125,24 @@ fn record(dir: &std::path::Path, lines: &str) { ); } -/// Run the REAL walk over a `cargo metadata` document, exactly as the producer -/// does. This is what keeps the four walk cases assertable after the port. -#[expect( - clippy::disallowed_types, - reason = "stays, and it is the subject under test rather than a convenience: the walk moved out of the dying program into `mise-tasks/evaluator-closure.py` precisely so a compiled case could drive it, and a harness that re-implemented it in Rust would be a second authority over the reachability the producer actually runs" -)] -fn walk(metadata: &str) -> String { - use std::io::Write as _; - use std::process::{Command, Stdio}; - - let mut child = Command::new("python3") - .arg("../../mise-tasks/evaluator-closure.py") - .stdin(Stdio::piped()) - .stdout(Stdio::piped()) - .stderr(Stdio::piped()) - .spawn() - .expect("the walk this tier exists for"); - child - .stdin - .as_mut() - .expect("stdin") - .write_all(metadata.as_bytes()) - .expect("feed the graph"); - let done = child.wait_with_output().expect("the walk completes"); +/// Drive the REAL walk over a `cargo metadata` document, through the REAL verb. +/// +/// The activated-edge reachability itself lives in +/// `crates/batten/src/cargo_graph.rs` and is asserted directly by that module's +/// own `#[cfg(test)] mod tests` — one walk, shared with `macos-link`, so the two +/// gates cannot drift apart the way their two copies did. +/// +/// What THIS tier drives is the COMPOSITION: the roots the consumer's +/// `evaluator-package` row selects, and the names its `evaluator-io-crate` row +/// looks for once there. That composition is the gate, and it is per-consumer. +fn walk(dir: &std::path::Path, metadata: &str) -> String { + let written = run_with_stdin(dir, &["record", "derive", "evaluator-closure"], metadata); assert!( - done.status.success(), + written.status.success(), "the walk reads its graph: {}", - String::from_utf8_lossy(&done.stderr) + String::from_utf8_lossy(&written.stderr) ); - String::from_utf8_lossy(&done.stdout).into_owned() + String::from_utf8_lossy(&written.stdout).into_owned() } /// A graph where `batten` depends on `regorus`, and `regorus` on `edge`. @@ -205,6 +205,7 @@ fn the_refusal_is_pointer_only_the_crate_name_never_the_path_that_reached_it() { // asserts the intermediate never appears. let dir = repo("pointer"); let reached = walk( + &dir, r#"{ "packages": [ {"id": "batten", "name": "batten", "features": {}, "dependencies": [{"name": "regorus"}]}, @@ -243,6 +244,7 @@ fn no_evaluator_node_at_all_is_could_not_look_not_a_clean_bill() { // CLOUD-251's vacuous pass in the one place it would be least visible. let dir = repo("absent"); let reached = walk( + &dir, r#"{"packages": [{"id": "batten", "name": "batten", "features": {}, "dependencies": []}], "workspace_members": ["batten"], "resolve": {"nodes": [{"id": "batten", "features": [], "deps": []}]}}"#, @@ -279,7 +281,8 @@ fn an_absent_record_says_nothing_rather_than_passing() { #[test] fn an_io_crate_reachable_from_the_evaluator_is_refused_at_exit_2() { - let reached = walk(&chain("reqwest", "null", false, "")); + let dir = repo("walk-reaches"); + let reached = walk(&dir, &chain("reqwest", "null", false, "")); assert!( reached.contains("crate reqwest"), "the walk reaches it\n{reached}" @@ -294,7 +297,9 @@ fn an_io_crate_the_workspace_depends_on_directly_but_the_evaluator_does_not_is_n // direct dependencies of `batten` itself, entering by paths that have nothing // to do with the evaluator. It would deny on `main` forever. This fixture is // that exact topology and the walk must be silent on it. + let dir = repo("not-the-evaluators"); let reached = walk( + &dir, r#"{ "packages": [ {"id": "batten", "name": "batten", "features": {}, @@ -326,7 +331,8 @@ fn an_unactivated_optional_io_dependency_of_the_evaluator_is_not_reported() { // `macos-link-check`'s `defmt` lesson applied here: an optional dependency // nobody enabled is in the resolve and is not in the build, and a gate that // fails on a crate the compiler never sees is not measuring what it names. - let reached = walk(&chain("reqwest", "null", true, r#""std""#)); + let dir = repo("dormant"); + let reached = walk(&dir, &chain("reqwest", "null", true, r#""std""#)); assert!( !reached.contains("crate reqwest"), "an unactivated optional dep is dormant\n{reached}" @@ -337,7 +343,8 @@ fn an_unactivated_optional_io_dependency_of_the_evaluator_is_not_reported() { fn the_same_optional_dependency_activated_is_reported() { // The other direction of the same filter. Without it the case above passes on // a walk that reports nothing at all. - let reached = walk(&chain("reqwest", "null", true, r#""std", "http""#)); + let dir = repo("activated"); + let reached = walk(&dir, &chain("reqwest", "null", true, r#""std", "http""#)); assert!( reached.contains("crate reqwest"), "activating the feature reaches it\n{reached}" @@ -349,7 +356,8 @@ fn a_dev_dependency_of_the_evaluator_is_not_in_the_built_closure() { // A dev-dependency of a DEPENDENCY is never built, so it is not in the // closure a policy module could reach. (A workspace member's dev-dependency // is built — the test binaries link — but that is the members' walk.) - let reached = walk(&chain("reqwest", r#""dev""#, false, "")); + let dir = repo("dev-edge"); + let reached = walk(&dir, &chain("reqwest", r#""dev""#, false, "")); assert!( !reached.contains("crate reqwest"), "a dev edge is not in the built closure\n{reached}" @@ -359,7 +367,7 @@ fn a_dev_dependency_of_the_evaluator_is_not_in_the_built_closure() { #[test] #[expect( clippy::disallowed_types, - reason = "stays: resolving the REAL graph is the whole of this case, and it is the only evidence that the fixtures above agree with the tree `Cargo.toml`'s pin comment makes its claim about" + reason = "stays: resolving the REAL graph is the whole of this case, and it is the only evidence that the fixtures above agree with the tree `Cargo.toml`'s pin comment makes its claim about. The spawn is `cargo metadata` — the producer's own effect, which house-style §5 keeps outside the engine — and never a reading this tier re-implements" )] fn the_repos_real_graph_is_clean_today() { // The anti-vacuity arm, against the tree as it actually resolves. Every case @@ -372,7 +380,8 @@ fn the_repos_real_graph_is_clean_today() { .expect("cargo metadata runs"); assert!(metadata.status.success(), "the lockfile is current"); - let reached = walk(&String::from_utf8_lossy(&metadata.stdout)); + let dir = repo("real-graph"); + let reached = walk(&dir, &String::from_utf8_lossy(&metadata.stdout)); assert!( reached.contains("closure "), "the evaluator is in the graph\n{reached}" diff --git a/crates/batten/tests/it/macos_link.rs b/crates/batten/tests/it/macos_link.rs index aaed17c10..1237f504d 100644 --- a/crates/batten/tests/it/macos_link.rs +++ b/crates/batten/tests/it/macos_link.rs @@ -18,8 +18,9 @@ //! Apple framework and never compiled, made this gate refuse a link //! `darwin-link` then completed on the same tree (CLOUD-718). //! -//! The walk is `mise-tasks/cargo_graph.py`, shared with `evaluator-closure`, and -//! `mise-tasks/macos-link.py` is this gate's roots and predicates over it. Both +//! The walk is `crates/batten/src/cargo_graph.rs`, shared with +//! `evaluator-closure`; this gate's own half is the roots it starts from and the +//! two `[[pattern]]` rows it reads once there. Both //! programs used to carry their own copy and both headers said "if one is //! corrected, correct both" — a rule with no mechanism. There is one walk now, //! and these cases drive it. @@ -33,15 +34,15 @@ // carried: tests/macos-link-check.bats policy/macos-link.rego kind:mechanism crates/batten/tests/it/macos_link.rs // carried: "the repo as it stands has no SDK-requiring dependency" policy/macos-link.rego kind:mechanism // carried: "a package declaring a native links key is caught without being listed" policy/macos-link.rego kind:mechanism -// carried: "an optional dependency nobody enabled is not reported" policy/macos-link.rego kind:mechanism -// carried: "the same optional dependency, once enabled, is reported" policy/macos-link.rego kind:mechanism -// carried: "A WEAK REFERENCE IS NOT AN ACTIVATION: dep-question-mark leaves the dep dormant" policy/macos-link.rego kind:mechanism +// carried: "an optional dependency nobody enabled is not reported" crates/batten/src/cargo_graph.rs kind:mechanism crates/batten/tests/it/macos_link.rs +// carried: "the same optional dependency, once enabled, is reported" crates/batten/src/cargo_graph.rs kind:mechanism crates/batten/tests/it/macos_link.rs +// carried: "A WEAK REFERENCE IS NOT AN ACTIVATION: dep-question-mark leaves the dep dormant" crates/batten/src/cargo_graph.rs kind:mechanism crates/batten/tests/it/macos_link.rs // carried: "rule 2 still fires through the reachability walk" policy/macos-link.rego kind:mechanism // carried: "a vendored-C links crate is exempt from rule 1" policy/macos-link.rego kind:mechanism // carried: "an unvetted links crate is still reported, so the exemption is a list not a switch" policy/macos-link.rego kind:mechanism // carried: "the walk starts at the workspace members" policy/macos-link.rego kind:mechanism // changed: "the graph is resolved for macOS, not for the host" mise.toml the `--filter-platform aarch64-apple-darwin` flag is a property of the SPAWN, so it moved to `[tasks.macos-link-record]` with the `cargo metadata` call it qualifies. The module reads whatever graph the producer recorded and cannot observe which platform it was resolved for; a case here would assert over input this surface cannot vary -// withdrawn: "the framework crate list covers the ones that actually bit us" the case grepped the shell program's FRAMEWORK_CRATES literal for four names. The list is `mise-tasks/macos-link.py`'s now and the assertion was over a spelling rather than a behaviour — `rule_2_still_fires_through_the_reachability_walk` pins what the list is FOR, and a second case re-reading its text would re-break on every legitimate addition +// withdrawn: "the framework crate list covers the ones that actually bit us" the case grepped the shell program's FRAMEWORK_CRATES literal for four names. The list is `crates/batten/src/cargo_graph.rs`'s now and the assertion was over a spelling rather than a behaviour — `rule_2_still_fires_through_the_reachability_walk` pins what the list is FOR, and a second case re-reading its text would re-break on every legitimate addition // withdrawn: "every vendored-links entry names a crate, so the pattern cannot be widened to a wildcard" the same shape one list over: it asserted that the VENDORED_LINKS regex contains no `.*`. `an_unvetted_links_crate_is_still_reported` is the behavioural statement of the same property — the exemption is a list rather than a switch — and it survives a rewrite of the pattern that the text assertion would not // Panicking on setup failure is the idiomatic way for a test to fail loudly. @@ -93,6 +94,18 @@ severity = "deny" [[record]] record = "macos-link" writer = "mise run macos-link-record" + +# THE TWO CONSUMER FACTS THE VERB RESOLVES, declared here rather than compiled +# into the engine (non-negotiable rule 1). `cargo_graph.rs` names no crate at +# all; which ones need an SDK and which vendor what they link are this +# consumer's to say. +[[pattern]] +id = "sdk-framework-crate" +regex = '^(security-framework|security-framework-sys|core-foundation|core-foundation-sys|native-tls|openssl-sys|cocoa|objc|objc2|system-configuration|system-configuration-sys)$' + +[[pattern]] +id = "vendored-links-crate" +regex = '^(tree-sitter|tree-sitter-language)$' "#, ); init_repo(&dir); @@ -110,36 +123,25 @@ fn record(dir: &std::path::Path, lines: &str) { ); } -/// Run the REAL walk over a `cargo metadata` document, exactly as the producer -/// does. This is what keeps the walk cases assertable after the port. -#[expect( - clippy::disallowed_types, - reason = "stays, and it is the subject under test rather than a convenience: the walk moved out of the dying program into `mise-tasks/macos-link.py` over the shared `cargo_graph.py` precisely so a compiled case could drive it, and a harness re-implementing it in Rust would be the second authority both programs' headers warned about" -)] -fn walk(metadata: &str) -> String { - use std::io::Write as _; - use std::process::{Command, Stdio}; - - let mut child = Command::new("python3") - .arg("../../mise-tasks/macos-link.py") - .stdin(Stdio::piped()) - .stdout(Stdio::piped()) - .stderr(Stdio::piped()) - .spawn() - .expect("the walk this tier exists for"); - child - .stdin - .as_mut() - .expect("stdin") - .write_all(metadata.as_bytes()) - .expect("feed the graph"); - let done = child.wait_with_output().expect("the walk completes"); +/// Drive the REAL walk over a `cargo metadata` document, through the REAL verb. +/// +/// The activated-edge reachability lives in `crates/batten/src/cargo_graph.rs` +/// and is asserted by that module's own `#[cfg(test)] mod tests` — ONE walk, +/// shared with `evaluator-closure`. Both programs used to carry their own copy +/// and both headers said so in prose: *"if one is corrected, correct both."* +/// That was a rule with no mechanism; this is the mechanism. +/// +/// What THIS tier drives is the COMPOSITION: the members it starts from, the +/// `links` key it reads, and the two `[[pattern]]` rows that decide which names +/// mean an SDK and which vendor what they link. +fn walk(dir: &std::path::Path, metadata: &str) -> String { + let written = run_with_stdin(dir, &["record", "derive", "macos-link"], metadata); assert!( - done.status.success(), + written.status.success(), "the walk reads its graph: {}", - String::from_utf8_lossy(&done.stderr) + String::from_utf8_lossy(&written.stderr) ); - String::from_utf8_lossy(&done.stdout).into_owned() + String::from_utf8_lossy(&written.stdout).into_owned() } /// A graph where the member depends on `edge`, optionally behind a feature. @@ -173,12 +175,10 @@ fn member_chain(edge: &str, extra: &str, optional: bool, member_features: &str) #[test] fn a_package_declaring_a_native_links_key_is_caught_without_being_listed() { let dir = repo("links"); - let reached = walk(&member_chain( - "openssl-sys", - r#", "links": "openssl""#, - false, - "", - )); + let reached = walk( + &dir, + &member_chain("openssl-sys", r#", "links": "openssl""#, false, ""), + ); assert!( reached.contains("links openssl-sys openssl"), "the walk names the library\n{reached}" @@ -205,7 +205,7 @@ fn rule_2_still_fires_through_the_reachability_walk() { // The named set rule 1 cannot see: a crate that links an Apple framework from // a build script and declares no `links` key. let dir = repo("framework"); - let reached = walk(&member_chain("core-foundation", "", false, "")); + let reached = walk(&dir, &member_chain("core-foundation", "", false, "")); assert!( reached.contains("framework core-foundation"), "the walk names it\n{reached}" @@ -228,12 +228,11 @@ fn a_vendored_c_links_crate_is_exempt_from_rule_1() { // ships C sources and builds them with `cc`, the second does not follow. // Measured 2026-08-21: this gate refused `tree-sitter`, and `darwin-link` // then linked the same tree with no SDK present. - let reached = walk(&member_chain( - "tree-sitter", - r#", "links": "tree-sitter""#, - false, - "", - )); + let dir = repo("a-vendored-c-links-crate-is-exempt-from-"); + let reached = walk( + &dir, + &member_chain("tree-sitter", r#", "links": "tree-sitter""#, false, ""), + ); assert!( !reached.contains("links "), "a vendored-C links crate is exempt\n{reached}" @@ -244,7 +243,11 @@ fn a_vendored_c_links_crate_is_exempt_from_rule_1() { fn an_unvetted_links_crate_is_still_reported_so_the_exemption_is_a_list_not_a_switch() { // The other direction of the same narrowing. Without this the case above // passes on a walk that stopped reading `links` at all. - let reached = walk(&member_chain("some-sys", r#", "links": "some""#, false, "")); + let dir = repo("an-unvetted-links-crate-is-still-reporte"); + let reached = walk( + &dir, + &member_chain("some-sys", r#", "links": "some""#, false, ""), + ); assert!( reached.contains("links some-sys some"), "an unknown links crate is still a finding\n{reached}" @@ -271,7 +274,11 @@ fn an_optional_dependency_nobody_enabled_is_not_reported() { // CLOUD-718's `defmt`: an unactivated optional dependency is in the resolve // and is not in the build, and a gate that fails on a crate the compiler // never sees is not measuring the thing it names. - let reached = walk(&member_chain("core-foundation", "", true, r#""default""#)); + let dir = repo("an-optional-dependency-nobody-enabled-is"); + let reached = walk( + &dir, + &member_chain("core-foundation", "", true, r#""default""#), + ); assert!( !reached.contains("framework "), "an unactivated optional dep is dormant\n{reached}" @@ -280,12 +287,11 @@ fn an_optional_dependency_nobody_enabled_is_not_reported() { #[test] fn the_same_optional_dependency_once_enabled_is_reported() { - let reached = walk(&member_chain( - "core-foundation", - "", - true, - r#""default", "tls""#, - )); + let dir = repo("the-same-optional-dependency-once-enable"); + let reached = walk( + &dir, + &member_chain("core-foundation", "", true, r#""default", "tls""#), + ); assert!( reached.contains("framework core-foundation"), "enabling the feature reaches it\n{reached}" @@ -296,7 +302,9 @@ fn the_same_optional_dependency_once_enabled_is_reported() { fn a_weak_reference_is_not_an_activation() { // `foo?/bar` applies only if something ELSE already activated `foo`, so // reading it as an activation walks back to the whole-resolve over-scan. + let dir = repo("a-weak-reference-is-not-an-activation"); let reached = walk( + &dir, r#"{ "packages": [ {"id": "batten", "name": "batten", "features": {"tls": ["core-foundation?/std"]}, @@ -320,7 +328,9 @@ fn a_weak_reference_is_not_an_activation() { fn the_walk_starts_at_the_workspace_members() { // Unlike `evaluator-closure`, whose roots are one package's nodes. A package // in the resolve that no member reaches is not in the built graph. + let dir = repo("the-walk-starts-at-the-workspace-members"); let reached = walk( + &dir, r#"{ "packages": [ {"id": "batten", "name": "batten", "features": {}, "dependencies": []}, @@ -365,7 +375,8 @@ fn the_repo_as_it_stands_has_no_sdk_requiring_dependency() { .expect("cargo metadata runs"); assert!(metadata.status.success(), "the macOS graph resolves"); - let reached = walk(&String::from_utf8_lossy(&metadata.stdout)); + let dir = repo("the-repo-as-it-stands-has-no-sdk-requiri"); + let reached = walk(&dir, &String::from_utf8_lossy(&metadata.stdout)); assert!(reached.contains("scanned "), "the walk ran\n{reached}"); assert!( !reached.contains("links ") && !reached.contains("framework "), diff --git a/mise-tasks/cargo_graph.py b/mise-tasks/cargo_graph.py deleted file mode 100644 index 27350c391..000000000 --- a/mise-tasks/cargo_graph.py +++ /dev/null @@ -1,127 +0,0 @@ -# The ACTIVATED dependency graph, read from a `cargo metadata` document. -# -# ONE WALK, TWO GATES, AND THAT IS THE WHOLE REASON THIS FILE EXISTS. -# `evaluator-closure` asks whether an IO-bearing crate is reachable from the -# evaluator's node; `macos-link` asks whether anything in the built graph needs a -# real macOS SDK to link. The questions differ only in their ROOTS and in what -# they look for once there — the reachability underneath is the same, down to the -# weak-dependency rule. -# -# Both programs carried their own copy, and both headers said so in prose: -# "The activation reading is deliberately IDENTICAL to `macos-link-check`'s, down -# to the weak-dependency rule, because the two gates ask the same structural -# question of the same graph and a second, subtly different walk is how a pair -# like this drifts. If one is corrected, correct both." -# -# That is a rule with no mechanism, which is half a change. This file is the -# mechanism: there is now one walk to correct, so the two cannot disagree. The -# `#MUTANT` rows that used to be stated twice are stated once, over the code they -# actually mutate. -# -# `.py` rather than a task body, and `mise-tasks/replay-pointers.py` is the -# precedent: `shell-retirement.rego:159-163` excludes `.py` from -# `under_mise_tasks`, so this adds no shell rule and both callers' compiled tiers -# can drive it directly. - -# Reverting the activation filter to the whole resolve is the defect this exists -# to prevent: an optional dependency nobody enabled reads as linked, and the gate -# refuses a link that succeeds. Measured on `defmt` — an embedded-logging crate, -# an unactivated optional dependency of `jiff`, reaching no Apple framework and -# never compiled — which made `macos-link-check` refuse a link `darwin-link` then -# completed on the same tree. -#MUTANT-SUITE crates/batten/tests/it/evaluator_closure.rs -#MUTANT graph-scans-unactivated|s@^ if (entry.get("rename") or entry\["name"\]) in enabled:$@ if True:@|an_unactivated_optional_io_dependency_of_the_evaluator_is_not_reported -#MUTANT graph-weak-dep-activates|s@^ head = token.split("/", 1)\[0\]$@ head = token.split("/", 1)[0].rstrip("?")@|a_weak_reference_is_not_an_activation -#MUTANT graph-keeps-dev-edges|s@^ if kinds == {"dev"} and not is_member:$@ if False:@|a_dev_dependency_of_the_evaluator_is_not_in_the_built_closure - - -def activated_keys(package, enabled): - """The dependency names this package's enabled features actually turn on. - - Three spellings reach a dependency and all three must be read, or an - activated dep looks dormant: the implicit feature (a bare 'foo'), the - namespaced form ('dep:foo'), and enabling one of the dep's own features - ('foo/bar'). The WEAK form ('foo?/bar') is deliberately not one — it applies - only if something else already activated the dep, and reading it as an - activation drifts back toward the whole-resolve scan this replaces.""" - keys = set(enabled) - declared = package.get("features", {}) - for feature in enabled: - for token in declared.get(feature, []): - if token.startswith("dep:"): - keys.add(token[4:]) - elif "/" in token: - head = token.split("/", 1)[0] - if not head.endswith("?"): - keys.add(head) - return keys - - -class Graph: - """A resolved `cargo metadata` document, walked along ACTIVATED edges only. - - `cargo metadata` lists every package the resolver CONSIDERED, including - optional dependencies nothing turned on. Scanning that asks "could some - configuration of this tree reach X" where both callers mean "does this one". - """ - - def __init__(self, meta): - self.packages = {p["id"]: p for p in meta["packages"]} - self.nodes = {n["id"]: n for n in (meta.get("resolve") or {}).get("nodes", [])} - self.members = set(meta.get("workspace_members", [])) - - def edges(self, node_id): - node = self.nodes.get(node_id) - package = self.packages.get(node_id) - if node is None or package is None: - return - enabled = activated_keys(package, set(node.get("features", []))) - is_member = node_id in self.members - for dep in node.get("deps", []): - target = self.packages.get(dep["pkg"]) - if target is None: - continue - # A dev-dependency of a *dependency* is never built. One of a - # workspace member is: the test binaries link too. - kinds = {k.get("kind") for k in dep.get("dep_kinds", [{}])} - if kinds == {"dev"} and not is_member: - continue - matching = [ - d - for d in package.get("dependencies", []) - if d["name"] == target["name"] - ] - if not matching: - # An edge the manifest does not explain: keep it rather than drop - # it. Unexplained means unmeasured, and unmeasured fails closed. - yield dep["pkg"] - continue - for entry in matching: - if not entry.get("optional"): - yield dep["pkg"] - break - if (entry.get("rename") or entry["name"]) in enabled: - yield dep["pkg"] - break - - def reachable(self, roots): - """Every package id reachable from `roots` along activated edges.""" - seen = set() - frontier = list(roots) - while frontier: - current = frontier.pop() - if current in seen: - continue - seen.add(current) - frontier.extend(self.edges(current)) - return seen - - def member_roots(self): - """The workspace members — `macos-link`'s starting set, because its - question is about everything this tree builds.""" - return [m for m in self.members if m in self.nodes] - - def named_roots(self, name): - """Every node for a package of this name — `evaluator-closure`'s - starting set, because its question is about ONE package's sub-closure.""" - return [i for i, p in self.packages.items() if p["name"] == name and i in self.nodes] diff --git a/mise-tasks/evaluator-closure.py b/mise-tasks/evaluator-closure.py deleted file mode 100644 index 219dd98aa..000000000 --- a/mise-tasks/evaluator-closure.py +++ /dev/null @@ -1,85 +0,0 @@ -# The evaluator's activated sub-closure, over a `cargo metadata` document on -# stdin (CLOUD-831, ported off `mise-tasks/evaluator-closure-check.sh` under -# CLOUD-1717). -# -# THIS IS THE STEP, NEVER THE DECISION. It walks a graph and names what it -# reached; whether reaching one of them is a refusal is -# `policy/evaluator-closure.rego`'s question. The split is forced rather than -# chosen: a reachability closure is not expressible in Rego — a self-referential -# rule is a compile error and `graph.reachable` is not in this build's regorus -# feature set (`Cargo.toml` is the one authority on that list). -# -# THE WALK ITSELF IS `cargo_graph.py`'s, SHARED WITH `macos-link`. Both gates ask -# the same structural question of the same graph and differ only in their roots -# and in what they look for; both headers used to say "if one is corrected, -# correct both", which is a rule with no mechanism. There is one walk now. -# -# Output is the record's own lines, on stdout, for `batten record named`: -# -# absent no evaluator node in the graph — could-not-look -# closure packages reached from the evaluator -# crate one per IO-bearing crate in that closure -# -# Pointer-only (non-negotiable rule 4): a crate NAME and a count. Never a -# dependency tree, never a path through the graph, never a version chain. - -import json -import os -import sys - -sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) - -from cargo_graph import Graph # noqa: E402 - -# The evaluator's package name. Named once: it is the node the walk starts from -# AND the word the refusal uses, and two spellings of it is how a rename turns -# this gate silent instead of red. -EVALUATOR = "regorus" - -# `Cargo.toml`'s pin names. Not a heuristic and not "crates that look networky": -# this is the list the manifest comment claims is absent, restated here as the -# thing that is looked for. The manifest comment points AT this file rather than -# repeating the list a third time. -IO_CRATES = frozenset( - "reqwest jsonschema hyper rustls openssl-sys native-tls ring globset glob".split() -) - - -def walk(meta): - graph = Graph(meta) - - # THE SCOPE IS THE EVALUATOR'S SUB-CLOSURE, NOT THE WORKSPACE'S, and that was - # measured before it was written because the obvious spelling is wrong. - # Walking from the workspace members instead reaches 281 packages including - # `globset` AND `jsonschema` — both direct dependencies of `batten` itself, - # entering by paths that have nothing to do with the evaluator. The wider - # spelling fired on all 5 lockfile-touching commits reachable from HEAD and - # every firing was a false positive. - roots = graph.named_roots(EVALUATOR) - if not roots: - # NOT a pass. The evaluator vanishing from the graph means the question - # could not be asked, and reporting 'nothing found' there is the vacuous - # pass this repository names CLOUD-251. - return ["absent"] - - reached = graph.reachable(roots) - found = sorted({graph.packages[i]["name"] for i in reached} & IO_CRATES) - return ["closure %d" % len(reached)] + ["crate %s" % name for name in found] - - -def main(): - try: - meta = json.load(sys.stdin) - except (ValueError, OSError) as failed: - # The producer writes NOTHING when it could not look, which is the record - # contract: an absent record is "the producer did not run", and it must - # not be spelled the same way as a graph that resolved. - sys.stderr.write("evaluator-closure: could not read the graph: %s\n" % failed) - return 1 - for line in walk(meta): - sys.stdout.write(line + "\n") - return 0 - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/mise-tasks/macos-link.py b/mise-tasks/macos-link.py deleted file mode 100644 index 3ae534d56..000000000 --- a/mise-tasks/macos-link.py +++ /dev/null @@ -1,102 +0,0 @@ -# What in the built macOS graph would need a real macOS SDK to link, over a -# `cargo metadata` document on stdin (ported off `mise-tasks/macos-link-check.sh` -# under CLOUD-1717). -# -# THIS IS THE STEP, NEVER THE DECISION. It names what it found; whether finding -# one is a refusal is `policy/macos-link.rego`'s question. The split is forced: -# §5 makes `check` `read` and incapable of spawning `cargo metadata`, and the -# reachability walk is not expressible in Rego at all. -# -# THE WALK IS `cargo_graph.py`'s, SHARED WITH `evaluator-closure`, which is what -# closes the drift both programs' headers warned about in prose and neither could -# enforce. -# -# TWO RULES, and rule 2 is a list and therefore incomplete by construction: -# -# 1. any package declaring a `links` key — the manifest's own statement that it -# links a native library, which is general and needs no list; minus the -# crates that vendor and compile their own C, for which "links a native -# library" does not imply "needs an SDK to find one"; -# 2. a named set of crates that link Apple frameworks from a build script -# WITHOUT declaring `links`, which rule 1 cannot see. -# -# A crate nobody has listed slips past rule 2. That residual gap is closed by -# actually linking the target, which `darwin-link` does; this pair is the fast, -# specific, early half of that, not a replacement for it. -# -# Output is the record's own lines, on stdout, for `batten record named`: -# -# scanned packages in the built macOS graph -# links rule 1: declares links= -# framework rule 2: links an Apple system framework -# -# Pointer-only (non-negotiable rule 4): a crate NAME and, for rule 1, the library -# the manifest itself names. Never a version chain, never a path through the -# graph. The VERSION the retired program printed is deliberately dropped — it is -# not needed to act on the finding and it dates the record against a lockfile -# the module never reads. - -import json -import os -import re -import sys - -sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) - -from cargo_graph import Graph # noqa: E402 - -# Crates that pull in an Apple system framework. Extend as new ones are met — and -# when you do, prefer proving it with a link (`darwin-link`) over trusting this -# list. -FRAMEWORK_CRATES = re.compile( - r"^(security-framework|security-framework-sys|core-foundation" - r"|core-foundation-sys|native-tls|openssl-sys|cocoa|objc|objc2" - r"|system-configuration|system-configuration-sys)$" -) - -# Crates whose `links` key names a library they VENDOR AND COMPILE THEMSELVES, -# reaching no Apple system framework — so rule 1's proxy is wrong about them. -# -# ADDING A NAME HERE REQUIRES A LINK, never a reading of the crate. Measured -# 2026-08-21: this gate refused `tree-sitter` and `tree-sitter-language`, and -# `darwin-link` then linked the same tree with no SDK present. A gate that -# refuses what the linker accepts is measuring something other than what it -# names. The fail-safe is unchanged in the direction that matters: an UNKNOWN -# `links` crate is still a finding, so this narrows the gate rather than opening -# it. -VENDORED_LINKS = re.compile(r"^(tree-sitter|tree-sitter-language)$") - - -def walk(meta): - graph = Graph(meta) - - # THE WALK STARTS AT THE WORKSPACE MEMBERS, because the question is about - # everything this tree builds — unlike `evaluator-closure`, whose question is - # about one package's sub-closure. That is the only difference between the - # two callers of this graph. - built = graph.reachable(graph.member_roots()) - - lines = ["scanned %d" % len(built)] - for package_id in sorted(built, key=lambda i: graph.packages[i]["name"]): - package = graph.packages[package_id] - name = package["name"] - if package.get("links") and not VENDORED_LINKS.match(name): - lines.append("links %s %s" % (name, package["links"])) - elif FRAMEWORK_CRATES.match(name): - lines.append("framework %s" % name) - return lines - - -def main(): - try: - meta = json.load(sys.stdin) - except (ValueError, OSError) as failed: - sys.stderr.write("macos-link: could not read the graph: %s\n" % failed) - return 1 - for line in walk(meta): - sys.stdout.write(line + "\n") - return 0 - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/mise.toml b/mise.toml index 32e2cbd09..0177f0a9c 100644 --- a/mise.toml +++ b/mise.toml @@ -617,7 +617,7 @@ CI_FANIN_WORKFLOW = ".github/workflows/ci.yml" BATS_TEST_TIMEOUT = "300" REGORUS_OPA_COMPLIANCE = "1.2.0" REGORUS_OPA_COMPLIANCE_FOR = "0.11" -MUTANT_GATES = "mise,attestation,timeout-drift,engine-checks-green,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure,evaluator-io-probe,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-callable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" +MUTANT_GATES = "mise,attestation,timeout-drift,engine-checks-green,engine-cargo-graph,engine-config,engine-doctor,engine-landed,engine-perf,engine-mcp,engine-pinned,engine-ready,engine-verdict,engine-wiring,engine-surface,agentic-experiment-record,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age,cap-drift,cfg-gated-test,ci-cache-declared,ci-hygiene,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,egress-fencing,evaluator-closure,evaluator-io-probe,filed-here,finding-sink-check,fixture-forks,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hk-plan-required,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land-divergence,landed-check,landing-loop,landing-roster-guarded,leased-push,license-table-check,linear-check,lock-complete,macos-link,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,suite-bench-check,suite-subject-retirable,task-callable,task-substitution,test-targets,timeout-check,token-bench-check,transcript-corpus,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,weakens-declared,worktree-registration,spawn-widening,nextest-slow,engine-lease,engine-handler,engine-speculation,engine-pipeline,engine-policy,rules-paths-trigger,skill-frontmatter-complete,engine-land,engine-hook,engine-rules" # --- GitHub reachability behind an egress proxy (Claude Code web sandbox etc.) --- # mise resolves every tool's release through GitHub's *API* host, api.github.com. @@ -2326,9 +2326,12 @@ if [ -n "${BATTEN_LINK_CHECK_METADATA:-}" ]; then else metadata=$(cargo metadata --format-version 1 --filter-platform aarch64-apple-darwin 2>/dev/null) || exit 0 fi +# THE SPAWN STAYS HERE (§5) and the WALK is the engine's: one activated-edge +# reading in `crates/batten/src/cargo_graph.rs`, shared with the other caller +# so the two cannot drift, and the crate names it looks for are `[[pattern]]` +# rows rather than constants in a repo-agnostic core (CLOUD-1717). printf '%s' "$metadata" \ - | python3 mise-tasks/macos-link.py \ - | cargo run --quiet -p batten -- record named macos-link + | cargo run --quiet -p batten -- record derive macos-link """ [tasks.evaluator-closure-record] @@ -2340,9 +2343,12 @@ if [ -n "${BATTEN_EVALUATOR_METADATA:-}" ]; then else metadata=$(cargo metadata --locked --format-version 1 2>/dev/null) || exit 0 fi +# THE SPAWN STAYS HERE (§5) and the WALK is the engine's: one activated-edge +# reading in `crates/batten/src/cargo_graph.rs`, shared with the other caller +# so the two cannot drift, and the crate names it looks for are `[[pattern]]` +# rows rather than constants in a repo-agnostic core (CLOUD-1717). printf '%s' "$metadata" \ - | python3 mise-tasks/evaluator-closure.py \ - | cargo run --quiet -p batten -- record named evaluator-closure + | cargo run --quiet -p batten -- record derive evaluator-closure """ [tasks.branch-age-record] diff --git a/policy/module-layering.rego b/policy/module-layering.rego index c9a19f55b..e2acf2ad8 100644 --- a/policy/module-layering.rego +++ b/policy/module-layering.rego @@ -563,6 +563,25 @@ declared_modules := { # payload. The producer still gathers the facts, because `git config` and # `git rev-list` are spawns §5 keeps outside; what they MEAN is composed here. "signer_posture", + # `cargo_graph` arrived with CLOUD-1717 and this rule named it, the coverage + # clause working an eighteenth time. + # + # It is `probe_verdict`'s and `environment`'s class: it reaches NOTHING, not + # even `error`. A `cargo metadata` document it cannot make sense of is an + # EMPTY graph, which the caller's own could-not-look arm decides over — a + # graph that answered "nothing is reachable" while claiming to have looked + # would be the vacuous pass, so the distinction is the caller's to draw and + # not a failure to report here. + # + # IT NAMES NO CRATE, and that is the placement rather than an accident of + # what it happens to contain. Which package is the evaluator, which crates + # bear IO, which need a platform SDK and which vendor what they link are + # CONSUMER facts (non-negotiable rule 1), so the roots are chosen by a + # predicate the caller supplies and the names live in `[[pattern]]` rows. + # An edge from here to `config` would be this module reading the consumer's + # authority itself, which is the direction that would make a repo-agnostic + # walk conditional on one repository's table. + "cargo_graph", } # THE FORBIDDEN EDGES, each traceable to prose already in the tree. From c3ca7748f4f7b50228905beb3894b23afb231555 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sun, 20 Sep 2026 07:27:39 +0000 Subject: [PATCH 28/32] fix(gate): the retirement's two dangling callers resolve, so verify can run MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `mise run verify` and `mise run commit-lint` both exited before running any gate, because this branch deleted two file tasks and left their callers: mise ERROR task not found: signing-posture `signing-posture` and `evaluator-io-check` were `mise-tasks/*.sh`, registered by filename. Deleting the file unregisters the name, and nothing asked whether a `depends` target or a task body's `mise run` still resolves — which is CLOUD-1833's own defect class, twice more, on the branch that files it. Both retired onto a producer/judge pair, because section 5 makes `check` `read` and incapable of spawning `git config` or a probe build. `verify` calls the pair directly. A `depends` entry names ONE task, so `signing-posture-check` composes the two halves rather than making four call sites carry both. Each caller guards its producer by hand: these bodies declare `shell = "bash -c"` and so do not run under `set -e`, and a producer that could not write its record must not fall through to a judge that reads the absence as silence. That is the failure CLOUD-1810 exists to close, and a bare two-line body would have reintroduced it at the call site. Refs: CLOUD-1833 --- mise.toml | 58 ++++++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 51 insertions(+), 7 deletions(-) diff --git a/mise.toml b/mise.toml index 0177f0a9c..8a9cb0899 100644 --- a/mise.toml +++ b/mise.toml @@ -2223,6 +2223,32 @@ cargo run --quiet -p batten -- record derive signing-posture \\ --input "gpgsign=$gpgsign" --input "signed=$signed" """ +# THE PAIR UNDER ONE NAME, and the name is what `commit-lint` depends on. +# +# `signing-posture.sh` was one program that gathered AND judged, and `commit-lint` +# named it in `depends`. The port split it (§5 makes `check` `read` and incapable +# of spawning `git config`), and the retirement deleted the program while leaving +# that `depends` entry naming it — `mise ERROR task not found: signing-posture`, +# so `commit-lint` could not run at all, in CI, in `verify`, and from the +# `commit-msg` hook. A `depends` entry names ONE task, so the two halves compose +# here rather than at the four call sites that would otherwise each need both. +# +# Guarded by hand: this body declares `shell = "bash -c"` and so does not run +# under `set -e`. A producer that could not write its record must not fall through +# to a judge that would read the absence as silence — which is the whole failure +# class CLOUD-1810 exists to close, and it would be reintroduced here by a bare +# two-line body. +[tasks.signing-posture-check] +description = "Gate: this checkout's signer is verifiable and no commit in range carries a key nobody can check (CLOUD-669)" +shell = "bash -c" +run = """ +if ! mise run signing-posture-record; then + echo "::error:: signing-posture-check: the posture could not be recorded, so there is nothing to judge." >&2 + exit 2 +fi +mise run batten -- check --rule 'commit grade unsafe' +""" + # THE WRITE, self-declared per house style §5 and kept a task because a module # cannot write. It fires ONLY against the broken configuration: disabling signing # unconditionally would also switch off a correctly configured signer, which is @@ -5531,8 +5557,23 @@ fi # `ready-guard` refuses `gh pr ready` without a verify receipt for this HEAD, so # a test that has quietly stopped discriminating blocks the PR before it can # spend a runner. -if ! mise run evaluator-io-check; then - echo "::error:: verify: evaluator-io-check refused — the IO-free evaluator test no longer discriminates, or its probe build could not run. No receipt written." >&2 +# THE SPLIT THE PORT FORCED (CLOUD-1717), and this caller is what the retirement +# left dangling — `mise run evaluator-io-check` named a program this branch had +# already deleted, so `verify` exited 1 before reaching anything below it and the +# branch could not verify itself. `evaluator-io-check` was one program making the +# probe build AND adjudicating its log; §5 makes `check` `read` and incapable of +# spawning, so the build is `evaluator-io-record`'s and the verdict is the +# engine's. +# +# BOTH HALVES GUARDED BY HAND, because this body declares `shell = "bash -c"` and +# so does not run under `set -e`. A producer that could not run must not fall +# through to a judge that would then read its absence as silence and pass. +if ! mise run evaluator-io-record; then + echo "::error:: verify: evaluator-io-record could not run the probe build, so there is no reading to judge. No receipt written." >&2 + exit 1 +fi +if ! mise run batten -- check --rule 'test cover never'; then + echo "::error:: verify: 'test cover never' refused — the IO-free evaluator test no longer discriminates. No receipt written." >&2 exit 1 fi # The latency regression gate (CLOUD-172). `perf-assert` holds the published @@ -5778,12 +5819,15 @@ description = "Validate every commit in BASE_SHA..HEAD_SHA is Conventional Commi # lights up the local pre-flight and the CI backstop at once, with no second # definition of which commits are in scope. It also fails closed for free: mise # stops before this body runs, so the no-`set -e` trap does not apply (CLOUD-274). -# `signing-posture` rides the same contract for the same reason, and it is the -# signature half of what `commit-attribution` does for the identity half +# `signing-posture-check` rides the same contract for the same reason, and it is +# the signature half of what `commit-attribution` does for the identity half # (CLOUD-669): both judge BASE_SHA..HEAD_SHA, so neither needs a second call -# site and `ci-local-parity` stays satisfied without editing a workflow. -depends = ["commit-attribution", "signing-posture", "commit-check"] -shell = "bash -c" # uses ${sha:0:8}; mise defaults tasks to /bin/sh (dash) +# site and `ci-local-parity` stays satisfied without editing a workflow. It is +# spelled `-check` since CLOUD-1717 split the gathering from the judging; the +# entry named the retired program for as long as nothing asked whether a +# `depends` target still resolves. +depends = ["commit-attribution", "signing-posture-check", "commit-check"] +shell = "bash -c" # uses ${sha:0:8}; mise defaults tasks to /bin/sh (dash) # The SERVER-SIDE half of CLOUD-431. The claim receipt lives in `.git/` and never # leaves the clone, so no reviewer and no workflow can see it; what a server CAN # see is whether each commit names the issue it serves. `verify` holds the From 4f34bbcb02a767b006816f4b4aedf25b8b2d5e39 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sun, 20 Sep 2026 07:44:43 +0000 Subject: [PATCH 29/32] fix(record): clippy is green, and three readings stop lying about themselves MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The branch did not pass its own `lint:clippy`, so `verify` could not have gone green even once the dangling callers resolved. Two errors, both in `record.rs`: a `.into()` onto the type it already was, and `run_derive` at 127 lines. The split is the shape the function wanted anyway. `derive_reading` gains an arm per producer; `store_derived` and `emit_derived` are fixed. The two graph-reading arms come out as siblings, because they differ only in their ROOTS and in what they look for once there — which is the reason `cargo_graph` exists rather than a walk per caller. Three readings corrected beside it: * `RecordTableRefused` was INSERTED mid-enum, shifting the discriminants of `ProvisionTableRefused`, `StartupTableRefused`, `OutcomeTableRefused`, `PlanReadStale` and `ConfigUnreadable`. The variant three below it documents that exact rule in capitals. Appended now; `Native::ALL` keeps the grouping, because position is API and reading order is not. * `CHECK_RULE`'s doc comment had `DERIVE_INPUT` spliced INSIDE it, so the CLOUD-1051 rationale documented the wrong constant and `CHECK_RULE` was undocumented. * `signing-posture-record` compared `commit.gpgsign` against the literals `true`/`false`, but git also accepts `1`/`on`/`yes`/`0`/`off`/`no` — wrong in both directions, so a global `1` graded a conflicting config clean and a local `off` recorded a conflict over a checkout with signing disabled. `--type=bool` normalises before either comparison. * `attestation-record` discarded both extractors' status, so a corrupt archive recorded `no-binary` — a release verdict where the honest reading is could-not-look. Refs: CLOUD-1717 --- crates/batten/src/record.rs | 249 ++++++++++++++++++++++------------- crates/batten/src/surface.rs | 28 ++-- crates/batten/src/verdict.rs | 21 ++- mise.toml | 28 +++- 4 files changed, 210 insertions(+), 116 deletions(-) diff --git a/crates/batten/src/record.rs b/crates/batten/src/record.rs index 624daf3db..3a1d6a30a 100644 --- a/crates/batten/src/record.rs +++ b/crates/batten/src/record.rs @@ -613,7 +613,6 @@ fn declared_pattern( UsageError::raise(format!( "record derive {family}: `[[pattern]]` row `{id}` will not compile" )) - .into() }) } @@ -651,16 +650,38 @@ pub fn run_derive( out: &mut dyn std::io::Write, ) -> Result { let inputs = derive_inputs(inputs)?; + let derived = derive_reading(family, &inputs, overrides)?; + let family = safe_component("family", family)?; + store_derived(&family, &derived)?; + emit_derived(&derived, out) +} + +/// The READING for one family, from its declared inputs and whatever is on stdin. +/// +/// Split out of [`run_derive`] because the two halves grow at different rates: +/// this one gains an arm per producer, and the store-and-emit tail below it is +/// fixed. Nothing here spawns — house-style §5 keeps a producer's effects in the +/// task, and what arrives is already a reading's worth of input. +/// +/// # Errors +/// +/// A [`UsageError`] for an unknown family, or a malformed, missing or +/// unaccepted input. +fn derive_reading( + family: &str, + inputs: &BTreeMap, + overrides: &Overrides, +) -> Result { let derived = match family { "evaluator-io-probe" => { - only_these_inputs(&inputs, family, &["status", "test"])?; - let raw = required_input(&inputs, family, "status")?; + only_these_inputs(inputs, family, &["status", "test"])?; + let raw = required_input(inputs, family, "status")?; let status: i32 = raw.parse().map_err(|_| { UsageError::raise(format!( "record derive {family}: status `{raw}` is not a whole number" )) })?; - let test = required_input(&inputs, family, "test")?; + let test = required_input(inputs, family, "test")?; let log = verdict_lines()?; format!( "{}\n", @@ -669,24 +690,24 @@ pub fn run_derive( } "signing-posture" => { only_these_inputs( - &inputs, + inputs, family, &["signingkey", "ssh-program", "gpgsign", "signed"], )?; - let signingkey = required_input(&inputs, family, "signingkey")?; - let program = required_input(&inputs, family, "ssh-program")?; + let signingkey = required_input(inputs, family, "signingkey")?; + let program = required_input(inputs, family, "ssh-program")?; // THE TWO ABSENT-MEANS-NOTHING INPUTS. A producer that found no // conflict and no signed commit still sends both, empty; treating an // omitted input as "no" here would make "the producer did not look" // and "the producer looked and found none" the same record. - let conflict = required_input(&inputs, family, "gpgsign")? == "conflict"; - let signed = required_input(&inputs, family, "signed")?; + let conflict = required_input(inputs, family, "gpgsign")? == "conflict"; + let signed = required_input(inputs, family, "signed")?; crate::signer_posture::record(signingkey, program, conflict, signed) } "transcript-corpus" => { - only_these_inputs(&inputs, family, &["root", "threshold", "exclude"])?; - let root = required_input(&inputs, family, "root")?; - let raw = required_input(&inputs, family, "threshold")?; + only_these_inputs(inputs, family, &["root", "threshold", "exclude"])?; + let root = required_input(inputs, family, "root")?; + let raw = required_input(inputs, family, "threshold")?; let threshold: usize = raw.parse().map_err(|_| { UsageError::raise(format!( "record derive {family}: threshold `{raw}` is not a whole number" @@ -711,82 +732,8 @@ pub fn run_derive( let sessions = crate::transcript::census(root, exclude); format!("sessions {sessions}\nthreshold {threshold}\n") } - "evaluator-closure" => { - only_these_inputs(&inputs, family, &[])?; - let config = resolve::resolve(Path::new("."), overrides)?; - let evaluator = declared_pattern(&config.patterns, family, "evaluator-package")?; - let bears_io = declared_pattern(&config.patterns, family, "evaluator-io-crate")?; - let graph = graph_on_stdin(family)?; - - // THE SCOPE IS THE EVALUATOR'S SUB-CLOSURE, NOT THE WORKSPACE'S, and - // that was measured before it was written because the obvious - // spelling is wrong: walking from the workspace members instead - // reached 281 packages, including two direct dependencies of the - // consumer itself entering by paths with nothing to do with the - // evaluator. The wider spelling fired on all five lockfile-touching - // commits reachable from HEAD and every firing was a false positive. - let roots = graph.roots_matching(|name| evaluator.is_match(name)); - if roots.is_empty() { - // NOT A PASS. The evaluator vanishing from the graph means the - // question could not be asked, and reporting "nothing found" - // there is the vacuous pass this repository names CLOUD-251. - "absent\n".to_owned() - } else { - let reached = graph.reachable(roots); - let mut lines = format!("closure {}\n", reached.len()); - let mut found: Vec<&str> = graph - .named_in_order(&reached) - .into_iter() - .map(|(name, _)| name) - .filter(|name| bears_io.is_match(name)) - .collect(); - found.dedup(); - for name in found { - lines.push_str("crate "); - lines.push_str(name); - lines.push('\n'); - } - lines - } - } - "macos-link" => { - only_these_inputs(&inputs, family, &[])?; - let config = resolve::resolve(Path::new("."), overrides)?; - let framework = declared_pattern(&config.patterns, family, "sdk-framework-crate")?; - let vendored = declared_pattern(&config.patterns, family, "vendored-links-crate")?; - let graph = graph_on_stdin(family)?; - - // THE WALK STARTS AT THE WORKSPACE MEMBERS, because the question is - // about everything this tree builds — unlike `evaluator-closure`, - // whose question is about one package's sub-closure. That is the - // only difference between the two callers of this graph. - let built = graph.reachable(graph.member_roots()); - let mut lines = format!("scanned {}\n", built.len()); - for (name, id) in graph.named_in_order(&built) { - match graph.links_of(id) { - // RULE 1's PROXY, minus the crates it is wrong about. A crate - // that VENDORS AND COMPILES the library it names reaches no - // platform framework — measured, after this gate refused a - // tree the linker then built with no SDK present. An UNKNOWN - // `links` crate is still a finding, so this narrows the gate - // rather than opening it. - Some(library) if !vendored.is_match(name) => { - lines.push_str("links "); - lines.push_str(name); - lines.push(' '); - lines.push_str(library); - lines.push('\n'); - } - _ if framework.is_match(name) => { - lines.push_str("framework "); - lines.push_str(name); - lines.push('\n'); - } - _ => {} - } - } - lines - } + "evaluator-closure" => evaluator_closure_reading(inputs, family, overrides)?, + "macos-link" => macos_link_reading(inputs, family, overrides)?, // AN UNKNOWN FAMILY IS A USAGE ERROR, never a record written under a name // nothing reads. A producer whose family was renamed would otherwise go on // writing happily into a key no module has looked at since. @@ -796,8 +743,119 @@ pub fn run_derive( ))); } }; + Ok(derived) +} - let family = safe_component("family", family)?; +/// The evaluator's own sub-closure, and which crates in it bear IO (CLOUD-831). +/// +/// # Errors +/// +/// A [`UsageError`] for an unaccepted input, an undeclared `[[pattern]]` row, or +/// stdin that is not a `cargo metadata` document. +fn evaluator_closure_reading( + inputs: &BTreeMap, + family: &str, + overrides: &Overrides, +) -> Result { + only_these_inputs(inputs, family, &[])?; + let config = resolve::resolve(Path::new("."), overrides)?; + let evaluator = declared_pattern(&config.patterns, family, "evaluator-package")?; + let bears_io = declared_pattern(&config.patterns, family, "evaluator-io-crate")?; + let graph = graph_on_stdin(family)?; + + // THE SCOPE IS THE EVALUATOR'S SUB-CLOSURE, NOT THE WORKSPACE'S, and + // that was measured before it was written because the obvious + // spelling is wrong: walking from the workspace members instead + // reached 281 packages, including two direct dependencies of the + // consumer itself entering by paths with nothing to do with the + // evaluator. The wider spelling fired on all five lockfile-touching + // commits reachable from HEAD and every firing was a false positive. + let roots = graph.roots_matching(|name| evaluator.is_match(name)); + let reading = if roots.is_empty() { + // NOT A PASS. The evaluator vanishing from the graph means the + // question could not be asked, and reporting "nothing found" + // there is the vacuous pass this repository names CLOUD-251. + "absent\n".to_owned() + } else { + let reached = graph.reachable(roots); + let mut lines = format!("closure {}\n", reached.len()); + let mut found: Vec<&str> = graph + .named_in_order(&reached) + .into_iter() + .map(|(name, _)| name) + .filter(|name| bears_io.is_match(name)) + .collect(); + found.dedup(); + for name in found { + lines.push_str("crate "); + lines.push_str(name); + lines.push('\n'); + } + lines + }; + Ok(reading) +} + +/// What this tree BUILDS, and which of it needs a platform SDK to link. +/// +/// [`evaluator_closure_reading`]'s sibling, and the two differ only in their +/// ROOTS and in what they look for once there — which is the whole reason +/// [`crate::cargo_graph`] exists rather than a walk per caller. +/// +/// # Errors +/// +/// A [`UsageError`] for an unaccepted input, an undeclared `[[pattern]]` row, or +/// stdin that is not a `cargo metadata` document. +fn macos_link_reading( + inputs: &BTreeMap, + family: &str, + overrides: &Overrides, +) -> Result { + only_these_inputs(inputs, family, &[])?; + let config = resolve::resolve(Path::new("."), overrides)?; + let framework = declared_pattern(&config.patterns, family, "sdk-framework-crate")?; + let vendored = declared_pattern(&config.patterns, family, "vendored-links-crate")?; + let graph = graph_on_stdin(family)?; + + // THE WALK STARTS AT THE WORKSPACE MEMBERS, because the question is + // about everything this tree builds — unlike `evaluator-closure`, + // whose question is about one package's sub-closure. That is the + // only difference between the two callers of this graph. + let built = graph.reachable(graph.member_roots()); + let mut lines = format!("scanned {}\n", built.len()); + for (name, id) in graph.named_in_order(&built) { + match graph.links_of(id) { + // RULE 1's PROXY, minus the crates it is wrong about. A crate + // that VENDORS AND COMPILES the library it names reaches no + // platform framework — measured, after this gate refused a + // tree the linker then built with no SDK present. An UNKNOWN + // `links` crate is still a finding, so this narrows the gate + // rather than opening it. + Some(library) if !vendored.is_match(name) => { + lines.push_str("links "); + lines.push_str(name); + lines.push(' '); + lines.push_str(library); + lines.push('\n'); + } + _ if framework.is_match(name) => { + lines.push_str("framework "); + lines.push_str(name); + lines.push('\n'); + } + _ => {} + } + } + Ok(lines) +} + +/// Write a derived reading into the policy-readable store for this branch. +/// +/// # Errors +/// +/// A [`UsageError`] for a tree that is not a repository or a detached HEAD with +/// no branch to key on; an internal error when the store cannot be written. +fn store_derived(family: &str, derived: &str) -> Result<()> { let root = Path::new("."); let git_dir = git::git_dir(root).map_err(|_| { UsageError::raise( @@ -811,10 +869,17 @@ pub fn run_derive( }; let claim = claim_of(&git_dir, &branch); store( - &crate::recorder::record_path(&git_dir, &family, &branch, claim.as_deref()), - &derived, - )?; + &crate::recorder::record_path(&git_dir, family, &branch, claim.as_deref()), + derived, + ) +} +/// Emit the derived reading on the data channel. +/// +/// # Errors +/// +/// An internal error when the output channel cannot be written. +fn emit_derived(derived: &str, out: &mut dyn std::io::Write) -> Result { // THE DERIVED RECORD GOES TO STDOUT TOO, and it stays pointer-only doing it: // what is emitted is the READING — a bounded set of tokens this verb // computed — never a byte of the input it read. That distinction is why diff --git a/crates/batten/src/surface.rs b/crates/batten/src/surface.rs index bdf18ce86..bce069e4d 100644 --- a/crates/batten/src/surface.rs +++ b/crates/batten/src/surface.rs @@ -1079,6 +1079,20 @@ const VERDICT_TOKEN: FlagDecl = FlagDecl { /// filter that matched nothing and exited 0 is the vacuous pass in its purest /// form: the caller would read "the gate passed" from a gate that was never /// selected, and a renamed row would silently stop being enforced. +const CHECK_RULE: FlagDecl = FlagDecl { + id: "rule", + long: Some("rule"), + short: None, + help: "Run only the declared rules with these ids (repeatable)", + env: EnvDecl::None, + global: false, + positional: false, + required: false, + hidden: false, + rung: Rung::None, + value: ValueDecl::StrMany, +}; + /// The non-document inputs a `record derive` family needs, as `=`. /// /// ONE DECLARED FLAG RATHER THAN A COLUMN PER FAMILY, and the alternative is @@ -1107,20 +1121,6 @@ const DERIVE_INPUT: FlagDecl = FlagDecl { value: ValueDecl::StrMany, }; -const CHECK_RULE: FlagDecl = FlagDecl { - id: "rule", - long: Some("rule"), - short: None, - help: "Run only the declared rules with these ids (repeatable)", - env: EnvDecl::None, - global: false, - positional: false, - required: false, - hidden: false, - rung: Rung::None, - value: ValueDecl::StrMany, -}; - /// `--rule ` on `enforce`: the same narrowing, on the verb that spawns. /// /// # This reverses a recorded decision, and that decision named its condition diff --git a/crates/batten/src/verdict.rs b/crates/batten/src/verdict.rs index b50e8028c..d372b4e45 100644 --- a/crates/batten/src/verdict.rs +++ b/crates/batten/src/verdict.rs @@ -1282,12 +1282,6 @@ pub enum Native { MintTableRefused, /// The `[[recorder]]` table would not load. RecorderTableRefused, - /// The `[[record]]` table would not load (CLOUD-1810). - /// - /// Its own class rather than the recorder's, on the reason the environment - /// pair one screen up already states: the two tables carry different rows and - /// a refusal has to name which one to edit. - RecordTableRefused, /// The `[[provision]]` table would not load. ProvisionTableRefused, /// The `[[startup]]` table would not load. @@ -1334,6 +1328,21 @@ pub enum Native { /// left, and the declared hatch is the recovery path. That asymmetry is why /// this class exists and why it is exactly one class wide. ConfigUnreadable, + /// The `[[record]]` table would not load (CLOUD-1810). + /// + /// Its own class rather than the recorder's, on the reason the environment + /// pair states: the two tables carry different rows and a refusal has to name + /// which one to edit. + /// + /// **APPENDED, BECAUSE THE VARIANT ABOVE SAYS SO AND THIS ONE DID NOT LISTEN.** + /// It landed beside the other config classes — between `RecorderTableRefused` + /// and `ProvisionTableRefused`, where it reads best — and that shifted the + /// discriminants of `ProvisionTableRefused`, `StartupTableRefused`, + /// `OutcomeTableRefused`, `PlanReadStale` and `ConfigUnreadable`, which is the + /// whole tail `enum_no_repr_variant_discriminant_changed` reads as broken. + /// Position is API; the reading order is not, and `Native::ALL` below is where + /// the grouping belongs. + RecordTableRefused, } impl Native { diff --git a/mise.toml b/mise.toml index 8a9cb0899..88b6ae361 100644 --- a/mise.toml +++ b/mise.toml @@ -1958,10 +1958,22 @@ fi name=${archive##*/} into="$scratch/x-$name" mkdir -p "$into" + # AN EXTRACTION THAT FAILED IS NOT AN ARCHIVE WITHOUT A BINARY. Both + # commands discarded their status with `|| true`, so a corrupt archive or a + # missing extractor left `find` with nothing to find and the producer wrote + # `no-binary` — a RELEASE VERDICT, where the honest reading is + # could-not-look. The WHOLE record is abandoned rather than the one line + # dropped, because a posture line plus a short archive list is a window the + # module reads as complete. + extracted=ok case "$name" in - *.zip) unzip -q -o "$archive" -d "$into" || true ;; - *) tar -xzf "$archive" -C "$into" || true ;; + *.zip) unzip -q -o "$archive" -d "$into" >/dev/null 2>&1 || extracted=failed ;; + *) tar -xzf "$archive" -C "$into" >/dev/null 2>&1 || extracted=failed ;; esac + if [ "$extracted" = failed ]; then + echo "::error:: attestation-record: could not extract $name, so its provenance is unread. Nothing recorded." >&2 + exit 2 + fi binary=$(find "$into" -type f \( -name batten -o -name batten.exe \) | head -n1) if [ -z "$binary" ]; then printf 'archive\t%s\tno-binary\n' "$name" @@ -2195,9 +2207,17 @@ git rev-parse --git-dir >/dev/null 2>&1 || exit 0 key=$(git config --get user.signingkey 2>/dev/null || true) prog=$(git config --get gpg.ssh.program 2>/dev/null || true) +# `--type=bool` IS LOAD-BEARING, NOT TIDINESS. `git config --get` returns the +# stored value verbatim and git accepts `1`, `on`, `yes`, `0`, `off` and `no` as +# booleans, so a comparison against the literals `true`/`false` is wrong in BOTH +# directions: a global `commit.gpgsign = 1` matched no `*true*` and recorded no +# conflict, grading a conflicting configuration clean; a local `off` was not +# equal to `false`, so the guard passed and a global `true` recorded a conflict +# over a checkout where signing is disabled. `--type=bool` makes git normalise +# both before either comparison runs. gpgsign=none -local_setting=$(git config --local --get commit.gpgsign 2>/dev/null || true) -inherited=$(git config --global --get commit.gpgsign 2>/dev/null || true) +local_setting=$(git config --type=bool --local --get commit.gpgsign 2>/dev/null || true) +inherited=$(git config --type=bool --global --get commit.gpgsign 2>/dev/null || true) if [ "$local_setting" != false ]; then case "$inherited$local_setting" in *true*) gpgsign=conflict ;; From 2c0812e693c112b682ab3671d46aa385c104f401 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sun, 20 Sep 2026 07:51:59 +0000 Subject: [PATCH 30/32] fix(record): the producers stop recording readings they did not take MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Five producers wrote a record that reads as a measurement over input they never read, or read only part of. Every one is silent on the decision surface, because a module cannot tell a short record from a clean one. * `timeout-drift-record` piped `emit` straight into `record named`, which buffers to EOF. A `return 2` mid-loop had already sent every workflow it had processed down the pipe, and `record named` REPLACES a family — so the store held a partial census the module adjudicates as whole. `pipefail` reds the task, which says nothing about the bytes left behind. Buffered to a file and written only on success. * `signing-posture-record` answered a non-repository with `|| exit 0`. `signing-posture.sh:87-89` exited 2 there and said why: *a gate that cannot look must not report the posture in force*. Silence is that reasoning inverted. Restored. * `evaluator-io-record` swallowed every `record derive` usage error with `|| exit 0` — a renamed family or an unwritable store became a successful run that wrote nothing. And a step-receipt cache HIT exited before writing at all, so the module went silent for the whole run on the strength of a receipt about an earlier tree. The receipt still skips the expensive half; the reading is re-derived. * `attestation-record` is in the previous commit. Two portability defects beside them: * the nonverdict roster used jq's `inside`, which on arrays is subset by `contains` and so SUBSTRING for strings — a non-required `action` job matched required `action (ubuntu-latest)`, entered the population and could produce a false `job answer missing`. Measured both ways before and after. * `timeout-drift-record` called `asort`, a gawk extension, with no awk declared in `[tools]` at all — so on BSD awk the program dies before emitting a single record. Replaced with a portable insertion sort over a window-bounded `n`. Refs: CLOUD-1717 --- mise.toml | 73 +++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 63 insertions(+), 10 deletions(-) diff --git a/mise.toml b/mise.toml index 88b6ae361..99e1238dd 100644 --- a/mise.toml +++ b/mise.toml @@ -2039,7 +2039,7 @@ emit() { " done awk -v file="$file" -v durations="$durations" ' - function flush( samples, n, i, sorted, idx, p95) { + function flush( samples, n, i, j, v, idx, p95) { if (job == "" || declared == "") return n = 0 split(durations, rows, "\n") @@ -2049,7 +2049,18 @@ emit() { } p95 = 0 if (n > 0) { - asort(samples) + # INSERTION SORT RATHER THAN `asort`, WHICH IS GAWK ONLY. `[tools]` + # declares no awk at all, so this runs whatever the host has, and + # on macOS that is BSD awk — where `asort` is not a function and + # the program dies before emitting a single record, for every job + # that has samples. `n` is bounded by the run window, so the + # quadratic sort is cheaper than declaring a tool to avoid it. + for (i = 2; i <= n; i++) { + v = samples[i] + j = i - 1 + while (j > 0 && samples[j] > v) { samples[j + 1] = samples[j]; j-- } + samples[j + 1] = v + } idx = int(((95 * n) + 99) / 100) if (idx < 1) idx = 1 p95 = samples[idx] @@ -2074,7 +2085,18 @@ emit() { done } -emit | cargo run --quiet -p batten -- record named timeout-drift +# THE RECORD IS WRITTEN ONLY AFTER `emit` FINISHES, and the pipe could not do +# that. `record named` buffers stdin and writes on EOF, so every workflow `emit` +# had already processed was down the pipe before its `return 2` — and `record +# named` REPLACES a family, so the store ended up holding a partial census that +# `policy/timeout-drift.rego` adjudicates as if it were whole. `pipefail` reds +# the task, which is honest about the run and says nothing about the bytes it +# left behind; the module reads the store on the next `check`, not this exit +# code. Buffer to a file, and write only on success. +census=$(mktemp) +trap 'rm -f "$census"' EXIT +emit >"$census" +cargo run --quiet -p batten -- record named timeout-drift <"$census" ''' # CLOUD-831's producer, ported off `mise-tasks/evaluator-closure-check.sh` under @@ -2192,13 +2214,21 @@ emit | cargo run --quiet -p batten -- record named timeout-drift # the signed commit this gate exists to catch would read as clean. # `pipefail-grep-check` caught what the retired suite could not. # -# NOTHING IS WRITTEN OUTSIDE A GIT REPOSITORY — could not look, which the module -# reads as silence rather than as a posture in force. +# OUTSIDE A GIT REPOSITORY THIS REFUSES, AND THAT IS THE RETIRED PROGRAM'S OWN +# REASON RESTORED. The port wrote `|| exit 0` here, which turns could-not-look +# into silence — and `signing-posture.sh:87-89` exited 2 on exactly this branch, +# saying why in as many words: *a gate that cannot look must not report the +# posture in force*. Silence is that reading inverted, because the module cannot +# tell an absent record from a clean one. Exit 2 keeps the reading loud on the +# one caller that can act on it. [tasks.signing-posture-record] description = "Effect: record this checkout's signer posture and any signed commit in range, for `commit grade unsafe`" shell = "bash -c" run = """ -git rev-parse --git-dir >/dev/null 2>&1 || exit 0 +if ! git rev-parse --git-dir >/dev/null 2>&1; then + echo "::error:: signing-posture-record: not a git repository, so the signer posture could not be read. Nothing recorded." >&2 + exit 2 +fi # THE GATHERING IS THIS TASK'S — `git config`, `git rev-list` and `git # cat-file` are spawns house-style §5 keeps outside the engine. What the facts @@ -2315,7 +2345,16 @@ if [ -n "${EVALUATOR_IO_PROBE_CMD:-}" ]; then # shellcheck disable=SC2086 # deliberate word-splitting: the override is an argv set -- $EVALUATOR_IO_PROBE_CMD else - if ./mise-tasks/step-receipt.sh check evaluator-io-check; then exit 0; fi + # A CACHE HIT MUST STILL LEAVE A RECORD. Exiting here wrote none, so the + # module went silent for the whole run on the strength of a receipt — a + # verdict about an earlier tree standing in for one about this one, spelled + # exactly like a producer that never ran. The receipt still buys the expensive + # half: the probe BUILD is skipped and the reading is re-derived from the + # verdict the receipt attests, which costs a `record derive` and nothing else. + if ./mise-tasks/step-receipt.sh check evaluator-io-check; then + printf 'probe failed\n' | cargo run --quiet -p batten -- record named evaluator-io-probe + exit $? + fi set -- cargo test --quiet -p batten --features probe-evaluator-io \ --test policy_modules "$probe_test" -- --include-ignored fi @@ -2329,8 +2368,17 @@ status=0 # where house-style §5 puts a producer's effects, and the classification of # what it wrote is `crates/batten/src/probe_verdict.rs` — Rust the engine # tests rather than a script beside this task. -verdict=$(cargo run --quiet -p batten -- record derive evaluator-io-probe \ - --input "status=$status" --input "test=$probe_test" <"$log") || exit 0 +# A DERIVATION THAT FAILED IS NOT A PROBE THAT SAID NOTHING. `|| exit 0` turned +# every usage error `record.rs` raises — a renamed family, a malformed input, an +# unwritable store — into a SUCCESSFUL producer run that wrote no record, and +# `policy/evaluator-io-probe.rego` reads that absence as silence. The loud +# could-not-look is the `probe unread` token the reading itself emits; a +# derivation that could not run has not reached a reading at all. +if ! verdict=$(cargo run --quiet -p batten -- record derive evaluator-io-probe \ + --input "status=$status" --input "test=$probe_test" <"$log"); then + echo "::error:: evaluator-io-record: the probe's reading could not be derived, so nothing was recorded." >&2 + exit 2 +fi if [ -z "${EVALUATOR_IO_PROBE_CMD:-}" ] && [ "$verdict" = 'probe failed' ]; then @@ -2687,7 +2735,12 @@ while IFS= read -r run; do | [.jobs[]? | select(.conclusion == "failure") | select(.name != $fan) - | select([.name] | inside($roster))] + # EXACT MEMBERSHIP, NEVER `inside`. jq's `inside` on arrays is subset + # by `contains`, which for strings is SUBSTRING — so `["action"]` was + # inside a roster carrying `action (ubuntu-latest)` and a non-required + # job entered the population, moving the window counts and producing a + # false `job answer missing` when it had no verdict step. + | select(.name as $n | $roster | index($n))] | sort_by(.name) | .[] | . as $job From 9fddd101fdf2d44a6d89ccaf6f5004eaec972f47 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sun, 20 Sep 2026 07:58:05 +0000 Subject: [PATCH 31/32] docs(record): the record stops citing files that never existed in the tree MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All five `mise-tasks/*.py` were created AND deleted inside this branch — `git log --all --diff-filter=A` puts every one of them on `29dd210`, `d407029`, `8f8f322`, `9eb1ed9` and `c66cdd5`. None ever reached `main`. So three modules were recorded in `mem:core` as "ported off" `signer_posture.py`, `cargo_graph.py` and `probe_verdict.py`, which is branch-internal churn written down as provenance, and a ledger arm in `signing_posture.rs` justified itself by naming `signer_posture.py` as the shared reading its tier drives. A ledger justification is evidence; this one pointed at nothing. Each now names the shell program it actually replaced, and records the `.py` detour where it belongs — on `shell-retirement.rego`'s arm F, which exists because of it and refuses the shape now. `evaluator-closure-record`'s header argued in the present tense that the walk IS a `.py` sibling and that `replay-pointers.py` is the precedent. Neither is true of this tree: the walk is `cargo_graph.rs`, and no `.py` is tracked under `mise-tasks/` at all. `bench/suites/RESULTS.md` said 94 suites over 84 rows. Regenerated from `test:bats`'s own report through `suite-bench`; `suite-bench-check` is green. `bench/gates/RESULTS.md` is deliberately NOT touched: its header declares it a frozen census taken 2026-08-23 and says in as many words that the corpus has shrunk since and a reader wanting today's count should take it. Refs: CLOUD-1717 --- .serena/memories/core.md | 20 +++- bench/suites/RESULTS.md | 140 +++++++++++----------- crates/batten/tests/it/signing_posture.rs | 2 +- mise.toml | 20 ++-- 4 files changed, 97 insertions(+), 85 deletions(-) diff --git a/.serena/memories/core.md b/.serena/memories/core.md index 6e1bb658b..996a59c3d 100644 --- a/.serena/memories/core.md +++ b/.serena/memories/core.md @@ -2466,9 +2466,13 @@ judge_fingerprint`, its own domain tag), so a caller can reference content it calls in a task body that nothing tested. It reaches NOTHING and never runs `git config`: the two values arrive as `&str`, which keeps the reading testable against a scratch path and a developer's real configuration out of - the tests. Ported off `mise-tasks/signer_posture.py`; `signing-posture-repair` + the tests. Ported off `mise-tasks/signing-posture.sh`; `signing-posture-repair` no longer classifies a second time but reads the posture off the record the - producer just wrote. + producer just wrote. **It never passed through a `.py`, whatever this entry + said first**: that sibling was created and deleted inside the same branch, so + naming it here recorded branch-internal churn as provenance. The detour is + real and belongs on `shell-retirement.rego`'s arm F, which now refuses it — + not in the module's own history. - `cargo_graph.rs` — the ACTIVATED dependency graph, read from a `cargo metadata` document (CLOUD-1717). ONE WALK, TWO GATES, and that is the whole reason it exists: `evaluator-closure` asks whether an IO-bearing crate is reachable from @@ -2489,8 +2493,11 @@ metadata` lists every package the resolver CONSIDERED, so scanning it asks IT — which package is the evaluator, which bear IO, which need an SDK and which vendor what they link are consumer facts in `[[pattern]]` rows (rule 1), so roots are chosen by a PREDICATE the caller supplies. Ported off - `mise-tasks/cargo_graph.py` and its two callers; carries the three `#MUTANT` + `mise-tasks/evaluator-closure-check.sh` and `macos-link-check.sh`, the two + callers that each carried a copy of the walk; carries the three `#MUTANT` rows that used to be stated twice, now stated once over the code they mutate. + A `cargo_graph.py` sibling existed for part of that branch and never reached + `main` — the detour belongs to `shell-retirement.rego`'s arm F, not here. - `probe_verdict.rs` — which of three things a probe build did, from its exit status and its log (CLOUD-418, CLOUD-1717). THE VERDICT IS THE HARNESS'S OWN LINE, NEVER THE EXIT CODE ALONE: `cargo test` exits non-zero for a compile @@ -2503,9 +2510,10 @@ metadata` lists every package the resolver CONSIDERED, so scanning it asks one total function to a three-valued enum, where a log it cannot make sense of is `Unread` — the could-not-look the caller already handles, so a `Result` would add a state with no distinct handling. Ported off - `mise-tasks/probe_verdict.py`, which was the campaign to delete bash routing a - reading into another interpreter; `shell-retirement.rego`'s arm F now refuses - that shape. Its caller is `record derive`, which SPAWNS NOTHING — the probe + `mise-tasks/evaluator-io-check.sh`. A `probe_verdict.py` sibling stood between + the two for part of that branch and never reached `main`: that was the + campaign to delete bash routing a reading into another interpreter, and + `shell-retirement.rego`'s arm F now refuses the shape. Its caller is `record derive`, which SPAWNS NOTHING — the probe build stays in the producer task (§5) and the log arrives on stdin. - `prune.rs` — the build tree's reclaim and its disk floor (CLOUD-766/861/1030), retired out of `mise-tasks/target-prune.sh` under CLOUD-1059. diff --git a/bench/suites/RESULTS.md b/bench/suites/RESULTS.md index bbb14e73a..7efef3fc7 100644 --- a/bench/suites/RESULTS.md +++ b/bench/suites/RESULTS.md @@ -6,92 +6,92 @@ runner measured it; the suite runs `--no-parallelize-within-files`, so a file's number is its own serial cost and is what an author adding a case to it pays. -- suites: 94 -- serial total: 156.0s +- suites: 84 +- serial total: 175.5s | seconds | share | suite | | ---: | ---: | --- | -| 17.9 | 11.3% | `tests/graph-check.bats` | -| 10.2 | 6.4% | `tests/board-diff-overlap.bats` | -| 7.3 | 4.6% | `tests/target-race.bats` | -| 7.1 | 4.5% | `tests/ready-lint.bats` | -| 6.5 | 4.1% | `tests/token-bench.bats` | -| 6.1 | 3.8% | `tests/released.bats` | -| 5.3 | 3.3% | `tests/board-sweep.bats` | -| 4.9 | 3.1% | `tests/release-tracking-check.bats` | -| 4.8 | 3.0% | `tests/release-assets-check.bats` | -| 4.1 | 2.6% | `tests/sbom.bats` | -| 4.0 | 2.5% | `tests/in-progress-drain.bats` | -| 3.6 | 2.3% | `tests/step-receipt.bats` | -| 3.5 | 2.2% | `tests/mcp-allow-check.bats` | -| 3.4 | 2.1% | `tests/doctor-race.bats` | -| 3.3 | 2.1% | `tests/ready-cites-check.bats` | -| 3.2 | 2.0% | `tests/hk-selection.bats` | -| 3.0 | 1.9% | `tests/ntia-check.bats` | -| 2.9 | 1.8% | `tests/target-ensure.bats` | -| 2.8 | 1.8% | `tests/with-lock.bats` | -| 2.2 | 1.4% | `tests/landed-check.bats` | -| 2.1 | 1.3% | `tests/install.bats` | -| 1.8 | 1.1% | `tests/closing-key-check.bats` | -| 1.8 | 1.1% | `tests/reclaim-census.bats` | -| 1.7 | 1.0% | `tests/finding-sink-check.bats` | -| 1.6 | 1.0% | `tests/suite-select.bats` | -| 1.5 | 0.9% | `tests/spec-ref-check.bats` | -| 1.2 | 0.8% | `tests/tree-clean.bats` | -| 1.2 | 0.8% | `tests/ci-slow-needed.bats` | +| 19.5 | 11.1% | `tests/graph-check.bats` | +| 12.6 | 7.2% | `tests/board-diff-overlap.bats` | +| 8.6 | 4.9% | `tests/target-race.bats` | +| 8.5 | 4.8% | `tests/board-sweep.bats` | +| 7.2 | 4.1% | `tests/ready-lint.bats` | +| 6.7 | 3.8% | `tests/token-bench.bats` | +| 6.5 | 3.7% | `tests/released.bats` | +| 5.8 | 3.3% | `tests/in-progress-drain.bats` | +| 5.0 | 2.8% | `tests/release-tracking-check.bats` | +| 4.8 | 2.8% | `tests/release-assets-check.bats` | +| 4.7 | 2.7% | `tests/mcp-allow-check.bats` | +| 4.4 | 2.5% | `tests/sbom.bats` | +| 4.0 | 2.3% | `tests/closing-key-check.bats` | +| 3.6 | 2.1% | `tests/doctor-race.bats` | +| 3.6 | 2.0% | `tests/step-receipt.bats` | +| 3.5 | 2.0% | `tests/landed-check.bats` | +| 3.5 | 2.0% | `tests/with-lock.bats` | +| 3.3 | 1.9% | `tests/hk-selection.bats` | +| 3.0 | 1.7% | `tests/ntia-check.bats` | +| 3.0 | 1.7% | `tests/ready-cites-check.bats` | +| 2.8 | 1.6% | `tests/target-ensure.bats` | +| 2.8 | 1.6% | `tests/lint-rego.bats` | +| 2.6 | 1.5% | `tests/install.bats` | +| 2.5 | 1.4% | `tests/spawn-census.bats` | +| 2.4 | 1.4% | `tests/ci-tools-check.bats` | +| 2.4 | 1.4% | `tests/lint-deno.bats` | +| 2.1 | 1.2% | `tests/finding-sink-check.bats` | +| 1.7 | 1.0% | `tests/ci-slow-needed.bats` | +| 1.6 | 0.9% | `tests/reclaim-census.bats` | +| 1.6 | 0.9% | `tests/deferral-check.bats` | +| 1.5 | 0.9% | `tests/suite-select.bats` | +| 1.4 | 0.8% | `tests/spec-ref-check.bats` | +| 1.3 | 0.7% | `tests/install-check.bats` | | 1.2 | 0.7% | `tests/ready-lint-deferral.bats` | -| 1.1 | 0.7% | `tests/ci-tools-check.bats` | -| 1.0 | 0.7% | `tests/verify.bats` | -| 1.0 | 0.6% | `tests/perf-record.bats` | -| 1.0 | 0.6% | `tests/install-check.bats` | -| 1.0 | 0.6% | `tests/linear-check.bats` | -| 1.0 | 0.6% | `tests/lint-rego.bats` | -| 0.9 | 0.6% | `tests/deferral-check.bats` | -| 0.9 | 0.6% | `tests/spawn-census.bats` | -| 0.9 | 0.5% | `tests/done-check.bats` | -| 0.8 | 0.5% | `tests/module-map-check.bats` | +| 1.2 | 0.7% | `tests/linear-check.bats` | +| 1.1 | 0.6% | `tests/tree-clean.bats` | +| 1.1 | 0.6% | `tests/verify.bats` | +| 1.1 | 0.6% | `tests/perf-record.bats` | +| 1.1 | 0.6% | `tests/done-check.bats` | +| 1.0 | 0.6% | `tests/commit-attribution.bats` | +| 1.0 | 0.6% | `tests/awk-regex-check.bats` | +| 0.9 | 0.5% | `tests/checksums.bats` | +| 0.9 | 0.5% | `tests/render-cli.bats` | | 0.8 | 0.5% | `tests/release-backfill.bats` | -| 0.8 | 0.5% | `tests/lint-deno.bats` | -| 0.7 | 0.4% | `tests/render-cli.bats` | +| 0.8 | 0.5% | `tests/module-map-check.bats` | +| 0.8 | 0.4% | `tests/mcp-attach-check.bats` | +| 0.8 | 0.4% | `tests/done-pr-check.bats` | +| 0.8 | 0.4% | `tests/doctor.bats` | +| 0.7 | 0.4% | `tests/sbom-binary.bats` | | 0.7 | 0.4% | `tests/pr-unsubscribed.bats` | -| 0.7 | 0.4% | `tests/awk-regex-check.bats` | -| 0.7 | 0.4% | `tests/doctor.bats` | -| 0.6 | 0.4% | `tests/commit-attribution.bats` | -| 0.6 | 0.4% | `tests/done-pr-check.bats` | -| 0.6 | 0.4% | `tests/sbom-binary.bats` | +| 0.7 | 0.4% | `tests/duplicate-close-check.bats` | | 0.6 | 0.4% | `tests/mcp-timeout-budget.bats` | -| 0.6 | 0.4% | `tests/mcp-attach-check.bats` | -| 0.5 | 0.3% | `tests/duplicate-close-check.bats` | | 0.5 | 0.3% | `tests/suite-bench-check.bats` | -| 0.4 | 0.3% | `tests/checksums.bats` | +| 0.5 | 0.3% | `tests/hook-pin-check.bats` | +| 0.5 | 0.3% | `tests/cap-drift.bats` | +| 0.5 | 0.3% | `tests/board-payloads.bats` | +| 0.5 | 0.3% | `tests/license-table-check.bats` | | 0.4 | 0.3% | `tests/publish-credential-check.bats` | -| 0.4 | 0.3% | `tests/msrv-pin-agreement.bats` | -| 0.4 | 0.2% | `tests/pipefail-grep-check.bats` | +| 0.4 | 0.2% | `tests/msrv-pin-agreement.bats` | | 0.4 | 0.2% | `tests/digest-major-agreement.bats` | +| 0.4 | 0.2% | `tests/pipefail-grep-check.bats` | +| 0.4 | 0.2% | `tests/sonar-gate.bats` | | 0.4 | 0.2% | `tests/connector-allow-guard.bats` | -| 0.4 | 0.2% | `tests/hook-pin-check.bats` | -| 0.4 | 0.2% | `tests/commit-convention.bats` | -| 0.3 | 0.2% | `tests/sonar-gate.bats` | -| 0.3 | 0.2% | `tests/no-doctests.bats` | +| 0.4 | 0.2% | `tests/serena-mcp.bats` | +| 0.4 | 0.2% | `tests/no-doctests.bats` | +| 0.3 | 0.2% | `tests/commit-convention.bats` | | 0.3 | 0.2% | `tests/report-only-check.bats` | -| 0.3 | 0.2% | `tests/license-table-check.bats` | -| 0.3 | 0.2% | `tests/board-payloads.bats` | | 0.3 | 0.2% | `tests/timeout-check.bats` | | 0.3 | 0.2% | `tests/release-due.bats` | -| 0.3 | 0.2% | `tests/serena-mcp.bats` | -| 0.2 | 0.2% | `tests/connector-allow-resolve.bats` | -| 0.2 | 0.2% | `tests/batten-glob-check.bats` | -| 0.2 | 0.2% | `tests/cap-drift.bats` | -| 0.2 | 0.1% | `tests/mise-action-floor.bats` | -| 0.2 | 0.1% | `tests/coderabbit-config-check.bats` | +| 0.3 | 0.2% | `tests/batten-glob-check.bats` | +| 0.3 | 0.2% | `tests/coderabbit-config-check.bats` | +| 0.3 | 0.1% | `tests/git-hook.bats` | | 0.2 | 0.1% | `tests/rust-paths-check.bats` | +| 0.2 | 0.1% | `tests/connector-allow-resolve.bats` | | 0.2 | 0.1% | `tests/container-preflight.bats` | -| 0.2 | 0.1% | `tests/git-hook.bats` | +| 0.2 | 0.1% | `tests/mise-action-floor.bats` | +| 0.2 | 0.1% | `tests/remedy-payload-source.bats` | +| 0.2 | 0.1% | `tests/dist.bats` | | 0.1 | 0.1% | `tests/token-bench-check.bats` | -| 0.1 | 0.1% | `tests/remedy-payload-source.bats` | -| 0.1 | 0.1% | `tests/task-fail-closed.bats` | -| 0.1 | 0.1% | `tests/dist.bats` | | 0.1 | 0.1% | `tests/egress-check.bats` | -| 0.0 | 0.0% | `tests/darwin-link.bats` | -| 0.0 | 0.0% | `tests/cross-check.bats` | +| 0.1 | 0.1% | `tests/task-fail-closed.bats` | +| 0.1 | 0.0% | `tests/darwin-link.bats` | +| 0.1 | 0.0% | `tests/cross-check.bats` | | 0.0 | 0.0% | `tests/zizmor-split.bats` | diff --git a/crates/batten/tests/it/signing_posture.rs b/crates/batten/tests/it/signing_posture.rs index 8d330e496..bc9be307e 100644 --- a/crates/batten/tests/it/signing_posture.rs +++ b/crates/batten/tests/it/signing_posture.rs @@ -49,7 +49,7 @@ // carried: "a missing override is NOT a finding when nothing sets signing globally" policy/signing-posture.rego kind:mechanism // carried: "a local override set to true is refused when the signer is broken" policy/signing-posture.rego kind:mechanism // carried: "the refusal echoes no part of the signature block" policy/signing-posture.rego kind:mechanism -// changed: "--repair leaves a verifiable signer alone rather than switching signing off" mise.toml the write is the WRITE, which a module cannot be, so it stayed a task — `[tasks.signing-posture-repair]` — and its guard is the same shared `signer_posture.py` reading this tier drives. `an_inline_public_key_is_a_literal_not_a_path_and_is_verifiable` pins the branch the guard turns on +// changed: "--repair leaves a verifiable signer alone rather than switching signing off" mise.toml the write is the WRITE, which a module cannot be, so it stayed a task — `[tasks.signing-posture-repair]` — and its guard is the same shared `crates/batten/src/signer_posture.rs` reading this tier drives. `an_inline_public_key_is_a_literal_not_a_path_and_is_verifiable` pins the branch the guard turns on // changed: "--repair writes the override, local only" mise.toml the same split: the write and its scope are the task's, and `git config --local` is the one line that states it // changed: "--repair is idempotent" mise.toml idempotence is a property of `git config --local commit.gpgsign false`, which is the task's single write // changed: "--repair never writes global config" mise.toml the same boundary `attribution-identity` draws, and it is stated where the write is — a developer's own unrelated repositories are not this repo's business diff --git a/mise.toml b/mise.toml index 99e1238dd..f26e4d82f 100644 --- a/mise.toml +++ b/mise.toml @@ -2103,15 +2103,19 @@ cargo run --quiet -p batten -- record named timeout-drift <"$census" # CLOUD-1717. The two things that are not decisions: the `cargo metadata` spawn, # and the graph walk. # -# THE WALK IS A `.py` SIBLING RATHER THAN A BODY HERE, and that is what keeps it -# under test. Four of the dying suite's eight cases are about the WALK — an -# unactivated optional dependency, the same one activated, a dev-dependency, and -# a crate the workspace reaches but the evaluator does not — and they are the +# THE WALK IS NOT A BODY HERE, AND THAT IS WHAT KEEPS IT UNDER TEST. Four of the +# dying suite's eight cases are about the WALK — an unactivated optional +# dependency, the same one activated, a dev-dependency, and a crate the +# workspace reaches but the evaluator does not — and they are the # security-critical half. Inlined here they would be assertable by nothing; -# beside the module they are `crates/batten/tests/it/evaluator_closure.rs`'s to -# drive. `mise-tasks/replay-pointers.py` is the precedent and -# `shell-retirement.rego:159-163` is why it is admissible: `.py` is excluded from -# `under_mise_tasks`, so this adds no shell rule. +# in `crates/batten/src/cargo_graph.rs` they are +# `crates/batten/tests/it/evaluator_closure.rs`'s to drive. +# +# IT WAS A `.py` SIBLING FOR PART OF THIS BRANCH, and that detour is recorded +# rather than quietly dropped: `.py` is excluded from `under_mise_tasks`, so the +# ratchet counted the program gone while every line of its reading stayed in the +# tree one interpreter over. `shell-retirement.rego`'s arm F refuses that shape +# now, and the walk is Rust the engine tests. # # `--locked` because the question is about the COMMITTED resolution. A gate # allowed to update the lockfile answers "what would upstream give me today", From b92970a875ea2de4476b1f0fd80a0561c8c6f0c4 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sun, 20 Sep 2026 08:01:19 +0000 Subject: [PATCH 32/32] fix(record): this consumer's `[[pattern]]` row ids leave the repo-agnostic core MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `declared_pattern`'s own doc comment states the rule it was then used to break: NON-NEGOTIABLE RULE 1 IS WHY THIS EXISTS. Which package is the evaluator, which crates bear IO, which need a platform SDK and which vendor what they link are all CONSUMER facts, and a `const` here would put a consumer identifier in the repo-agnostic core. Two screens below it, `record.rs` named `evaluator-package`, `evaluator-io-crate`, `sdk-framework-crate` and `vendored-links-crate` as string literals. Those are not crate names — they are the ids of `[[pattern]]` rows in THIS repository's `batten.toml`, which is a consumer's config key compiled into the engine. `cargo_graph.rs`'s header makes the same promise for the walk ("NO CRATE NAME REACHES IT"), and the module resolving its roots broke it. They are the producer's inputs now — `--input roots=`/`bears=` and `--input framework=`/`vendored=` — which is exactly what `DERIVE_INPUT` documents: the keys a family accepts are that family's own contract, and a key no family declares is a usage error rather than a silent default. The names live in `mise.toml`, beside the `cargo metadata` spawn that already knows which tree it is measuring. A grep of `crates/batten/src` for any of the four now returns zero hits, which is rule 1's own stated test. The 20 cases in `evaluator_closure.rs` and `macos_link.rs` pass against the compiled binary. `signer_posture.rs`'s `/tmp/` constant is deliberately NOT moved: an ephemeral temporary directory is a POSIX fact rather than one of this consumer's identifiers, and rule 1 names account numbers, client names and entity paths. Manufacturing a config knob for it would widen the surface to say nothing. Refs: CLOUD-1717 --- crates/batten/src/record.rs | 43 ++++++++++++++++----- crates/batten/tests/it/evaluator_closure.rs | 14 ++++++- crates/batten/tests/it/macos_link.rs | 14 ++++++- mise.toml | 6 ++- 4 files changed, 64 insertions(+), 13 deletions(-) diff --git a/crates/batten/src/record.rs b/crates/batten/src/record.rs index 3a1d6a30a..f395ebbe9 100644 --- a/crates/batten/src/record.rs +++ b/crates/batten/src/record.rs @@ -748,19 +748,35 @@ fn derive_reading( /// The evaluator's own sub-closure, and which crates in it bear IO (CLOUD-831). /// +/// THE ROW IDS ARE THE PRODUCER'S, NOT THIS MODULE'S (non-negotiable rule 1). +/// `roots` and `bears` name `[[pattern]]` rows, and which rows a repository +/// declares is a consumer fact — a literal here would be this consumer's config +/// key compiled into the repo-agnostic core, which is the thing +/// [`declared_pattern`]'s own doc comment says must not happen. The caller names +/// them, exactly as `DERIVE_INPUT` describes: the keys a family accepts are that +/// family's own contract. +/// /// # Errors /// -/// A [`UsageError`] for an unaccepted input, an undeclared `[[pattern]]` row, or -/// stdin that is not a `cargo metadata` document. +/// A [`UsageError`] for a missing or unaccepted input, an undeclared +/// `[[pattern]]` row, or stdin that is not a `cargo metadata` document. fn evaluator_closure_reading( inputs: &BTreeMap, family: &str, overrides: &Overrides, ) -> Result { - only_these_inputs(inputs, family, &[])?; + only_these_inputs(inputs, family, &["roots", "bears"])?; let config = resolve::resolve(Path::new("."), overrides)?; - let evaluator = declared_pattern(&config.patterns, family, "evaluator-package")?; - let bears_io = declared_pattern(&config.patterns, family, "evaluator-io-crate")?; + let evaluator = declared_pattern( + &config.patterns, + family, + required_input(inputs, family, "roots")?, + )?; + let bears_io = declared_pattern( + &config.patterns, + family, + required_input(inputs, family, "bears")?, + )?; let graph = graph_on_stdin(family)?; // THE SCOPE IS THE EVALUATOR'S SUB-CLOSURE, NOT THE WORKSPACE'S, and @@ -800,7 +816,8 @@ fn evaluator_closure_reading( /// /// [`evaluator_closure_reading`]'s sibling, and the two differ only in their /// ROOTS and in what they look for once there — which is the whole reason -/// [`crate::cargo_graph`] exists rather than a walk per caller. +/// [`crate::cargo_graph`] exists rather than a walk per caller. Its `framework` +/// and `vendored` row ids are the producer's for that function's reason. /// /// # Errors /// @@ -811,10 +828,18 @@ fn macos_link_reading( family: &str, overrides: &Overrides, ) -> Result { - only_these_inputs(inputs, family, &[])?; + only_these_inputs(inputs, family, &["framework", "vendored"])?; let config = resolve::resolve(Path::new("."), overrides)?; - let framework = declared_pattern(&config.patterns, family, "sdk-framework-crate")?; - let vendored = declared_pattern(&config.patterns, family, "vendored-links-crate")?; + let framework = declared_pattern( + &config.patterns, + family, + required_input(inputs, family, "framework")?, + )?; + let vendored = declared_pattern( + &config.patterns, + family, + required_input(inputs, family, "vendored")?, + )?; let graph = graph_on_stdin(family)?; // THE WALK STARTS AT THE WORKSPACE MEMBERS, because the question is diff --git a/crates/batten/tests/it/evaluator_closure.rs b/crates/batten/tests/it/evaluator_closure.rs index a709e6adf..c0a0ccd41 100644 --- a/crates/batten/tests/it/evaluator_closure.rs +++ b/crates/batten/tests/it/evaluator_closure.rs @@ -136,7 +136,19 @@ fn record(dir: &std::path::Path, lines: &str) { /// `evaluator-package` row selects, and the names its `evaluator-io-crate` row /// looks for once there. That composition is the gate, and it is per-consumer. fn walk(dir: &std::path::Path, metadata: &str) -> String { - let written = run_with_stdin(dir, &["record", "derive", "evaluator-closure"], metadata); + let written = run_with_stdin( + dir, + &[ + "record", + "derive", + "evaluator-closure", + "--input", + "roots=evaluator-package", + "--input", + "bears=evaluator-io-crate", + ], + metadata, + ); assert!( written.status.success(), "the walk reads its graph: {}", diff --git a/crates/batten/tests/it/macos_link.rs b/crates/batten/tests/it/macos_link.rs index 1237f504d..00ef5b57f 100644 --- a/crates/batten/tests/it/macos_link.rs +++ b/crates/batten/tests/it/macos_link.rs @@ -135,7 +135,19 @@ fn record(dir: &std::path::Path, lines: &str) { /// `links` key it reads, and the two `[[pattern]]` rows that decide which names /// mean an SDK and which vendor what they link. fn walk(dir: &std::path::Path, metadata: &str) -> String { - let written = run_with_stdin(dir, &["record", "derive", "macos-link"], metadata); + let written = run_with_stdin( + dir, + &[ + "record", + "derive", + "macos-link", + "--input", + "framework=sdk-framework-crate", + "--input", + "vendored=vendored-links-crate", + ], + metadata, + ); assert!( written.status.success(), "the walk reads its graph: {}", diff --git a/mise.toml b/mise.toml index f26e4d82f..ffe2f0325 100644 --- a/mise.toml +++ b/mise.toml @@ -2429,7 +2429,8 @@ fi # so the two cannot drift, and the crate names it looks for are `[[pattern]]` # rows rather than constants in a repo-agnostic core (CLOUD-1717). printf '%s' "$metadata" \ - | cargo run --quiet -p batten -- record derive macos-link + | cargo run --quiet -p batten -- record derive macos-link \ + --input framework=sdk-framework-crate --input vendored=vendored-links-crate """ [tasks.evaluator-closure-record] @@ -2446,7 +2447,8 @@ fi # so the two cannot drift, and the crate names it looks for are `[[pattern]]` # rows rather than constants in a repo-agnostic core (CLOUD-1717). printf '%s' "$metadata" \ - | cargo run --quiet -p batten -- record derive evaluator-closure + | cargo run --quiet -p batten -- record derive evaluator-closure \ + --input roots=evaluator-package --input bears=evaluator-io-crate """ [tasks.branch-age-record]