feat(analysis): wire calibration in (#48); fix both red CI checks - #107
Merged
Merged
Conversation
Two long-standing red checks on `main`, plus the blocking per-file debt behind one of them. CI --- * Hypatia's single high/critical finding was research_extensions RE008: `dependabot-automerge.yml` gated auto-merge on the dispatch actor rather than the PR author. The PR-author gate is kept, the actor conjunct is gone. The explanatory comment describes the pattern instead of quoting it, because RE008 is a raw-text scan with no comment filtering and quoting it would re-arm the finding. * `Publish Image` failed with `cargo build --release --locked -p oikosbot-cli` exit 101: tree-sitter 0.27.0 declares rust-version 1.90 / edition 2024 while the builder was rust:1.88-slim. CI on ubuntu-latest passed because the runner ships a newer rustc. Builder is now the rolling rust:1-slim. Calibration (#48) ----------------- `Analyzer::estimate_resources()` maps a detected pattern onto an `OperationKind` via the new `calibration::operation_for_pattern` and prices it with `estimate_operation()`, whose `ResourceRange` carries the min/typical/max band and the confidence that row has earned — `Calibrated` for the measured kinds, `Estimated` for the host-dependent ones. Units with no recognised pattern keep the naive complexity path, labelled `Estimated` with no band claimed. `ResourceRange` moves to `oikosbot-metrics` and rides on `AnalysisResult` (serde-default, skipped when absent), and SARIF emits it as `properties.resource_range`. `calibrated_estimate()` and its duplicated multiplier table are deleted; `standard_objectives()` is deleted as dead code. Tests ----- * calibration: per-kind confidence ladder, min<=typical<=max on every axis, the pattern→kind mapping (including the deliberate `redundant-allocation` hole). * analyzer: a recognised pattern earns Calibrated + a band; unrecognised code stays Estimated with no band; the calibrated path changes the numbers. * `crates/oikosbot-cli/tests/check_gate.rs`: end-to-end proof that `compare --check` can block — an undocumented calibrated regression exits 1, the same regression without `--check` exits 0, a documented `Pareto-Trade-off:` is accepted, and a heuristic regression refuses to block *and says so*. Docs ---- `docs/usage.adoc` and `docs/ci-runbook.adoc` (both indexed), per-crate guides, and `analyzers/code-haskell/README.adoc`. The "`--check` cannot block a merge" caveats in README/QUICKSTART/STATUS/action.yml/COMPARISON now describe what is true, and name the eco-score saturation that follows from calibrated microjoule-scale estimates. Also: delete `policy-engine/deepproblog/eco_problog.pl` (dead by the 2026-07-28 ruling) and record `instant-sync.yml`'s settings-level disabled state in its header. NOT verified in CI by this change: there is no Rust toolchain in the authoring environment, so `cargo fmt/test` results are unknown until the PR runs. Before/after figures in the docs are derived by reasoning and labelled as such. Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
arena-ai-coding-agent
Bot
requested a review
from hyperpolymath
as a code owner
September 25, 2026 23:51
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
hyperpolymath
approved these changes
Sep 25, 2026
hyperpolymath
added a commit
that referenced
this pull request
Sep 26, 2026
## What `DEBT.adoc` re-verified against the tree. Six entries listed fixes that had **already landed** as still open; each is now marked resolved with its date and the evidence, and the Summary rows are corrected to match. | Entry | Evidence it is resolved | |---|---| | `LICENSES/AGPL-3.0-or-later.txt` orphaned | absent from the working tree *and* from the base commit — `git ls-tree -r 7781489 -- LICENSES/` lists only `CC-BY-SA-4.0.txt` and `MPL-2.0.txt` | | `GOVERNANCE.md` no SPDX header | file deleted; the surviving `GOVERNANCE.adoc` carries `CC-BY-SA-4.0` on line 1 | | `ARCHITECTURE.md` boilerplate | deleted; `ARCHITECTURE.adoc` is the only architecture document | | `GOVERNANCE.md` same defect | deleted | | `ARCHITECTURE.adoc` names unbuilt components | the `[WARNING]` block now names each one inline (OCaml docs analyzer, superseded "Python + Datalog" line, fourth bot role, Praxis Loop / DeepProbLog) | | `push-email-notify.yml` dormant | gone from `.github/workflows/` (15 workflows, none so named), removed under the dual-use ruling | The **wiki stub** is recorded as *claimed* resolved rather than verified: wiki pages are not part of the repository tree, and this register's own rule is that an unverifiable claim says so. Two entries were corrected rather than closed: * the licence-identifier counts were re-measured — **159 MPL-2.0 / 65 CC-BY-SA-4.0** occurrences via `grep -rhoP "SPDX-License-Identifier: \K[A-Za-z0-9.+-]+"`, with the two headers quoted inside `docs/superpowers/plans/` called out as quotations rather than declarations; * the #16/#17 entry now names what the 2026-09-26 documentation pass actually delivered (`docs/usage.adoc`, `docs/ci-runbook.adoc`, the per-crate `README.adoc` set) instead of pointing at a bare `docs/README.adoc`. One deliberate restraint: the AGPL entry does **not** restate the licence identifier verbatim. A quoted `SPDX-License-Identifier:` line inside a document is indistinguishable from a real declaration to `grep`, and this register has already recorded that class of defect once (the duplicated header in `docs/README.adoc`). Writing the identifier out in full would have re-created it. `ROADMAP.adoc` follows: candidate (B) is marked landed, with its two consequences recorded rather than hidden (the naive path is still collinear and still `Estimated`; calibrated energies saturate `calculate_eco_score` near 100, so the `check` eco-threshold gate is near-vacuous and needs an owner ruling on the EnergyScore scale). "Policy rules in Datalog and DeepProbLog" no longer claims the DeepProbLog asset retired by the 2026-07-28 ruling. `CHANGELOG.adoc` and `STATE.a2ml` record the same. ## Why now PR #107 landed and CI verified it — the debt register is the repo's authority on what is outstanding, so a register that over-reports open debt is its own kind of dishonesty, and it hides the entries that genuinely are open. ## Verification * `tools/ci/linter-verify.sh` → PASS on all four steps (SPDX headers, permissions declaration, actions lockfile coverage, `lockcheck.sh`). * `grep -rhoP "SPDX-License-Identifier: \K[A-Za-z0-9.+-]+" | sort | uniq -c` → `65 CC-BY-SA-4.0`, `159 MPL-2.0`, `2 MPL-2.0+` (the two quoted in `docs/superpowers/plans/`). No third licence is declared. * Every claim above was re-checked against the tree before being written. No code changes; documentation only. Co-authored-by: hyperpolymath <6759885+hyperpolymath@users.noreply.github.com> Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the two red checks on
mainand lands the blocking per-file debt behind one of them (issue #48).Please read "What this change cannot prove" before merging.
1. Hypatia — the one high/critical finding, diagnosed and fixed
Hypatia Neurosymbolic Analysishas failed on every run since 2026-09-15 with exactly one high/critical finding. It isresearch_extensionsRE008 (critical):.github/workflows/dependabot-automerge.ymlgated auto-merge ongithub.actoris the run-triggering user, which a fork/branch/rerun event can set without the PR being Dependabot's. RE008 matches it with a raw-text regex over.github/workflows/*(no YAML parse, no comment filtering), which is why the explanatory comment in this change describes the pattern instead of quoting it — quoting it would re-arm the finding.Fix: drop the
github.actor ==conjunct, keep the PR-author gate. I re-ran RE008's own regex over all 15 workflows after the edit: zero matches.2. Publish Image — root cause confirmed
Build and pushexited 101 in 21s.tree-sitter0.27.0 declaresrust-version = "1.90"/edition = "2024"; theContainerfilebuilder wasrust:1.88-slim. CI passed becauseubuntu-latestships a newer rustc — the container is the only place the floor is enforced. Builder is now the rollingrust:1-slim(arrow-rs 59.3.0 needs only 1.88, so tree-sitter is the binding constraint).3. #48 — calibration wired in
calibration::operation_for_patternmaps a detected pattern onto theOperationKindthat best explains its cost:nested-loops→Sort,busy-wait→MathCompute,string-concat-in-loop→StringOp,clone-in-loop→Allocation,unbuffered-io→FileIO,large-allocation→Allocation.redundant-allocationis deliberately unmapped (multiplier 1.2) so a marginal finding cannot inherit calibrated confidence.estimate_operation()prices recognised units; itsResourceRangecarries min/typical/max and the confidence that row has earned —CalibratedforHashLookup/Sort/Allocation/MathCompute,Estimatedfor the host-dependentFileIO/StringOp/NetworkCall,UnknownforGeneric.complexity × constantpath, labelledEstimated, with no band claimed.ResourceRangemoves tooikosbot-metricsand rides onAnalysisResult(#[serde(default, skip_serializing_if = "Option::is_none")]— additive, old JSON still loads). SARIF emitsproperties.resource_range.calibrated_estimate()(the unwired, duplicate multiplier table) andstandard_objectives()(dead code).Falsifier tests
calibration: the per-kind confidence ladder, min ≤ typical ≤ max on all four axes for every row, and the pattern→kind mapping including the deliberate hole.analyzer: a recognised pattern earnsCalibrated+ a band; unrecognised code staysEstimatedwith no band; the calibrated path actually changes the number.crates/oikosbot-cli/tests/check_gate.rs(new, runs the real binary): an undocumented calibrated regression exits 1; the same regression without--checkexits 0; a documentedPareto-Trade-off:is accepted; a heuristic regression refuses to block and says so.Before/after (derived by reasoning, NOT measured — see below)
A depth-3 nested-loop unit of ~63 AST nodes:
EstimatedSort)CalibratedEvery resource figure and every downstream score moves with it.
What this change cannot prove
cargo fmt --check,cargo test --workspace --locked,cargo clippyand the container build have not been run. This PR's CI is the first compile. The changes were written to berustfmt-canonical (100-col, existing brace style) but that is a claim, not a result.EcoScoreis100 − 10·ln(E), anchored at 1 J. Calibrated energies are microjoule-scale, so most units clamp at 100 andcheck's eco threshold discriminates far less than it did. The Pareto verdicts (base vs head) are unaffected. This is documented inSTATUS.adoc,QUICKSTART.adoc,README.adocandaction.yml, and it needs an owner ruling on the score scale — I did not invent a new formula.Also in this change
docs/usage.adoc(end-user guide) anddocs/ci-runbook.adoc(gate inventory,actions.lockrules, how to read a red check from annotations,justtargets, pre-release checklist), both indexed indocs/README.adoc.analyzers/code-haskell/README.adoc.policy-engine/deepproblog/eco_problog.pl(dead by the 2026-07-28 ruling;EXPLAINME.adocupdated).instant-sync.yml's header now records its settings-level disabled state, which is invisible from the repository contents.DEBT.adoc,CHANGELOG.adocandSTATE.a2mlupdated: resolved entries marked, the eco-saturation caveat recorded.Still open (not in this PR)
mainruleset (repo settings — owner only).action.yml's pinned image digest must be refreshed after the next successful image build.