Close the baseline: plain-prose README, orientation and findings write-ups, and a switch panel on the explainer - #3
Merged
Conversation
…gs write-ups README.md is reworded for voice only. Every number, link target and fenced block is unchanged against the previous revision; the numeric claims were compared token by token. The "Honest gaps" heading is now "Known gaps" and its one anchor is updated. docs/EXPLAINED.md is a public orientation for a reader who does not know what an AI-control evaluation is. It corrects four statements from the private note it was written from: checkpoint B is 10 scenarios by 3 configurations rather than 30 scenarios, the approval token is not described as cryptographic because lab/approval.py says it is not, the FastMCP servers are stated as in-process, and T1's fault is a checkout 503 rather than a rollback. docs/FINDINGS.md is a first-person write-up for the AI-control audience, dated as a draft. Every figure names the file it was read from. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…correct the explainer footer docs/provider-policy-and-approvals.md now records that the application went out on 2026-08-27 and carries the sent text, which differs from the draft by an opening summary and by a spend figure that predates the repository's correction. The pre-send checklist is rewritten as a record. HANDOVER-07-EXECUTOR-STATE.md moves to docs/archive/ unchanged; the link in docs/findings/T4-human-gate-reached.md and the sentence in docs/HANDOFF.md that named its location are updated. The plain-text mentions in docs/verification.md are left as they are. explainer/build.py no longer emits a footer sentence saying README.md is stale; that stopped being true on 2026-08-27. The generated page is regenerated and build.py --check passes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…red configurations The page now opens with a panel where a visitor picks T1 or T4 and sets three switches for the policy gate, the monitor and human approval. The switches are tied so the reachable states are exactly the three arms the evaluation lane runs; no other combination has a published row, and the panel says so. All six states are rendered by build.py from the committed artifacts and the inline script only chooses which block is shown. T1 states come from derive_arms(); T4 states come from a new derive_t4_arms() over artifacts/model-monitor/T4-v4-nemo-dev-qwen3.json. The T4 full-chain block quotes the qwen3-14b step-0 rationale, cross-checked against its capture body the same way the Route B quotation is. A gate that recorded no decision reads "no decision recorded" rather than the artifact's null. Two helpers, _step_record() and _capture_with_rationale(), were extracted from existing derive functions so the new code shares them rather than copying them. Four tests pin the panel: exactly six blocks over the three arm names, every shown value equal to its artifact value, the rationale byte for byte, and only the T1 ungated block visible by default. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Team Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🟡 Changes recommended
A newly added test helper (_t1_rows) can silently ignore per-arm disagreements across attacker envelopes, undermining the intended artifact-pinning coverage.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR “closes the baseline” by (1) reworking the public-facing docs for clearer orientation and a first-person findings write-up, and (2) extending the explainer to include a new “switch panel” UI whose six rendered states are derived from committed artifacts and pinned by tests.
Changes:
- Add a switch panel to the explainer (server-rendered states, minimal client-side switching) and add tests that pin displayed values to the underlying artifacts.
- Reword README into plain prose while keeping the core technical claims and references intact, and add two new docs (
docs/EXPLAINED.md,docs/FINDINGS.md). - Update provider approval recordkeeping and archive/retarget the executor handover document.
File summaries
| File | Description |
|---|---|
| tests/test_explainer_is_derived_from_artifacts.py | Adds artifact-pinning tests for the new switch panel states and rationale quoting. |
| README.md | Plain-prose rewrite + pointers to new orientation doc; keeps technical content but updates phrasing/structure. |
| explainer/index.html | Generated explainer HTML updated to include the new switch panel and its styling/JS. |
| explainer/build.py | Adds derivation + rendering for the switch panel, plus shared helpers for step records and rationale provenance. |
| docs/provider-policy-and-approvals.md | Updates OpenRouter application section to record the sent message and current status. |
| docs/HANDOFF.md | Updates reference to the archived executor-state document location. |
| docs/findings/T4-human-gate-reached.md | Updates the handover document link to the archived path. |
| docs/FINDINGS.md | New AI-control-audience findings write-up (draft dated 2026-09-06) with file-cited figures. |
| docs/EXPLAINED.md | New reader-oriented “explained” document with file-cited figures. |
| docs/archive/HANDOVER-07-EXECUTOR-STATE.md | Moves/archives the executor handover snapshot into docs/archive/. |
Review details
- Files reviewed: 10/11 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+461
to
+468
| def _t1_rows() -> dict[str, dict]: | ||
| """The T1 model-attacker rows, one per arm, read straight off the envelopes.""" | ||
| rows: dict[str, dict] = {} | ||
| for path in sorted((ROOT / "artifacts/model-attacker").glob("*.json")): | ||
| for row in json.loads(path.read_text(encoding="utf-8"))["results"]: | ||
| if row["attacker_source"].startswith("model:"): | ||
| rows.setdefault(row["condition"], row) | ||
| return rows |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three commits that close the project at a baseline.
Plain-prose README, plus two new documents.
README.mdis reworded for voice only: every number, link target and fenced block is unchanged againstmain, compared token by token.docs/EXPLAINED.mdis a public orientation for a reader who does not know what an AI-control evaluation is, and corrects four statements from the private note it came from.docs/FINDINGS.mdis the first-person write-up for the AI-control audience, dated as a draft, with every figure naming the file it was read from.Records brought in line with what happened.
docs/provider-policy-and-approvals.mdrecords the OpenRouter application as sent on 2026-08-27 and carries the sent text.HANDOVER-07-EXECUTOR-STATE.mdmoves todocs/archive/with its two references updated. The explainer footer no longer says the README is stale.A switch panel on the explainer. The hosted page now opens with a panel where a visitor picks T1 or T4 and sets the three gates. The switches only reach the three configurations the lane runs; all six states are rendered from the artifacts by
build.py, and four new tests pin every shown value to its artifact, including the T4 rationale byte for byte.Verification on this branch: full suite 2525 passed, 2 skipped against Postgres;
ruff checkclean;explainer/build.py --checkclean. Merging redeploys the explainer through the existing Pages workflow.🤖 Generated with Claude Code