Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .agent-loop/CURRENT_STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ authority; these records do not grant or withhold it.
| [WS-CI-001](initiatives/WS-CI-001-backend-ci-acceleration/STATUS.md) | Semantic distributed backend lanes complete | Treat further CI optimization as a fresh measured bounded change |
| [WS-CI-002](initiatives/WS-CI-002-deterministic-agent-gates/STATUS.md) | `WS-CI-002-01` complete through PR #311; Agent Gates is deterministic per PR head | Preserve protected-branch review as the independent approval authority |
| [WS-CI-003](initiatives/WS-CI-003-atomic-chunk-state/STATUS.md) | `WS-CI-003-01` complete | Require every chunk PR to land its final contract and initiative state atomically |
| [WS-CI-004](initiatives/WS-CI-004-review-evidence-integrity/STATUS.md) | `WS-CI-004-PLAN` lands the design; later implementation steps are proposed only and have no active contracts | After explicit approval, create the smallest contract for the first implementation step only |
| [WS-DB-001](initiatives/WS-DB-001-v01-schema-baseline/STATUS.md) | v0.1 schema baseline complete through PRs #316 and #317 | Extend `0001_v01_baseline` only through future bounded migrations |
| [WS-SEC-001](initiatives/WS-SEC-001-dependency-alert-remediation/STATUS.md) | `WS-SEC-001-01` complete on merge with patched runtime and tooling dependencies | Handle future security alerts through fresh bounded dependency changes |
| [WS-DOCS-001](initiatives/WS-DOCS-001-current-v01-documentation/STATUS.md) | Current v0.1 entry documentation complete | Keep current pages synchronized with merged capability changes |
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Chunk Map: WS-CI-004 Review Evidence Integrity

| Chunk | Goal | Risk | State |
|---|---|---:|---|
| `WS-CI-004-PLAN` | First-principles discovery, threat model, design, and bounded delivery contracts | L1 | Planned on merge; no implementation behavior |
| Proposed step 1 | Shared Reviewer Evidence Protocol and deterministic review-target command, reusing `scripts/git_delta.py` | L1 | No contract; not active |
| Proposed step 2 | Adopt the shared protocol across all nine reviewer agents and matching skills, with the shared evaluation harness | L1 | No contract; not active |
| Proposed step 3 | Add local exact-target session convergence and orchestration rules | L1 | No contract; not active |

This is sequencing guidance, not an implementation queue. Create one bounded
contract only when a human explicitly starts that step. One implementation
contract equals one pull request, and there is no automatic successor.
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Decisions: WS-CI-004 Review Evidence Integrity

## D1: Review evidence is not authority

GitHub permissions and branch protection govern contribution. Human approval
governs merge. Internal review records cannot start, block, recover, or complete
initiative work by themselves.

## D2: Bind every pass to a subject

The minimum review subject is base SHA, merge-base SHA, and reviewed head SHA.
A verdict without that identity is not final-head evidence.

## D3: Separate subject, evidence, and verdict

Commit identity proves what was reviewed, evidence provenance explains what was
observed, and the reviewer verdict expresses a bounded conclusion. None may be
inferred from another.

## D4: Reuse the old evidence core, not the old control system

Exact SHA, timestamp, run references, closed results, and stale-delta detection
are reusable. Signed starts, leases, merge intents, loop memory, recovery
certificates, successor dispatch, and machine authorization are excluded.

## D5: One shared protocol, specialty extensions

Universal reviewer rules live in one mandatory skill. Specialty agents and
skills add focused questions and must not fork the provenance/freshness model.

## D6: Proportionate invalidation and convergence

A head change invalidates deterministic evidence and every reviewer whose paths,
boundaries, findings, or evidence claims are affected. Unaffected specialties do
not repeat ceremony. Base changes always require effective-delta reconciliation.
Readiness requires all required results to converge on one
`{base_sha, merge_base_sha, head_sha}` triple.

## D7: No hosted receipt gate

This initiative ends at local session convergence. It does not create durable
receipt custody, a trusted issuer, a hosted validator, or a blocking evidence
gate.

## D8: Human review remains essential

Exact-head provenance prevents stale or ambiguous passes; it does not prove the
reviewer was correct. Independent sensors and human judgment remain mandatory.

## D9: Session receipt custody

The orchestrator alone writes create-once JSON receipts beneath the Git common
directory, keyed by review target, run, and specialty. Target changes create new
receipts; prior receipts are never rewritten. GitHub remains durable custody.

## D10: One receipt shape and one target model

The existing internal-review-evidence and review-finding templates are extended
in place as the session receipt/ledger shape. The shared protocol skill is the
instruction source. The proposed convergence step must consume the target model
from the first implementation step; it must not reimplement `scripts/git_delta.py`
resolution or closed-result semantics.
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
# Discovery: WS-CI-004 Review Evidence Integrity

## Current behavior

The active Agent Gates workflow checks out the exact pull-request head and runs
repository integrity checks, but it does not validate internal reviewer evidence.
Reviewer routing is risk-based and human merge approval remains mandatory.

The nine `.codex/agents/*.toml` files share nearly identical boilerplate. They
require comparison with a base branch but do not require base/head SHAs,
merge-base resolution, dirty-state inspection, end-of-review freshness, prior
finding replay, or evidence provenance in their output.

The problem is not only missing provenance. Generic prompts let a reviewer claim
its specialty without proving that it detects the defects it owns. Documentation
review has missed stale current-state wording; architecture review has missed
private imports, public-port ownership, and ledger asymmetry. No specialty has a
complete isolated must-find and false-positive evaluation suite, so prompt
presence is currently being mistaken for reviewer effectiveness.

## Relevant files and modules

| Path | Current role | Gap |
|---|---|---|
| `.codex/agents/*.toml` | Nine specialty reviewer prompts | Universal review-subject and evidence rules are absent and duplicated |
| `.agents/skills/*-review/SKILL.md` | Specialty review checklists | Broad focus questions; no shared exact-head protocol |
| `.agents/skills/evidence-gate/SKILL.md` | Pre-fanout deterministic checklist | Does not bind results to a revision or verify evidence claims |
| `.agents/skills/task-chunk-loop/SKILL.md` | Bounded implementation workflow | Reruns failed reviewers only; a prior pass may become stale |
| `.agents/skills/plan-review/SKILL.md` | L1 plan review | Does not require consumer/import/ledger/historical-state tracing |
| `.agents/skills/pr-trust-bundle/SKILL.md` | PR evidence summary guidance | Does not require run provenance or one final reviewed head |
| `.agent-loop/templates/INTERNAL_REVIEW_EVIDENCE.md` | Optional durable review template | Records SHA/run IDs, but the active process does not enforce or clearly interpret them |
| `.agent-loop/templates/PR_TRUST_BUNDLE.md` | Canonical trust-bundle template | Has review fields but no freshness/replay rules |
| `.github/workflows/agent-gates.yml` | Exact-head deterministic repository checks | Correctly avoids making AI review merge authority; no internal review freshness check |
| `scripts/workstream_agent_gate.py` | Read-only risk sensor | Useful diff inventory; not a review attestation tool |

## Failure replay

PR #338 initially recorded all internal tracks and deterministic checks as
passing. Five later commits were needed to correct issues that record missed:

1. preserve the tracked CP03 contract path;
2. declare atomic `planned` outcomes rather than future implementation as
complete;
3. expose ACTORS/PROJECTS public owner ports and keep CON lifecycle truth in CON;
4. preserve completed CP02 history rather than rewriting it;
5. expose `ServiceIdentity` through `actors.api` and retire the exact private
AUTH import-ledger edges.

Recent PRs #331, #334, and #336 also contain reviewer-pass summaries without one
uniform exact-head/run-provenance format. PR #336 explicitly records that an
earlier pass was superseded by later exact-head QA.

## Historical implementation

The repository previously contained `scripts/check_internal_review_evidence.py`.
Its reusable core required a resolvable reviewed commit, UTC timestamp, reviewer
run references, closed result tokens, no unresolved blocking findings, and
rejected relevant changes after review.

Commit `baffbe8a` removed that checker together with signed starts, active leases,
merge intents, loop memory, recovery certificates, chunk-scope enforcement, and
more than 20,000 lines of combined process machinery. The removal restored the
simple contribution loop because derived state and self-authorizing recovery had
made contribution circular and brittle.

The exact-revision core is useful. The coupled authorization/orchestration system
is not.

## External principles reviewed

- [SLSA's attestation model](https://slsa.dev/attestation-model) (SLSA v1.2,
accessed 2026-08-15)
separates the exact subject from the predicate that makes claims about it;
[SLSA provenance](https://slsa.dev/provenance) (SLSA v1.2, accessed
2026-08-15) treats provenance as
verifiable information about where an artifact came from. Applied here, a
review result needs an unambiguous subject digest: the reviewed Git revision.
- [SLSA specification](https://slsa.dev/spec/) (v1.2, accessed 2026-08-15)
separates an attestation's
subject and verified properties. Applied here, commit identity and reviewer
conclusions must be separate fields rather than one prose claim.
- [GitHub protected-branch guidance](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches)
(accessed 2026-08-15)
records approvals against a diff and supports dismissing stale approvals or
requiring approval of the latest reviewable push. Internal review should use
the same freshness principle without replacing GitHub authority.
- [NIST SSDF](https://csrc.nist.gov/projects/ssdf) (accessed 2026-08-15)
recommends risk-based secure
development, provenance, documented security requirements/risks/design
decisions, and continuous improvement rather than a ceremonial checklist.
- [OpenSSF Scorecard](https://github.com/ossf/scorecard) (accessed 2026-08-15)
treats code review,
branch protection, and CI as distinct supply-chain signals. Workstream should
preserve that independence.

## Existing tests and gaps

Current lightweight Agent Gates test repository checks and atomic chunk state.
There is no active test suite for:

- reviewer start/end SHA equality;
- relevant-delta invalidation after a pass;
- reviewer output provenance;
- prior-finding replay;
- false trust-bundle evidence;
- ownership/import/debt-ledger trace expectations;
- all required reviewer results converging on one final head.
- isolated must-find and must-not-flag evaluations for each reviewer specialty;
- cross-specialty routing without invented conclusions.

## Risks discovered

| Risk | Why it matters | Proposed handling |
|---|---|---|
| Self-referential committed evidence | Adding evidence changes the head it describes | Keep internal receipts outside Git; GitHub owns durable evidence |
| Universal fanout | Slows contribution and creates ceremony | Preserve risk routing and invalidate only affected tracks |
| Prompt drift | Nine TOMLs duplicate universal rules | Put universal rules in one mandatory shared skill |
| Prose passes | Claims can outrun proof | Require explicit evidence source and observed result |
| Diff-only review | Unchanged consumer or ledger can invalidate design | Require impact-cone tracing into relevant base code and ledgers |

## Unknowns requiring measurement

- How often final-head changes affect only evidence/docs versus reviewer-owned
behavior.
- Whether a compact structured review receipt materially improves human review.
- Whether session receipts improve review enough to justify keeping them.

## Existing conventions to preserve

- Review is evidence, not contribution authority.
- Human approval owns merge.
- Internal and external review remain distinct.
- Reviewer routing remains proportionate to risk.
- GitHub open PRs represent transient work; durable files do not imitate a queue.
- No second post-merge memory PR.
110 changes: 110 additions & 0 deletions .agent-loop/initiatives/WS-CI-004-review-evidence-integrity/INTENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
# Intent: WS-CI-004 Review Evidence Integrity

## Problem being solved

Internal reviewer results can currently say `PASS` without proving which exact
revision, merge base, unchanged implementation, prior findings, or raw evidence
the reviewer inspected. A later push can make an earlier pass stale without a
mandatory replay. Review prose can also claim that a command passed when the
exact-head check did not.

## Why this work matters

Workstream is critical governed-contribution infrastructure. Authorization,
artifact, review, contribution, and compensation boundaries cannot depend on
review records whose subject or proof is ambiguous. A false pass can be more
dangerous than no review because it gives humans confidence that was not earned.

## Current behavior

- GitHub Actions binds repository checks to the pull-request head.
- GitHub permissions, branch protection, and human approval control merge.
- Reviewer agents have useful specialties but only generic shared instructions.
- Optional review templates can record a SHA, but the reviewer workflow does not
require or validate it.
- Head changes do not automatically invalidate affected internal reviewer passes.
- Prior findings have no stable replay/closure identity.

## Target behavior

Every applicable internal reviewer result identifies one exact review subject,
proves what was inspected, distinguishes executed evidence from claims, replays
prior findings, traces affected ownership and boundaries, and becomes stale when
the relevant pull-request delta changes. All final reviewer results and hosted
checks must refer to the same final head before merge readiness is reported.

## Design chosen

Use a small shared Reviewer Evidence Protocol plus deterministic Git target
inspection. Specialty agents keep their focused review responsibilities while
the shared protocol owns revision identity, evidence provenance, prior-finding
replay, final-head freshness, and common output fields.

Adopt the protocol in measured stages without a committed or hosted receipt
gate. Internal receipts remain user-private, out-of-tree session evidence;
GitHub checks/reviews and merged Git history remain durable evidence.

## Alternatives considered

- **Restore the deleted internal-review gate:** rejected. Its exact-SHA core was
sound, but it became coupled to signed starts, leases, merge intents, loop
memory, recovery certificates, chunk-scope authorization, and a 7,870-line
mixed regression suite.
- **Prompt-only reminders in nine agent files:** rejected. Universal rules would
drift and remain difficult to test.
- **Require a committed exact-head attestation immediately:** rejected. Evidence
created after review changes the commit it attempts to attest to and recreates
evidence-only commit ceremony unless a carefully bounded subject model is
proven first.
- **Treat all head changes as requiring all nine reviews:** rejected. Review must
be conservative but proportionate; unaffected specialties need not repeat
ceremonial work.
- **Let CI or AI approval authorize merge:** rejected. Review is evidence, not
repository authority.

## Boundaries preserved

- No Workstream product lifecycle, API, schema, migration, authorization grant,
payment, artifact, review, or contribution behavior changes.
- No signed-start, loop-memory, merge-intent, lease, recovery, or post-merge
automation returns.
- No automated merge or replacement of GitHub branch protection.
- No universal reviewer fanout for low-risk work.
- No external service, secret, token, or new hosted infrastructure is required.

## Expected risks

- Reintroducing the self-blocking process complexity removed in PR #207.
- Creating review ceremony without improving defect detection.
- Allowing a reviewer to attest only to the changed lines while missing relevant
unchanged consumers, owners, or debt ledgers.
- Treating a commit SHA as sufficient proof without evidence provenance.
- Letting mutable PR text silently substitute for reviewer execution.
- Over-invalidating unaffected reviews after documentation-only evidence updates.

## What must not change

The simple loop remains:

```text
Intent -> Plan -> Bounded Change -> Tests -> Review -> PR -> Human Merge
```

GitHub remains contribution and merge authority. Internal agents, CodeRabbit,
and CI remain independent sensors. Historical review evidence never becomes an
active-work queue or implementation authorization source.

## How this will be proven

- Deterministic Git fixtures for head/base/merge-base identity, dirty state, and
head-change invalidation.
- Reviewer contract tests proving required provenance and prior-finding replay.
- Negative tests for stale passes, false evidence claims, missing commands,
unknown findings, private-import/ownership omissions, and final-head mismatch.
- Replay of the five defects missed during PR #338 planning review.
- Out-of-tree create-once receipt fixtures and exact-target convergence tests.

## Human decisions required

- Approve this plan and each implementation chunk separately.
- Continue to own every merge and every accepted remaining risk.
Loading
Loading