Skip to content

feat(code-review)!: two rosters in the default, and a manifest a repo can start from - #85

Merged
pedromvgomes merged 7 commits into
mainfrom
feature/code-review-manifest-init
Sep 9, 2026
Merged

feat(code-review)!: two rosters in the default, and a manifest a repo can start from#85
pedromvgomes merged 7 commits into
mainfrom
feature/code-review-manifest-init

Conversation

@pedromvgomes

Copy link
Copy Markdown
Owner

Follows #84. Five things, one of them breaking.

The default reviews pull requests on codex

The change most likely to surprise, because it applies to every repo that has not written a manifest — which is most of them, since working without one is what makes adoption cheap.

The default now carries a roster per provider with the same panels and rules for each: the local pass is Claude (quick), the pull request is codex (standard-codex). A change is read by two models trained differently before anyone else sees it, rather than twice by the model that already read it. The codex panels judge and validate on codex too, so a PR is reconciled without the model that reviewed it locally. Every model is named — a reviewer on the CLI's own default is a model nobody chose.

run --pr therefore needs codex on PATH. A machine without it does not get a silent half-review: the codex runs report they could not start, and if nothing answered at all the run refuses rather than reporting an empty finding set.

The escalation criteria changed with it:

criterion panel why
auth, crypto, concurrency, sensitive-data, fix-revert deep exploitable, or lands on somebody not in the room, or indistinguishable from correct until production
referencing_files >= 20 deep unchanged
migrations, ci-cd, iac standard irreversible or wide, but the risk is not in the diff
changed_files >= 20 standard unchanged

auth and migrations escalated on path globs, which duplicated — badly — signals the binary already detects and tests. Demonstrated: request-gating middleware under internal/session/ carries the auth signal, matched no glob, and got the shallowest panel. They are signals now. migrations drops to standard because its risk is the table it locks, not anything legible in the diff.

context, so a rule can say where it applies

A rule names one panel, so a two-roster manifest could not say "this criterion, but on the other roster for a pull request". Writing it twice does not work either — equal cost is not a raise, so the first-written rule wins in both contexts, decided by an order the design says carries no meaning.

  - to: deep-codex
    all:
      - signals: {in: [auth]}
      - context: {in: [pr]}

in and not_in over worktree/pr; unknown names refused at parse. The one condition that tests the run rather than the change, so it can never be unavailable.

It exposed a bug: an all: rule stopped at its first unreadable condition, so a rule guarded to the other context looked unreadable rather than inapplicable. A repo whose language has no symbol extractor saw every PR-only rule reported as skipped on every local review, and a repo-written manifest refused the review outright.

agtk code-review init, and initializeregister

A manifest was never required — a repo without one uses the default in the binary, which is what makes adoption work. What was missing was a way to start from it. init writes it to .agents/code-review/manifest.yaml, refuses to overwrite without --force, and --dry-run reports the path.

That forced a rename: initialize stores a GitHub App private key, an unrelated job, and init beside initialize is two near-identical words where the shorter is a strict prefix — a typo runs the other. It is register now, which its own summary always called it. initialize still resolves as an unlisted alias and says so on stderr.

Skill: panel and commit targets

/panel-code-review quick and /panel-code-review the last commit both work now. Neither was documented; the commit target was a regression from deep-code-review, which took ranges.

A named panel silences the rules--panel wins outright, so an escalation that fired does not raise past it. The skill now reports that rather than printing the panel as though the rules chose it. And it always passes --head with a commit, since that flag defaults to the working tree: --base HEAD~1 alone reviews the last commit plus anything uncommitted.

Reviewed, and what it found

/code-review on this branch returned seven findings. All seven were verified against running code and fixed. Two mattered:

  • init broke reviewing outright. It promised "the default verbatim, so nothing changes", but Builtin is provenance: the default's referencing_files rules are skipped only while they ship in the binary, and refused once a repo owns them. Same bytes, panel: quick before and agtk: escalate[2]: … after. The refusal is correct — a repo with a manifest can act on the advice — so init now names the rule it just made a refusal, and the remedy. The original test only asserted the bytes parse; parsing is not selection.
  • The escalation fix went too wide. Suppressing the refusal on any readable no meant a repo's broken rule passed for a change outside its globs and refused for one inside — failing by mood. Only a context guard settles a rule now.

Plus: --dry-run refused exactly when the path question was worth asking, gofmt (which would have failed CI), a comment claiming two rules "cannot fire" when they fire and merely cannot raise, and release notes that never mentioned the codex requirement.

Verification

gofmt -s, go build, go vet, full suite — clean. New tests cover the context condition and its guards, the narrowed refusal, init (verbatim write, overwrite refusal, dry-run with and without an existing manifest, and that it names an unrunnable rule), and the register/init split. Each was mutation-tested — nine mutations across these commits, each caught by its intended test, code restored.

Behaviour verified end to end against the built binary, not only in tests: both rosters and their context guards, init before/after on a repo whose language has no symbol extractor, commit and range targets, and --panel overriding a fired rule.

Open, and yours to call

  • The codex model picks are inferred. astra on security and judge, sol elsewhere — version number is my only evidence; the driver refuses to map families across vendors, so there is no opus-equivalent to lean on.
  • The two standard-codex rules cannot raise while that is also the PR default. Kept so both rosters read identically, and so lowering the default does not silently drop a criterion.
  • --panel quick on a PR runs the Claude roster, crossing the local/PR split. Correct for an explicit override, but both quick and quick-codex are reachable from either target.
  • feature-flags, message-consumers, public-api, shared-kernel still do not escalate — too loose for a shipped default (shared-kernel fires on all of pkg/, public-api on any Go struct tag).

An escalation names one panel, so a manifest holding a roster per provider
could not say "this criterion, but reviewed by the other roster on a pull
request". Writing the criterion twice does not work either: `deeper` is strict
and equal cost is not a raise, so of two rules pointing at equal-cost panels the
first written wins in both contexts, and the second ladder is unreachable —
decided by an order the design says carries no meaning.

`context: {in: [pr]}` is the missing clause. It takes `in` and `not_in` over the
contexts this build runs in, and an unknown name is refused when the manifest is
read rather than becoming a rule that silently never fires.

It is the one condition that tests the run rather than the change, which is why
it can never be unavailable: the context is what the caller asked for, not
something read off a diff. Evaluate takes it alongside the profile for that
reason.
An `all:` rule stopped at its first unreadable condition, so a clause after
that one was never evaluated. A rule guarded to the other context therefore
looked like a rule that could not be read, when it was simply not applicable:
a repo whose language has no symbol extractor saw every pull-request-only rule
reported as skipped on every local review, and a repo-written manifest refused
the review outright over a rule that would not have fired.

Every condition is evaluated now, and what stopped the rule decides what is
reported. An `all:` rule that a readable condition already answered "no" does
not apply, so an unreadable sibling is beside the point; an `any:` rule is
settled by a readable "yes". Only a rule left undecided by the unreadable
clause is reported, or refused for a manifest a repo wrote.

The existing built-in-default test passes unchanged: exactly one rule is still
reported, and the pull-request copy of it no longer is.
The default assumed one provider, so a repo using it got the same model reading
a change locally and reading it again on the pull request — a second look that
shares the first one's blind spots.

It now carries a roster per provider and the same panels and rules for each.
The local pass is Claude and the pull request is codex, so a change is read by
two models trained differently before anyone else sees it. The codex panels
judge and validate on codex, so a pull request is reconciled without the model
that already reviewed it. Every model is named: a reviewer on the CLI's default
is a model nobody chose.

Each rule is written once per roster and kept on its own side by `context`.

The escalation criteria change with it. auth and migrations were matched by path
globs, which is a weaker duplicate of signals the binary already detects and
tests: a request-gating middleware under internal/session/ carries the auth
signal and matched no glob, so the change these rules exist for got the
shallowest panel. They are signals now.

migrations drops from deep to standard. A migration is irreversible, but its
risk is the table it locks rather than anything legible in the diff, so every
axis twice buys opinions that cannot see what matters; the security reviewer is
the one with something to say. ci-cd and iac join it for the same reason.
sensitive-data joins auth at deep: harm there lands on people who are not in
the room.

BREAKING CHANGE: a repo with no manifest of its own now reviews pull requests
on codex. A machine without it gets a review that reports what could not run
rather than one that silently ran half. Panel names quick-codex, standard-codex
and deep-codex are new, and `--panel` accepts them.
…egister

A manifest was never required — a repo without one is reviewed by the default
in the binary, which is what lets code-review work on a repo that has just
adopted the toolkit. What was missing was a way to start from that default
without copying it out of the source. `init` writes it to the manifest path.

It writes the default verbatim, so nothing about a repo's reviews changes until
somebody edits it: explain reports the same panel before and after, and only
the manifest line differs. An existing manifest is not overwritten without
--force, because that file is where a repo tuned its own reviews and replacing
it with the default swaps every rule for none of them.

That forces the rename. `initialize` stores this machine's App private key, an
unrelated job, and `init` beside `initialize` is two near-identical words for
which the shorter is a strict prefix of the longer — so a typo runs the other,
and one of them handles a credential. `register` is what this command's own
summary always called it. `initialize` stays as an alias to the same command
rather than a second one, unlisted so the vocabulary a reader learns is the
current one, and it says on stderr which name it now has.
…ences a rule

A named panel wins outright: an escalation that fired does not raise past it,
so asking for `quick` on a change the rules send to `deep` is a downgrade the
person is entitled to make. The engine reports it — "named on the command line;
the rules above did not decide" — and the skill now has to pass that on rather
than print the panel as though the rules chose it.

The argument was described only as a name asked for in words, which left the
bare form a reader could not tell was accepted.
… failed by mood

Review of the branch found seven defects. Two mattered.

`init` promised the default verbatim changes nothing, and in a repo whose
language has no symbol extractor it broke reviewing outright: the same bytes
are skipped while they ship in the binary and refused once a repo owns them,
because a repo with a manifest can act on the advice a refusal gives. The
refusal is right; handing it over in silence was not. init now names the rule
it just made a refusal, and what to do about it, and its help states the rule
rather than promising nothing changes.

The escalation fix went too wide. Suppressing the refusal whenever any readable
condition said no meant a repo's broken rule passed for a change outside its
globs and refused for one inside — a manifest that looks healthy until the day
somebody touches the wrong directory. Only a context guard settles a rule now:
a rule this context does not run in was never addressed to this review, while
one whose glob merely did not match is still a protection the repo asked for.

Also: `--dry-run` refused instead of reporting the path when a manifest already
existed, which is when the question is worth asking; the default's comment said
two rules cannot fire when they fire and merely cannot raise; and gofmt.

The release notes never mentioned that a repo with no manifest now reviews
pull requests on codex and needs that CLI, nor that `context` is a new key.
`run` and `explain` take --base and --head, so reviewing one commit or a range
has always worked. The skill offered a pull request or the working tree and
nothing else, so "review the last commit" landed on the branch instead — the
one target it was not asked for.

It also has to pass --head. That flag defaults to the working tree, so
`--base HEAD~1` alone reviews the last commit plus whatever is uncommitted,
and the range line is the only place that difference shows.
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

bulwark

  • scan — no findings

  • coverage — go: 81.3% (baseline 81.2%); go patch: 87.9% (131/149 new lines; baseline 81.2%)

📦 Full bulwark output — complete scan and coverage logs.

@pedromvgomes
pedromvgomes merged commit 07f30f7 into main Sep 9, 2026
10 checks passed
@pedromvgomes
pedromvgomes deleted the feature/code-review-manifest-init branch September 9, 2026 03:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant