feat(code-review)!: two rosters in the default, and a manifest a repo can start from - #85
Merged
Merged
Conversation
An escalation names one panel, so a manifest holding a roster per provider
could not say "this criterion, but reviewed by the other roster on a pull
request". Writing the criterion twice does not work either: `deeper` is strict
and equal cost is not a raise, so of two rules pointing at equal-cost panels the
first written wins in both contexts, and the second ladder is unreachable —
decided by an order the design says carries no meaning.
`context: {in: [pr]}` is the missing clause. It takes `in` and `not_in` over the
contexts this build runs in, and an unknown name is refused when the manifest is
read rather than becoming a rule that silently never fires.
It is the one condition that tests the run rather than the change, which is why
it can never be unavailable: the context is what the caller asked for, not
something read off a diff. Evaluate takes it alongside the profile for that
reason.
An `all:` rule stopped at its first unreadable condition, so a clause after that one was never evaluated. A rule guarded to the other context therefore looked like a rule that could not be read, when it was simply not applicable: a repo whose language has no symbol extractor saw every pull-request-only rule reported as skipped on every local review, and a repo-written manifest refused the review outright over a rule that would not have fired. Every condition is evaluated now, and what stopped the rule decides what is reported. An `all:` rule that a readable condition already answered "no" does not apply, so an unreadable sibling is beside the point; an `any:` rule is settled by a readable "yes". Only a rule left undecided by the unreadable clause is reported, or refused for a manifest a repo wrote. The existing built-in-default test passes unchanged: exactly one rule is still reported, and the pull-request copy of it no longer is.
The default assumed one provider, so a repo using it got the same model reading a change locally and reading it again on the pull request — a second look that shares the first one's blind spots. It now carries a roster per provider and the same panels and rules for each. The local pass is Claude and the pull request is codex, so a change is read by two models trained differently before anyone else sees it. The codex panels judge and validate on codex, so a pull request is reconciled without the model that already reviewed it. Every model is named: a reviewer on the CLI's default is a model nobody chose. Each rule is written once per roster and kept on its own side by `context`. The escalation criteria change with it. auth and migrations were matched by path globs, which is a weaker duplicate of signals the binary already detects and tests: a request-gating middleware under internal/session/ carries the auth signal and matched no glob, so the change these rules exist for got the shallowest panel. They are signals now. migrations drops from deep to standard. A migration is irreversible, but its risk is the table it locks rather than anything legible in the diff, so every axis twice buys opinions that cannot see what matters; the security reviewer is the one with something to say. ci-cd and iac join it for the same reason. sensitive-data joins auth at deep: harm there lands on people who are not in the room. BREAKING CHANGE: a repo with no manifest of its own now reviews pull requests on codex. A machine without it gets a review that reports what could not run rather than one that silently ran half. Panel names quick-codex, standard-codex and deep-codex are new, and `--panel` accepts them.
…egister A manifest was never required — a repo without one is reviewed by the default in the binary, which is what lets code-review work on a repo that has just adopted the toolkit. What was missing was a way to start from that default without copying it out of the source. `init` writes it to the manifest path. It writes the default verbatim, so nothing about a repo's reviews changes until somebody edits it: explain reports the same panel before and after, and only the manifest line differs. An existing manifest is not overwritten without --force, because that file is where a repo tuned its own reviews and replacing it with the default swaps every rule for none of them. That forces the rename. `initialize` stores this machine's App private key, an unrelated job, and `init` beside `initialize` is two near-identical words for which the shorter is a strict prefix of the longer — so a typo runs the other, and one of them handles a credential. `register` is what this command's own summary always called it. `initialize` stays as an alias to the same command rather than a second one, unlisted so the vocabulary a reader learns is the current one, and it says on stderr which name it now has.
…ences a rule A named panel wins outright: an escalation that fired does not raise past it, so asking for `quick` on a change the rules send to `deep` is a downgrade the person is entitled to make. The engine reports it — "named on the command line; the rules above did not decide" — and the skill now has to pass that on rather than print the panel as though the rules chose it. The argument was described only as a name asked for in words, which left the bare form a reader could not tell was accepted.
… failed by mood Review of the branch found seven defects. Two mattered. `init` promised the default verbatim changes nothing, and in a repo whose language has no symbol extractor it broke reviewing outright: the same bytes are skipped while they ship in the binary and refused once a repo owns them, because a repo with a manifest can act on the advice a refusal gives. The refusal is right; handing it over in silence was not. init now names the rule it just made a refusal, and what to do about it, and its help states the rule rather than promising nothing changes. The escalation fix went too wide. Suppressing the refusal whenever any readable condition said no meant a repo's broken rule passed for a change outside its globs and refused for one inside — a manifest that looks healthy until the day somebody touches the wrong directory. Only a context guard settles a rule now: a rule this context does not run in was never addressed to this review, while one whose glob merely did not match is still a protection the repo asked for. Also: `--dry-run` refused instead of reporting the path when a manifest already existed, which is when the question is worth asking; the default's comment said two rules cannot fire when they fire and merely cannot raise; and gofmt. The release notes never mentioned that a repo with no manifest now reviews pull requests on codex and needs that CLI, nor that `context` is a new key.
`run` and `explain` take --base and --head, so reviewing one commit or a range has always worked. The skill offered a pull request or the working tree and nothing else, so "review the last commit" landed on the branch instead — the one target it was not asked for. It also has to pass --head. That flag defaults to the working tree, so `--base HEAD~1` alone reviews the last commit plus whatever is uncommitted, and the range line is the only place that difference shows.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follows #84. Five things, one of them breaking.
The default reviews pull requests on codex
The change most likely to surprise, because it applies to every repo that has not written a manifest — which is most of them, since working without one is what makes adoption cheap.
The default now carries a roster per provider with the same panels and rules for each: the local pass is Claude (
quick), the pull request is codex (standard-codex). A change is read by two models trained differently before anyone else sees it, rather than twice by the model that already read it. The codex panels judge and validate on codex too, so a PR is reconciled without the model that reviewed it locally. Every model is named — a reviewer on the CLI's own default is a model nobody chose.run --prtherefore needscodexon PATH. A machine without it does not get a silent half-review: the codex runs report they could not start, and if nothing answered at all the run refuses rather than reporting an empty finding set.The escalation criteria changed with it:
auth,crypto,concurrency,sensitive-data,fix-revertreferencing_files >= 20migrations,ci-cd,iacchanged_files >= 20authandmigrationsescalated on path globs, which duplicated — badly — signals the binary already detects and tests. Demonstrated: request-gating middleware underinternal/session/carries theauthsignal, matched no glob, and got the shallowest panel. They are signals now.migrationsdrops to standard because its risk is the table it locks, not anything legible in the diff.context, so a rule can say where it appliesA rule names one panel, so a two-roster manifest could not say "this criterion, but on the other roster for a pull request". Writing it twice does not work either — equal cost is not a raise, so the first-written rule wins in both contexts, decided by an order the design says carries no meaning.
inandnot_inoverworktree/pr; unknown names refused at parse. The one condition that tests the run rather than the change, so it can never be unavailable.It exposed a bug: an
all:rule stopped at its first unreadable condition, so a rule guarded to the other context looked unreadable rather than inapplicable. A repo whose language has no symbol extractor saw every PR-only rule reported as skipped on every local review, and a repo-written manifest refused the review outright.agtk code-review init, andinitialize→registerA manifest was never required — a repo without one uses the default in the binary, which is what makes adoption work. What was missing was a way to start from it.
initwrites it to.agents/code-review/manifest.yaml, refuses to overwrite without--force, and--dry-runreports the path.That forced a rename:
initializestores a GitHub App private key, an unrelated job, andinitbesideinitializeis two near-identical words where the shorter is a strict prefix — a typo runs the other. It isregisternow, which its own summary always called it.initializestill resolves as an unlisted alias and says so on stderr.Skill: panel and commit targets
/panel-code-review quickand/panel-code-review the last commitboth work now. Neither was documented; the commit target was a regression fromdeep-code-review, which took ranges.A named panel silences the rules —
--panelwins outright, so an escalation that fired does not raise past it. The skill now reports that rather than printing the panel as though the rules chose it. And it always passes--headwith a commit, since that flag defaults to the working tree:--base HEAD~1alone reviews the last commit plus anything uncommitted.Reviewed, and what it found
/code-reviewon this branch returned seven findings. All seven were verified against running code and fixed. Two mattered:initbroke reviewing outright. It promised "the default verbatim, so nothing changes", butBuiltinis provenance: the default'sreferencing_filesrules are skipped only while they ship in the binary, and refused once a repo owns them. Same bytes,panel: quickbefore andagtk: escalate[2]: …after. The refusal is correct — a repo with a manifest can act on the advice — soinitnow names the rule it just made a refusal, and the remedy. The original test only asserted the bytes parse; parsing is not selection.nomeant a repo's broken rule passed for a change outside its globs and refused for one inside — failing by mood. Only a context guard settles a rule now.Plus:
--dry-runrefused exactly when the path question was worth asking,gofmt(which would have failed CI), a comment claiming two rules "cannot fire" when they fire and merely cannot raise, and release notes that never mentioned the codex requirement.Verification
gofmt -s,go build,go vet, full suite — clean. New tests cover the context condition and its guards, the narrowed refusal,init(verbatim write, overwrite refusal, dry-run with and without an existing manifest, and that it names an unrunnable rule), and theregister/initsplit. Each was mutation-tested — nine mutations across these commits, each caught by its intended test, code restored.Behaviour verified end to end against the built binary, not only in tests: both rosters and their context guards,
initbefore/after on a repo whose language has no symbol extractor, commit and range targets, and--paneloverriding a fired rule.Open, and yours to call
astraon security and judge,solelsewhere — version number is my only evidence; the driver refuses to map families across vendors, so there is no opus-equivalent to lean on.standard-codexrules cannot raise while that is also the PR default. Kept so both rosters read identically, and so lowering the default does not silently drop a criterion.--panel quickon a PR runs the Claude roster, crossing the local/PR split. Correct for an explicit override, but bothquickandquick-codexare reachable from either target.feature-flags,message-consumers,public-api,shared-kernelstill do not escalate — too loose for a shipped default (shared-kernelfires on all ofpkg/,public-apion any Go struct tag).