feat(review): adversarial code-review workflow + findings convention (ADR 0077)#1851
Conversation
|
👀 Quinn is reviewing — verdict (PASS / WARN / FAIL) + findings to follow. |
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 4 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (10)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![protoquinn[bot]](https://avatars.githubusercontent.com/in/3282126?s=60&v=4){kind=small}
There was a problem hiding this comment.
QA Audit — PR #1851 | feat(review): adversarial code-review workflow + findings convention (ADR 0077)
VERDICT: WARN (pending CI — approve-on-green policy promotes to APPROVED once all checks settle terminal-green)
CI Status
- All 8 checks in_progress: build, Lint, Python tests, A2A live smoke, Fleet integration, Web E2E smoke, Verify workspace config, gitleaks
Diff Review
- 10 files, +755/−43: ADR 0077 doc,
graph/review/findings.py(findings convention module), newreview-finder/review-synthesizerroles,code-review.yamlrecipe, craft skill update - Design is sound: the findings contract (schema → prompt snippet → tolerant parser → renderer) is exactly the right shape for an LLM-facing string-based engine
- Recipe mirrors the proven deep-research pattern (parallel gather → dedup → verify → report); per-finding verify fan-out deliberately parked per ADR text
Observations
- LOW: Clawpatch structural review timed out — diff-based review only
- Gap: Companion PR
protoLabsAI/github-plugin#20(providesgithub_pr_diff— the finders' diff source) not verified; finders depend on it - Gap:
findings.pytruncated at ~209 lines in the diff preview —parse_findings/render_findings_markdownimplementations not visible to this review pass - Gap: Tests referenced (15 new, 3304 passed) but not visible in the diff
- Note: The visible
findings.pycode is clean —_VERDICT_ALIASESmapping,SEVERITIEStuple,FINDINGS_CONTRACTprompt snippet, and dataclass shape are all well-structured
— Quinn, QA Engineer
|
Submitted COMMENT review on #1851. |
…(ADR 0077)
The delivery loop's review step, codified (plan M1):
- graph/review/findings.py — THE findings convention: schema [{file, line,
severity, category, claim, evidence, verdict?}], FINDINGS_CONTRACT (the
prompt snippet producers embed — schema written down once), tolerant
parse_findings (JSON-in-prose: fenced preferred, bare arrays, coerced
fields, verifier-vocabulary normalization), render_findings_markdown.
- review-finder / review-synthesizer subagent roles — one diff-reading lens
per finder (angle passed per-step), synthesizer dedups/ranks and never adds
findings; both allow_skill_emission=False. verifier gains a code-findings
mode + github read tools so verdicts come from re-reading the diff.
- code-review.yaml — 4 finder angles ∥ (correctness / removed-behavior /
cross-file / conventions) → dedup → ONE verify pass over the merged list
(per-finding fan-out parked; needs a data-driven engine primitive) →
report ending in the canonical fenced findings JSON.
- craft /code-review skill — now a thin driver for the workflow; the old
two-axis inline review survives as the no-plugin fallback.
- ADR 0077 + index row + nav regen; 15 new tests.
Companion: github-plugin#20 adds github_pr_diff (the finders' diff source).
Consumed headlessly by the projectBoard review gate next (plan M5).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
mabry1985
force-pushed
the
feat/adversarial-review-workflow
branch
from
31a1bef to
9e20987
Compare
There was a problem hiding this comment.
QA Audit — PR #1851 | feat(review): adversarial code-review workflow + findings convention (ADR 0077)
VERDICT: WARN (CI still settling — approve-on-green policy would promote to APPROVED once all checks are terminal-green, but build was observed as failure in the last snapshot. If build remains red the policy will not promote; the author must fix the build before merge.)
CI Status (last snapshot)
- build: failure ⬅️
- Lint (ruff + import contracts): success
- Python tests: success
- A2A live smoke: success
- Fleet integration: success
- Web E2E smoke: success
- Verify workspace config: success
- gitleaks: success
- deploy: skipped
Diff Review
- 10 files, +755/−43: ADR 0077,
graph/review/findings.py,review-finder/review-synthesizerroles,code-review.yamlrecipe, craft skill update - Design sound: tolerant JSON-in-prose parser, schema-in-prompt-snippet pattern, parallel-finder recipe mirrors proven deep-research shape
- Clawpatch structural: no features touched (docs + config + new module — expected)
Observations
- HIGH:
buildCI check observed as failure. Likely an import/packaging issue in the newgraph/review/package or recipe config — check build logs. A red build blocks the approve-on-green promotion even if all other checks pass. - Gap: Companion PR
protoLabsAI/github-plugin#20(github_pr_diff) not verified; finders depend on it - Gap:
findings.pytruncated at ~209 lines in diff preview — full file not inspected
— Quinn, QA Engineer
|
Submitted COMMENT review on #1851. |
|
✅ CI went terminal-green with no blockers on the prior review — promoting it to APPROVED per the approve-on-green policy (#748). |
There was a problem hiding this comment.
CI terminal-green, no blockers on prior review — auto-approving on green (#748).
1 participant
M1 of the codified-delivery-loop plan (#1850). The review step of the delivery loop as a first-class, headlessly-callable workflow.
What ships
FINDINGS_CONTRACTprompt snippet, tolerantparse_findings,render_findings_markdowngraph/review/findings.pyreview-finder(one angle per dispatch, github-diff tools) +review-synthesizer(dedup/rank, never adds) rolesgraph/subagents/config.pyverifiergrows a code-findings mode + github read tools (verdicts from re-reading the diff, not vibes)graph/subagents/config.pycode-reviewrecipe: 4 finders ∥ → dedup → one verify pass → report ending in canonical fenced findings JSONplugins/workflows/recipes/code-review.yaml/code-reviewbecomes a thin workflow driver (old two-axis review = fallback)plugins/craft/skills/code-review/SKILL.mddocs/adr/Design notes
allow_skill_emission=False— per-invocation verdicts must not pollute the skills index.Verification
ruffclean,lint-imports3 contracts kept,live_smokepassedgithub_pr_diff— the finders' diff source)Acceptance (
/code-review <PR#>on a real PR) exercised after the github-plugin PR merges and the working-tree plugin is synced.🤖 Generated with Claude Code