Skip to content

docs(adr): add ADR 0089 — PR-level risk assessment scoring - #5765

Merged
maruiz93 merged 3 commits into
fullsend-ai:mainfrom
maruiz93:adr-pr-risk-assessment
Aug 18, 2026
Merged

docs(adr): add ADR 0089 — PR-level risk assessment scoring#5765
maruiz93 merged 3 commits into
fullsend-ai:mainfrom
maruiz93:adr-pr-risk-assessment

Conversation

@maruiz93

@maruiz93 maruiz93 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds ADR 0078 deciding how the review pipeline computes a composite PR risk score (1–5) from metadata, git history, and linked-issue context signals.
  • Updates docs/architecture.md with a "Decided:" annotation under the review agent runtime section.
  • Annotates the rubber-stamp open question in docs/problems/code-review.md with a link to ADR 0078.

Related Issue

Relates to #4698

Changes

File Change
docs/ADRs/0078-pr-risk-assessment-scoring.md New ADR — context, 3 rejected options, decision (revised option B), consequences
docs/architecture.md "Decided:" line under §11 review agent runtime
docs/problems/code-review.md Parenthetical annotation on rubber-stamp open question

Checklist

  • ADR follows 0000-adr-template.md format
  • Frontmatter relates_to references valid problem doc filenames
  • ADR number checked against main and open PRs (0078 is next available)
  • docs/architecture.md updated per writing-adrs skill
  • Problem doc annotated per writing-adrs skill
  • make lint passes (all ADR linters + markdown links)
  • DCO sign-off present

🤖 Generated with Claude Code

@maruiz93
maruiz93 requested a review from a team as a code owner July 30, 2026 14:49
@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Add ADR 0078 for PR-level risk assessment scoring

📝 Documentation 🕐 20-40 Minutes

Grey Divider

AI Description

• Add ADR 0078 defining a 3-tier, weighted PR risk score (1–5) and integration points.
• Annotate the review agent runtime architecture section with the newly decided approach.
• Link the “rubber stamp” open question to risk scoring as a future mitigation input.
Diagram

graph TD
  A(["PR review orchestrator"]) --> B(["Risk pre-pass sub-agent"]) --> C["Tier1 metadata script"] --> D{{"risk_assessment JSON"}} --> E["post-review.sh"] --> F(["PR labels + comment"])
  B --> G["Tier2/3 LLM signals"] --> D
  subgraph Legend
    direction LR
    _svc(["Service/Agent"]) ~~~ _file["Script/Doc"] ~~~ _out{{"Output"}}
  end
Loading
High-Level Assessment

The PR’s approach—capturing the decision in an ADR and adding cross-references from architecture and problem statements—is the right mechanism for long-lived design decisions. The only notable alternative (keeping this solely in skills/implementation docs) would reduce discoverability and weaken decision traceability, so it’s not recommended.

Files changed (3) +167 / -1

Documentation (3) +167 / -1
0078-pr-risk-assessment-scoring.mdAdd ADR 0078 defining PR risk scoring model and pipeline integration +160/-0

Add ADR 0078 defining PR risk scoring model and pipeline integration

• Introduces an accepted ADR specifying a composite 1–5 PR risk score built from three weighted signal tiers (metadata, git history, linked-issue context). Documents rejected alternatives, constraints around pre-script sandbox propagation, expected JSON output shape, post-review label/comment integration, and a feature flag default.

docs/ADRs/0078-pr-risk-assessment-scoring.md

architecture.mdRecord the risk-scoring decision in the review agent runtime section +6/-0

Record the risk-scoring decision in the review agent runtime section

• Adds a short “Decided:” note under the review agent runtime describing PR-level risk assessment scoring and links to ADR 0078 for details.

docs/architecture.md

code-review.mdLink rubber-stamp concern to ADR 0078 risk scoring rationale +1/-1

Link rubber-stamp concern to ADR 0078 risk scoring rationale

• Annotates the open question about agents being rubber stamps with a reference to PR-level risk scoring as an input for stricter review thresholds, while explicitly deferring outcome gating until confidence is established.

docs/problems/code-review.md

@fullsend-ai-review

fullsend-ai-review Bot commented Jul 30, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 2:51 PM UTC · Completed 3:08 PM UTC
Commit: dca2060 · View workflow run →

@github-actions

github-actions Bot commented Jul 30, 2026

Copy link
Copy Markdown

Site preview

Preview: https://2cac488a-site.fullsend-ai.workers.dev

Commit: adad2e1bb928387f5f4952ff5c48aaa4c055aa78

@qodo-code-review

qodo-code-review Bot commented Jul 30, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (1)

Grey Divider


Remediation recommended

1. Mislinked prioritize agent doc ✓ Resolved 🐞 Bug ⚙ Maintainability
Description
ADR 0078 cites the prioritize agent’s RICE scoring but the link target points to the external repo
root instead of the specific document being referenced, making the cross-reference hard to follow
and easy to drift. The repo already contains the relevant doc (docs/agents/prioritize.md), so this
should be a relative link to that file.
Code

docs/ADRs/0078-pr-risk-assessment-scoring.md[R38-40]

+The prioritize agent's RICE scoring
+([agents/agents/prioritize.md](https://github.com/fullsend-ai/agents)) provides
+a proven pattern: agent produces structured JSON → post-script applies labels
Relevance

●●● Strong

Broken/mispointed documentation links are commonly fixed in review.

PR-#226
PR-#601

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The ADR currently links to the external repo root rather than the named file, while the referenced
prioritize doc exists locally in this repo and documents the RICE scoring behavior.

docs/ADRs/0078-pr-risk-assessment-scoring.md[38-40]
docs/agents/prioritize.md[1-12]
PR-#226

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`docs/ADRs/0078-pr-risk-assessment-scoring.md` references the prioritize agent’s RICE scoring, but the markdown link text implies a specific file path while the URL points only to the `fullsend-ai/agents` repository root. This makes the reference non-specific and harder for readers to navigate.

### Issue Context
The repo already has a local prioritize-agent doc at `docs/agents/prioritize.md`, which describes the RICE scoring behavior. ADRs should prefer stable intra-repo relative links when the target exists in this repo.

### Fix Focus Areas
- docs/ADRs/0078-pr-risk-assessment-scoring.md[38-40]

### Suggested change
Replace the link with a relative link to the local doc, e.g.:
- `([prioritize agent RICE scoring](../agents/prioritize.md))`
(or keep the existing link text but change the target to `../agents/prioritize.md`).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Context missing required links ✓ Resolved 📜 Skill insight ⚙ Maintainability
Description
ADR 0078's Context section does not link to relevant problem docs and does not cross-reference the
related ADR it cites later, which reduces traceability of rationale and dependencies.
Code

docs/ADRs/0078-pr-risk-assessment-scoring.md[R29-48]

+## Context
+
+The review pipeline has no quantitative risk signal. Protected-path checks in
+`post-review.sh` provide a binary gate, and the orchestrator's scope
+classification (trivial/small/standard) captures size but not risk. There is no
+composite score that accounts for path sensitivity, git history churn, author
+context, or linked-issue complexity — signals that would inform review effort,
+model selection, and auto-merge eligibility.
+
+The prioritize agent's RICE scoring
+([agents/agents/prioritize.md](https://github.com/fullsend-ai/agents)) provides
+a proven pattern: agent produces structured JSON → post-script applies labels
+and posts a breakdown comment.
+
+A key constraint is that the fullsend harness expands `env.sandbox` values
+before the pre-script runs ([#5756](https://github.com/fullsend-ai/fullsend/issues/5756)),
+so pre-script-computed values cannot flow into the sandbox. Sub-agents currently
+share the parent sandbox ([#3978](https://github.com/fullsend-ai/fullsend/issues/3978));
+a future sub-agent harness schema ([#3982](https://github.com/fullsend-ai/fullsend/issues/3982))
+may enable native skill loading per sub-agent.
Relevance

●●● Strong

Team routinely accepts ADR/doc cross-reference and traceability link fixes.

PR-#1549

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The Context section contains narrative motivation but no links to problem docs, violating the
requirement to link problem docs rather than restate them. The file also references ADR 0045 in
References, indicating a related ADR exists but is not cross-referenced in Context as required.

docs/ADRs/0078-pr-risk-assessment-scoring.md[29-48]
docs/ADRs/0078-pr-risk-assessment-scoring.md[154-160]
Skill: writing-adrs

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The ADR Context section should link to relevant problem docs (instead of restating them) and should cross-reference related ADRs in the Context when they are dependencies/precursors.

## Issue Context
This ADR declares `relates_to` entries (`code-review`, `agent-architecture`) but the Context section itself contains no links to `docs/problems/...`. It also references a related ADR (ADR 0045) only in References, not in Context.

## Fix Focus Areas
- docs/ADRs/0078-pr-risk-assessment-scoring.md[29-48]
- docs/ADRs/0078-pr-risk-assessment-scoring.md[154-160]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Informational

3. ADR 0078 too long 📜 Skill insight ⚙ Maintainability
Description
docs/ADRs/0078-pr-risk-assessment-scoring.md exceeds the 100-line ADR content limit (excluding
frontmatter), making it harder to review and increasing the chance it is deciding/defining too much
inline.
Code

docs/ADRs/0078-pr-risk-assessment-scoring.md[R118-160]

+```json
+{
+  "score": 3,
+  "level": "elevated",
+  "tier1_signals": [{"dimension": "...", "value": "..."}],
+  "tier2_signals": [...],
+  "tier3_signals": [...],
+  "rationale": "..."
+}
+```
+
+`score` (1–5), `level` (enum: low/moderate/elevated/high/critical), and
+`rationale` are required within the object. Signal arrays are optional for
+graceful degradation.
+
+**Post-review integration:** `post-review.sh` reads the risk assessment from
+the result JSON, applies a `risk/*` label (removing any prior `risk/*` label),
+and appends a breakdown table to the PR comment. Risk level is informational
+only — it does not gate the review outcome. The protected-path check remains
+the sole blocking mechanism.
+
+**Feature flag:** `FULLSEND_RISK_ASSESSMENT_ENABLED` env var, default `true`.
+
+## Consequences
+
+- Review pipeline gains a quantitative risk signal visible via labels and PR
+  comments, enabling risk-informed triage and review prioritization.
+- Risk scoring adds one sonnet-model sub-agent call per PR (cost-effective
+  relative to the opus dimension sub-agents).
+- Metadata tier signals are deterministic and auditable via bash script output.
+- Risk level is decoupled from review outcome — gating can be added later once
+  scoring confidence is established.
+- The docs-currency skill-loading pattern adds an implicit coupling between the
+  orchestrator and the skill directory; this coupling is eliminated when sub-agent
+  sandbox isolation lands ([#3978](https://github.com/fullsend-ai/fullsend/issues/3978)).
+
+## References
+
+- [#4698](https://github.com/fullsend-ai/fullsend/issues/4698) — PR-level risk assessment scoring
+- [#5756](https://github.com/fullsend-ai/fullsend/issues/5756) — Pre-script → sandbox data flow
+- [#3978](https://github.com/fullsend-ai/fullsend/issues/3978) — Sub-agent sandbox isolation
+- [#3982](https://github.com/fullsend-ai/fullsend/issues/3982) — Sub-agent harness schema ADR
+- [ADR 0045](0045-forge-portable-harness-schema.md) — Forge-portable harness schema
Relevance

● Weak

Close precedent: request to shorten ADRs to ≤100 lines was rejected.

PR-#2582

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The rule requires ADR content (excluding frontmatter) to be ≤100 lines. This ADR has content running
from line 14 through line 160 (147 lines), exceeding the limit.

docs/ADRs/0078-pr-risk-assessment-scoring.md[14-160]
Skill: writing-adrs

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
ADR 0078 content length exceeds 100 lines (excluding YAML frontmatter), violating the ADR size constraint.

## Issue Context
The ADR currently contains extensive inline detail (e.g., tier tables, output example, post-review behavior, feature flag specifics, and a full references list). The rule expects ADRs to stay concise and push deep detail into referenced docs/specs.

## Fix Focus Areas
- docs/ADRs/0078-pr-risk-assessment-scoring.md[14-160]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context
✅ Compliance rules (platform): 54 rules

Grey Divider

Tip of the day
💡 Did you know, you can keep summaries lean with Finding overflow, which tucks the rest behind 'View more'

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread docs/ADRs/0089-pr-risk-assessment-scoring.md
Comment thread docs/ADRs/0078-pr-risk-assessment-scoring.md Outdated
@codecov

codecov Bot commented Jul 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@fullsend-ai-review

fullsend-ai-review Bot commented Jul 30, 2026

Copy link
Copy Markdown

Looks good to me

Previous run

Review

Findings

Low

  • [algorithm specification gap] docs/ADRs/0078-pr-risk-assessment-scoring.md:107 — The scoring model specifies three signal tiers with weights (50/30/20) and states each tier contains multiple sub-signals scored 1–5, but does not define how sub-signals within a tier are aggregated into the tier-level score (e.g., average, weighted average, max). The ADR explicitly defers this to skills/pr-risk-assessment/SKILL.md, which is appropriate for an ADR-level decision record — but the implementation PR should specify the intra-tier aggregation method to avoid ambiguity in the scoring model.
Previous run (2)

Review

Findings

Low

  • [undefined mapping / internal ambiguity] docs/ADRs/0078-pr-risk-assessment-scoring.md:125 — The ADR defines a 5-value level enum (low/moderate/elevated/high/critical) and a 5-value score range (1–5) but never specifies the explicit mapping between them. The example JSON (score 3, level "elevated") strongly implies a 1:1 sequential mapping (1=low, 2=moderate, 3=elevated, 4=high, 5=critical), but an explicit table would remove all ambiguity.
  • [missing-doc] docs/agents/review.md:12 — The orchestrator workflow description omits the risk assessment pre-pass decided in ADR 0078. Since the feature is not yet implemented (sub-agent files and scripts do not exist yet), updating docs now would be premature — this is follow-up work for the implementation PR.

Labels: PR adds an ADR for the review pipeline's risk scoring feature and annotates related docs

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge documentation agent/review Review agent labels Jul 30, 2026

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review findings (dedup-checked against existing review comments on this PR).

Comment thread docs/ADRs/0089-pr-risk-assessment-scoring.md
Comment thread docs/ADRs/0078-pr-risk-assessment-scoring.md Outdated
Comment thread docs/ADRs/0078-pr-risk-assessment-scoring.md Outdated
Comment thread docs/ADRs/0078-pr-risk-assessment-scoring.md Outdated
Comment thread docs/ADRs/0078-pr-risk-assessment-scoring.md Outdated
@maruiz93
maruiz93 force-pushed the adr-pr-risk-assessment branch from dca2060 to 468bc7e Compare July 31, 2026 10:24
@fullsend-ai-review

fullsend-ai-review Bot commented Jul 31, 2026

Copy link
Copy Markdown

🤖 Review · ⚠️ Cancelled · Started 10:25 AM UTC · Ended 10:33 AM UTC
Commit: 468bc7e · View workflow run →

@fullsend-ai-review

fullsend-ai-review Bot commented Jul 31, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 10:34 AM UTC · Completed 10:50 AM UTC
Commit: 341e925 · View workflow run →

fullsend-ai-review[bot]

This comment was marked as outdated.

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review findings (dedup-checked against existing review comments and prior fix threads on this PR).

Comment thread docs/ADRs/0078-pr-risk-assessment-scoring.md Outdated
Comment thread docs/ADRs/0078-pr-risk-assessment-scoring.md Outdated
Comment thread docs/ADRs/0078-pr-risk-assessment-scoring.md Outdated
Comment thread docs/ADRs/0078-pr-risk-assessment-scoring.md Outdated

@ralphbean ralphbean left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left one clarifying question inline, otherwise this looks solid.

@ralphbean

Copy link
Copy Markdown
Member

Since we're here — ADR 0078 documents behaviors that live entirely in fullsend-ai/agents (skills, sub-agents, scripts), even though the ADR itself is filed in this repo. Makes me wonder whether ADRs about specific agent behaviors should start living in fullsend-ai/agents directly, with this repo's ADRs reserved for harness/pipeline-level decisions. Curious what folks think.

@ben-alkov

Copy link
Copy Markdown
Member

whether ADRs about specific agent behaviors should start living in fullsend-ai/agents directly

That occurred to me as well.

IG it depends on whether "agents" is only a "repo" or is an actual separate "project"...

@maruiz93
maruiz93 force-pushed the adr-pr-risk-assessment branch from 341e925 to 4b5258a Compare August 18, 2026 00:39
@maruiz93

Copy link
Copy Markdown
Contributor Author

Good question. This ADR is very agents-repo specific, but other ADRs (harness schema, sandbox model, dispatch routing) span both repos or are squarely about fullsend infrastructure. For now I'd keep ADRs centralized in fullsend — if we end up with more agent-only ADRs we can revisit migrating them to fullsend-ai/agents.

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 18, 2026

Copy link
Copy Markdown

🤖 Review · ⚠️ Cancelled · Started 12:41 AM UTC · Ended 12:48 AM UTC

Commit: 4b5258a · View workflow run →

maruiz93 and others added 3 commits August 18, 2026 02:47
Decides how the review pipeline computes a composite PR risk score
(1–5) from metadata, git history, and linked-issue signals.

Closes fullsend-ai#4698

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Marta Anon <manon@redhat.com>
- Add links to problem docs and ADR 0045 in Context section
- Use relative link for prioritize agent doc
- Add explicit score-to-level mapping table
- Qualify component paths with target repo (fullsend-ai/agents)
- Rename feature flag to REVIEW_RISK_ASSESSMENT_ENABLED per ADR 0049
- Mark scoring weights as initial/tunable pending calibration
- Acknowledge tier-1 LLM pass-through non-determinism in Consequences

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Marta Anon <manon@redhat.com>
ADR 0078 was merged to main as "Simplified mint authorization policy"
while this PR was open. Renumber risk assessment scoring ADR to 0089.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Marta Anon <manon@redhat.com>
@maruiz93
maruiz93 force-pushed the adr-pr-risk-assessment branch from 4b5258a to adad2e1 Compare August 18, 2026 00:47
@maruiz93 maruiz93 changed the title docs(adr): add ADR 0078 — PR-level risk assessment scoring docs(adr): add ADR 0089 — PR-level risk assessment scoring Aug 18, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 18, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 12:49 AM UTC · Completed 1:05 AM UTC

Commit: adad2e1 · View workflow run →

@maruiz93
maruiz93 added this pull request to the merge queue Aug 18, 2026
Merged via the queue into fullsend-ai:main with commit 1c55863 Aug 18, 2026
16 checks passed
@maruiz93
maruiz93 deleted the adr-pr-risk-assessment branch August 18, 2026 01:07
@fullsend-ai-retro

fullsend-ai-retro Bot commented Aug 18, 2026

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 1:09 AM UTC · Completed 1:22 AM UTC

Commit: adad2e1 · View workflow run →

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #5765 — ADR 0089 (PR-level risk assessment scoring)

Timeline

  • Jul 30 14:49: PR opened by maruiz93, adding ADR 0078 (later renumbered 0089) for PR-level risk assessment scoring. Three files changed: new ADR (+185 lines), docs/architecture.md annotation, docs/problems/code-review.md annotation.
  • Jul 30 ~15:08: Review agent (run 30553664963) approved with 2 low findings (undefined score-to-level mapping; premature orchestrator doc update). qodo-code-review found 2 medium findings (mislinked doc; missing context links).
  • Jul 30 17:06: waynesun09 posted automated review findings — 5 issues (1 HIGH: unqualified cross-repo paths; 4 MEDIUM: issue linkage scope mismatch, naming convention violation per ADR 0049, unsupported weight calibration, non-determinism contradiction).
  • Jul 31 10:33: Author addressed all round-1 feedback (commit 341e925).
  • Jul 31 ~10:50: Review agent (run 30623945029) approved with 1 new low finding (algorithm specification gap).
  • Jul 31 13:07: waynesun09 posted second automated review round — 4 more issues (1 HIGH: fabricated "docs-currency convention" term; 3 MEDIUM: post-review script location, rejected→adopted wording, unsupported cost claim).
  • Jul 31 13:21: ralphbean approved with a clarifying question confirming the "docs-currency convention" concern.
  • Aug 18 00:40: Author addressed round-2 feedback, renumbered ADR 0078→0089 to avoid collision with a merged ADR.
  • Aug 18 ~01:05: Review agent (run 32085875217) approved with no findings.
  • Aug 18 01:07: PR merged.

Review quality delta

The review agent found 3 low-severity findings across 3 runs (~$3.61 per run). Human reviewers found 12 substantive findings (2 HIGH, 8 MEDIUM, 1 clarifying question, plus 2 medium from qodo-code-review). The agent missed 100% of MEDIUM and HIGH findings.

Gap category Severity Human finding Agent behavior
Cross-repo path qualification HIGH ADR listed component paths without specifying they live in fullsend-ai/agents Not flagged
Issue linkage scope MEDIUM PR said "Closes #4698" but only delivered 1 of 3 required behaviors Not flagged
Cross-ADR convention compliance MEDIUM Feature flag name violated ADR 0049 naming convention Not flagged
Fabricated conventions HIGH "docs-currency convention" described a nonexistent generic mechanism Not flagged
Unsupported quantitative claims MEDIUM Scoring weights and cost claims asserted without evidence Not flagged
Internal logical contradictions MEDIUM Rejected option adopted under a revised name without updating the rejection verdict Not flagged

What the agent did well

Existing issues that cover the gaps

All identified improvement opportunities are substantially covered by existing open issues. This retro provides new supporting evidence for the following (highest-impact first):

  1. Add ADR design-depth checklist to AGENTS.md review guidance #3030 — ADR design-depth checklist in review guidance. Would have caught 4–5 of the human findings (unsupported claims, missing calibration basis, logical contradictions, rejected→adopted wording). This docs-only ADR PR is strong evidence the checklist is needed.
  2. Review orchestrator should promote docs sub-agent budget priority for docs-only PRs #5298 — Docs sub-agent budget priority for docs-only PRs. The agent spent ~$10+ across 3 runs but under-invested in deep ADR analysis relative to that cost.
  3. Review agent should flag PRs that auto-close investigation issues with partial scope coverage agents#167 — Flag PRs that auto-close issues with partial scope. This retro provides evidence the check should extend beyond investigation issues to ADR/design PRs that partially deliver on feature requests (issue Add PR-level risk assessment score to the review pipeline #4698 had 3 requirements; the PR delivered 1).
  4. Review agent should cross-reference governance docs when reviewing convention-encoding files #3285 — Cross-reference governance docs when reviewing convention-encoding files. Would have caught the ADR 0049 naming violation.
  5. Review agent should escalate terminology-precision findings on governance docs with explicit terminology conventions #3449 — Escalate terminology-precision findings on governance docs. Would have caught the fabricated "docs-currency convention" term.
  6. Review agent should verify implementation against ADR rejected alternatives #2812 — Verify against ADR rejected alternatives. Would have caught the non-determinism contradiction (the chosen architecture reintroduces the non-determinism cited to reject Option A).

No novel proposals are warranted — filing duplicates would add noise to an already well-tracked issue backlog. Prioritizing #3030 and #5298 would address the broadest set of gaps observed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent/review Review agent documentation ready-for-merge All reviewers approved — ready to merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants