docs(adr): add ADR 0089 — PR-level risk assessment scoring - #5765
Conversation
PR Summary by QodoAdd ADR 0078 for PR-level risk assessment scoring
AI Description
Diagram
High-Level Assessment
Files changed (3)
|
|
🤖 Finished Review · ✅ Success · Started 2:51 PM UTC · Completed 3:08 PM UTC |
Site previewPreview: https://2cac488a-site.fullsend-ai.workers.dev Commit: |
Code Review by Qodo
1.
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
Looks good to me Previous runReviewFindingsLow
Previous run (2)ReviewFindingsLow
Labels: PR adds an ADR for the review pipeline's risk scoring feature and annotates related docs |
waynesun09
left a comment
There was a problem hiding this comment.
Automated review findings (dedup-checked against existing review comments on this PR).
dca2060 to
468bc7e
Compare
|
🤖 Review · |
|
🤖 Finished Review · ✅ Success · Started 10:34 AM UTC · Completed 10:50 AM UTC |
waynesun09
left a comment
There was a problem hiding this comment.
Automated review findings (dedup-checked against existing review comments and prior fix threads on this PR).
ralphbean
left a comment
There was a problem hiding this comment.
Left one clarifying question inline, otherwise this looks solid.
|
Since we're here — ADR 0078 documents behaviors that live entirely in fullsend-ai/agents (skills, sub-agents, scripts), even though the ADR itself is filed in this repo. Makes me wonder whether ADRs about specific agent behaviors should start living in fullsend-ai/agents directly, with this repo's ADRs reserved for harness/pipeline-level decisions. Curious what folks think. |
That occurred to me as well. IG it depends on whether "agents" is only a "repo" or is an actual separate "project"... |
341e925 to
4b5258a
Compare
|
Good question. This ADR is very agents-repo specific, but other ADRs (harness schema, sandbox model, dispatch routing) span both repos or are squarely about fullsend infrastructure. For now I'd keep ADRs centralized in fullsend — if we end up with more agent-only ADRs we can revisit migrating them to fullsend-ai/agents. |
|
🤖 Review · Commit: |
Decides how the review pipeline computes a composite PR risk score (1–5) from metadata, git history, and linked-issue signals. Closes fullsend-ai#4698 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Marta Anon <manon@redhat.com>
- Add links to problem docs and ADR 0045 in Context section - Use relative link for prioritize agent doc - Add explicit score-to-level mapping table - Qualify component paths with target repo (fullsend-ai/agents) - Rename feature flag to REVIEW_RISK_ASSESSMENT_ENABLED per ADR 0049 - Mark scoring weights as initial/tunable pending calibration - Acknowledge tier-1 LLM pass-through non-determinism in Consequences Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Marta Anon <manon@redhat.com>
ADR 0078 was merged to main as "Simplified mint authorization policy" while this PR was open. Renumber risk assessment scoring ADR to 0089. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Marta Anon <manon@redhat.com>
4b5258a to
adad2e1
Compare
|
🤖 Finished Review · ✅ Success · Started 12:49 AM UTC · Completed 1:05 AM UTC Commit: |
|
🤖 Finished Retro · ✅ Success · Started 1:09 AM UTC · Completed 1:22 AM UTC Commit: |
Retro: PR #5765 — ADR 0089 (PR-level risk assessment scoring)Timeline
Review quality deltaThe review agent found 3 low-severity findings across 3 runs (~$3.61 per run). Human reviewers found 12 substantive findings (2 HIGH, 8 MEDIUM, 1 clarifying question, plus 2 medium from qodo-code-review). The agent missed 100% of MEDIUM and HIGH findings.
What the agent did well
Existing issues that cover the gapsAll identified improvement opportunities are substantially covered by existing open issues. This retro provides new supporting evidence for the following (highest-impact first):
No novel proposals are warranted — filing duplicates would add noise to an already well-tracked issue backlog. Prioritizing #3030 and #5298 would address the broadest set of gaps observed here. |
Summary
docs/architecture.mdwith a "Decided:" annotation under the review agent runtime section.docs/problems/code-review.mdwith a link to ADR 0078.Related Issue
Relates to #4698
Changes
docs/ADRs/0078-pr-risk-assessment-scoring.mddocs/architecture.mddocs/problems/code-review.mdChecklist
0000-adr-template.mdformatrelates_toreferences valid problem doc filenamesmainand open PRs (0078 is next available)docs/architecture.mdupdated per writing-adrs skillmake lintpasses (all ADR linters + markdown links)🤖 Generated with Claude Code