Skip to content

Add silent-failure severity escalation guidance to code-review skill and correctness sub-agent #33

Description

@fullsend-ai-retro

What happened

Issue #1144 in fullsend-ai/fullsend proposed adding silent-failure severity escalation guidance to the review agent's code-review skill. Two concrete instances validated the need: PR #1079 (awk path truncation rated [low], later confirmed as a real bug) and PR #2947 (annotated-tag silent 404 rated [low] across 4+ review passes while a human squad rated it [HIGH]).

PR #3156 implemented the fix but targeted the wrong repo (internal/scaffold/fullsend-repo/ in fullsend-ai/fullsend). ralphbean closed both PR #3156 and issue #1144, redirecting to the agents repo. However, no corresponding issue was created in fullsend-ai/agents, and issue #3153 remains open in the wrong repo with no path to resolution.

What could go better

When ralphbean closed #1144 with a redirect comment, no automated or manual process ensured the work was re-filed in the target repo. The well-specified content from #1144 (proposed text, validation criteria, triage analysis, RICE prioritization) and the concrete implementation from PR #3156 (88 lines of tested, well-placed guidance) are stranded — neither is discoverable from the agents repo.

Confidence: High that this is a gap — the work has two data points and a ready implementation but no tracking issue in the correct repo. Medium that the PR #3156 content can be applied directly to the agents repo's skills/code-review/SKILL.md and skills/pr-review/sub-agents/correctness.md with minimal adaptation.

Proposed change

File a new issue in fullsend-ai/agents to add silent-failure severity escalation guidance to the code-review skill (Section 4, before "Severity anchoring") and the correctness sub-agent. The content from PR #3156's diff provides a ready implementation: a subsection instructing reviewers to escalate severity by one level when a correctness bug produces silent failures (no error, no warning, appears to succeed but produces wrong results), with explicit examples of silent vs loud failures and guidance that probability-based reasoning must not override failure-mode-based severity.

The issue should reference the originating evidence: fullsend-ai/fullsend#1144, fullsend-ai/fullsend#3153, PR #2947 (annotated-tag bug), and PR #1079 (awk path truncation).

Validation criteria

  1. After the guidance is added: re-run the review agent on PR #2947's 3ce5eb7 diff. The annotated-tag finding should be rated [medium] or higher in at least 4/5 trials, with severity justification referencing the silent failure mode rather than tag-type probability.
  2. On a test PR containing a loud-failure bug (nil pointer, index out of bounds), the review agent should not over-escalate severity.
  3. Close fullsend-ai/fullsend#3153 once the corresponding agents-repo issue is filed and tracked.

Generated by retro agent from fullsend-ai/fullsend#3156

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions