What happened
On PR #475, the review agent's correctness sub-agent (Claude Opus, run 30295479886) analyzed a new bash linter script (hack/lint-agent-docs) and found 2 LOW-severity items (grep flag, shebang convention). However, it missed a correctness bug that Qodo independently identified: the linter only validates H1 heading format when an H1 is found (if [[ -n "$h1" ]]; then check_suffix; fi), so documents without any H1 heading silently pass the check. This same bug persists in successor PR #476. The review agent's findings were never posted (PR closed before posting), so the gap had no practical impact on this PR, but the detection gap is the concern.
What could go better
The correctness sub-agent already has 'runtime mechanism verification' guidance that asks whether callers 'silently proceed' when a mechanism fails. But the H1 bug is a different pattern: validation code that checks a property only when present, silently accepting items where the property is entirely absent. This is a common bug class in validators, linters, and config checkers — distinct from shell-specific pitfalls (#131), from severity escalation after detection (#33), and from the bidirectional permission checking gap in #46). The correctness sub-agent's existing 'runtime mechanism' guidance applies to inter-component failures, not to the validator's own conditional logic. Confidence: medium — based on a single instance, but the pattern generalizes across languages and validation contexts.
Proposed change
Add an 'input-completeness verification' check to skills/pr-review/sub-agents/correctness.md. When reviewing validation, linting, or enforcement code, the sub-agent should verify: for each property the code validates, what happens when the property is entirely absent (not just malformed)? If a validation check is guarded by a presence conditional (if found: validate()), the sub-agent should flag absent-input silent pass-through unless there is a separate check ensuring the property exists. An anchoring example: 'A linter checks that H1 ends with Agent — but only when H1 is found. Documents without H1 silently pass. The absent-input path must also be validated.'
Validation criteria
On the next 3 review agent runs that analyze validation or linter code with conditional checks, the correctness sub-agent's findings should include analysis of what happens when the validated input is absent. Specifically: re-running on PR #476's diff should flag the missing-H1 detection gap in at least 3 out of 5 trials.
Generated by retro agent from #475
What happened
On PR #475, the review agent's correctness sub-agent (Claude Opus, run 30295479886) analyzed a new bash linter script (
hack/lint-agent-docs) and found 2 LOW-severity items (grep flag, shebang convention). However, it missed a correctness bug that Qodo independently identified: the linter only validates H1 heading format when an H1 is found (if [[ -n "$h1" ]]; then check_suffix; fi), so documents without any H1 heading silently pass the check. This same bug persists in successor PR #476. The review agent's findings were never posted (PR closed before posting), so the gap had no practical impact on this PR, but the detection gap is the concern.What could go better
The correctness sub-agent already has 'runtime mechanism verification' guidance that asks whether callers 'silently proceed' when a mechanism fails. But the H1 bug is a different pattern: validation code that checks a property only when present, silently accepting items where the property is entirely absent. This is a common bug class in validators, linters, and config checkers — distinct from shell-specific pitfalls (#131), from severity escalation after detection (#33), and from the bidirectional permission checking gap in #46). The correctness sub-agent's existing 'runtime mechanism' guidance applies to inter-component failures, not to the validator's own conditional logic. Confidence: medium — based on a single instance, but the pattern generalizes across languages and validation contexts.
Proposed change
Add an 'input-completeness verification' check to
skills/pr-review/sub-agents/correctness.md. When reviewing validation, linting, or enforcement code, the sub-agent should verify: for each property the code validates, what happens when the property is entirely absent (not just malformed)? If a validation check is guarded by a presence conditional (if found: validate()), the sub-agent should flag absent-input silent pass-through unless there is a separate check ensuring the property exists. An anchoring example: 'A linter checks that H1 ends with Agent — but only when H1 is found. Documents without H1 silently pass. The absent-input path must also be validated.'Validation criteria
On the next 3 review agent runs that analyze validation or linter code with conditional checks, the correctness sub-agent's findings should include analysis of what happens when the validated input is absent. Specifically: re-running on PR #476's diff should flag the missing-H1 detection gap in at least 3 out of 5 trials.
Generated by retro agent from #475