docs(fix-review): tighten weak language and add trigger phrase - #751
Conversation
PR Summary by QodoTighten Fix Review skill wording and suppress a skillsaw false positive
AI Description
Diagram
High-Level Assessment
Files changed (1)
|
|
🤖 Finished Review · ✅ Success · Started 5:00 PM UTC · Completed 5:12 PM UTC Commit: |
Code Review by Qodo
1. Protected skills/ file modified
|
ReviewFindingsMedium
Previous runReviewFindingsMedium
Low
Previous run (2)ReviewFindingsMedium
Low
Previous run (3)ReviewFindingsMedium
Low
Previous run (4)ReviewFindingsMedium
Low
Labels: PR modifies skill documentation (skills/fix-review/SKILL.md) with no code changes |
|
/fs-fix address @waynesun09's findings. |
|
🤖 Finished Fix · ✅ Success · Started 8:47 PM UTC · Completed 8:52 PM UTC Commit: |
🔧 Fix agent — iteration 1 (human-triggered)Addressed 1 review finding from @waynesun09. Indented the skillsaw HTML comment to fix markdown list rendering without changing directive semantics. Fixed (1):
Tests: passedNext steps:
|
Indent the skillsaw-disable-next-line HTML comment by 2 spaces so it becomes continuation content of the preceding list item rather than breaking the markdown list into two separate <ul> elements on GitHub. Addresses review feedback on #751
|
🤖 Finished Review · ✅ Success · Started 8:53 PM UTC · Completed 9:02 PM UTC Commit: |
cca982f to
6def28e
Compare
|
🤖 Finished Review · ✅ Success · Started 9:39 PM UTC · Completed 9:51 PM UTC Commit: |
waynesun09
left a comment
There was a problem hiding this comment.
Reviewed via 4-agent squad (claude-coder, claude-researcher, grok-review-agent, cursor-code-review) + verification pass at head 6def28e — no CRITICAL/HIGH/MEDIUM findings survived verification.
|
/fs-fix move the "Use when implementing" phrase to the beginning of the description |
|
🤖 Finished Fix · ✅ Success · Started 10:46 PM UTC · Completed 10:50 PM UTC Commit: |
🔧 Fix agent — iteration 2 (human-triggered)Moved the 'Use when implementing' trigger phrase from the end of the skill description to the beginning, as requested by the human reviewer. Fixed (1):
Tests: passedNext steps:
|
|
🤖 Finished Review · ✅ Success · Started 10:51 PM UTC · Completed 11:02 PM UTC Commit: |
Ran skillsaw (evaluating it per #188) against the repo's skills. Adds a "Use when..." trigger phrase to the description, rewrites the trigger_source field explanation as a positive instruction instead of a "must be / do NOT" pair, and suppresses a false-positive placeholder-text match on the phrase "TODO comments" in a checklist item. Related to #188 Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ralph Bean <rbean@redhat.com>
Indent the skillsaw-disable-next-line HTML comment by 2 spaces so it becomes continuation content of the preceding list item rather than breaking the markdown list into two separate <ul> elements on GitHub. Addresses review feedback on #751
9097caf to
585c707
Compare
|
🤖 Finished Review · ✅ Success · Started 3:38 PM UTC · Completed 3:51 PM UTC Commit: |
|
🤖 Finished Retro · ✅ Success · Started 4:11 PM UTC · Completed 4:23 PM UTC Commit: |
Retro: PR #751 —
|
| Phase | What happened |
|---|---|
| PR opened (Aug 11) | Review agent ran in 12 min, found Low structure issue ("Use when" phrase at end vs beginning) and Medium protected-path gate. qodo-code-review flagged protected-path compliance and a debatable "comment-only" finding. |
| Human review (Aug 11, +2h) | waynesun09 identified the only actual bug: the unindented <!-- skillsaw-disable-next-line --> HTML comment between list items broke the markdown into two separate <ul> elements. Verified via GitHub's markdown API. |
| Fix 1 (Aug 12) | Author triggered /fs-fix address @waynesun09's findings. Fix agent correctly indented the HTML comment by 2 spaces to preserve list continuity. Completed in 5 min — surgical and correct. |
| Fix 2 (Aug 13) | Author triggered /fs-fix move the "Use when implementing" phrase to the beginning of the description. Fix agent relocated the phrase as instructed. Completed in 4 min. |
| Approval (Aug 13) | waynesun09 approved after a 4-agent squad verification pass found no surviving CRITICAL/HIGH/MEDIUM findings. |
| Merge (Aug 14) | Merged after final rebase and review. |
What went well
- Fix agent quality was strong. Both
/fs-fixiterations produced surgical, correct changes with clear commit messages. The fix agent correctly parsed a human reviewer's inline comment (fix 1) and explicit instructions (fix 2). - Protected-path gate worked as designed. The
skills/path triggeredrequires-manual-review, ensuring a human reviewed the change — which is exactly where the rendering bug was caught. - The review agent's "Use when" placement finding was valid and led to fix iteration 2, improving consistency with the repo's skill description conventions.
What could go better
- The review agent missed the only actual bug across 3 pre-fix runs. The unindented HTML comment breaking markdown list rendering was detectable (it requires understanding GFM list continuation rules) and was in-diff. The correctness sub-agent did not surface it. This is the same class of bug that occurred on sibling PR docs(code-implementation): tighten weak language and add trigger phrase #748 from the same skillsaw batch.
- 5 review runs for a 6-line doc change is excessive. Runs 2 and 3 reproduced the same structure finding from run 1 verbatim. The protected-path finding was repeated in all 5 runs. Total: 7 agent runs (5 reviews + 2 fixes) for a change that could have been reviewed once.
- A non-obvious tone finding appeared only on run 4. Investigation showed this was diff-dependent, not LLM non-determinism — the structure finding was fixed by that point, so the agent surfaced a secondary observation. This is expected behavior but highlights that the review agent does not guarantee finding completeness on any single pass.
Evidence for existing issues (no new proposals needed)
- agents#802 (Correctness sub-agent should detect GFM list-breaking insertions): This PR provides direct corroborating evidence — the same rendering bug class was missed by the review agent on this sibling PR, confirming the gap is systematic, not a one-off on PR docs(code-implementation): tighten weak language and add trigger phrase #748.
- fullsend#5139, fullsend#4681, agents#108, agents#323: 5 review runs on a 6-line doc PR with runs 2–3 producing zero new information. This is a concrete cost data point for the review dedup effort.
- fullsend#2794: The protected-path finding was posted identically across all 5 runs with no change in status or context.
- agents#721: Stable finding identifiers would have enabled dedup of the repeated structure finding across runs 1–3.
- agents#373: All 6 review dimensions were dispatched for a single-file markdown edit. Security, cross-repo-contracts, and potentially style-conventions sub-agents produced no findings — a lighter review path for docs-only changes would have saved tokens.
No novel proposals are warranted. All identified improvement opportunities are already tracked by open issues with clear remediation paths.
Indent the skillsaw-disable-next-line HTML comment by 2 spaces so it becomes continuation content of the preceding list item rather than breaking the markdown list into two separate <ul> elements on GitHub. Addresses review feedback on #751
Indent the skillsaw-disable-next-line HTML comment by 2 spaces so it becomes continuation content of the preceding list item rather than breaking the markdown list into two separate <ul> elements on GitHub. Addresses review feedback on fullsend-ai#751
Move the "Use when implementing fixes for review comments left on an open PR" sentence to the beginning of the skill description, per human review instruction on PR fullsend-ai#751. Addresses review feedback on fullsend-ai#751
Ran skillsaw (evaluating it per #188) against the repo's skills.
Adds a "Use when..." trigger phrase to the description, rewrites the
trigger_source field explanation as a positive instruction instead of
a "must be / do NOT" pair, and suppresses a false-positive
placeholder-text match on the phrase "TODO comments" in a checklist
item.
Related to #188
Assisted-by: Claude Opus 4.6 noreply@anthropic.com
Signed-off-by: Ralph Bean rbean@redhat.com
Stack created with GitHub Stacks CLI • Give Feedback 💬