What happened
On PR #6804 (fixing #6803), the code agent (run 33404920262, $1.79) implemented a stage-aware role check in routeSlashCommand that distinguishes observation stages (triage, review) from mutation stages. The production code was correct. The test code used table-driven tests with two tables: one for triage-role access and one for entity-author bypass.
However, the code agent only tested one member of the observation-stage set (/fs-triage) in the entity-author bypass table, and omitted the /fs-review + change_proposal combination from the triage-role table. The review agent ($4.11) caught both gaps as low-severity test-adequacy findings. The fix agent ($0.73) added the 2 missing test cases (13 lines), followed by a re-review ($2.52) that approved. Total review-fix cycle cost for these 13 lines: $7.36.
What could go better
The code agent demonstrated it understood the two-member observation set — it tested both /fs-triage and /fs-review in one table — but didn't systematically cross every member against every relevant dimension (entity kind, bypass condition) in the other table. This is a recurring pattern in table-driven tests: the agent picks a representative member of a small set and treats it as covering the whole set, rather than enumerating all members.
Confidence: moderate. This is a single data point, but the pattern (incomplete enumeration in table-driven tests) is a known class of test-adequacy gap that the review agent reliably catches. Front-loading the instruction would shift the cost from the review-fix cycle ($7.36) to the code agent's initial pass (marginal cost increase). The risk is that an overly aggressive combinatorial instruction could produce bloated test tables for larger input sets, so the instruction should be scoped to small enumerated sets (2-4 members).
Proposed change
Add an instruction to the code agent's test-writing guidance (in agents/code.md or skills/code-implementation/SKILL.md) along these lines:
When writing table-driven tests for behavior that varies by a small enumerated set of inputs (2-4 members, e.g., observation stages = {triage, review}, entity kinds = {work_item, change_proposal}), include test cases for every member of the set against every relevant behavioral dimension. Do not rely on one representative member to implicitly cover the others — each member should appear in at least one test case per dimension where it produces a distinct outcome or exercises a distinct code path.
This instruction is intentionally scoped to small sets to avoid encouraging combinatorial explosion for larger input spaces.
Validation criteria
Over the next 10 code agent PRs in fullsend-ai/fullsend that involve table-driven tests with small enumerated input sets, fewer than 20% should receive review findings about missing test cases for untested members of those sets. Currently, the review agent catches this pattern reliably (as it did here), so a reduction in test-adequacy findings of this specific type indicates the code agent is front-loading the coverage.
Generated by retro agent from fullsend-ai/fullsend#6804
What happened
On PR #6804 (fixing #6803), the code agent (run 33404920262, $1.79) implemented a stage-aware role check in
routeSlashCommandthat distinguishes observation stages (triage,review) from mutation stages. The production code was correct. The test code used table-driven tests with two tables: one for triage-role access and one for entity-author bypass.However, the code agent only tested one member of the observation-stage set (
/fs-triage) in the entity-author bypass table, and omitted the/fs-review+change_proposalcombination from the triage-role table. The review agent ($4.11) caught both gaps as low-severity test-adequacy findings. The fix agent ($0.73) added the 2 missing test cases (13 lines), followed by a re-review ($2.52) that approved. Total review-fix cycle cost for these 13 lines: $7.36.What could go better
The code agent demonstrated it understood the two-member observation set — it tested both
/fs-triageand/fs-reviewin one table — but didn't systematically cross every member against every relevant dimension (entity kind, bypass condition) in the other table. This is a recurring pattern in table-driven tests: the agent picks a representative member of a small set and treats it as covering the whole set, rather than enumerating all members.Confidence: moderate. This is a single data point, but the pattern (incomplete enumeration in table-driven tests) is a known class of test-adequacy gap that the review agent reliably catches. Front-loading the instruction would shift the cost from the review-fix cycle ($7.36) to the code agent's initial pass (marginal cost increase). The risk is that an overly aggressive combinatorial instruction could produce bloated test tables for larger input sets, so the instruction should be scoped to small enumerated sets (2-4 members).
Proposed change
Add an instruction to the code agent's test-writing guidance (in
agents/code.mdorskills/code-implementation/SKILL.md) along these lines:This instruction is intentionally scoped to small sets to avoid encouraging combinatorial explosion for larger input spaces.
Validation criteria
Over the next 10 code agent PRs in fullsend-ai/fullsend that involve table-driven tests with small enumerated input sets, fewer than 20% should receive review findings about missing test cases for untested members of those sets. Currently, the review agent catches this pattern reliably (as it did here), so a reduction in test-adequacy findings of this specific type indicates the code agent is front-loading the coverage.
Generated by retro agent from fullsend-ai/fullsend#6804