What happened
On PR #115 (code agent) vs PR #114 (human, merged): Issue #111 and the triage summary both stated the WMCO operator version does not exist in any CI build-log.txt file — the data source is fundamentally wrong. The human's comment at 16:02 said 'Replace _fetch_build_log_text + _extract_operator_version with a new method.' Despite these signals, the agent kept the old build-log extraction code as a fallback and added CSV extraction as a primary method, wiring both into _process_run_single_pass. This resulted in 775 additions across 5 files including a 657-line test file covering both paths, config.yaml pattern additions, and maintained backward compatibility for code that will never succeed. The human's PR removed the dead code (54 deletions) and replaced it with the CSV method (48 additions, 2 files). The repo's AGENTS.md rule #2 says 'Simplicity first. Make the smallest change that solves the problem.'
What could go better
The code agent should recognize when an issue establishes that existing code targets a fundamentally wrong data source (not a logic bug, but the target data literally does not exist) and replace the old code cleanly rather than keeping it as a fallback. The issue said 'WMCO version is not present in any standard CI build artifact.' The triage said '_fetch_build_log_text works correctly but has no data to parse.' The human comment said 'Replace.' All three signals pointed to removal, not fallback retention. The agent's additive approach violated the repo's simplicity-first rule and created a PR that was 16x larger than the human's equivalent. Confidence is high that the signals were clear and the agent should have followed them — the issue was exceptionally well-investigated with a table of verified runs showing the data source. This is distinct from agents#64 (option ordering) because the agent was too conservative (retaining dead code) rather than too destructive.
Proposed change
Add guidance to the code-implementation skill (skills/code-implementation/SKILL.md in fullsend-ai/agents) in the planning step: 'When the issue or triage context establishes that existing code targets a data source that does not contain the expected data (e.g., searching file A when the data only exists in file B), replace the old extraction code with the new approach. Do not keep the old code as a fallback — fallback to code that targets a nonexistent data source adds complexity without value. If the issue author or triage summary uses the word Replace, treat that as a directive to remove the old code and substitute the new approach. A clean replacement with fewer total lines is preferable to an additive approach that retains dead code paths.'
Validation criteria
On future issues where the issue body or triage context identifies a fundamentally wrong data source, the code agent should remove the old extraction code and replace it with the new approach rather than keeping both. The resulting PR should have a comparable or smaller diff than a human would produce for the same change. Verify on the next 3 code agent PRs involving data source replacement that the agent does not retain dead fallback paths.
Generated by retro agent from redhat-community-ai-tools/ci-failure-tracker#115
What happened
On PR #115 (code agent) vs PR #114 (human, merged): Issue #111 and the triage summary both stated the WMCO operator version does not exist in any CI build-log.txt file — the data source is fundamentally wrong. The human's comment at 16:02 said 'Replace _fetch_build_log_text + _extract_operator_version with a new method.' Despite these signals, the agent kept the old build-log extraction code as a fallback and added CSV extraction as a primary method, wiring both into _process_run_single_pass. This resulted in 775 additions across 5 files including a 657-line test file covering both paths, config.yaml pattern additions, and maintained backward compatibility for code that will never succeed. The human's PR removed the dead code (54 deletions) and replaced it with the CSV method (48 additions, 2 files). The repo's AGENTS.md rule #2 says 'Simplicity first. Make the smallest change that solves the problem.'
What could go better
The code agent should recognize when an issue establishes that existing code targets a fundamentally wrong data source (not a logic bug, but the target data literally does not exist) and replace the old code cleanly rather than keeping it as a fallback. The issue said 'WMCO version is not present in any standard CI build artifact.' The triage said '_fetch_build_log_text works correctly but has no data to parse.' The human comment said 'Replace.' All three signals pointed to removal, not fallback retention. The agent's additive approach violated the repo's simplicity-first rule and created a PR that was 16x larger than the human's equivalent. Confidence is high that the signals were clear and the agent should have followed them — the issue was exceptionally well-investigated with a table of verified runs showing the data source. This is distinct from agents#64 (option ordering) because the agent was too conservative (retaining dead code) rather than too destructive.
Proposed change
Add guidance to the code-implementation skill (skills/code-implementation/SKILL.md in fullsend-ai/agents) in the planning step: 'When the issue or triage context establishes that existing code targets a data source that does not contain the expected data (e.g., searching file A when the data only exists in file B), replace the old extraction code with the new approach. Do not keep the old code as a fallback — fallback to code that targets a nonexistent data source adds complexity without value. If the issue author or triage summary uses the word Replace, treat that as a directive to remove the old code and substitute the new approach. A clean replacement with fewer total lines is preferable to an additive approach that retains dead code paths.'
Validation criteria
On future issues where the issue body or triage context identifies a fundamentally wrong data source, the code agent should remove the old extraction code and replace it with the new approach rather than keeping both. The resulting PR should have a comparable or smaller diff than a human would produce for the same change. Verify on the next 3 code agent PRs involving data source replacement that the agent does not retain dead fallback paths.
Generated by retro agent from redhat-community-ai-tools/ci-failure-tracker#115