Skip to content

Review agent should cross-reference sibling commands for codebase consistency when reviewing new CLI subcommands #3965

Description

@fullsend-ai-retro

What happened

On PR #3033, the human Review Squad (waynesun09, 4 agents: Claude x2, Gemini, Codex) caught two codebase-consistency bugs that the fullsend-ai-review bot missed across 13 successful review runs:

  1. HIGH: repos install (repos.go:365) used gh.New(token) directly while sibling repos status (repos.go:75) uses newGitHubLiveClient(token) — breaking GITHUB_API_URL support for test and enterprise environments. Every other CLI command in the codebase uses newGitHubLiveClient.
  2. MEDIUM: repos install used StringSliceVar for the --repo flag while sibling repos status (repos.go:52) uses StringArrayVar — causing inconsistent comma-splitting behavior between sibling commands.

Both issues are in the same file (repos.go) with the existing repos status command visible in the diff context. The review bot never flagged either pattern despite having access to the full file. Additionally, the bot missed a CRITICAL duplicate repos command registration in root.go (line 51 and 55) that would cause the command to appear twice in help output — a cross-file integration issue introduced when PR #3032 merged while #3033 was in flight.

What could go better

The review agent appears to focus on the new/changed code in isolation without systematically comparing it to sibling patterns in the same file or package. On this PR, two of the most impactful bugs were simple consistency mismatches with code visible in the same file. The AGENTS.md already has strong guidance about forge.Client usage and newGitHubLiveClient, but the review agent didn't cross-reference the existing repos status constructor pattern. A similar gap was observed on the sibling PR #3032 (per #3948), where the review agent spent its budget on low-severity style issues while missing CLI UX pattern violations that human reviewers caught. Confidence: high for the sibling-comparison gap — the evidence is unambiguous across two PRs. Medium confidence on whether AGENTS.md guidance alone will fix it, since the forge abstraction rule already exists but wasn't applied.

Proposed change

Add CLI subcommand review guidance to AGENTS.md (in the 'Go code' section or a new 'CLI commands' subsection). When reviewing a PR that adds a new CLI subcommand to an existing command group (e.g., adding repos install alongside repos status), the review agent should compare the new command's constructor and configuration patterns against sibling commands in the same file. Specific patterns to check:

  1. GitHub client construction: all commands in a group should use the same client factory (e.g., newGitHubLiveClient vs gh.New).
  2. Flag types: sibling flags with the same semantic (e.g., --repo across install and status) should use the same Cobra flag type (StringArrayVar vs StringSliceVar).
  3. Command registration: after adding a new subcommand, verify root.go doesn't register the parent group twice.
  4. Testing hook patterns: new commands should follow the same testing override style as siblings.

This also provides evidence for #3948 (CLI UX pattern review guidance).

Validation criteria

On the next 3 PRs that add a new CLI subcommand alongside existing siblings in fullsend-ai/fullsend, the review agent should flag any divergences in client construction, flag types, or registration patterns compared to sibling commands. Baseline: 0/3 such issues caught by the review bot on PR #3033 (two consistency bugs and one duplicate registration missed).


Generated by retro agent from #3033

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/reviewReview agentcomponent/skillsfeatureFeature-category issue awaiting human prioritizationpriority/mediumNormal priority, plan for next cycletriagedTriaged but awaiting human prioritizationtype/featureNew capability request

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions