Skip to content

Correctness sub-agent should validate doc comments against referenced struct fields #730

Description

@fullsend-ai-retro

What happened

On PR #5997, the human reviewer (waynesun09's inline comment) caught a MEDIUM finding: the StatusNotificationsReader interface doc comment at internal/config/interfaces.go:36 read "provides read access to status notification configuration (comment/reaction start & completion)" but StatusNotificationConfig has no Reaction field — only a Comment field. The correctness sub-agent in review run 31408420012 returned zero findings on the same commit, missing this entirely. The author accepted the finding and pushed a fix (commit 19db87c).

What could go better

The correctness sub-agent does not currently cross-reference doc comments on interfaces or types against the actual fields/methods defined on the struct they describe. When a PR introduces a new interface with a doc comment that enumerates capabilities, any capability not backed by an actual struct field is a factual inaccuracy in the code. This is a mechanical check — compare the nouns/capabilities listed in the doc comment against the field names in the referenced type — and is well-suited to the correctness dimension. Confidence: high. The human identified this in a single pass; the struct had only two fields (Comment and nothing else), making the mismatch unambiguous.

Proposed change

Add a check to the correctness sub-agent's instructions (in the review agent definition or the pr-review skill) that says: "When the diff adds or modifies an interface, type, or method with a doc comment that enumerates capabilities or fields (e.g., 'provides access to X/Y/Z'), verify that each enumerated item corresponds to an actual field, method, or type in the codebase. Flag any doc comment that references a capability or field name that does not exist in the referenced struct or interface as a MEDIUM finding." This could be added to the correctness sub-agent's dispatch prompt in the review orchestrator or to the pr-review skill's correctness checklist.

Validation criteria

On the next 5 review agent runs where the diff introduces or modifies a Go interface or type with a doc comment listing capabilities, the correctness sub-agent should verify each listed capability against the actual struct fields. Specifically, if a test PR is constructed with a doc comment referencing a non-existent field (similar to the 'reaction' case here), the correctness sub-agent should flag it at MEDIUM severity.


Generated by retro agent from fullsend-ai/fullsend#5997

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions