feat(#3697): add on_failure mode for comment.completion status notifications

[ralphbean]

Summary

• Add on_failure as a valid value for status_notifications.comment.completion
• When set, completion comments are posted only on failure/cancellation — suppressed on success
• On success with on_failure, the start comment is silently cleaned up (deleted)
• Config validation rejects on_failure for comment.start (no outcome to evaluate yet)

Phase 1 of #3697 — comment-only changes. Reaction support is a follow-up.

Also extends orphan-comment synthesis (previously on_failure-only) to the default enabled mode: a hard crash before any status comment is posted now surfaces as a synthesized "Interrupted" comment there too. This affects every existing install using the default config, not just on_failure opt-ins. See docs/guides/user/customizing-agents.md's Completion modes section.

Test plan

• Unit tests: config validation accepts on_failure for completion, rejects for start
• Unit tests: PostCompletion with on_failure suppresses on success, fires on failure/cancelled
• Unit tests: cleanup of start comment when completion suppressed
• All existing tests pass (no regressions)
• Functional test: fullsend run triage with on_failure config on a test issue

🤖 Generated with Claude Code

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 5:44 PM UTC · Ended 5:46 PM UTC
Commit: 1c8b95b · View workflow run →

[github-actions]

Site preview

Preview: https://31c7f779-site.fullsend-ai.workers.dev

Commit: fa6d15ad7d9d5b7689eddb7b5b10db94c8b0b773

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 5:47 PM UTC · Completed 6:01 PM UTC
Commit: 2829847 · View workflow run →

[codecov]

Codecov Report

❌ Patch coverage is 91.80328% with 5 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/statuscomment/statuscomment.go	83.87%	2 Missing and 3 partials ⚠️

📢 Thoughts on this report? Let us know!

[fullsend-ai-review]

Review

Findings

Low

• [edge-case] action.yml:432 — The reconcile step passes ${{ job.status }} as JOB_STATUS, but job.status reflects the cumulative outcome of ALL prior steps in the job (including the newly reordered Upload fullsend artifacts step, which now runs before reconciliation). If the upload step fails after a successful fullsend run, job.status will be "failure" and ReconcileOrphaned will synthesize a spurious "Interrupted" comment when completion mode is on_failure. The PR author's code comment documents this as an intentional design choice ("job.status reflects the job's true final outcome"), and the scenario requires three conditions to align (on_failure mode configured, upload step failure, successful agent run).

• [api-shape] internal/statuscomment/statuscomment.go — ReconcileOrphaned has 13 positional parameters (up from 9), adding completionMode, jobStatus, wasSkipped, and agentDescription. The codebase uses options struct patterns elsewhere (e.g., ComposeOpts, LoadOpts). A high parameter count remains a maintenance concern for callers.

• [missing-documentation] docs/guides/infrastructure/layered-config-reference.md:283 — The layered config reference documents status_notifications.comment.start/comment.completion settings but does not specify the valid values for these fields. With this PR adding on_failure as a new valid value for completion, the reference could document all three valid completion values: enabled (default), on_failure, and disabled. Note: the primary user-facing documentation has already been comprehensively updated in docs/guides/user/customizing-agents.md.

Previous run

Review

Findings

Low

• [edge-case] action.yml:431 — The reconcile step passes ${{ job.status }} as JOB_STATUS, but job.status reflects the cumulative outcome of ALL prior steps in the job (including the newly reordered Upload fullsend artifacts step, which now runs before reconciliation). If the upload step fails after a successful fullsend run, job.status will be "failure" and ReconcileOrphaned will synthesize a spurious "Interrupted" comment when completion mode is on_failure. The PR author's code comment documents this as an intentional design choice ("job.status reflects the job's true final outcome"), and the scenario requires three conditions to align (on_failure mode configured, upload step failure, successful agent run).

• [api-shape] internal/statuscomment/statuscomment.go:507 — ReconcileOrphaned has 13 positional parameters (up from 9), adding completionMode, jobStatus, wasSkipped, and agentDescription. The codebase uses options struct patterns elsewhere (e.g., ComposeOpts, LoadOpts). A high parameter count remains a maintenance concern for callers.

Previous run (2)

Review

Findings

Low

• [edge-case] action.yml:431 — The reconcile step passes ${{ job.status }} as JOB_STATUS, but job.status reflects the cumulative outcome of ALL prior steps in the job. If the Upload fullsend artifacts step fails after a successful fullsend run, job.status will be "failure" and ReconcileOrphaned will synthesize a spurious "Interrupted" comment when completion mode is on_failure. The PR author's code comment documents this as an intentional design choice ("job.status reflects the job's true final outcome"), and the scenario requires three conditions to align (on_failure mode configured, upload step failure, successful agent run).

• [edge-case] internal/statuscomment/statuscomment.go:155 — PostStart suppresses the start comment when completion is on_failure by checking n.cfg.Comment.Completion != "on_failure". A user who sets start: enabled and completion: on_failure will not see a start comment, which may be surprising. The behavior is deliberate, documented in the code comment, and tested — PostStart's suppression prevents a GitHub notification that would point to a comment that gets deleted on success.

• [edge-case] internal/statuscomment/statuscomment.go:532 — The synthesis condition completionMode == "on_failure" && (wasSkipped || (jobStatus != "" && jobStatus != "success")) accepts any non-empty, non-"success" jobStatus value. Any unexpected value (e.g., a typo in the --job-status flag) would trigger synthesis. The risk is low since the flag value is controlled by action.yml, not user input.

• [api-shape] internal/statuscomment/statuscomment.go:495 — ReconcileOrphaned now has 13 positional parameters (up from 9), adding completionMode, jobStatus, wasSkipped, and agentDescription. The codebase uses options struct patterns elsewhere (e.g., ComposeOpts, LoadOpts). Pre-existing concern worsened by four parameters.

Previous run (3)

Review

Findings

Medium

• [logic-error] internal/cli/reconcilestatus.go:98 — The code type-asserts ConfigWriter to OrgConfigReader to call StatusNotifications(), but ConfigWriter already embeds StatusNotificationsReader (via ConfigReader), so writer.StatusNotifications() works directly for both orgConfig and perRepoConfig. The type assertion fails for per-repo installations (perRepoConfig does not implement OrgConfigReader), silently falling through to empty completionMode. A per-repo installation with completion: on_failure will not have ReconcileOrphaned detect the mode, potentially missing synthesized interrupt comments on failure. The test only exercises the org-config path.
  Remediation: Call writer.StatusNotifications() directly instead of type-asserting to OrgConfigReader. Add a test with a per-repo config that sets completion: on_failure.

Low

• [edge-case] action.yml:431 — The reconcile step passes ${{ job.status }} as JOB_STATUS, but job.status reflects the cumulative outcome of ALL prior steps in the job. If the Upload fullsend artifacts step fails after a successful fullsend run, job.status will be "failure" and ReconcileOrphaned will synthesize a spurious "Interrupted" comment when completion mode is on_failure. The PR author's code comment documents this as an intentional design choice ("job.status reflects the job's true final outcome"), and the scenario requires three conditions to align (on_failure mode configured, upload step failure, successful agent run).

• [edge-case] internal/statuscomment/statuscomment.go:155 — PostStart suppresses the start comment when completion is on_failure by checking n.cfg.Comment.Completion != "on_failure". A user who sets start: enabled and completion: on_failure will not see a start comment, which may be surprising. The behavior is deliberate, documented in the code comment, and tested — PostStart's suppression prevents a GitHub notification that would point to a comment that gets deleted on success.

• [edge-case] internal/statuscomment/statuscomment.go:532 — The synthesis condition completionMode == "on_failure" && (wasSkipped || (jobStatus != "" && jobStatus != "success")) accepts any non-empty, non-"success" jobStatus value. Any unexpected value (e.g., a typo in the --job-status flag) would trigger synthesis. The risk is low since the flag value is controlled by action.yml, not user input.

• [api-shape] internal/statuscomment/statuscomment.go:495 — ReconcileOrphaned now has 13 positional parameters (up from 9), adding completionMode, jobStatus, wasSkipped, and agentDescription. The codebase uses options struct patterns elsewhere (e.g., ComposeOpts, LoadOpts). Pre-existing concern worsened by four parameters.

• [error-messages] internal/cli/reconcilestatus.go:103 — The PR introduces INFO: prefix for diagnostic messages on stderr (lines 103, 105). The existing codebase uses only WARNING: prefix for stderr diagnostics (20+ instances). This creates an inconsistent diagnostic vocabulary.

---

Labels: PR touches core harness internals (internal/cli, internal/config, internal/statuscomment), the GitHub Actions composite action (action.yml), and Go source files.

---

Next steps:

• /fs-fix — agent addresses review findings automatically
• /fs-fix <your instruction> — agent fixes with your specific guidance
• Push commits directly — review re-runs automatically on push
• /fs-fix-stop — disable automatic fix runs for this PR

Previous run (4)

Review

Findings

Low

• [logic-error] action.yml:431 — The reconcile step passes ${{ job.status }} as JOB_STATUS, but job.status reflects the cumulative outcome of ALL prior steps in the job. If the Upload fullsend artifacts step fails after a successful fullsend run, job.status will be "failure" and ReconcileOrphaned will synthesize a spurious "Interrupted" comment when completion mode is on_failure. The PR author's code comment documents this as an intentional design choice ("job.status reflects the job's true final outcome"), and the scenario requires three conditions to align (on_failure mode configured, upload step failure, successful agent run). Consider using ${{ steps.run.outcome }} to limit synthesis to agent-level failures.

• [edge-case] internal/statuscomment/statuscomment.go:148 — PostStart suppresses the start comment when completion is on_failure by checking n.cfg.Comment.Completion != "on_failure". A user who sets start: enabled and completion: on_failure will not see a start comment, which may be surprising. The behavior is deliberate, documented in the code comment, and tested — PostStart's suppression prevents a GitHub notification that would point to a comment that gets deleted on success.

• [edge-case] internal/statuscomment/statuscomment.go:525 — The synthesis condition completionMode == "on_failure" && (wasSkipped || (jobStatus != "" && jobStatus != "success")) accepts any non-empty, non-"success" jobStatus value. Any unexpected value (e.g., a typo in the --job-status flag) would trigger synthesis. The risk is low since the flag value is controlled by action.yml, not user input.

• [api-shape] internal/statuscomment/statuscomment.go:488 — ReconcileOrphaned now has 13 positional parameters (up from 9), adding completionMode, jobStatus, wasSkipped, and agentDescription. The codebase uses options struct patterns elsewhere (e.g., ComposeOpts, LoadOpts). Pre-existing concern worsened by four parameters.

• [error-messages] internal/cli/reconcilestatus.go — The PR introduces INFO: prefix for diagnostic messages alongside the existing WARNING: pattern. This prefix is new to the codebase's CLI diagnostic conventions.

• [design-direction] docs/guides/user/customizing-agents.md — status_notifications is documented as org-level only ("not available in per-repo configs — notification style is an org-wide UX decision"), but no ADR or problem-doc captures the rationale for this constraint. Not a blocker — it's documenting existing behavior.

Previous run (5)

Review

Findings

Low

• [logic-error] action.yml:431 — The reconcile step passes ${{ job.status }} as JOB_STATUS, but job.status reflects the cumulative outcome of ALL prior steps in the job. If the Upload fullsend artifacts step fails after a successful fullsend run, job.status will be "failure" and ReconcileOrphaned will synthesize a spurious "Interrupted" comment when completion mode is on_failure. The PR author's code comment documents this as an intentional design choice ("job.status reflects the job's true final outcome"), and the scenario requires three conditions to align (on_failure mode configured, upload step failure, successful agent run). Consider using ${{ steps.run.outcome }} to limit synthesis to agent-level failures.

• [edge-case] internal/statuscomment/statuscomment.go:148 — PostStart suppresses the start comment when completion is on_failure by checking n.cfg.Comment.Completion != "on_failure". A user who sets start: enabled and completion: on_failure will not see a start comment, which may be surprising. The behavior is deliberate, documented in the code comment, and tested — PostStart's suppression prevents a GitHub notification that would point to a comment that gets deleted on success.

• [edge-case] internal/statuscomment/statuscomment.go:525 — The synthesis condition completionMode == "on_failure" && (wasSkipped || (jobStatus != "" && jobStatus != "success")) accepts any non-empty, non-"success" jobStatus value. Any unexpected value (e.g., a typo in the --job-status flag) would trigger synthesis. The risk is low since the flag value is controlled by action.yml, not user input.

• [api-shape] internal/statuscomment/statuscomment.go:488 — ReconcileOrphaned now has 13 positional parameters (up from 9), adding completionMode, jobStatus, wasSkipped, and agentDescription. The codebase uses options struct patterns elsewhere (e.g., ComposeOpts, LoadOpts). Pre-existing concern worsened by four parameters.

• [missing-doc] docs/guides/dev/cli-internals.md:122 — The --was-skipped flag was added to the reconcile-status command but was not added to the CLI tree documentation. The PR added --fullsend-dir and --job-status to the CLI tree but omitted --was-skipped.

Previous run (6)

Review

Findings

Medium

• [logic-error] action.yml:432 — The reconcile step passes ${{ job.status }} as JOB_STATUS, but job.status reflects the cumulative outcome of ALL prior steps in the job. This PR moves the Upload fullsend artifacts step before the reconcile step (both use if: always()), so if the upload fails after a successful fullsend run, job.status will be "failure" and ReconcileOrphaned will synthesize a spurious "Interrupted" comment — falsely claiming the agent was terminated when it completed normally. The Run fullsend step has id: run, so steps.run.outcome is available and would accurately reflect only the fullsend run's result.
  Remediation: Pass ${{ steps.run.outcome }} instead of ${{ job.status }} for the JOB_STATUS env var.

Low

• [edge-case] internal/statuscomment/statuscomment.go:145 — PostStart suppresses the start comment when completion is on_failure by checking n.cfg.Comment.Completion != "on_failure". A user who sets start: enabled and completion: on_failure will not see a start comment, which may be surprising. The behavior is deliberate and documented: posting then deleting a start comment on success still triggers a GitHub notification pointing to a deleted comment, which defeats the noise-reduction purpose.

• [edge-case] internal/statuscomment/statuscomment.go:521 — The synthesis condition completionMode == "on_failure" && (wasSkipped || (jobStatus != "" && jobStatus != "success")) accepts any non-empty, non-"success" jobStatus value. Any unexpected value (e.g., a typo in the --job-status flag) would trigger synthesis. The risk is low since the flag value is controlled by action.yml, not user input.

• [api-shape] internal/statuscomment/statuscomment.go:484 — ReconcileOrphaned now has 13 positional parameters (up from 9), adding completionMode, jobStatus, wasSkipped, and agentDescription. The codebase uses options struct patterns elsewhere (e.g., config.LoadOpts). Pre-existing concern worsened by four parameters.

• [missing-doc] docs/guides/dev/cli-internals.md:122 — The --was-skipped flag was added to the reconcile-status command but was not added to the CLI tree documentation. The PR added --fullsend-dir and --job-status to the CLI tree but omitted --was-skipped.

---

Next steps:

• /fs-fix — agent addresses review findings automatically
• /fs-fix <your instruction> — agent fixes with your specific guidance
• Push commits directly — review re-runs automatically on push
• /fs-fix-stop — disable automatic fix runs for this PR

Previous run (7)

Review

Findings

Medium

• [logic-error] action.yml:431 — The reconcile step passes ${{ job.status }} as JOB_STATUS, but job.status reflects the cumulative outcome of ALL prior steps in the job. This PR moves the Upload fullsend artifacts step before the reconcile step (both use if: always()), so if the upload fails after a successful fullsend run, job.status will be "failure" and ReconcileOrphaned will synthesize a spurious "Interrupted" comment — falsely claiming the agent was terminated when it completed normally. The Run fullsend step has id: run, so steps.run.outcome is available and would accurately reflect only the fullsend run's result.
  Remediation: Pass ${{ steps.run.outcome }} instead of ${{ job.status }} for the JOB_STATUS env var.

Low

• [edge-case] internal/statuscomment/statuscomment.go:146 — PostStart suppresses the start comment when completion is on_failure by checking n.cfg.Comment.Completion != "on_failure". A user who sets start: enabled and completion: on_failure will not see a start comment, which may be surprising. The behavior is deliberate and documented: posting then deleting a start comment on success still triggers a GitHub notification pointing to a deleted comment, which defeats the noise-reduction purpose.

• [edge-case] internal/statuscomment/statuscomment.go:514 — The synthesis condition completionMode == "on_failure" && jobStatus != "" && jobStatus != "success" accepts any non-empty, non-"success" jobStatus value. Any unexpected value (e.g., a typo in the --job-status flag) would trigger synthesis. The risk is low since the flag value is controlled by action.yml, not user input.

• [diagnostic-message-prefix] internal/cli/reconcilestatus.go:102 — The diff introduces INFO: prefix in diagnostic messages (lines 102, 104). This prefix does not appear anywhere else in internal/cli/*.go files. The codebase uses WARNING: extensively (20+ occurrences) for stderr prefixes; non-warning diagnostics use no prefix.

• [api-shape] internal/statuscomment/statuscomment.go:478 — ReconcileOrphaned now has 11 positional parameters (up from 9), adding completionMode and jobStatus. The codebase uses options struct patterns elsewhere (e.g., config.LoadOpts). Pre-existing concern worsened by two parameters.

---

Next steps:

• /fs-fix — agent addresses review findings automatically
• /fs-fix <your instruction> — agent fixes with your specific guidance
• Push commits directly — review re-runs automatically on push
• /fs-fix-stop — disable automatic fix runs for this PR

Previous run (8)

Review

Findings

Medium

• [logic-error] action.yml:431 — The reconcile step passes ${{ job.status }} as JOB_STATUS, but job.status reflects the cumulative outcome of ALL prior steps in the job. This PR moves the Upload fullsend artifacts step before the reconcile step (both use if: always()), so if the upload fails after a successful fullsend run, job.status will be "failure" and ReconcileOrphaned will synthesize a spurious "Interrupted" comment — falsely claiming the agent was terminated when it completed normally. The Run fullsend step has id: run, so steps.run.outcome is available and would accurately reflect only the fullsend run's result.
  Remediation: Pass ${{ steps.run.outcome }} instead of ${{ job.status }} for the JOB_STATUS env var.

Low

• [edge-case] internal/statuscomment/statuscomment.go:146 — PostStart suppresses the start comment when completion is on_failure by checking n.cfg.Comment.Completion != "on_failure". A user who sets start: enabled and completion: on_failure will not see a start comment, which may be surprising. The behavior is deliberate and documented: posting then deleting a start comment on success still triggers a GitHub notification pointing to a deleted comment, which defeats the noise-reduction purpose.

• [edge-case] internal/statuscomment/statuscomment.go:514 — The synthesis condition completionMode == "on_failure" && jobStatus != "" && jobStatus != "success" accepts any non-empty, non-"success" jobStatus value. Any unexpected value (e.g., a typo in the --job-status flag) would trigger synthesis. The risk is low since the flag value is controlled by action.yml, not user input.

• [diagnostic-message-prefix] internal/cli/reconcilestatus.go:106 — The diff introduces INFO: prefix in diagnostic messages (lines 106, 108). This prefix does not appear anywhere else in internal/cli/*.go files. The codebase uses WARNING: extensively (20+ occurrences) for stderr prefixes; non-warning diagnostics use no prefix.

• [api-shape] internal/statuscomment/statuscomment.go:478 — ReconcileOrphaned now has 11 positional parameters (up from 9), adding completionMode and jobStatus. The codebase uses options struct patterns elsewhere (e.g., config.LoadOpts). Pre-existing concern worsened by two parameters.

---

Next steps:

• /fs-fix — agent addresses review findings automatically
• /fs-fix <your instruction> — agent fixes with your specific guidance
• Push commits directly — review re-runs automatically on push
• /fs-fix-stop — disable automatic fix runs for this PR

Previous run (9)

Review

Findings

Medium

• [logic-error] action.yml — The reconcile step uses job.status (via JOB_STATUS) to decide whether to synthesize an "Interrupted" comment when completionMode == "on_failure". However, job.status reflects the outcome of ALL previous steps in the job, not just the fullsend run step. If the fullsend run succeeds (agent completes normally, PostCompletion suppresses the comment as designed) but the Upload fullsend artifacts step fails, job.status will be "failure" and ReconcileOrphaned will create a spurious "Interrupted" comment — falsely claiming the agent was terminated when it actually completed normally.
  Remediation: Pass the fullsend run step's outcome (e.g., steps.<run-step-id>.outcome) instead of job.status for the --job-status flag, or have PostCompletionWithDetail write a sentinel file when it suppresses the comment so ReconcileOrphaned can distinguish "completed normally" from "hard-killed."

Low

• [edge-case] internal/statuscomment/statuscomment.go:146 — PostStart suppresses the start comment when completion is on_failure by checking n.cfg.Comment.Completion != "on_failure". A user who sets start: enabled and completion: on_failure will not see a start comment, which may be surprising. The behavior is deliberate and documented: posting then deleting a start comment on success still triggers a GitHub notification pointing to a deleted comment, which defeats the noise-reduction purpose. See also: [scope-creep] finding at this location.

• [scope-creep] internal/statuscomment/statuscomment.go:146 — The implementation couples start and completion settings: completion=on_failure auto-suppresses start comments. A user cannot configure start:enabled + completion:on_failure. This is a deliberate tradeoff to avoid sending GitHub notifications for comments that would be deleted on success. See also: [edge-case] finding at this location.

• [undisclosed-scope] internal/cli/reconcilestatus.go:95 — The PR adds reconciliation infrastructure for on_failure mode (config loading, --fullsend-dir and --job-status flags, workflow integration, completion-mode-aware synthesis logic) that is not mentioned in the PR description's claimed scope ("Phase 1 of Triage agent causes unnecessary notifications - should skip initial comment #3697 — comment-only changes").

• [diagnostic-message-prefix] internal/cli/reconcilestatus.go — The diff introduces INFO: prefix in diagnostic messages. This prefix does not appear anywhere else in internal/cli/*.go files. The codebase uses WARNING: extensively (20+ occurrences) but never INFO:.

• [api-shape] internal/statuscomment/statuscomment.go:480 — ReconcileOrphaned now has 11 positional parameters (up from 9), adding completionMode and jobStatus. The codebase uses options struct patterns elsewhere (e.g., config.LoadOpts). Pre-existing concern worsened by two parameters.

---

Next steps:

• /fs-fix — agent addresses review findings automatically
• /fs-fix <your instruction> — agent fixes with your specific guidance
• Push commits directly — review re-runs automatically on push
• /fs-fix-stop — disable automatic fix runs for this PR

Previous run (10)

Review

Findings

Low

• [edge-case] internal/statuscomment/statuscomment.go:148 — PostStart suppresses the start comment when completion is on_failure by checking n.cfg.Comment.Completion != "on_failure". A user who sets start: enabled and completion: on_failure will not see a start comment, which may be surprising. The behavior is deliberate and documented: posting then deleting a start comment on success still triggers a GitHub notification pointing to a deleted comment, which defeats the noise-reduction purpose.

• [api-shape] internal/statuscomment/statuscomment.go:480 — ReconcileOrphaned now has 11 positional parameters (up from 9), adding completionMode and jobStatus. The codebase uses options struct patterns elsewhere (e.g., config.LoadOpts). Pre-existing concern worsened by two parameters.

• [config-loading-pattern] internal/cli/reconcilestatus.go:94 — Config loading uses a nested switch-case pattern that differs from simpler patterns elsewhere but is warranted for defensive error handling since config loading is optional and errors should not abort the reconcile operation.

Previous run (11)

Review

Findings

Low

• [implicit-coupling] internal/statuscomment/statuscomment.go:148 — Setting completion: on_failure silently overrides start: enabled. PostStart checks n.cfg.Comment.Completion != "on_failure" and suppresses start comments regardless of the start setting. The coupling is documented in user-facing docs (customizing-agents.md completion modes table and prose), the code comment on PostStart, and is a deliberate design choice — posting a start comment that gets deleted on success would still trigger a GitHub notification pointing to a deleted comment. See also: [scope-interpretation] finding at this location.

• [scope-interpretation] internal/statuscomment/statuscomment.go:146 — Issue Triage agent causes unnecessary notifications - should skip initial comment #3697 requested skipping the start comment to reduce notification noise. The PR implements a broader feature (on_failure completion mode) that solves the notification problem by suppressing both start and completion comments on success. The implementation scope is wider than the simplest interpretation of the issue, but achieves the same UX outcome and provides value for other agent types beyond triage. See also: [implicit-coupling] finding at this location.

• [scope-creep] internal/statuscomment/statuscomment.go:516 — Orphan synthesis logic extends beyond the simplest interpretation of issue Triage agent causes unnecessary notifications - should skip initial comment #3697. However, synthesis is a necessary correctness measure for on_failure mode — without it, a hard-killed agent in on_failure mode would be completely invisible (no start comment was posted, process died before PostCompletion). The logic is conservative: triggers only when completionMode == "on_failure" && jobStatus != "" && jobStatus != "success". See also: [edge-case] finding at this location.

• [edge-case] internal/statuscomment/statuscomment.go:516 — In ReconcileOrphaned, synthesized interrupted comments use the reason parameter (terminated vs. cancelled) for the status label, not jobStatus. A job that GitHub reports as "failure" will get a "Terminated" label if the process was SIGKILL'd. This is semantically correct for the orphaned-process scenario (the label describes how the process died, not the job outcome). See also: [scope-creep] finding at this location.

• [platform-coupling] internal/statuscomment/statuscomment.go:467 — The jobStatus parameter docstring references GitHub Actions job status values. While the string comparisons themselves are generic CI concepts (success/failure/cancelled), the docstring creates a documentation-level coupling to GHA semantics. Currently moot since the platform is exclusively github-actions (config validation enforces this).

• [api-shape] internal/statuscomment/statuscomment.go:480 — ReconcileOrphaned now has 11 positional parameters (up from 9), adding completionMode and jobStatus. The codebase uses options struct patterns elsewhere (e.g., config.LoadOpts). Pre-existing concern worsened by two parameters.

• [naming-consistency] internal/statuscomment/statuscomment.go:130 — New shouldPostCompletion helper uses a different naming convention (should*) than the existing commentEnabled helper. The difference reflects a genuine semantic distinction — commentEnabled is a simple config check while shouldPostCompletion encodes richer logic depending on both config and runtime status.

• [runtime-dependency] internal/cli/reconcilestatus.go:94 — The reconcile-status command loads config at runtime to determine completion mode. This is well-defended: loading is skipped when --fullsend-dir is not passed, MissingOK: true handles absent configs, errors print a WARNING and fall through to default (enabled) behavior, and the nil check on the writer handles the MissingOK path.

• [config-loading-verbosity] internal/cli/reconcilestatus.go:693 — Config loading code uses a nested type assertion pattern (13 lines). Functionally correct with clear, linear logic: check flag, load config, handle error, type-assert, read value. Single call site — extracting a helper would add indirection without meaningful reuse.

• [naming-clarity] internal/config/config.go:116 — The on_failure value describes when to post completion comments but also controls start comment behavior. The name doesn't hint at the broader suppression effect on start comments, though this is documented in user-facing docs.

• [validation-pattern] internal/config/config.go:454 — Config validation now splits into validStartValues and validCompletionValues where previously a single validCommentValues was reused. The split is a correctness requirement since start and completion now accept different value sets.

• [config-scope-restriction] docs/guides/user/customizing-agents.md:496 — status_notifications is org-wide only, not per-repo. This is a pre-existing design constraint (not introduced by this PR) and is reasonable from a UX consistency perspective.

Previous run (12)

Review

Findings

Low

• [implicit-coupling] internal/statuscomment/statuscomment.go:146 — Setting completion: on_failure silently overrides start: enabled. PostStart checks n.cfg.Comment.Completion != "on_failure" and suppresses start comments regardless of the start setting. The coupling is documented in user-facing docs (customizing-agents.md completion modes table and prose), the code comment on PostStart, and is a deliberate design choice — posting a start comment that gets deleted on success would still trigger a GitHub notification pointing to a deleted comment.

• [scope-creep] internal/statuscomment/statuscomment.go:514 — Orphan synthesis logic (lines 507–520) extends beyond the simplest interpretation of issue Triage agent causes unnecessary notifications - should skip initial comment #3697, which asked to skip posting the initial comment. However, synthesis is a necessary correctness measure for on_failure mode — without it, a hard-killed agent in on_failure mode would be completely invisible (no start comment was posted, process died before PostCompletion). The logic is conservative: triggers only when completionMode == "on_failure" && jobStatus != "" && jobStatus != "success".

• [edge-case] internal/statuscomment/statuscomment.go:514 — In ReconcileOrphaned, synthesized interrupted comments use the reason parameter (terminated vs. cancelled) for the status label, not jobStatus. A job that GitHub reports as "failure" will get a "Terminated" label if the process was SIGKILL'd. This is semantically correct for the orphaned-process scenario (the label describes how the process died, not the job outcome) but may confuse operators. See also: [scope-creep] finding at this location.

• [platform-coupling] internal/statuscomment/statuscomment.go:467 — The jobStatus parameter docstring references GitHub Actions job status values. While the string comparisons themselves are generic CI concepts (success/failure/cancelled), the docstring creates a documentation-level coupling to GHA semantics. Currently moot since the platform is exclusively github-actions (config validation enforces this).

• [runtime-dependency] internal/cli/reconcilestatus.go:94 — The reconcile-status command loads config at runtime to determine completion mode. This is well-defended: loading is skipped when --fullsend-dir is not passed, MissingOK: true handles absent configs, errors print a WARNING and fall through to default (enabled) behavior, and the nil check on the writer handles the MissingOK path.

• [api-shape] internal/statuscomment/statuscomment.go:478 — ReconcileOrphaned now has 11 positional parameters (up from 9), adding completionMode and jobStatus. The codebase uses options struct patterns elsewhere (e.g., config.LoadOpts). Pre-existing concern worsened by two parameters.

• [naming-clarity] internal/config/config.go:116 — The on_failure value describes when to post completion comments but also controls start comment behavior. The name doesn't hint at the broader suppression effect on start comments, though this is documented in user-facing docs.

Previous run (13)

Review

Findings

Medium

• [implicit-coupling] internal/statuscomment/statuscomment.go:146 — Setting completion: on_failure silently overrides start: enabled. PostStart adds a hard-coded check n.cfg.Comment.Completion != "on_failure" that suppresses start comments regardless of the start setting. A user who sets start: enabled, completion: on_failure expecting start comments would not get them. The coupling is documented in the user-facing docs and the code comment explains the rationale (posting a start comment that gets deleted on success would trigger a notification pointing to a deleted comment), but the design still allows one config field to silently override another's explicit value.

Low

• [missing-test] internal/statuscomment/statuscomment.go:128 — shouldPostCompletion includes "timeout" as a posting trigger for on_failure mode, but there is no test exercising PostCompletion with status="timeout" under on_failure mode. Tests cover success, failure, cancelled, and skipped but not timeout.

• [edge-case] internal/statuscomment/statuscomment.go — In ReconcileOrphaned, when synthesizing an interrupted comment for on_failure mode with a non-success job status, the termination reason defaults to "terminated", so the synthesized comment reads "Terminated" even when jobStatus is "failure". This is correct for the orphaned-process scenario but may confuse operators.

• [api-shape] internal/statuscomment/statuscomment.go — ReconcileOrphaned now has 11 positional parameters (up from 9), adding completionMode and jobStatus. The codebase uses options struct patterns elsewhere (e.g., config.LoadOpts). Pre-existing concern worsened by two parameters.

• [naming-clarity] internal/config/config.go — The on_failure value describes when to post completion comments but also controls start comment behavior. The name doesn't hint at the broader suppression effect on start comments.

• [naming-conventions] internal/config/config.go — Renaming validCommentValues to validStartValues and introducing validCompletionValues is reasonable given the now-different valid sets, but creates two arrays where the Start values are identical to the original shared array.

Previous run (14)

Review

Findings

Medium

• [implicit-coupling] internal/statuscomment/statuscomment.go:146 — Setting completion: on_failure silently overrides start: enabled. PostStart adds a hard-coded check n.cfg.Comment.Completion != "on_failure" that suppresses start comments regardless of the start setting. A user who sets start: enabled, completion: on_failure expecting start comments would not get them. The coupling is documented in the user-facing docs and the code comment explains the rationale (posting a start comment that gets deleted on success would trigger a notification pointing to a deleted comment), but the design still allows one config field to silently override another's explicit value.

Low

• [api-shape] internal/statuscomment/statuscomment.go:468 — ReconcileOrphaned now has 11 positional parameters (up from 9), adding completionMode and jobStatus. The codebase uses options struct patterns elsewhere (e.g., config.LoadOpts). Pre-existing concern worsened by two parameters.

• [error-handling] internal/cli/reconcilestatus.go:96 — Config loading warning omits fallback behavior context. When loading fails, completionMode defaults to empty string (which means enabled behavior). Adding "using default completion mode" to the warning would improve operator clarity.

• [naming-clarity] internal/config/config.go — The on_failure value describes when to post completion comments but also controls start comment behavior. The name doesn't hint at the broader suppression effect on start comments.

• [naming-conventions] internal/config/config.go:454 — Renaming validCommentValues to validStartValues and introducing validCompletionValues is reasonable given the now-different valid sets, but creates two arrays where the Start values are identical to the original shared array.

Previous run (15)

Review

Findings

Medium

• [edge-case] internal/statuscomment/statuscomment.go:504 — ReconcileOrphaned synthesizes an interrupted comment when completionMode == "on_failure" && jobStatus != "success", but there is no guard against jobStatus being an empty string. The --job-status flag is optional (not marked required), so omitting it produces jobStatus == "", which satisfies != "success" and triggers a spurious synthesized comment even though the job outcome is unknown. In the current action.yml, --job-status is always passed, so this path is not triggered in practice today — but the CLI contract allows omission.
  Remediation: Add an empty-string check: if completionMode == "on_failure" && jobStatus != "" && jobStatus != "success". Alternatively, make --job-status required when --fullsend-dir is provided.

Low

• [api-shape] internal/statuscomment/statuscomment.go:478 — ReconcileOrphaned now has 11 positional parameters (up from 9), adding completionMode and jobStatus. The codebase uses options struct patterns elsewhere (e.g., config.LoadOpts). Pre-existing concern worsened by two parameters.

Previous run (16)

Review

Findings

Medium

• [error-handling-idiom] internal/cli/reconcilestatus.go:91 — Config loading warning uses lowercase warning: prefix. The codebase convention in internal/cli uses uppercase WARNING: consistently across run.go, bootstrap_scan.go, mint.go, and repos.go (20+ instances). The lowercase prefix is inconsistent with the established pattern.
  Remediation: Change to uppercase: fmt.Fprintf(os.Stderr, "WARNING: could not load config from %s: %v\n", fullsendDir, err)

Low

• [scope-creep] internal/statuscomment/statuscomment.go:511 — Auto-suppression of start comment when completion is on_failure adds an implicit coupling between start and completion settings. When comment.completion is "on_failure", PostStart skips the start comment regardless of the start setting. This is a deliberate design choice by the feature author (who also authored the "independent and composable" requirement on Triage agent causes unnecessary notifications - should skip initial comment #3697), but documenting the coupling in the config reference would improve discoverability.

• [api-shape] internal/statuscomment/statuscomment.go:460 — ReconcileOrphaned's parameter list grew from 9 to 11 positional parameters (adding completionMode and jobStatus). The codebase uses options struct patterns for similar functions. Pre-existing concern incremented by two.

• [naming-consistency] internal/cli/reconcilestatus.go:90 — The local variable completionMode uses a Mode suffix not present in the config field Comment.Completion. The term "mode" is not used elsewhere for this concept.

Previous run (17)

Review

Findings

Critical

• [logic-error] internal/statuscomment/statuscomment.go:498 — ReconcileOrphaned synthesizes a false "Interrupted" comment on every successful run when completionMode is "on_failure". The flow: (1) PostStart is suppressed because completion == "on_failure", so no start comment marker exists. (2) The agent completes successfully; PostCompletionWithDetail suppresses the completion comment (shouldPostCompletion returns false for "success"). (3) The post-job reconcile step runs unconditionally (if: always()), reads the config, passes completionMode="on_failure", finds no marker comment, and creates a synthesized "Interrupted" comment. This directly undermines the noise-reduction intent — every successful on_failure run leaves a false "Terminated" comment on the issue/PR.
  Remediation: ReconcileOrphaned needs a way to distinguish "completed successfully, no comment needed" from "hard-killed before PostCompletion ran." Simplest option: have the action.yml post step check JOB_STATUS and skip the synthesis when the job succeeded (the shell already has $JOB_STATUS from job.status). Alternatively, have PostCompletionWithDetail write a lightweight sentinel (e.g., a terminal-tagged hidden comment) when it suppresses a completion comment.

Medium

• [error-handling-idiom] internal/cli/reconcilestatus.go:91 — Config loading errors are silently swallowed (if writer, err := ...; err == nil). A config parse error causes completionMode to default to empty string, silently disabling the on_failure synthesis path. This differs from the repo's error-handling idiom where non-fatal issues are at minimum warned about.
  Remediation: Log a warning to stderr when config loading fails (e.g., fmt.Fprintf(os.Stderr, "warning: could not load config from %s: %v\n", fullsendDir, err)).

• [missing-cli-flag] docs/guides/dev/cli-internals.md:151 — The CLI tree for reconcile-status lists all flags but is missing the new --fullsend-dir flag added in this PR.
  Remediation: Add --fullsend-dir to the reconcile-status flag list.

Low

• [edge-case] internal/statuscomment/statuscomment.go:131 — shouldPostCompletion includes "timeout" in the on_failure allowlist, but the user-facing documentation states completion is posted "only when the agent fails or is cancelled." Including timeout is defensible (it is failure-adjacent), but the documentation should mention it for consistency.

• [code-organization] internal/statuscomment/statuscomment.go:136 — PostStart now couples start notification logic to completion configuration via n.cfg.Comment.Completion != "on_failure". The doc comment explains the rationale and the coupling is intentional. Noting for context.

• [api-shape] internal/statuscomment/statuscomment.go:460 — ReconcileOrphaned's parameter list grew to 10 positional parameters. The codebase uses options struct patterns for similar functions (e.g., config.LoadOpts, mintclient.MintRequest). Pre-existing concern incremented by one.

• [naming-consistency] internal/statuscomment/statuscomment.go:460 — The new parameter is named completionMode while the config field is Comment.Completion. The term "mode" is not used elsewhere for this concept.

---

Next steps:

• /fs-fix — agent addresses review findings automatically
• /fs-fix <your instruction> — agent fixes with your specific guidance
• Push commits directly — review re-runs automatically on push
• /fs-fix-stop — disable automatic fix runs for this PR

Previous run (18)

Review

Findings

Medium

• [logic-error] internal/statuscomment/statuscomment.go:131 — shouldPostCompletion treats "skipped" as a non-success status, so on_failure mode will post a completion comment for skipped runs. The PR body says completion comments are posted "only on failure/cancellation — suppressed on success," and the user-facing documentation states on_failure will "post only when the agent fails or is cancelled." However, the code uses status != "success" which is a broader predicate that also matches "skipped." A skipped run is a benign pre-script outcome (e.g., the pre-script decided no work is needed), and posting a completion comment for it contradicts the noise-reduction intent. No test covers the on_failure + skipped combination.
  Remediation: Confirm design intent — should on_failure suppress skipped as well? If so, change the predicate to return status == "failure" || status == "cancelled" (allowlist of failure outcomes rather than blocklist of success). Add a test for on_failure + skipped to codify the decision either way.

Low

• [code-organization] internal/statuscomment/statuscomment.go — PostStart now couples start notification logic to completion configuration via n.cfg.Comment.Completion != "on_failure". The doc comment explains the rationale (posting a start comment that gets deleted on success would still trigger a GitHub notification pointing to a deleted comment), and the coupling is intentional. No action needed — noting for context.

Previous run (19)

Review

Findings

Medium

• [logic-error] internal/statuscomment/statuscomment.go:131 — shouldPostCompletion uses status != "success" as the on_failure predicate, but the documentation states on_failure posts "only when the agent fails or is cancelled." The "skipped" status (set in internal/cli/run.go:680 when a pre-script decides no work is needed) would trigger a completion comment under on_failure mode. A skipped run is a benign outcome — posting a completion comment for it contradicts the noise-reduction intent. No test covers the on_failure + skipped combination.

Low

• [scope-authorization-mismatch] internal/statuscomment/statuscomment.go:138 — Issue Triage agent causes unnecessary notifications - should skip initial comment #3697 requests skipping start comments to reduce notifications. This PR implements on_failure mode for completion comments as Phase 1, coupling start comment suppression to completion mode. The implementation is internally consistent and the phased approach is explicitly described, but the issue author (deboer-tim) differs from the PR author (ralphbean); confirmation that this approach meets the requirement would validate scope alignment.

Previous run (20)

Review

Findings

Low

• [scope-authorization-mismatch] internal/config/config.go:115 — The PR implements on_failure mode for completion comments as Phase 1 of Triage agent causes unnecessary notifications - should skip initial comment #3697, rather than the start-comment-skip described in the issue. The PR author and issue author are the same person, and the PR description explicitly frames this as a phased approach. The implementation is internally consistent and correct; the question is purely about project planning priorities.

Previous run (21)

Review

Findings

Low

• [technical documentation accuracy] internal/config/config.go:115 — The doc comment on CommentNotificationConfig states "Valid values: "enabled" (default when parent is set), "disabled"" but the Completion field now also accepts "on_failure". The doc comment should reflect per-field valid values.

• [stale-documentation] docs/guides/getting-started/operations.md:159 — The status_notifications example in operations.md only lists "enabled" and "disabled" for completion, missing the new "on_failure" value. Consider updating the example or redirecting to the comprehensive docs in customizing-agents.md (matching the pattern used in the running-agents-locally.md update).

• [documentation-style] internal/statuscomment/statuscomment.go:126 — The shouldPostCompletion function has a doc comment while the similar helper commentEnabled does not. Minor style inconsistency in doc comment coverage for unexported boolean helpers.

---

Labels: PR adds a new user-facing config option (on_failure mode) with documentation updates

[qodo-code-review]

PR Summary by Qodo

Add on_failure mode for completion status comments

✨ Enhancement 🧪 Tests 📝 Documentation 🕐 20-40 Minutes

AI Description

• Add on_failure option to completion status notifications to reduce success noise.
• Suppress completion comments on success while cleaning up the start comment.
• Extend config validation and add unit tests for new completion behavior.

Diagram

graph TD
  A[/"config.yaml"/] --> B["Config validation"] --> C["Notifier.PostCompletion"] --> D{"Completion mode?"}
  D -->|"enabled / on_failure+non-success"| E["Forge client"] --> F["Issue/PR comment"]
  D -->|"disabled / on_failure+success"| G["Delete start comment"] --> E
  subgraph Legend
    direction LR
    _io[/"Config input"/] ~~~ _svc["Component"] ~~~ _dec{"Decision"}
  end

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Introduce structured enum/typed mode instead of string literals

• ➕ Avoids scattering string values like "on_failure"/"enabled" across code
• ➕ Makes future expansion (e.g., on_success, on_cancelled) safer and more discoverable
• ➖ Requires broader refactor across config parsing/serialization and existing APIs
• ➖ Higher migration/compatibility surface for a small behavioral addition

2. Generalize to a single notification policy function for start+completion

• ➕ Centralizes all status-notification logic (start/completion) in one place
• ➕ Easier to enforce consistent semantics across notification types
• ➖ More refactoring than needed for a phase-1 feature
• ➖ Risk of behavior changes to existing start-comment logic

Recommendation: Current approach is appropriate for a targeted feature: extend completion-only validation, add a small predicate helper (shouldPostCompletion), and keep behavior localized to PostCompletion with strong unit test coverage. If notification modes expand further (e.g., reaction support or more outcome-based rules), consider migrating to a typed enum/policy layer to reduce stringly-typed branching.

Files changed (6) +201 / -8

Enhancement (2) +20 / -7

config.goAccept on_failure for comment.completion and reject it for comment.start +5/-4

Accept on_failure for comment.completion and reject it for comment.start

• Splits validation into start vs completion allowed values. Extends completion validation to allow 'on_failure' while keeping start restricted to enabled/disabled.

internal/config/config.go

statuscomment.goGate completion comment posting with on_failure-aware predicate +15/-3

Gate completion comment posting with on_failure-aware predicate

• Introduces 'shouldPostCompletion(val, status)' to interpret 'on_failure' as "post unless success". Updates 'PostCompletion' to suppress completion comments when appropriate and clean up the start comment to avoid leaving an orphaned "Started" comment.

internal/statuscomment/statuscomment.go

Tests (2) +156 / -0

config_test.goAdd config validation/parsing tests for on_failure completion +55/-0

Add config validation/parsing tests for on_failure completion

• Adds unit tests verifying 'on_failure' is accepted for 'status_notifications.comment.completion' and rejected for '.start'. Includes a YAML parse test to ensure the value round-trips through config parsing.

internal/config/config_test.go

statuscomment_test.goAdd notifier tests covering on_failure success/failure/cancelled outcomes +101/-0

Add notifier tests covering on_failure success/failure/cancelled outcomes

• Adds unit tests ensuring 'on_failure' suppresses completion on success (and deletes the start comment), while still posting completion on failure and cancellation. Also covers the case where start comments are disabled and completion is on_failure.

internal/statuscomment/statuscomment_test.go

Documentation (2) +25 / -1

customizing-agents.mdDocument Status Notifications and add on_failure completion mode +24/-0

Document Status Notifications and add on_failure completion mode

• Adds a new user-facing "Status Notifications" section describing 'status_notifications.comment.start' and '.completion'. Documents 'on_failure' behavior (post only on failure/cancellation; delete start comment on success) alongside enabled/disabled modes.

docs/guides/user/customizing-agents.md

running-agents-locally.mdUpdate local-running guide to link to new Status Notifications docs +1/-1

Update local-running guide to link to new Status Notifications docs

• Repoints the status notification reference from the operations guide to the new section in 'customizing-agents.md'. Keeps guidance user-focused where configuration is described.

docs/guides/user/running-agents-locally.md

[fullsend-ai-review]

🤖 Finished Review · ❌ Failure · Started 6:03 PM UTC · Completed 6:04 PM UTC
Commit: 2829847 · View workflow run →

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (0)

1. Docs imply per-repo support ✓ Resolved 🐞 Bug ≡ Correctness

Description

Docs state status_notifications in config.yaml controls status comments, but per-repo configs do
not have a status_notifications field and YAML unmarshalling ignores unknown fields, so
configuring it in per-repo mode will have no effect.

Code

docs/guides/user/customizing-agents.md[R499-507]

+Agent workflows post status comments on issues and PRs when they start and complete. Control this with `status_notifications` in `config.yaml`:
+
+```yaml
+defaults:
+  status_notifications:
+    comment:
+      start: enabled      # "enabled" (default) | "disabled"
+      completion: enabled  # "enabled" (default) | "on_failure" | "disabled"
+```

Relevance

●●● Strong

Team often fixes doc/behavior mismatches to avoid misleading operators, especially per-repo vs
org-mode details.

PR-#5454

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The docs claim the setting applies to config.yaml generally, but the code only defines and reads
it for org-mode; per-repo parsing uses yaml.Unmarshal into a struct without that field so the key
is ignored.

docs/guides/user/customizing-agents.md[497-520]
internal/config/config.go[108-129]
internal/config/config.go[493-573]
internal/cli/run.go[2893-2904]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The docs introduce `status_notifications` without clarifying that it is only supported in org-mode configuration. In per-repo mode, the field is not part of the schema/struct and will be silently ignored, so users can apply the documented config and see no change.

### Issue Context
- Org-mode carries `defaults.status_notifications`.
- Per-repo config struct has no `StatusNotifications` field.
- YAML parsing uses `yaml.Unmarshal` (non-strict), so unknown keys are ignored.
- The CLI only reads `StatusNotifications()` when the loaded config implements `OrgConfigReader`.

### Fix Focus Areas
- docs/guides/user/customizing-agents.md[497-520]
- internal/config/config.go[108-129]
- internal/config/config.go[493-573]
- internal/cli/run.go[2893-2904]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Misleading cleanup warning ✓ Resolved 🐞 Bug ◔ Observability

Description

PostCompletion now suppresses completion comments for on_failure+success, but cleanup warnings
still claim completion was “disabled”, which will mislead debugging when start-comment deletion
fails under on_failure suppression.

Code

internal/statuscomment/statuscomment.go[R172-178]

+	if !shouldPostCompletion(n.cfg.Comment.Completion, status) {
+		// Completion comment suppressed (disabled or on_failure with success) —
+		// clean up the start comment so it doesn't remain orphaned in its
+		// "Started" state.
		if n.startCommentID != 0 {
			if err := n.refreshClient(ctx); err != nil {
				n.warnf("failed to mint token for start comment cleanup: %v", err)

Relevance

●●● Strong

They’ve accepted aligning warnings/messages with actual behavior to prevent misleading debugging
output.

PR-#697

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The new helper suppresses completion for on_failure when status == "success", routing through
the same cleanup block that logs "when completion disabled" warnings on errors.

internal/statuscomment/statuscomment.go[126-184]
internal/cli/run.go[665-676]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
When completion comments are suppressed (either `disabled` or `on_failure` on `success`), the start-comment cleanup path can emit warnings. Those warnings currently hardcode wording for the `disabled` case, which is now inaccurate for the new `on_failure` suppression path.

### Issue Context
This affects only warning text (emitted on error), but it is triggered by the new `on_failure` behavior added in this PR.

### Fix Focus Areas
- internal/statuscomment/statuscomment.go[126-184]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Outdated config GoDoc ✓ Resolved 🐞 Bug ⚙ Maintainability

Description

The GoDoc for CommentNotificationConfig still describes only enabled/disabled, but this PR
adds on_failure as a valid value for comment.completion, leaving the type documentation
incorrect.

Code

internal/config/config.go[R457-459]

+	validCompletionValues := []string{"", "enabled", "disabled", "on_failure"}
+	if !slices.Contains(validCompletionValues, cfg.Comment.Completion) {
+		return fmt.Errorf("invalid status_notifications.comment.completion %q: must be \"enabled\", \"on_failure\", or \"disabled\"", cfg.Comment.Completion)

Relevance

●●● Strong

They routinely update GoDoc/comments when semantics change to keep config types accurate.

PR-#5625

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The validation change explicitly allows on_failure, while the nearby type comment still lists only
the old values.

internal/config/config.go[108-119]
internal/config/config.go[449-461]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`validateStatusNotifications` now accepts `on_failure` for `comment.completion`, but the type-level comment on `CommentNotificationConfig` hasn’t been updated and still implies only `enabled`/`disabled` are valid.

### Issue Context
This is a documentation/maintainability issue that can mislead future maintainers and any generated/internal docs.

### Fix Focus Areas
- internal/config/config.go[114-119]
- internal/config/config.go[449-461]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Context

✅ Compliance rules (platform): 54 rules

✅ Skills: writing-user-docs, writing-adrs

Tip of the day

💡 Did you know, you can group findings by type and pick your Finding display, from Minimal to Full

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

[waynesun09]

Review Squad findings (medium+ severity)

Posting the medium+ findings from a 4-agent review pass (claude-coder, claude-researcher, grok-review-agent, gemini-code-review). Two additional medium findings couldn't be attached inline since they're in files outside this PR's diff:

[MEDIUM] operations.md still documents the old enabled/disabled-only completion values
docs/guides/getting-started/operations.md:150-162 still shows completion: enabled # "enabled" (default) | "disabled", unchanged by this PR — no mention of on_failure. Meanwhile running-agents-locally.md's cross-reference was repointed to the new customizing-agents.md#status-notifications section, leaving two docs pages disagreeing about valid values for the same key. Suggest replacing operations.md's section with a pointer to the new canonical location instead of maintaining two copies.

[MEDIUM] reconcile-status/ReconcileOrphaned ignores status_notifications entirely
internal/cli/reconcilestatus.go and internal/statuscomment/statuscomment.go's ReconcileOrphaned take no config input and always finalize an orphaned marker to "Interrupted" regardless of completion's configured value. This is pre-existing behavior, but this PR turns "silent on success" into a documented guarantee without addressing the narrow race where a process succeeds but is hard-killed before its deferred PostCompletion runs — the reconciler would still surface a false "Interrupted" notice under on_failure. Worth a doc note or a test capturing this known interaction.

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 8:48 PM UTC · Ended 8:49 PM UTC
Commit: f832f7a · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 8:50 PM UTC · Completed 9:04 PM UTC
Commit: 1999cd4 · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 9:38 PM UTC · Completed 9:55 PM UTC
Commit: 6131423 · View workflow run →

[waynesun09]

[HIGH] Required behaviour CI check is red at current HEAD — PR is currently BLOCKED

(Noting this in the review body rather than as an inline comment since the referenced test file, e2e/behaviour/features/dispatch/url-dispatch.feature:26, is not part of this PR's diff.)

The behaviour required check is failing on the PR's current HEAD (6131423) and mergeStateStatus is BLOCKED. This isn't a stale/cached result: commit 6131423 ("fix(#3697): auto-suppress start comment when completion is on_failure") triggered a fresh CI run (run 30493021917, job 90715292701) that ran to completion and failed again — this time with 4 scenarios failing (pr-ping, fork-pr-sync, url-ping, enabled-ping, all failing with a generic "did not complete successfully" error, clustered within a ~26s window around the ~13–14 minute mark of the run) versus only 1 scenario failing on the prior commit's run (1999cd47, run 30489904095).

The changing failure count/set across consecutive runs of the same suite is a strong flake/infrastructure signature (e.g. rate-limiting, a stuck dispatch queue, or the suite's own time-boxed guard pending the harness CEL cutover) rather than a deterministic regression — this PR's diff (docs, internal/config/config.go, internal/statuscomment/statuscomment.go + tests) has zero file overlap with e2e/behaviour or dispatch/harness-resolution code in either commit, which reinforces that this is very unlikely to be caused by this change.

Regardless of root cause: this is a currently-red required check on a PR that GitHub reports as BLOCKED, and it hasn't been raised elsewhere in this PR's comment/review history yet.

Suggestion: Don't merge on a red required check. Re-run behaviour once more (or loop in CI/infra to rule out GitHub API rate-limiting or runner contention) — given the failing scenario set changed between runs and now spans 4 unrelated harness scenarios simultaneously, this looks like shared test-infrastructure flakiness rather than something to fix in this PR's code, but it needs an explicit green run (or a documented infra ticket) before merging.

Superseded by updated review

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 8:45 PM UTC · Completed 8:59 PM UTC

Commit: 46c5d58 · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 9:06 PM UTC · Completed 9:18 PM UTC

Commit: 9199342 · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 2:29 PM UTC · Completed 2:45 PM UTC

Commit: fa6d15a · View workflow run →

[fullsend-ai-retro]

🤖 Finished Retro · ✅ Success · Started 3:34 PM UTC · Completed 4:31 PM UTC

Commit: fa6d15a · View workflow run →

[fullsend-ai-retro]

Retro: PR #5736 — on_failure mode for completion status notifications

Workflow Overview

PR #5736 (Phase 1 of #3697) adds an on_failure completion mode for status notification comments. Authored by ralphbean (human), it touched 11 files (+853/−67) across action.yml, internal/statuscomment/, internal/cli/, internal/config/, and docs. The feature required 22 commits over 13 days (Jul 29 – Aug 11), 22 review agent runs, 2 fix agent runs (both failed), 7 rounds of human review by waynesun09, and 3 merge queue attempts before merging.

Review Quality: Significant Gap Between Agent and Human

The review agent found 21 distinct findings (after deduplication across 21 runs), including 1 critical bug (false "Interrupted" comments on successful on_failure runs). The human reviewer found 17 unique findings the review agent never identified, including 3 critical/high bugs that drove major architectural changes:

1. Start comment deletion doesn't undo the GitHub notification — led to auto-suppressing the start comment entirely under on_failure (commit 75a56e1).
2. on_failure defeats the hard-kill orphan-reconciliation safety net — removing start comments breaks ReconcileOrphaned's assumption that a marker always exists. Led to building the entire reconciliation synthesis infrastructure.
3. jobStatus==success masks failed skip-comment post — indistinguishable from "nothing to report". Led to plumbing --was-skipped through action.yml.

The review agent's critical finding (R3) was valuable and triggered a fix agent commit before the human reviewed. But the human subsequently discovered a deeper root cause (the invariant violation in ReconcileOrphaned) that the agent's shallow jobStatus-guard patch didn't address. The human reviewer had zero false positives; the review agent had 2–3.

The gap is primarily in cross-component chain-of-consequence reasoning: the human traced behavior chains across action.yml → reconcilestatus.go → statuscomment.go → GitHub notification semantics, while the review agent analyzed each file/function independently.

Fix Agent: Wasted Runs

Both fix agent runs (31051443741, 31189937381) failed at the "Pre-fetch review body" step with: "Bot-triggered run but review body is empty — nothing to fix." The shim dispatched fix runs on pull_request_review.submitted events without filtering on review state. By the time the fix agent ran, no CHANGES_REQUESTED review existed (the triggering review was either APPROVED/COMMENTED, or a prior CHANGES_REQUESTED review had been dismissed).

Token Cost

22 review agent runs re-raised the same 5–6 findings across iterations without incorporating context from the human reviewer's responses or the author's explanations.

Merge Queue

Three merge queue attempts were needed due to flaky E2E/behaviour tests (missing workflow file in test org halfsend-07, cancelled harness runs, flaky TestAdminInstallUninstall assertion). All failures were infrastructure-related.

Evidence for Existing Open Issues

• #1556 (review agent misses mode-specific behavioral implications): Strong evidence — the review agent missed that on_failure mode's start-comment suppression invalidated ReconcileOrphaned's core invariant, and that deleting start comments doesn't undo GitHub notifications.
• #1525 (review agent misses cross-file impact analysis): Strong evidence — the human traced chains across 4+ components; the review agent stayed within single files.
• agents#653 (correctness sub-agent should trace error/exception paths): The review agent found the symptom (false Interrupted comments) but not the root invariant violation in ReconcileOrphaned's marker-search logic.
• #5967 (filter fix dispatch to CHANGES_REQUESTED state): Both fix runs were wasted because the shim dispatched on non-CHANGES_REQUESTED review events.
• #5863 (post comment when fix agent guard-rail blocks): Both fix failures were silent — no PR comment was posted.
• #2959 / #1013 (review finding dedup across iterations): 21 runs with high repetition of the same findings.
• #4970 (detect all findings in a single pass): Many of the human's 17 unique findings were discoverable from the initial diff through first-principles analysis.

Autonomy Readiness

The review agent is not ready for autonomous merge authority on complex multi-component features. The human reviewer's 17 unique findings — especially the 3 critical/high bugs driving architectural changes — demonstrate that cross-component reasoning and mode-specific invariant analysis remain significant gaps. The review agent performs well on surface-level code pattern matching (naming, docs, simple predicate bugs) but cannot yet trace cascading behavioral effects across components.

Proposals filed

• Review agent should flag premature issue-closing keywords when PR describes partial work (in fullsend-ai/agents)