Skip to content

fix(#6678): preserve per-repo config on re-onboarding - #6679

Closed
fullsend-ai-coder[bot] wants to merge 1 commit into
mainfrom
agent/6678-preserve-config-on-reonboard
Closed

fix(#6678): preserve per-repo config on re-onboarding#6679
fullsend-ai-coder[bot] wants to merge 1 commit into
mainfrom
agent/6678-preserve-config-on-reonboard

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

Summary

Preserve existing per-repo config (custom agents, roles, runtime, per-agent settings) when re-onboarding a project via admin install. Previously, BuildScaffoldFiles always generated a fresh config.yaml when PerRepoConfig was nil, silently overwriting any user customizations.

Changes

  • Read the existing .fullsend/config.yaml via loadExistingPerRepoConfig when the repo is already installed, reusing the same helper the github setup path already uses
  • Pass the existing config as PerRepoConfig in both the dry-run and live install paths so BuildScaffoldFiles preserves it instead of generating defaults
  • Apply --runtime flag on top of the existing config when explicitly provided
  • Gracefully degrade to generating a fresh config if the existing one cannot be read (warns instead of failing)
  • Add two tests: one verifying full config preservation, one verifying --runtime override with preservation

Testing

  • TestRunPerRepoInstall_AlreadyInstalled_PreservesExistingConfig — verifies custom agents, roles, runtime, and per-agent settings survive re-onboarding
  • TestRunPerRepoInstall_AlreadyInstalled_RuntimeOverride — verifies --runtime updates only the runtime while keeping everything else
  • All existing TestRunPerRepoInstall_* tests pass
  • All existing TestRunGitHubSetup* tests pass

Closes #6678

Post-script verification

  • Branch is not main/master (agent/6678-preserve-config-on-reonboard)
  • Secret scan passed (gitleaks — 73ce2b2cf5f046fd554a8c29698e7d53dc7173de..HEAD)
  • PR body secret scan passed (gitleaks — no-git)

BuildScaffoldFiles always generated a fresh config.yaml via
NewPerRepoConfig when PerRepoConfig was nil, which is the
standard admin install path. Re-onboarding a project that
already had custom agent configurations (agents, roles,
per-agent settings) would overwrite them with defaults.

Read the existing .fullsend/config.yaml via
loadExistingPerRepoConfig (the same helper the github setup
path uses) when the repo is already installed, and pass it
through to BuildScaffoldFiles as PerRepoConfig. The --runtime
flag still applies on top when explicitly provided. Both the
dry-run and live install paths are covered.

Closes #6678
@fullsend-ai-coder
fullsend-ai-coder Bot requested a review from a team as a code owner August 27, 2026 13:25
@fullsend-ai-coder fullsend-ai-coder Bot added the ready-for-review Agent PR ready for human review label Aug 27, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 27, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 1:27 PM UTC · Completed 1:45 PM UTC

Commit: ac5576c · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $5.89

@codecov

codecov Bot commented Aug 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.00000% with 3 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
internal/cli/admin.go 75.00% 1 Missing and 2 partials ⚠️

📢 Thoughts on this report? Let us know!

@fullsend-ai-review fullsend-ai-review Bot added the risk/moderate PR risk: moderate label Aug 27, 2026
@fullsend-ai-review

Copy link
Copy Markdown

Risk Assessment: moderate (2/5)

Details

Small, well-tested bot-authored bug fix with excellent test coverage ratio (0.50), but the target file (admin.go) shows high churn (18 commits/30d), broad author diversity (9 authors/90d), and extensive regression history (49 fix/revert commits/90d), elevating overall risk to moderate.

@fullsend-ai-review

Copy link
Copy Markdown

Review

Findings

Medium

  • [error-handling idiom divergence] internal/cli/admin.go:749 — The established pattern in github.go treats loadExistingPerRepoConfig errors as hard failures in non-dry-run mode, only downgrading to a warning for dry runs. The new code unconditionally downgrades the error to StepWarn and falls through to regenerate a fresh config. This means a parse error on a corrupted/invalid config.yaml — which loadExistingPerRepoConfig intentionally returns to prevent silent overwrite (per its doc comment: “a re-run must not silently regenerate over a file the repo edited”) — is swallowed, and the admin install path will overwrite the repo’s config with scaffold defaults.
    Remediation: Consider matching the github.go error-handling idiom — return the error in non-dry-run mode so invalid configs are not silently overwritten; in dry-run mode, downgrade to a warning.

Low

  • [naming convention] internal/cli/admin.go:744 — The variable is named existingPerRepoCfg, while the established name in github.go for the same type (config.PerRepoConfigWriter) returned from the same function (loadExistingPerRepoConfig) is existingCfg. Consider renaming for consistency across call sites.

@fullsend-ai-review fullsend-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the review comment for full details.

Comment thread internal/cli/admin.go
var existingPerRepoCfg config.PerRepoConfigWriter
if alreadyInstalled {
cfg, cfgErr := loadExistingPerRepoConfig(ctx, client, owner, repo)
if cfgErr != nil {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[medium] error-handling idiom divergence

The established pattern in github.go treats loadExistingPerRepoConfig errors as hard failures in non-dry-run mode, only downgrading to a warning for dry runs. The new code unconditionally downgrades the error to StepWarn and falls through to regenerate a fresh config. This means a parse error on a corrupted/invalid config.yaml — which loadExistingPerRepoConfig intentionally returns to prevent silent overwrite (per its doc comment: 'a re-run must not silently regenerate over a file the repo edited') — is swallowed, and the admin install path will overwrite the repo's config with scaffold defaults.

Suggested fix: Consider matching the github.go error-handling idiom: return the error in non-dry-run mode so invalid configs are not silently overwritten; in dry-run mode, downgrade to a warning.

Comment thread internal/cli/admin.go
// configurations (agents, roles, runtime, etc.) are not overwritten
// with scaffold defaults (#6678). When the repo already has a
// config.yaml, load it and pass it through to BuildScaffoldFiles;
// only apply the --runtime flag on top when it was explicitly

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] naming convention

The variable is named existingPerRepoCfg, while the established name in github.go for the same type (config.PerRepoConfigWriter) returned from the same function (loadExistingPerRepoConfig) is existingCfg. Consider renaming for consistency across call sites.

@fullsend-ai-review fullsend-ai-review Bot added the requires-manual-review Review requires human judgment label Aug 27, 2026
@Roming22

Roming22 commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator

@rh-hemartin The approach seems to be too simplistic. What if a new version of fullsend removes an agent? I'm wondering if we should store the default config separately from the users customizations, as it would make the upgrade process easier to deal with. WDYT?

@Roming22

Copy link
Copy Markdown
Collaborator

Closing in favor of #6809

@Roming22 Roming22 closed this Aug 31, 2026
@Roming22
Roming22 deleted the agent/6678-preserve-config-on-reonboard branch August 31, 2026 16:37
@fullsend-ai-retro

fullsend-ai-retro Bot commented Aug 31, 2026

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 4:38 PM UTC · Completed 4:53 PM UTC

Commit: ac5576c · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $5.59

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #6679 — preserve per-repo config on re-onboarding (closed, not merged)

Timeline

  1. 2026-08-27 12:55 — Issue #6678 filed: re-onboarding with fullsend v0.37 destroys custom agent configuration in .fullsend/config.yaml.
  2. 2026-08-27 13:00 — Human runs /fs-triage. Triage agent (run 33074639685) applies ready-to-code in ~8 minutes.
  3. 2026-08-27 13:08–13:25 — Code agent (run 33075303479) produces PR #6679 in ~17 minutes. Approach: load the existing config.yaml wholesale via loadExistingPerRepoConfig and pass it through to BuildScaffoldFiles, preserving it verbatim.
  4. 2026-08-27 13:25–13:45 — Review agent (run 33076737661, $5.89) posts two findings: a medium-severity error-handling idiom divergence (the new code silently downgrades config-read errors to warnings, diverging from the hard-failure pattern in github.go) and a low-severity naming inconsistency. Verdict: COMMENTED.
  5. 2026-08-27 13:28 — Codecov reports 75% patch coverage (below 80% threshold).
  6. 2026-08-27 15:12 — Human reviewer (Roming22) flags the fundamental architectural flaw: wholesale config preservation doesn't handle fullsend version upgrades that add or remove agents. Suggests separating scaffold defaults from user customizations.
  7. 2026-08-31 — Human updates issue Re-onboarding with fullsend v0.37 removes custom agent configuration #6678 with detailed architectural specification for a layered config approach (base + overlay, per ADR-0069). Re-invokes /fs-triage and /fs-code.
  8. 2026-08-31 16:37 — PR fix(#6678): preserve per-repo config on re-onboarding #6679 closed in favor of PR #6809, which implements the layered config split (+320/-5 across 4 files, 94.87% patch coverage).

What went well

  • Review agent caught a real bug. The error-handling divergence finding was accurate and important — the code agent's fix would have silently overwritten corrupted configs, which is the same class of bug the PR was trying to fix.
  • Code agent produced clean, tested code quickly. 17-minute turnaround with 2 well-structured tests using existing patterns. The implementation itself was correct for the approach chosen.
  • Human-agent iteration worked. After the human provided architectural guidance in the issue body, the second code agent attempt (PR fix(#6678): split scaffold config into base layer and user overlay #6809) implemented the correct layered approach. The pipeline's ability to re-trigger with better context is functioning as designed.

What could go better

  • Triage agent applied ready-to-code without recognizing design complexity. The bug fix required understanding the three-tier config architecture (ADR-0069) and making a design decision about how to handle version upgrades. The triage agent assessed this in 8 minutes and sent it straight to coding. Evidence for #6183 (implement needs-design label in triage pipeline).
  • Code agent didn't consult relevant ADRs. The code agent read github.go extensively and found loadExistingPerRepoConfig, but never followed the ADR references in that file to understand the layered config architecture (ADR-0069). If it had, it might have recognized that wholesale preservation was insufficient and the layered approach was already documented. Evidence for #3158 (code agent should check relevant ADRs during planning phase).
  • Code agent didn't articulate alternatives or trade-offs. It converged on the simplest approach without considering upgrade semantics (what happens when agents are added/removed between versions). The PR description and commit message describe what was done but not what was considered and rejected. Evidence for #2185 (code agent should follow triage's recommended implementation strategy).
  • Review agent missed the design-level gap. The review agent correctly identified a code-level error-handling bug but did not flag that the wholesale preservation approach would break on version upgrades — the exact concern the human reviewer raised. Evidence for #2325 (review agent should escalate architectural-coherence findings when a design-level alternative exists).
  • Review agent verdict didn't match finding severity. The medium-severity error-handling divergence — which would silently overwrite user data, the very problem the PR aimed to fix — was posted as COMMENTED rather than CHANGES_REQUESTED. Evidence for #4415 (review agent should escalate verdict for findings requiring major rework).

Waste quantification

The rejected PR cost approximately $10–12 in agent compute (triage + code + review) plus human review time. This was avoidable if the triage agent had flagged the need for design input before coding.

E2E test flakiness (unrelated)

The E2E behaviour tests failed on the PR branch (run 33076736829) due to infrastructure flakiness — harness agent workflows in test repos failed before producing artifacts. The same pattern appeared on main and multiple unrelated branches on the same day. Not caused by this PR's changes.

Agents repo

Both code and review agents resolved from fullsend-ai/agents@main (commit 19f8eaba9d48).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-review Agent PR ready for human review requires-manual-review Review requires human judgment risk/moderate PR risk: moderate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Re-onboarding with fullsend v0.37 removes custom agent configuration

1 participant