test(attribution): fix order-dependent CI flake in getAttributionTexts#1435
Open
0xfandom wants to merge 1 commit into
Open
test(attribution): fix order-dependent CI flake in getAttributionTexts#14350xfandom wants to merge 1 commit into
0xfandom wants to merge 1 commit into
Conversation
…er env `getAttributionTexts` resolves the active provider and model through `getAPIProvider` (→ `resolveActiveRouteIdFromEnv`) and `getMainLoopModel`, both of which read process env. The suite's `beforeEach` pinned `CLAUDE_CODE_USE_OPENAI=1` + `OPENAI_MODEL=gpt-5.5` but left the competing route-selection flags untouched. Sibling test files (gemini / mistral / foundry / codex shim, provider config) set those flags and an incomplete restore there flips our resolved provider under interleaved `bun test` ordering: the route becomes gemini / mistral / foundry, the co-author name changes, and preserves includeCoAuthoredBy true as an explicit old-default opt-in fails with a mismatched/empty commit trailer. The failure is order-dependent, which is why it surfaced intermittently across unrelated PR branches. Capture and clear the routing env (`CLAUDE_CODE_USE_GEMINI` / `_MISTRAL` / `_GITHUB` / `_BEDROCK` / `_VERTEX` / `_FOUNDRY`, `NVIDIA_NIM`, `OPENAI_BASE_URL` / `OPENAI_API_BASE`, `CLAUDE_CODE_PROVIDER_PROFILE_ENV_APPLIED`, and the alternate `*_MODEL` vars) in `beforeEach`, restore them in `afterEach`. The route now resolves deterministically to plain OpenAI + gpt-5.5 regardless of run order. Verified by reproducing the failure (`CLAUDE_CODE_USE_GEMINI=1`, `CLAUDE_CODE_USE_MISTRAL=1`, `CLAUDE_CODE_USE_FOUNDRY=1` each made the suite fail before the change) and confirming all 20 tests pass with each of those env vars set after it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
src/utils/attribution.test.tshas an order-dependent failure that has been intermittently red-lightingsmoke-and-testsacross unrelated PR branches:Root cause
getAttributionTexts()resolves the active provider + model throughgetAPIProvider()(→resolveActiveRouteIdFromEnv) andgetMainLoopModel(), both of which readprocess.env. The suite'sbeforeEachpinnedCLAUDE_CODE_USE_OPENAI=1+OPENAI_MODEL=gpt-5.5but left the competing route-selection flags untouched. Sibling test files (gemini / mistral / foundry / codex shim, provider-config) set those flags, and an incomplete env restore there leaks into this suite under interleavedbun testfile ordering — the route resolves to gemini/mistral/foundry instead of openai, the co-author name changes, and theCo-Authored-By: OpenClaude (gpt-5.5) …expectation fails.Because it depends on which file ran immediately before, it surfaced on some PR CI runs and not others (same commit base).
Fix
Capture and clear the full set of routing env vars in
beforeEach(CLAUDE_CODE_USE_GEMINI/_MISTRAL/_GITHUB/_BEDROCK/_VERTEX/_FOUNDRY,NVIDIA_NIM,OPENAI_BASE_URL/OPENAI_API_BASE,CLAUDE_CODE_PROVIDER_PROFILE_ENV_APPLIED, alternate*_MODEL), restore inafterEach. The route now resolves deterministically to plain OpenAI + gpt-5.5 regardless of run order. No production code changes.Test plan
Reproduced + verified each leaker:
CLAUDE_CODE_USE_GEMINI=1CLAUDE_CODE_USE_MISTRAL=1CLAUDE_CODE_USE_FOUNDRY=1bun test src/utils/attribution.test.ts— 20/20 pass clean and under each leaked-env scenario above.