Stabilize flaky pro CSV-import e2e test - #806
Merged
Merged
Conversation
pro.spec.ts:118 ("Create chart from imported data") flaked ~4x across recent
PRs, always in the import->confirm flow against the Vercel preview: sometimes
the "9 nodes and 10 edges" confirmation didn't appear within the 5s default
(line 158), sometimes the confirm click hung to the 120s test timeout (line
160). Both are preview-timing issues, not product bugs.
- Bump the confirmation toBeVisible timeout to 30s and wait for the confirm
button to be actionable before clicking.
- Add CI retries (retries: isCI ? 2 : 0) — the standard mitigation for e2e
against a live preview. Each failure was a single-attempt flake, so a retry
absorbs it; a real break still fails every attempt. Local retries stay off
so flakes remain visible while developing.
Note: can't be proven in one run (it only flakes against CI/preview, passes
locally); verification is a reduced flake rate over subsequent CI runs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
pro.spec.ts:118("Create chart from imported data") has flaked ~4x across recent PRs (#804, #805), always in the CSV import → confirm flow against the Vercel preview — never a product bug:Fix
toBeVisibleto 30s (import parsing is slow against the preview) and wait for the confirm button to be actionable before clicking (so it can't hang to the test timeout).retries: isCI ? 2 : 0) — the standard mitigation for e2e against a live preview. Every failure was a single-attempt flake, so a retry absorbs it; a real break still fails all attempts and stays red. Local retries stay off so flakes remain visible while developing.Honest caveat
This can't be proven in a single run — the test only flakes against CI/preview and passes locally. Verification is a reduced flake rate over subsequent CI runs. Retries alone would have turned all 4 recent failures green.
This is a pragmatic stabilizer; the deeper fix (less preview-timing-dependent e2e) comes with the eventual CRA→Vite/framework migration.
🤖 Generated with Claude Code