fix: deflake CI test runs - #61
Merged
Merged
Conversation
Fixes the four recurring failure classes from the last month of "Run
tests" workflow history:
- Seeded active rule now wraps past midnight instead of clamping to
23:59. The clamp left <15 minutes on the block near UTC midnight, so
RulePolicy.canPause hid the pause button and the pause UI tests failed
in every run between ~23:44Z and midnight. Covered by a parameterized
regression test (fails on the old clamp at 23:50/23:58).
- setTextVerified replaces blind typeText where a typed value is later
asserted: CI runs showed dropped keystrokes ("My", "My Foc" for
"My Focus"). Types, verifies the field value, and retries.
- tap(untilAppears:) re-taps navigation rows against a postcondition,
generalizing the retap loop goToSection already used; applied to the
AppListUITests sites that flaked on dropped taps.
- New "Ensure simulator exists" workflow step: some macos-26 images
intermittently lack the named simulator (3 jobs since 07-21 died in
~80s with "Unable to find a device"); the step waits, creates the
device if needed, and dumps diagnostics instead of failing cryptically.
The name match is anchored on " (" so "iPhone 17 Pro Max" can't
satisfy a check for "iPhone 17 Pro".
- Raise timeout-minutes 45 -> 60 for both device legs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y9652c13kpQrUyj7CZrHZg
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Diagnosed the last month of "Run tests" failures (24 failed/cancelled runs) and fixed each recurring failure class at its root cause, rather than re-running:
RulePolicy.canPause's 15-minute floor hidpauseRuleButton. The seed now wraps the window past midnight (schedules supportend <= start), so the rule always has ~6h remaining. Guarded by a parameterized unit test that fails on the old clamp at 23:50/23:58.testRenameRuleInEditorobserved"My"/"My Foc"instead of"My Focus"—typeTextdrops keystrokes on loaded runners. NewsetTextVerifiedhelper types, verifies the field value, and retries before submitting.tap(untilAppears:)helper re-taps against a postcondition — the same workaroundgoToSectionalready used for sidebar rows — applied at the flaking sites." ("soiPhone 17 Pro Maxcan't satisfy a check foriPhone 17 Pro.Test plan
SampleRulesTestspausability test: red on the old clamp (23:50/23:58 args fail), green after the fix; full suite passes.Executed 3 tests, with 0 failures).build-for-testingsucceeds; workflow YAML validated.🤖 Generated with Claude Code
https://claude.ai/code/session_01Y9652c13kpQrUyj7CZrHZg