Skip to content

test_gap_cron_cronjob is timing-dependent and flakes in CI: each side has its own 10 s tick deadline, so a slow moment flips a boolean and the byte comparison fails #10563

Description

@proggeramlug

Found while triaging a false "regression" on PR #10530. test-files/test_gap_cron_cronjob.ts waits up to 10 s for a
* * * * * * CronJob to tick twice and prints manual ticked at least twice: <bool> / auto ticked at least twice: <bool>. Node and Perry each run the program under their own deadline, so a slow moment on EITHER side (including the
Node oracle) flips one boolean and the harness reports an output mismatch — attributed to whatever PR happened to be
running.

Evidence

  • CI: across the last 40 test.yml runs, 20 gap-suite jobs ran this test; it passed in 19 and failed in 1 (a PR whose
    change provably cannot affect it).
  • That PR's proof: compiling the fixture with --trace llvm on the base commit and on the PR build produces
    byte-identical IR (5,438 lines; the only difference is the embedded absolute compile-directory path). cron takes
    the native-binding route (crates/perry-ext-cron), so the only TypeScript compiled is the test itself.
  • Local reruns on both builds: 12/12 pass each, 24 concurrent instances identical, 5/5 pass pinned to one core against
    three busy loops.
  • Failure signature: both exit 0, the first line matches, a later line differs — i.e. a flipped boolean, not a crash.

Impact

A timing-dependent fixture in the ratchet gate makes the gap suite an unreliable signal: a green PR can be blocked and
a real regression can be dismissed as "probably the cron flake". It cost a full investigation cycle on #10530.

Suggested fix

Make the assertions independent of wall-clock progress, for example:

  • drive the job deterministically (fire the callback via the API instead of waiting on a real schedule), or
  • assert only facts that do not depend on how many ticks happened within a deadline (e.g. that a never-started job
    never fired, that start()/stop() toggle a state flag), printing counts only as >= 1/0, or
  • give both sides the same generous deadline AND make the printed value insensitive to the deadline being hit
    (e.g. wait indefinitely for the first two ticks with a hard timeout that fails loudly rather than printing false).

The same review applies to other timer-driven gap fixtures (test_gap_backoff_options, node-cron variants).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    package-auditFound by the 2026 package audit: compiling real npm packages from source instead of native bindings

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions