Skip to content

e2e_verify: step-9 cancel wait uses 30s default, too tight for a real checker #331

Description

@mithro

Summary

The e2e_verify harness's step 9 ("Cancelling precheck...") waits for the
check to reach the Cancelled state using the default 30s timeout. Against
a real checker VM, cancelling a running precheck reliably takes longer than 30s,
so step 9 fails with:

TimeoutError: Timed out waiting for cancellation (30000 ms)

This blocks the harness from ever reaching the manufacturability verdict
(steps 10–13) unless the timeout is bumped locally.

Root cause

scripts/e2e_verify/main.py:299 calls wait_for_precheck_cancelled() with no
timeout, so it uses the short default:

log(9, "Cancelling precheck...")
detail_page.click_cancel_precheck()
detail_page.wait_for_precheck_cancelled()   # <- defaults to 30_000 ms

wait_for_precheck_cancelled (scripts/e2e_verify/pages/project_detail.py:304)
defaults to 30s:

def wait_for_precheck_cancelled(self, timeout_ms: int | None = None) -> None:
    """Wait for precheck cancellation to be confirmed."""
    timeout = timeout_ms or 30_000
    ...

The preflight cancel path already uses 180s and works fine —
cancel_check_if_running (same file, line 369) calls
wait_for_precheck_cancelled(timeout_ms=180_000). Only the step-9 call site
uses the short default.

Suggested fix

Pass a 180s timeout at the step-9 call site to match the working preflight path:

detail_page.wait_for_precheck_cancelled(timeout_ms=180_000)

(Alternatively, raise the default in wait_for_precheck_cancelled itself so
both call sites get the longer window.)

Impact / context

Test tooling only — no production impact. Surfaced repeatedly while running
quarter-tile (0p5x0p5) e2e_verify runs against both stage and prod on
2026-07-30; a local 180s bump let the run complete to a manufacturable verdict.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions