Problem
The PR #515 remote controller loses the actionable polling failure and collects results before cancellation has reached a terminal state. A bounded iOS attempt therefore ended with no test execution, collection=COMPLETE, and only a generic StateError diagnostic. Qualification correctly stayed false and cleanup succeeded.
Reproduction evidence
- Runner commit:
21f89f94d is the current reviewed head; the submitted run began on b92a0e1df. Both retain the same controller code.
- Clean signed bundle:
31867c10c, native v0.4.1, Qwen3.5 GGUF Metal.
- Target: Firebase iPhone 16 Pro / iOS 18.3, Xcode 26.2, eight-minute test timeout, no flaky retries.
- Matrix:
matrix-3k4vi693lyhyc; run: qa-1789730285358167.
- Journal: FAILED, cleanup VERIFIED, collection COMPLETE. Error:
Remote operation failed (StateError); inspect bounded local diagnostics.
- Collected
matrix.json is still PENDING; only the uploaded tests.zip was available at collection time.
- Fresh provider read: matrix FINISHED / outcome INCONCLUSIVE; execution CANCELLED. Tool Results: complete,
abortedByUser=true, no testProcessDuration, no XCTest output. The controller's cleanup issued cancellation; this is not evidence of an iOS runtime/model failure.
- Android's earlier run completed collection normally but failed its model download deadline: 2,285,638,612 / 2,588,147,712 bytes in ten minutes, before inference. That is a separate model-delivery gap.
Expected follow-up
- Persist safe structured polling diagnostics (phase, operation, HTTP status or typed failure code) without tokens, URLs with credentials, or arbitrary secret-bearing error text.
- Exercise transient polling failures explicitly; any bounded retry must retain the existing absolute deadline and must never resubmit the matrix.
- Refresh the matrix/result inventory after terminal cleanup and distinguish a transferred pre-terminal snapshot from complete final test artifacts.
- Keep cancelled/preparation-only runs unqualified. Preserve verified cleanup and fail-closed allowance handling when test-process usage is absent.
- Add production controller-path positive and negative tests before another bounded iOS attempt.
No automatic rerun is requested. Both matrices are terminal and no GCE VM was created.
Problem
The PR #515 remote controller loses the actionable polling failure and collects results before cancellation has reached a terminal state. A bounded iOS attempt therefore ended with no test execution,
collection=COMPLETE, and only a genericStateErrordiagnostic. Qualification correctly stayed false and cleanup succeeded.Reproduction evidence
21f89f94dis the current reviewed head; the submitted run began onb92a0e1df. Both retain the same controller code.31867c10c, native v0.4.1, Qwen3.5 GGUF Metal.matrix-3k4vi693lyhyc; run:qa-1789730285358167.Remote operation failed (StateError); inspect bounded local diagnostics.matrix.jsonis still PENDING; only the uploaded tests.zip was available at collection time.abortedByUser=true, notestProcessDuration, no XCTest output. The controller's cleanup issued cancellation; this is not evidence of an iOS runtime/model failure.Expected follow-up
No automatic rerun is requested. Both matrices are terminal and no GCE VM was created.