Skip to content

fix(limits): Bound idle probing and preserve partial results - #45

Merged
korya merged 2 commits into
masterfrom
korya-fix-idle-timeout
Sep 10, 2026
Merged

fix(limits): Bound idle probing and preserve partial results#45
korya merged 2 commits into
masterfrom
korya-fix-idle-timeout

Conversation

@korya

@korya korya commented Sep 10, 2026

Copy link
Copy Markdown
Owner

Problem

A stalled idle probe can keep a measurement running indefinitely, even when discovery and load phases have finite time budgets.

The CLI supplies no parent deadline, so neither stalled headers nor an unfinished idle response body is bounded. Caller cancellation also discards completed idle samples, and a caller deadline during load is reported as the direction's own duration cap.

Solution

Bound the whole idle phase with a configurable timeout, retain completed samples, and continue selected load measurements when only idle expires.

Options.IdleTimeout and nq --idle-timeout default to 10 seconds. Non-positive values select that default; negative IdleProbes still skips idle measurement.

The cap does not scale with IdleProbes: healthy slow paths and larger sample sets may require a larger IdleTimeout to reach the requested percentile thresholds. Ten seconds is a configurable product default, not a network-health threshold. Timeout warnings retain the preceding probe error when available, and a deadline observed after every requested sample completed does not produce an idle-timeout warning.

Stop condition Idle result Remaining load phases
Idle timeout Completed samples plus one warning with successful/requested counts Run with their own budgets
Caller cancellation or earlier deadline Completed samples in a cancelled partial result Omitted
Direction's own timeout Existing measurements retained; reason=duration_cap Next selected direction runs

Discovery, idle, and load deadlines use the existing injectable clock. The combined default phase budget is 44 seconds for both directions, 32 seconds for one, or 34 seconds for both with idle skipped. Return time also includes local orchestration and prompt cleanup; caller-supplied transports, dialers, body closers, sinks, and log handlers must cooperate. MaxDuration retains its per-direction meaning, and the result JSON shape is unchanged.

The budget regression also covers an HTTP/2 upload peer that sends an early response and stops reading. Cancellation explicitly closes that response body to wake the stream writer, and joins the close callback before transport teardown. This avoids waiting indefinitely on HTTP/2 flow control while preserving the existing socket cleanup order.

Known teardown limitation: Go's HTTP/2 response-body close can still wait on the connection write mutex with unread response data and a peer that stops reading its socket entirely. This pre-existing case needs a separate transport-abort investigation; abandoning the close callback would violate the worker ownership contract.

Actual CLI transcript rendered for review: one idle sample survives a stalled response, the warning reports the 500 ms timeout, download completes, and the process exits successfully.

CLI continues to download after idle timeout

When an earlier probe failed, the CLI now retains that error in the timeout warning:

CLI retains the preceding HTTP 403 in its timeout warning

Validated with go build ./..., go vet ./..., golangci-lint run ./..., and go test -race -count=3 ./.... Offline tests cover HTTP/1.1 and HTTP/2 header/body stalls, zero and partial samples, preceding errors, expiry after the final sample, cancellation from the phase event sink, caller deadlines, selected/skipped phases, deadline cleanup, owned sockets, and the built CLI binary. Phase-budget tests also assert elapsed return time with a generous scheduling/teardown allowance. The custom-TLS socket assertion follows the existing eventual-cleanup contract and passes 30 repetitions under the race detector. Specs, changelog, and the test matrix are updated together.

Related:

Add a whole-phase idle timeout so a stalled response cannot prevent load measurements.
Preserve completed samples when idle expires or the caller cancels, and distinguish
caller deadlines from per-direction caps.

Close early upload responses on cancellation to wake HTTP/2 flow-control waits,
joining the cleanup callback before transport teardown. Document the combined phase
budget and caller cooperation contract, with offline library and CLI regression coverage.

Closes #39

Co-Authored-By: GPT-6 <noreply@openai.com>
@korya
korya marked this pull request as ready for review September 10, 2026 02:17
Document that the fixed idle budget can truncate healthy slow paths and larger
sample sets. Preserve the preceding probe error when a later probe times out,
and avoid timeout warnings after every requested sample has completed.

Align the custom-TLS socket assertion with the existing eventual cleanup
contract, while preserving the immediate assertion for ordinary dials. Exercise
phase-event cancellation and elapsed return time alongside exact phase budgets.

Co-Authored-By: GPT-6 <noreply@openai.com>
@korya
korya merged commit b8fd6cd into master Sep 10, 2026
12 checks passed
@korya
korya deleted the korya-fix-idle-timeout branch September 10, 2026 02:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bound idle probes and define an overall run deadline

1 participant