make: re-run a fuzz target once when it hits the deadline with no crasher - #313
Merged
Conversation
…sher The nightly failed on common/vml FuzzShapeRoundTrip with "context deadline exceeded" and no crasher. It is the second time this has cost a nightly, after FuzzDocxMailMerge, and neither reproduces: this one survived four runs at the CI settings on go1.25.0, the toolchain CI actually uses, after three on the newer one that turned out not to be the same evidence. The log says what it is. The execution rate was 37901/sec at 2m0s of a 120s budget and the failure landed 80ms later, while the sibling target in the same job ran 121.0s and passed. Nothing stalled; the coordinator gave up waiting on a worker at shutdown. The parse in that target is already bounded by a fuzzbound budget, so a genuinely slow input would have failed deterministically and named itself instead. So that exact pair — the deadline message and no new crasher — is re-run once. Both halves keep it narrow. A crasher means a reproducible defect, so a target that wrote one is never re-run however it phrased its failure; any other message is never re-run; and a target that hits the deadline twice still fails the sweep. Each of those four paths was checked with a planted target, including the one that writes a crasher while saying the deadline words. A target that passes the second time is listed under PASSED ON RE-RUN rather than passing quietly. A sweep that hides its own flakiness is how a nightly stops being read, which is the failure this exists to prevent, not to cause.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the nightly failure in run 32551816165 (
common/vml FuzzShapeRoundTrip).Not a defect
Full execution rate at 2m0s of a 120s budget; the failure lands 80ms later. Nothing stalled — the coordinator gave up waiting on a worker at shutdown. The sibling target in the same job ran 121.0s and passed, so the difference is sub-second timing.
Three further reasons it is not the code under test:
fuzzboundbudget, so a genuinely slow input would fail deterministically and name itself.Second time this class has cost a nightly, after
FuzzDocxMailMerge.The change
The exact pair — deadline message and no new crasher — is re-run once. Both halves keep it narrow:
All four checked with planted targets, including one that writes a crasher while saying the deadline words — it is not re-run.
A target that passes the second time is named, not silently passed. A sweep that hides its own flakiness is how a nightly stops being read, which is the failure this exists to prevent rather than cause.
Verification
build, vet,
golangci-lint0 issues. Makefile-only change.