test(relay): deflake trace_context_lookup test via rebuild_interest_cache - #4100
Open
iroiro147 wants to merge 1 commit into
Open
test(relay): deflake trace_context_lookup test via rebuild_interest_cache#4100iroiro147 wants to merge 1 commit into
iroiro147 wants to merge 1 commit into
Conversation
…ache trace_context_lookup_does_not_enable_callsites flakes ~1/6 in the full buzz-relay lib suite (clean main), because tracing's per-callsite `interest` cache is global and outlives the thread-local `with_default` dispatcher — a stale cache can bypass our `LevelFilter::OFF` layer and make the `enabled!` assertion race. Call `tracing::callsite::rebuild_interest_cache()` once inside the `with_default` block so the per-test filter is authoritative, and once afterward to restore default interest for downstream tests. This is the tracing-idiomatic escape hatch and avoids both ENV_LOCK (which cannot catch every global-state race) and any per-run naming hack. Verified: 6/6 green in the full lib suite; was 1/6 failing previously. Fixes block#3929 Signed-off-by: Sarthak Singh <sarthak.singh@juspay.in>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #3929.
What
telemetry::tests::trace_context_lookup_does_not_enable_callsiteswas intermittently failing in the parallelbuzz-relaylib suite — 1/6 of full-suite runs on cleanmain. Standalone the test passes 100% of the time, but under parallel scheduling thetracing::enabled!assertion could race with other tests that install a global default dispatcher.Root cause
tracingmaintains a per-callsiteinterestcache that is global and process-wide, not per-subscriber.with_defaultonly installs a thread-local dispatcher. If another test (or any prior thread) has primed a static callsite'sinteresttoalways, a later assertion on the same static callsite can consult stale cache and bypass our newly-installedLevelFilter::OFFlayer.Change
Call
tracing::callsite::rebuild_interest_cache():with_default— forces all callsites to be re-evaluated against our subscriber, making theLevelFilter::OFFfilter authoritative for this assertion.This is the tracing-idiomatic escape hatch (tracing-rs/tracing#1527) and is more direct than the alternates:
ENV_LOCKonly serializes against other tests that take it — it can't catch a non-participating racer (the flake materialised precisely because the interferer isn't ENV_LOCK-adjacent).&'static str, whichtracing::event!/enabled!cannot take.Why this is safe
rebuild_interest_cache()is documented intracing-coreand is the standard escape hatch for stale-callsite flakes.rebuild_interest_cache()call restores the default interest — it doesn't lock in ourOFFfilter. Verified by running the full suite 6x post-fix.Verification
cargo clippy -p buzz-relay --all-targets -- -D warnings— clean.