probe(vm): comment-only control for the repeat A/B noise floor - #709
probe(vm): comment-only control for the repeat A/B noise floor#709mparrett wants to merge 1 commit into
Conversation
Measurement control. Do not merge. The entire diff is a comment. Verified that it changes no code: built base and head with -trimpath and compared `go tool nm` output -- 12907 symbols at identical addresses in both, and __text is 0x338184 bytes in both. The binaries differ only in the Go build ID, which hashes the source bytes. So any per-family delta the repeat A/B lane reports on this PR is the lane's own noise at N=7, measured with code and layout held constant. That number does not exist today, which is what makes the 5% ratchet budget uncalibrated (#705). Prompted by #706, where a semantically inert change measured +8.30% on SeqIteration/List/01000 and +4.97% on FrameDispatch, both consistent across all seven cycles. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Closing this — it did its job. Numbers below, and the combined analysis of all three A/A runs is on #705. This run, comment-only diff so the machine code was identical (verified: same symbol addresses, same
Taken with the two accidental A/A runs on #708, the conclusion is that the floor is per-family and spans roughly 0.3% to 13.4% on this tier. The allocation-heavy families at sizes 1000 and 10000 are where nearly all of it lives; 29 of 53 families stay under 2%. Details and the per-family table are on #705. The one thing I would carry forward from building this: a deliberate control was not needed. Both of the other samples came from PRs that happened to touch no Go source, and this repo produces those routinely. Recording A/A results when they occur would accumulate the per-family floor for free. |
Measurement control. Do not merge. Closing it once the numbers land, same as #691.
The entire diff is a comment. The point is to push a build with identical machine code through the repeat A/B lane, so whatever deltas come back are the lane's own noise at N=7 rather than a property of any change.
The code really is identical
Worth checking rather than assuming, since the whole value of the control rests on it. Built base and head with
-trimpathand compared:go tool nm: 12907 symbols in both, at identical addresses — same layout, nothing moved.__textsection:0x338184bytes in both.So this holds code and placement constant, which is what makes any reported delta attributable to the harness and the runner alone.
Why now
#706 is a semantically inert change — it deletes a
*Frameparameter that was never read. On this lane it measuredSeqIteration/List/01000+8.30% andFrameDispatch+4.97%, each consistent across all seven cycles, alongside −0.75% onFuncInvoke/Direct. I could not tell from that run whether those are real or whether the lane can produce numbers that size on its own, and that is not a question #706 should have to answer.It is also the measurement #705 §1 is about. The 5% ratchet budget has never been checked against a measured floor, and #564 already found byte-identical binaries landing 3.7% and 1.1% apart on two packages.
Reading the result
Either way the number is worth having, and it belongs on #705 rather than buried in a PR thread. I will post it there too.