Skip to content

Fix compiled distribution sampling#422

Merged
pedro-stanaka merged 1 commit into
Shopify:mainfrom
maxveldink:fix-compiled-distribution-sampling
Jun 23, 2026
Merged

Fix compiled distribution sampling#422
pedro-stanaka merged 1 commit into
Shopify:mainfrom
maxveldink:fix-compiled-distribution-sampling

Conversation

@maxveldink

@maxveldink maxveldink commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

✅ What

Fixes CompiledMetric::Distribution so sampled metrics only make one sampling decision.

CompiledMetric::Distribution already samples in the generated latency/block handler. The precompiled distribution client emit path then sampled again, which made sampled compiled distributions emit with probability sample_rate² while still sending datagrams marked with only |@sample_rate.

This PR routes pre-sampled compiled distributions to a dedicated private emit helper so the generated method's sampling decision is reused for both direct emission and aggregation.

Also bumps the patch version to 3.11.1 for release.

Fixes #421.

🤔 Why

A compiled distribution with sample_rate: 0.001 should emit about 1 in 1000 calls and let the backend scale by 1000. Instead it emitted about 1 in 1,000,000 calls and still told the backend to scale by only 1000, under-reporting by roughly the sample rate.

Counters and gauges were not affected because they do not use the generated latency/block handler.

👩🔬 How to validate

Tests added coverage that fails on main because sample? is called twice:

  • sampled compiled distribution value path
  • sampled compiled distribution block path
  • sampled compiled distribution with aggregation enabled

Validation run locally:

Full test suite:
390 runs, 1098 assertions, 0 failures, 0 errors

RuboCop changed files:
4 files inspected, no offenses detected

Benchmark

Temporary benchmark script based on benchmark/send-metrics-to-dev-null-log, focused on compiled distributions. Higher i/s is better.

Environment: Ruby 3.4.9, arm64-darwin24.

main → this branch:

compiled distribution value static rate=1
2.351M -> 2.579M i/s (+9.7%)

compiled distribution value dynamic tags rate=1
1.437M -> 1.509M i/s (+5.0%)

compiled distribution value static rate=0.5 sampled-in
2.055M -> 2.364M i/s (+15.0%)

compiled distribution block static rate=1
1.669M -> 1.746M i/s (+4.6%)

Checklist

  • I documented the changes in the CHANGELOG file.

@maxveldink maxveldink marked this pull request as ready for review June 22, 2026 18:38

@pedro-stanaka pedro-stanaka left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG - Apart from the comment, it would be nice if the sampling test was probabilistic since the beginning. It would have covered this.

I think the mocked test is enough for now, you may want to bump the version already to avoid a PR just for the release.

Comment thread lib/statsd/instrument/compiled_metric.rb
CompiledMetric::Distribution already samples in its generated latency handler so sampled-out calls can skip timing block overhead. The precompiled distribution emit path sampled again, making sampled distributions emit at rate^2 while still advertising @Rate. Route pre-sampled compiled distributions to a dedicated emit helper so each call makes one sampling decision.

Bump the patch version to 3.11.1 for release.

Benchmark (Ruby 3.4.9, temporary compiled distribution benchmark, main -> this branch):

- value static rate=1: 2.351M -> 2.579M i/s (+9.7%)

- value dynamic tags rate=1: 1.437M -> 1.509M i/s (+5.0%)

- value static rate=0.5 sampled-in: 2.055M -> 2.364M i/s (+15.0%)

- block static rate=1: 1.669M -> 1.746M i/s (+4.6%)

Tests:

- full test suite: 390 runs, 1098 assertions, 0 failures

- rubocop changed files: no offenses
@maxveldink maxveldink force-pushed the fix-compiled-distribution-sampling branch from cd9ae2c to e44cacd Compare June 23, 2026 11:28
@maxveldink maxveldink marked this pull request as draft June 23, 2026 11:29
@maxveldink maxveldink marked this pull request as ready for review June 23, 2026 11:30
@pedro-stanaka pedro-stanaka merged commit 7fe6261 into Shopify:main Jun 23, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CompiledMetric::Distribution applies the sample rate twice

2 participants