Skip to content

Wire per-component cardinality limits from Config#11732

Open
dougqh wants to merge 5 commits into
masterfrom
dougqh/cardinality-limits-config
Open

Wire per-component cardinality limits from Config#11732
dougqh wants to merge 5 commits into
masterfrom
dougqh/cardinality-limits-config

Conversation

@dougqh

@dougqh dougqh commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

What Does This Do

Adds the ability to configure cardinality limits per component / tag for client-side stats

Motivation

Avoid problems with high cardinality leading to excessive resource consumption while informing the customer (via sentinel value) when a limit was reached

Additional Notes

Unlike #11387 which introduced the limiting mechanism, this PR enables the limits by default.

  • Adds a single parameterized Config.getTraceStatsCardinalityLimit(tagName, defaultLimit) method following the RFC naming pattern DD_TRACE_STATS_{tag_name}_CARDINALITY_LIMIT
  • Wires all 9 per-field PropertyCardinalityHandler instances and the TagCardinalityHandler in PeerTagSchema to read their limits from Config at class-load time, with defaults from MetricCardinalityLimits
  • Activates sentinel substitution (useBlockedSentinel = true) now that limits are configurable; previously hardcoded to false pending this wiring
  • Registers all 10 config keys in supported-configurations.json

Stacked on #11387.

Test plan

  • Existing CardinalityHandlerTest and AggregateEntryTest cover sentinel substitution behavior
  • Verify default limits match MetricCardinalityLimits constants
  • Confirm DD_TRACE_STATS_RESOURCE_CARDINALITY_LIMIT etc. are picked up at startup via Config

tag: no release note
tag: ai generated

🤖 Generated with Claude Code

@dougqh dougqh requested review from a team as code owners June 24, 2026 17:03
@dougqh dougqh requested review from AlexeyKuznetsov-DD and ygree and removed request for a team June 24, 2026 17:03
@dd-octo-sts dd-octo-sts Bot added the tag: ai generated Largely based on code generated by an AI or LLM label Jun 24, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 75f72165fd

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +3853 to +3854
return configProvider.getInteger(
"trace.stats." + tagName + ".cardinality.limit", defaultLimit);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Validate stats cardinality limits before returning them

When any of the new DD_TRACE_STATS_*_CARDINALITY_LIMIT values is set to 0, a negative number, or a value above 2^29, this method returns it unchanged; the callers added in AggregateEntry and PeerTagSchema pass it directly into PropertyCardinalityHandler/TagCardinalityHandler, whose constructors throw for those ranges. In that misconfigured environment the stats classes fail initialization instead of falling back to the default/logging, so the accessor should reject invalid limits before handing them to the handlers.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my ideal world, config would probably handle this and report a warning immediately at start-up.
Unfortunately, there's no precedence for that in config today.
I'm in full agreement with the sentiment, but I think that's better left to another PR.

Comment thread metadata/supported-configurations.json Outdated
"DD_TRACE_STATS_RESOURCE_CARDINALITY_LIMIT": [
{
"version": "A",
"type": "integer",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use int for the new config metadata type

The supported-configurations metadata uses type: "int" for the existing integral settings (including the tracer metrics limits), while these ten new cardinality entries are the only ones using "integer". The local supported-configurations validation/publishing jobs consume this file, so the new configs can be rejected or omitted by downstream metadata consumers even though the agent code reads integers; please use int consistently for these entries.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, int is used instead of integer in the file.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mhlidd WDYT to add a test that will check types and failed if there someone made a typo?
OR probably we can support some sort of aliases like int, integer should be treated as the same type...
But I like the test better.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for aliasing, I'll just fix it. I'm curious why Claude decided to use the wrong value though.

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD Jul 1, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, Just thinking how to prevent issues in future.
Fir example if there will be a test - Claude with high probability will run it, test will fail, Claude will fix it.

I'm curious why Claude decided to use the wrong value though.

I think it is a nature of LLMs, somehow integer has a bigger weight in model that int and LLM produced wrong output.

@dd-octo-sts

dd-octo-sts Bot commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.06 s 13.93 s [-0.2%; +2.0%] (no difference)
startup:insecure-bank:tracing:Agent 12.93 s 12.97 s [-1.0%; +0.5%] (no difference)
startup:petclinic:appsec:Agent 16.91 s 16.91 s [-0.8%; +0.9%] (no difference)
startup:petclinic:iast:Agent 16.95 s 17.02 s [-1.2%; +0.4%] (no difference)
startup:petclinic:profiling:Agent 16.80 s 16.87 s [-1.4%; +0.5%] (no difference)
startup:petclinic:sca:Agent 16.95 s 16.76 s [+0.1%; +2.2%] (maybe worse)
startup:petclinic:tracing:Agent 15.63 s 16.20 s [-7.7%; +0.7%] (no difference)

Commit: e558d99c · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but:

  1. Code like:
Config.get()
                  .getTraceStatsCardinalityLimit(
                      "peer_tag", MetricCardinalityLimits.PEER_TAG_VALUE))

looks a bit noisy, probably can be improved if refactored to enum?
2. JSON descriptor should be fixed integer -> int.

Comment on lines +38 to +39
"resource",
Config.get().getTraceStatsCardinalityLimit("resource", MetricCardinalityLimits.RESOURCE));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QQ: just curious if this MetricCardinalityLimits can be an enum and no need to pass string and int const. Just enum and get the rest from it where needed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we might be able to do that. The only complication is that we might have to move the enum next to Config.

@dougqh dougqh force-pushed the dougqh/cardinality-limits-config branch 2 times, most recently from 1741d4f to b23f870 Compare July 1, 2026 01:58
@datadog-prod-us1-3

This comment has been minimized.

@dougqh dougqh force-pushed the dougqh/cardinality-limits-config branch from b23f870 to 2418d84 Compare July 1, 2026 02:31
Base automatically changed from dougqh/control-tag-cardinality to master July 1, 2026 05:12
dougqh and others added 3 commits July 1, 2026 08:46
…its RFC

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@dougqh dougqh force-pushed the dougqh/cardinality-limits-config branch from 2418d84 to 5b550dd Compare July 1, 2026 12:48
@dd-octo-sts

dd-octo-sts Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@dougqh dougqh added comp: core Tracer core type: enhancement Enhancements and improvements labels Jul 1, 2026
dougqh and others added 2 commits July 1, 2026 13:08
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…y limits

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: core Tracer core tag: ai generated Largely based on code generated by an AI or LLM type: enhancement Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants