[NOT READY] feat(profiling): add --show-from filter to profiling aggregate#470
Closed
long2mao1 wants to merge 1 commit into
Closed
[NOT READY] feat(profiling): add --show-from filter to profiling aggregate#470long2mao1 wants to merge 1 commit into
long2mao1 wants to merge 1 commit into
Conversation
The Datadog Continuous Profiler UI exposes a `show_from(<function>)` flame-graph filter that zooms the displayed graph to subtrees rooted at a chosen function. The filter is applied client-side — the API still returns the full aggregated profile — so reproducing it from the CLI required hand-rolled jq, which is awkward for AI agents and humans alike. This adds an equivalent native flag. `pup profiling aggregate --show-from=<function_name>` does, in order: 1. Resolves the function name to string IDs (exact match against `data.strings`). 2. Walks the response's frame table to find every `frame_id` whose `function` field (`frameSchema[4]`) refers to one of those string IDs. A single logical function commonly maps to many frame IDs after Go inlining / generics. 3. Walks `flameGraph` and collects the topmost subtree rooted at any matching frame on each call path (does not descend into nested re-entries — matches UI semantics). 4. Merges the collected subtrees into a single root by recursively collapsing children that share the same function name. Without this, the same logical function appears as N siblings at depth 0 (one per inlined frame ID), which doesn't match the UI. 5. Prunes `frames` and `strings` to only the entries the trimmed subtree references (with consistent ID remapping), and drops UI aggregation fields (`metadata`, `summaryTable`, `endpointCounts`, etc.) that describe the unfiltered profile rather than the filtered subtree. Concrete impact on a 14-hour `service:logs-event-store-reader` query: unfiltered response is 43 MB; with `--show-from=vectorEQStringNotMulti` the response drops to ~107 KB and shows the same call tree as the UI. - New `--show-from` flag on `ProfilingActions::Aggregate` (src/main.rs) - `apply_show_from`, `merge_subtrees_by_function`, and `prune_aggregate_response` helpers in src/commands/profiling.rs Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Collaborator
|
closing for #492 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Datadog Continuous Profiler UI exposes a
show_from(<function>)flame-graph filter that zooms the displayed graph to subtrees rooted at a chosen function. The filter is applied client-side — the API still returns the full aggregated profile — so reproducing it from the CLI required hand-rolled jq, which is awkward for AI agents and humans alike. This adds an equivalent native flag.Behavior
pup profiling aggregate --show-from=<function_name>does, in order:strings).frame_idwhosefunctionfield (frameSchema[4]) refers to one of those string IDs. A single logical function commonly maps to many frame IDs after Go inlining / generics.flameGraphand collects the topmost subtree rooted at any matching frame on each call path (does not descend into nested re-entries — matches UI semantics).framesandstringsto only the entries the trimmed subtree references (with consistent ID remapping), and drops UI aggregation fields (metadata,summaryTable,endpointCounts, etc.) that describe the unfiltered profile rather than the filtered subtree.The output is the same wire-format shape as the unfiltered API response (positional tuples +
frames+stringsinterning tables, withframeSchemaandnodeSchemapreserved). The intent is that anything an agent already knows about Datadog's profiler API still applies —--show-fromonly narrows the data, it doesn't change the schema.Concrete impact
Tested end-to-end against a 14-hour aggregate query on a real Go service. Order-of-magnitude numbers:
--show-from--show-from=<function>frames/stringstable sizesThe resulting
flameGraphshape matches the UI'sshow_from(...)view exactly: a single root for the chosen function, with its merged callee subtree below.Changes
--show-fromflag onProfilingActions::Aggregate(src/main.rs).src/commands/profiling.rs:apply_show_from— exact-match filter + topmost subtree collection.merge_subtrees_by_function— recursive same-name sibling merge.prune_aggregate_response— frames/strings remap + UI-field drop.collect_show_from_subtrees,collect_used_frame_ids,remap_frame_ids— small recursive walkers.Design notes (for reviewer feedback)
--show-fromis set, not behind its own flag. The argument: if a user opted into a focused subtree view, the unfiltered metadata/summary fields are deadweight. Happy to gate behind--pruneif the reviewer prefers.{ name, value_ms, children }record tree (noframes/stringslookup), since that's much friendlier for jq and LLMs. Rejected on the grounds that it diverges from the documented Datadog API shape and would force any existing agent/script to learn a second representation. Open to revisiting if the reviewer disagrees.Testing
show_from(...)view, including merged sibling collapse, with the same per-minute CPU figure the UI displays.apply_show_fromandmerge_subtrees_by_functionagainst synthetic flame-graph fixtures before the draft moves out of draft, if the reviewer wants them inline.Follow-ups out of scope
--show-from-mode=substring|regexfor non-exact matching.pup profiling callgraphonce that endpoint's own scaling issues (504s on long windows) are addressed.🤖 Generated with Claude Code