Skip to content

Split threads by default in memray transform csv - #1056

Merged
godlygeek merged 1 commit into
bloomberg:mainfrom
godlygeek:transform-csv-splits-threads
Oct 17, 2026
Merged

godlygeek merged 1 commit into
bloomberg:mainfrom
godlygeek:transform-csv-splits-threads

Conversation

@godlygeek

Copy link
Copy Markdown
Contributor

Closes: #996

memray transform always retrieved allocation records with threads merged, so allocations made on different threads at the same call stack collapsed into a single CSV row and the tid / thread_name columns no longer identified a single thread.

The point of the csv format is to be machine-parseable, so rather than adding an opt-in flag, make memray transform csv retrieve records with merge_threads=False by default. Each thread's allocations now stay on their own row carrying that thread's real tid and thread_name, which is also what the csv column documentation has always described. The gprof2dot and speedscope formats have no per-thread dimension and are unchanged.

@godlygeek

Copy link
Copy Markdown
Contributor Author

@BhargavKumarNath this is your work from #1006, lightly edited. I simplified the tests quite a lot, dropping some things that I think were already adequately exercised by other tests, and switching to building a capture file "by hand" rather than running a test program to generate one to simplify the remaining test. All of the implementation and docs match yours, except that I changed the NotImplementedError to be raised whenever merge_threads == (self.format == "csv") - so we raise a NotImplementedError if merge_threads is true and the format is CSV, or when merge_threads is false and the format isn't CSV.

Sorry for not getting around to reviewing #1006 before you deleted your fork 😅 but I'm leaving the commit attribution as you since you did basically all of the work here, and I just tweaked the tests and one exception check / message.

@codecov-commenter

codecov-commenter commented Oct 16, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.66667% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.31%. Comparing base (c3be2a3) to head (1ead335).

Files with missing lines Patch % Lines
src/memray/commands/transform.py 0.00% 1 Missing ⚠️
src/memray/reporters/transform.py 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1056      +/-   ##
==========================================
- Coverage   92.35%   92.31%   -0.04%     
==========================================
  Files         103      103              
  Lines       13363    13385      +22     
  Branches      483      484       +1     
==========================================
+ Hits        12341    12357      +16     
- Misses       1022     1028       +6     
Flag Coverage Δ
cpp 92.31% <91.66%> (-0.04%) ⬇️
python_and_cython 92.31% <91.66%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@godlygeek
godlygeek requested a review from a team October 16, 2026 22:36
@BhargavKumarNath

Copy link
Copy Markdown
Contributor

Thanks for picking this up, @godlygeek, and no worries about #1006. I'm glad it's getting in. The simplified tests and the merge_threads == (self.format == "csv") check both look good to me. That condition covers both invalid combinations in one place, which is cleaner than what I had. Let me know if you'd like anything else changed on my side.

`memray transform` always retrieved allocation records with threads
merged, so allocations made on different threads at the same call stack
collapsed into a single CSV row and the `tid` / `thread_name` columns no
longer identified a single thread.

The point of the csv format is to be machine-parseable, so rather than
adding an opt-in flag, make `memray transform csv` retrieve records with
`merge_threads=False` by default. Each thread's allocations now stay on
their own row carrying that thread's real `tid` and `thread_name`, which
is also what the csv column documentation has always described. The
gprof2dot and speedscope formats have no per-thread dimension and are
unchanged.

Signed-off-by: Bhargav kumar Nath <bhargavkumarnathh@gmail.com>
@godlygeek
godlygeek force-pushed the transform-csv-splits-threads branch from ba51f5c to 1ead335 Compare October 17, 2026 19:11
@godlygeek
godlygeek enabled auto-merge (rebase) October 17, 2026 19:11
@godlygeek
godlygeek merged commit 1be0a98 into bloomberg:main Oct 17, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add --split-threads support to memray transform csv

4 participants