Skip to content

Report writing utility file - #978

Open
kyle-hoffmeyer wants to merge 8 commits into
mainfrom
cleanup/report_writing_utility
Open

kyle-hoffmeyer wants to merge 8 commits into
mainfrom
cleanup/report_writing_utility

Conversation

@kyle-hoffmeyer

@kyle-hoffmeyer kyle-hoffmeyer commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR creates a report writing utility file and creates utility functions for exporting dataframes to csvs/excel files.

Every report generator in TraceLens/Reporting/ ended with its own copy of the same "write these DataFrames out" boilerplate: make the CSV directory, loop and write one CSV per sheet, check that openpyxl is importable, open an ExcelWriter, and loop again writing one sheet per DataFrame. Ten generators carried a near-identical version of this block — roughly 300 lines of duplicated logic.

Because each copy drifted independently, they had also become inconsistent in ways that were quietly buggy:

  • Sheet-name truncation. Excel caps sheet names at 31 characters. Some generators truncated with a bare [:31] (which silently collides when two long names share a prefix), some did collision-safe truncation, and several did no truncation at all — meaning a long sheet name would make the whole report fail.
  • Missing-openpyxl handling. Half the generators prompted the user to pip install openpyxl; the other half raised ImportError; a couple didn't check at all. openpyxl is in fact a declared install dependency, so all of this was dead defensive code.
  • CSV vs. Excel exclusivity. Some generators wrote both CSV and Excel when both output paths were given; five others silently suppressed the Excel output the moment a CSV directory was supplied.

This PR consolidates all of that into a single shared utility and migrates every generator onto it, which both removes the duplication and makes the behavior uniform.

Implementation

New write_report_outputs() in TraceLens/Reporting/reporting_utils.py. This is the one place report DataFrames get written. It takes the {sheet_name: DataFrame} mapping plus optional xlsx_path and csvs_dir, and writes whichever outputs were requested (both, if both are given). Two smaller behaviors are folded in:

  • hide_columns — an optional {sheet_name: [column, ...]} mapping for columns that should be hidden in the Excel view (they stay in the file, just collapsed). This exists so the comparison report, the one generator that post-processes its worksheets, can express its column-hiding declaratively instead of reaching into the writer itself.
  • skip_empty — drops None/empty DataFrames from all outputs. Off by default (an empty-but-present sheet is often a meaningful "we looked and found nothing" signal), opted into only by the genesis report.

Sheet-name sanitization is handled once, in the new _safe_sheet_name() helper (promoted from the genesis report, which already had the only collision-safe implementation): truncate to 31 chars, and if the name of the sheet already exists append _1, _2, ... to the end of the name while staying within the 31 char limit.

New derive_pftrace_output_path() in TraceLens/Reporting/pftrace_utils.py. The three pftrace generators each had an identical block for turning an input trace path into a default .xlsx path (stripping .pftrace / .json.gz / other suffixes). That block is now a single helper parameterized by the report-specific suffix.

Tests

  • Added TestWriteReportOutputs, TestSafeSheetName, and TestDerivePftraceOutputPath to tests/test_reporting_utils.py (18 cases). Coverage includes CSV-only, Excel-only, and both-at-once output; the no-op case; long-name truncation and dedup; skip_empty behavior in both directions; hide_columns hiding the right column while leaving the data intact and ignoring column names that don't exist; and a regression test for the "sheet"-name collision described above.
  • Moved the sheet-name tests out of tests/test_genesis.py (where they imported the helper from the genesis module) into tests/test_reporting_utils.py, so they now live next to the function's actual home.
  • Ran the affected generators' existing suites (genesis, comparison, rocprof, pftrace memory-copy) to confirm the migration and the both-outputs behavior change don't regress them.

@kyle-hoffmeyer
kyle-hoffmeyer marked this pull request as ready for review August 28, 2026 22:10
@kyle-hoffmeyer kyle-hoffmeyer changed the title create report writing utility file Report writing utility file Aug 28, 2026
@codecov-commenter

codecov-commenter commented Aug 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@kyle-hoffmeyer
kyle-hoffmeyer force-pushed the cleanup/report_writing_utility branch from d55c0ef to 8ecf927 Compare September 1, 2026 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants