Problem
The analysis run completed but its output CSVs deviate from the golden reference: ops_summary.csv is missing the required column 'total_subtree_kernel_time_ms', and ops_summary_by_category.csv / ops_unique_args.csv have substantially different row counts (41 vs 12 and 317 vs 307), causing the quality_eval_1 check to fail.
Reproduction context
- Model:
GLM-5.2-MXFP4
- Framework:
sglang
- GPU:
MI355X
- Trace SHA256:
1c858d3c2de4a204e482287c5769a873d01a54f62846c6dd0c82ff5d99ef81c0
- Harness exit:
0
Evidence
- The run failed at the eval stage with exactly one failing check out of 54:
error_class='eval_failed', fail_count=1, pass_count=53, check_count=54 (check/result_summary)
- The failing check is a data-quality comparison of analysis output against the golden reference:
check_index='quality_eval_1', category='Quality', result='FAIL', root_cause='data' (check/failed_checks)
- Analysis output ops_summary.csv is missing a required column present in the reference:
ops_summary.csv: missing required columns: {'total_subtree_kernel_time_ms'} (check/failed_checks details)
- Analysis output row counts diverge significantly from the golden reference:
ops_summary_by_category.csv: row count 41 vs ref 12; ops_unique_args.csv: row count 317 vs ref 307 (check/failed_checks details)
- The analysis phase itself ran to completion and produced output; the harness exited cleanly:
harness_exit_code=0, analysis_produced=true, 'Phase 1 complete.' and 'Phase 2 complete.' logged; Summary: 55/56 PASS (check/log/harness.log)
- Golden reference was generated successfully in the same run, so the comparison baseline is valid:
Golden Reference Generation Complete: Total: 1 | Generated: 1 | Failed: 0 (check/log/harness.log)
Failed checks
quality_eval_1 [data]: ops_summary.csv: missing required columns: {'total_subtree_kernel_time_ms'}; ops_summary_by_category.csv: row count 41 vs ref 12; ops_unique_args.csv: row count 317 vs ref 307
Problem
The analysis run completed but its output CSVs deviate from the golden reference: ops_summary.csv is missing the required column 'total_subtree_kernel_time_ms', and ops_summary_by_category.csv / ops_unique_args.csv have substantially different row counts (41 vs 12 and 317 vs 307), causing the quality_eval_1 check to fail.
Reproduction context
GLM-5.2-MXFP4sglangMI355X1c858d3c2de4a204e482287c5769a873d01a54f62846c6dd0c82ff5d99ef81c00Evidence
error_class='eval_failed', fail_count=1, pass_count=53, check_count=54(check/result_summary)check_index='quality_eval_1', category='Quality', result='FAIL', root_cause='data'(check/failed_checks)ops_summary.csv: missing required columns: {'total_subtree_kernel_time_ms'}(check/failed_checks details)ops_summary_by_category.csv: row count 41 vs ref 12; ops_unique_args.csv: row count 317 vs ref 307(check/failed_checks details)harness_exit_code=0, analysis_produced=true, 'Phase 1 complete.' and 'Phase 2 complete.' logged; Summary: 55/56 PASS(check/log/harness.log)Golden Reference Generation Complete: Total: 1 | Generated: 1 | Failed: 0(check/log/harness.log)Failed checks
quality_eval_1[data]: ops_summary.csv: missing required columns: {'total_subtree_kernel_time_ms'}; ops_summary_by_category.csv: row count 41 vs ref 12; ops_unique_args.csv: row count 317 vs ref 307