When using report-path with a glob pattern that matches multiple CTRF report files (e.g. from sharded Playwright runs), the action correctly merges data for:
- The uploaded artifact (correct totals)
- Annotations (all failures annotated)
- Insights section (aggregated historical data)
However, the Summary table in the PR comment only reflects one of the matched files instead of the merged totals.
Reproduction
3 Playwright shards each produce a ctrf-report.json, uploaded as separate artifacts and downloaded into e2e-artifacts/:
| Shard |
Tests |
Passed |
Failed |
Skipped |
| 1 |
53 |
24 |
3 |
26 |
| 2 |
49 |
34 |
0 |
15 |
| 3 |
49 |
36 |
3 |
10 |
| Expected merged |
151 |
94 |
6 |
51 |
Config:
report-path: "e2e-artifacts/**/ctrf-report.json"
Logs confirm all 3 reports are read (Read 3 CTRF reports), and 6 failures are annotated via ##[error]. But the Summary table shows shard 2's numbers only (49 tests, 0 failed).
Expected behavior
The Summary table should reflect the merged totals across all matched CTRF files (151 tests, 6 failed).
Workaround
Pre-merge the CTRF reports into a single file before passing to the action, so report-path only matches one file.
When using
report-pathwith a glob pattern that matches multiple CTRF report files (e.g. from sharded Playwright runs), the action correctly merges data for:However, the Summary table in the PR comment only reflects one of the matched files instead of the merged totals.
Reproduction
3 Playwright shards each produce a
ctrf-report.json, uploaded as separate artifacts and downloaded intoe2e-artifacts/:Config:
Logs confirm all 3 reports are read (
Read 3 CTRF reports), and 6 failures are annotated via##[error]. But the Summary table shows shard 2's numbers only (49 tests, 0 failed).Expected behavior
The Summary table should reflect the merged totals across all matched CTRF files (151 tests, 6 failed).
Workaround
Pre-merge the CTRF reports into a single file before passing to the action, so
report-pathonly matches one file.