Skip to content

perf(scan): scan once and write both artifacts - #50

Open
joeymussalli wants to merge 1 commit into
trustabl:mainfrom
joeymussalli:perf/single-scan-both-outputs
Open

perf(scan): scan once and write both artifacts#50
joeymussalli wants to merge 1 commit into
trustabl:mainfrom
joeymussalli:perf/single-scan-both-outputs

Conversation

@joeymussalli

Copy link
Copy Markdown

The script ran two full scans of the target: one redirected to trustabl.sarif, one to trustabl.json. That is not needed. trustabl has --json-out and --sarif-out, which write their documents independently of --format, so a single run produces both.

Cost aside, the two-scan arrangement had a correctness edge. Rules are fetched at scan time, so the two runs could resolve different rulesets -- the SARIF that gets uploaded and shown in the CodeCatalyst Reports tab and the JSON that decides the gate were describing two separate scans, with nothing checking that they agreed. On a large repository it also simply doubled the wall-clock and the compute bill of every build.

There is a side benefit. Both runs previously redirected stdout to a file, so trustabl's own output never reached the build log at all -- everything a user saw came from this script's report box. With the machine documents going to --json-out/--sarif-out, --format human puts the scanner's own summary in the log where it belongs.

  • One invocation: --format human --json-out "$JSON_FILE" --sarif-out "$SARIF_FILE".
  • NATIVE_CODE still comes from that single run, so the gate is unaffected.

Verified with a stub scanner that records its invocations: two full scans before (--format sarif, then --format json), one after, with both output files written and valid. Exit codes 0, 1 and 2 propagate to NATIVE_CODE exactly as before. bash -n passes.

One compatibility note for reviewers: this requires a trustabl release that has --json-out/--sarif-out. VERSION defaults to "latest" so most users are covered, but anyone pinning an older tag would have the scan fail on the unknown flag. That fails closed -- a non-zero exit the gate reports -- rather than silently mis-reporting, which is the right direction, but it is a real consideration if you support older pinned versions and may warrant a documented minimum.

The script ran two full scans of the target: one redirected to trustabl.sarif,
one to trustabl.json. That is not needed. trustabl has --json-out and
--sarif-out, which write their documents independently of --format, so a single
run produces both.

Cost aside, the two-scan arrangement had a correctness edge. Rules are fetched
at scan time, so the two runs could resolve different rulesets -- the SARIF that
gets uploaded and shown in the CodeCatalyst Reports tab and the JSON that
decides the gate were describing two separate scans, with nothing checking that
they agreed. On a large repository it also simply doubled the wall-clock and the
compute bill of every build.

There is a side benefit. Both runs previously redirected stdout to a file, so
trustabl's own output never reached the build log at all -- everything a user
saw came from this script's report box. With the machine documents going to
--json-out/--sarif-out, --format human puts the scanner's own summary in the
log where it belongs.

- One invocation: --format human --json-out "$JSON_FILE" --sarif-out "$SARIF_FILE".
- NATIVE_CODE still comes from that single run, so the gate is unaffected.

Verified with a stub scanner that records its invocations: two full scans before
(`--format sarif`, then `--format json`), one after, with both output files
written and valid. Exit codes 0, 1 and 2 propagate to NATIVE_CODE exactly as
before. bash -n passes.

KNOWN CONFLICT. This touches the two scan invocations, and
chore/remove-dead-code deletes the SCAN_END line immediately after them, so
those two branches collide on one line. It is the only collision in the
series and the resolution is to take both changes: no second scan, no
SCAN_END. I tried stacking this on that branch instead, which removes the
collision here but creates two larger ones against the gate and severity
branches -- strictly worse, so this is left standalone.

One compatibility note for reviewers: this requires a trustabl release that has
--json-out/--sarif-out. VERSION defaults to "latest" so most users are covered,
but anyone pinning an older tag would have the scan fail on the unknown flag.
That fails closed -- a non-zero exit the gate reports -- rather than silently
mis-reporting, which is the right direction, but it is a real consideration if
you support older pinned versions and may warrant a documented minimum.
@joeymussalli
joeymussalli force-pushed the perf/single-scan-both-outputs branch 2 times, most recently from 9705439 to 6865e94 Compare August 24, 2026 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant