Summary
Makefile audit tests exercise only dry-run command strings; no end-to-end or behavioural tests exist for the validate-audit.js workflow or for actual Makefile target execution.
Problems
scripts/makefile-audit.test.mjs verifies only that expected command strings appear in make --dry-run output; it does not execute the targets with real tools.
validate-audit.js has no tests that exercise the assertValidSchema and assertMitigated orchestration paths with real or mock advisories.
- There are no behavioural tests that verify audit execution failure paths (e.g. unexpected advisory detected, schema validation error).
Acceptance criteria
- Add integration or behavioural tests for the
validate-audit.js orchestration, covering:
assertValidSchema with valid and invalid exception-ledger JSON.
assertMitigated with a match between advisories and the exception ledger (no error), and with an unmatched advisory (error + non-zero exit).
- Add behavioural tests that execute relevant Makefile targets (
audit-node, rust-audit) with actual (or suitably shimmed) tool behaviour rather than dry-run string checks.
Context
Raised during review of PR #354 by @leynos.
Summary
Makefile audit tests exercise only dry-run command strings; no end-to-end or behavioural tests exist for the
validate-audit.jsworkflow or for actual Makefile target execution.Problems
scripts/makefile-audit.test.mjsverifies only that expected command strings appear inmake --dry-runoutput; it does not execute the targets with real tools.validate-audit.jshas no tests that exercise theassertValidSchemaandassertMitigatedorchestration paths with real or mock advisories.Acceptance criteria
validate-audit.jsorchestration, covering:assertValidSchemawith valid and invalid exception-ledger JSON.assertMitigatedwith a match between advisories and the exception ledger (no error), and with an unmatched advisory (error + non-zero exit).audit-node,rust-audit) with actual (or suitably shimmed) tool behaviour rather than dry-run string checks.Context
Raised during review of PR #354 by @leynos.