Tooling & DevX: add fuzzing, cargo-deny, Miri, CodeQL & GitHub Actions hardening

[pmatos]

Part of a portfolio-wide SOTA developer-experience audit. This issue proposes additive tooling only — everything already in place is acknowledged below so recommendations are strictly incremental.

Audit summary

Languages: Rust (engine), Python (test/utility scripts, run via uv), JavaScript (test262 harness/benchmarks), Shell.
Project: an agent-coded, from-scratch JS engine (lexer → recursive-descent parser → tree-walking interpreter + bytecode VM) chasing 100% test262. Large, actively maintained, security-conscious repo.

Already in place (do not re-add):

• Formatting: rustfmt (cargo fmt --check in CI + PostToolUse fmt-hook.sh).
• Linting: clippy -D warnings in CI and scripts/lint.sh.
• Testing: cargo test, test262 harness (smoke + 10% seeded sample in CI, full suite nightly), Python unittest, custom tests/ + test262-extra/.
• Mutation testing: cargo-mutants (local, .cargo/mutants.toml + scripts/run-mutants.sh).
• Coverage: cargo-llvm-cov nightly → Codecov.
• Static analysis: Semgrep in CI (config: auto) + scripts/semgrep.sh.
• Dependencies: Dependabot (cargo + github-actions, daily).
• Agentic: rich CLAUDE.md/AGENTS.md, .claude/ PostToolUse fmt hook + performance-* skills, docs/agents/, claude-code-action for @claude and automated PR review (with a well-documented trust-gate threat model; both jobs already scope permissions: contents: read + id-token: write).

This is already a top-decile setup. The gaps below are the ones that matter most for a parser/interpreter with 29 unsafe blocks (src/interpreter/types.rs, src/interpreter/builtins/atomics.rs) and secret-holding, id-token: write workflows.

Recommendations

Testing

• cargo-fuzz (libFuzzer) — the single highest-value gap. A lexer/parser/VM is the canonical fuzz target, and there is currently no fuzzing at all. Start with a parser target (feed arbitrary bytes, assert no panic/UB), then a differential target comparing jsse output against node/boa on the same source (directly serves the 100%-test262 goal by surfacing divergences test262 misses).

  cargo install cargo-fuzz
  cargo fuzz init && cargo fuzz add parse_roundtrip
  # target body (adjust to the real parser entry point):
  #   let _ = std::str::from_utf8(data).ok().map(jsse::parser::parse);
  cargo fuzz run parse_roundtrip -- -max_total_time=120
  

  Add a short-budget cargo fuzz run smoke job to CI and/or a nightly deep-fuzz job; commit a seed corpus from tests/.
• cargo-nextest — faster, better-isolated test runner (per-test process isolation catches global-state leaks a shared cargo test binary hides). Swap the CI cargo test step:

  cargo install cargo-nextest --locked
  cargo nextest run --release
  

Static analysis & security

• cargo-deny — Dependabot bumps versions but does not scan the RUSTSEC advisory DB, licenses, or duplicate/banned crates. Add deny.toml and a CI job:

  cargo install cargo-deny --locked
  cargo deny check advisories bans licenses sources
  

  (Use EmbarkStudios/cargo-deny-action for CI.)
• Miri — the 29 unsafe blocks in src/interpreter/types.rs and src/interpreter/builtins/atomics.rs are exactly what Miri is for: it detects UB (invalid aliasing, uninit reads, OOB) that clippy/semgrep cannot. Run it on a targeted, fast subset of unit tests (full test262 under Miri is impractical, and icu/chrono FFI paths won't run):

  rustup +nightly component add miri
  cargo +nightly miri test property_map gc object_arena atomics
  

• actionlint + zizmor — these workflows are security-critical (trust gates, id-token: write, CLAUDE_CODE_OAUTH_TOKEN). zizmor audits Actions for injection, over-broad permissions, and unpinned refs; actionlint catches YAML/expression bugs. Add a lint job:

  zizmor .github/workflows/
  actionlint
  

• CodeQL (Rust) — now GA for Rust; adds interprocedural dataflow beyond Semgrep's auto ruleset. Add github/codeql-action on push/PR + a weekly schedule.

CI/CD & PR gates

• SHA-pin all third-party actions — every action is float-tagged (actions/checkout@v7, dtolnay/rust-toolchain@stable, Swatinem/rust-cache@v2, astral-sh/setup-uv@v7, codecov/codecov-action@v7, semgrep/semgrep-action@v1, anthropics/claude-code-action@v1, …). A moved tag on a secret-holding workflow (claude.yml, claude-code-review.yml) is a live supply-chain risk. Pin to full commit SHAs (… @<sha> # v7); Dependabot keeps them current. zizmor flags these automatically.
• Least-privilege permissions: — ci.yml and nightly-test262-coverage.yml have no top-level permissions: block (they inherit the repo default). semgrep.yml and both Claude workflows already scope theirs — bring the other two in line by adding permissions: contents: read at the top of each.
• Concurrency cancellation — no workflow sets a concurrency: group, so superseded pushes keep burning runners (costly given the release build + test262 sample). Add to ci.yml (and the Claude workflows):

  concurrency:
    group: ${{ github.workflow }}-${{ github.ref }}
    cancel-in-progress: true

Agentic coding

Setup is already excellent — only small additions:

• Package a fuzzing/differential-testing skill under .claude/skills/ (mirroring the existing performance-* skills) so agents can drive cargo fuzz runs and triage crashes/divergences against node as a first-class workflow toward the 100% goal.
• Extend the PostToolUse hook (currently fmt-hook.sh only) to also run cargo clippy (changed-file scoped) alongside rustfmt, so lint failures surface at edit time rather than in CI.

Suggested next steps

• Add cargo-fuzz with a parser target + short-budget CI smoke job (highest value).
• Add cargo-deny (deny.toml) advisories/licenses/bans CI job.
• Add zizmor + actionlint job, then SHA-pin all actions and add permissions: contents: read to ci.yml + nightly-test262-coverage.yml.
• Add concurrency: cancel-in-progress to CI to stop redundant runs.
• Wire Miri over the unsafe-touching unit tests.
• Switch CI test runner to cargo nextest.
• (Later) CodeQL Rust workflow + a nightly deep-fuzz job with a committed corpus.

[pmatos]

Opened #197 covering the GitHub Actions hardening + static-analysis portion of this audit:

• Least-privilege permissions, concurrency: cancel-in-progress, and SHA-pinned actions across all workflows (persist-credentials: false on read-only checkouts).
• cargo-deny (deny.toml + weekly job), CodeQL (Rust), and an actionlint + zizmor workflow-lint job.

Validated locally (actionlint, zizmor online+offline, cargo-deny all pass).

The remaining audit items are split into their own issues so this stays green and reviewable:

• Tooling: add cargo-fuzz (parser + differential-vs-node targets) #193 — cargo-fuzz (parser + differential-vs-node)
• Tooling: run Miri over the unsafe-adjacent modules #194 — Miri over the unsafe-adjacent modules
• Tooling: cargo-nextest in CI + clippy in the PostToolUse hook #195 — cargo-nextest + clippy PostToolUse hook
• CI: migrate off the archived semgrep/semgrep-action #196 — migrate semgrep off its archived action

I've deliberately not used a closing keyword on #197, so this issue stays open to track the follow-ups.