Skip to content

Snapshots break under test bundling; support them cross-platform #17

Description

@brainkim

The bug

libuild test bundles each test file into .libuild-test/bundle-{platform}-{id}.js (test-runner.js, ~line 90) and, on bun, passes expect through to bun:test. But bun keys snapshot storage off the executing file's path, so toMatchSnapshot() inside a bundle resolves to .libuild-test/__snapshots__/bundle-bun-29.js.snap — the committed tests/__snapshots__/*.snap files are never found.

The failure mode is nasty in both directions:

  • CI: bun refuses to create snapshots (Snapshot creation is disabled in CI environments), so every toMatchSnapshot call fails. In termdom this surfaced as 67 failures that looked like rendering regressions — the received output was pixel-perfect.
  • Local: bun silently creates fresh snapshots keyed to the ephemeral bundle path and compares against nothing. Tests pass vacuously, every run, forever. Any libuild-tested project using bun snapshots has this hole today.

termdom's stopgap is a canSnapshot gate (skip snapshot assertions when import.meta.path contains .libuild-test), with plain bun test from source as the enforcement point: bikeshaving/termdom@70e3fdc see tests/test-utils.ts.

Minimal fix: preserve snapshot identity under bundling

The per-file bundle structure (id suffix) is already there, so:

  1. Name each bundle after its source test file (color-rendering.test.js, not bundle-bun-29.js).
  2. Copy the source's adjacent __snapshots__/<file>.snap into .libuild-test/__snapshots__/ before the run — bun then finds and compares with zero format reimplementation.
  3. On --update-snapshots, copy the updated .snap back to the source tree.

The real ask: cross-platform snapshots

Snapshot testing shouldn't be a bun-only feature. libuild already owns the expect shim per platform (test-bun.js / test-node.js / test-browser.js), which is exactly the right place for a portable toMatchSnapshot:

  • One snapshot format, one resolver, keyed to source file paths — identical results on bun, node, and browser runs.
  • Node/browser platforms currently have no snapshot support at all; a shim-level implementation gives all platforms parity instead of inheriting bun's path-keyed semantics.
  • Cross-runtime snapshot agreement becomes a testable property: the same DOM render snapshotted under bun (native Bun.stringWidth) and node (pure-JS fallbacks) should produce identical output, and a shared snapshot file makes any divergence a test failure instead of an invisible skip. That's precisely the class of bug termdom's node smoke test exists to catch, and snapshots would cover it far more broadly.

The format is simple enough to own (exports[name 1] = ...; à la jest/bun), and owning it removes the dependency on each runtime's snapshot quirks (bun's CI-creation policy included).

Filed from a real-world hit: bikeshaving/termdom CI was red across 5 pushes before the cause was identified as test identity rather than rendering.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions