Integration suite: ~1-in-13 unattributed flake against live BookStack #15

Filed for honesty rather than diagnosis: I could not reproduce this and cannot attribute it.

What was seen

During the final verification of the Bun migration branch, one full live run reported:

467 pass / 1 fail / 1 error

It did not recur in:

• 5 subsequent full live runs by the same agent
• 3 stress runs of the specific suite that had just gained spawned-process tests
• 2 further full live runs (473 pass / 0 fail each)

That is roughly 1 failure in 13 runs. The failing test's name was not captured, so there is nothing to point at.

Why it is being filed rather than dismissed

The suite drives a single shared live BookStack instance and includes spawned processes, a rate limiter with real timing, and health checks with a monotonic clock. Any of those can produce a rare ordering-dependent failure. Two concrete candidates, neither confirmed:

• Earlier in the same session, two separate agents observed transient failures caused by a concurrent edit landing mid-run (a compile error in src/types.ts written while the suite was executing). Both passed on re-run. If the flake was that, it cannot happen in CI or a clean checkout.
• The suite intentionally exercises throttling and waiter-shedding, which are timing-sensitive by nature.

Reproduction

Unknown. To hunt it:

docker compose up -d db bookstack
for i in $(seq 1 20); do
  RUN_INTEGRATION=1 bun test 2>&1 | tee /tmp/run-$i.log | tail -3
done
grep -l "fail" /tmp/run-*.log

Capture the failing test name and stderr when it fires.

Notes

• CI is unaffected: CI runs RUN_INTEGRATION=0, which skips the live suite entirely (501 tests, exit 0, consistently).
• The live suite is a local/manual tool (bun run test:integration), so a rare flake does not gate merges today — but it does undermine trust in the suite, which is the point of having it.