Replies: 1 comment
|
Quick follow-up: I've published a broader writeup that includes Prisma's numbers alongside nine other projects. Happy to adjust how the project is characterized if anything reads wrong: https://fixture.dev/writing/we-mutation-tested-10-projects |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
While mutation-testing this repo (I'm building a tool in this space), I ran the vitest suites of the 28 hermetic packages against 1,038 mutants. 866 were caught, which is strong. The 169 survivors are heavily clustered, and three files hold about a third of them:
packages/client-engine-runtime/src/parameterization/parameterize.ts(27 survivors): swapping&&/||in the selection guards (e.g.:154,:224) and negatingsel.selection(:158) all pass. The parameterization logic has broad line coverage but the tests mostly don't distinguish these branch outcomes.packages/param-graph/src/serialization.ts(17 survivors): serialize and deserialize each have coverage, but mutants that corrupt one direction survive because no test round-trips a value through both and compares.packages/get-platform/src/getPlatform.ts(15 survivors): the platform and libssl detection branches are permissive under mutation. Examples: negatingplatform === 'freebsd'(:76), flipping the libssl filename guards (:371,:378,:382), and inverting the linux distro check (:507) all pass. On CI runners these branches mostly collapse to one path, so the others go unobserved.Smaller clusters:
generator-helper/src/GeneratorProcess.ts(10) andadapter-mssql/src/connection-string.ts(9).The round-trip suggestion for serialization.ts is probably the cheapest win: a single property-style test (serialize, deserialize, deep-equal) would kill most of that file's survivors at once.
Scope note: this covers only the hermetic packages. client, migrate, cli, and the engine-integration paths run under separate DB-backed suites that I did not execute, so nothing here speaks to those.
Happy to share the full survivor list or turn the round-trip test into a PR if that's welcome. No action needed otherwise, 83.7% with this operator set is comfortably above most of what I've measured!
All reactions