Context
PR #5 pins the direct dependencies for each engine so runs are reproducible, but a few build inputs still float. Several code comments (in ruby/Gemfile and scripts/requirements.txt) reference a "compiled lockfile" follow-up. This issue tracks deciding and finishing that work, per engine.
The guiding principle: a lockfile only earns its keep when the manifest expresses ranges that resolve at install time and can resolve differently later. Where every version is already pinned exactly, the manifest is the lockfile.
Per-engine status
Java (java/pom.xml) — already reproducible, no action.
Every dependency and plugin version is pinned exactly (jedis 5.2.0, lettuce 6.5.0.RELEASE, valkey-glide 2.2.3, jackson 2.18.0, …). No ranges or LATEST/RELEASE keywords, no dependency SNAPSHOTs. Maven released artifacts/poms are immutable and transitive resolution is deterministic, so an all-exact pom already provides what a lockfile would. Maven also has no idiomatic lockfile mechanism.
Ruby (ruby/Gemfile) — the one real gap.
redis "~> 5.0" still floats across 5.x, and transitive gems float without a checked-in Gemfile.lock. Since the redis client is a driver under measurement, a minor bump can move benchmark numbers. Options:
- (a) Pin
redis to an exact version in the Gemfile (matches the Java approach; one-line change, no lockfile machinery).
- (b) Check in
Gemfile.lock (Bundler's recommendation for app-style tools; also pins transitives).
Python (scripts/requirements.txt) — lowest value.
Direct deps (matplotlib, numpy) are pinned exactly; transitives (contourpy, pillow, fonttools, …) float. These scripts do graph generation / matrix orchestration — off the measurement path — so their float is cosmetic at most. A compiled lockfile (pip-compile / uv) is optional and low priority.
Decision needed
Choose the locking strategy per ecosystem (exact-pin vs checked-in lockfile), then reconcile the in-code comments that currently advertise this follow-up so they match whatever we decide.
Follow-up to #5.
Context
PR #5 pins the direct dependencies for each engine so runs are reproducible, but a few build inputs still float. Several code comments (in
ruby/Gemfileandscripts/requirements.txt) reference a "compiled lockfile" follow-up. This issue tracks deciding and finishing that work, per engine.The guiding principle: a lockfile only earns its keep when the manifest expresses ranges that resolve at install time and can resolve differently later. Where every version is already pinned exactly, the manifest is the lockfile.
Per-engine status
Java (
java/pom.xml) — already reproducible, no action.Every dependency and plugin version is pinned exactly (
jedis 5.2.0,lettuce 6.5.0.RELEASE,valkey-glide 2.2.3,jackson 2.18.0, …). No ranges orLATEST/RELEASEkeywords, no dependency SNAPSHOTs. Maven released artifacts/poms are immutable and transitive resolution is deterministic, so an all-exact pom already provides what a lockfile would. Maven also has no idiomatic lockfile mechanism.Ruby (
ruby/Gemfile) — the one real gap.redis "~> 5.0"still floats across 5.x, and transitive gems float without a checked-inGemfile.lock. Since the redis client is a driver under measurement, a minor bump can move benchmark numbers. Options:redisto an exact version in the Gemfile (matches the Java approach; one-line change, no lockfile machinery).Gemfile.lock(Bundler's recommendation for app-style tools; also pins transitives).Python (
scripts/requirements.txt) — lowest value.Direct deps (
matplotlib,numpy) are pinned exactly; transitives (contourpy, pillow, fonttools, …) float. These scripts do graph generation / matrix orchestration — off the measurement path — so their float is cosmetic at most. A compiled lockfile (pip-compile / uv) is optional and low priority.Decision needed
Choose the locking strategy per ecosystem (exact-pin vs checked-in lockfile), then reconcile the in-code comments that currently advertise this follow-up so they match whatever we decide.
Follow-up to #5.