Skip to content

ci: require crates.io dependencies to be at least 48h old - #9312

Merged
wjones127 merged 3 commits into
lance-format:mainfrom
wjones127:ci-dependency-min-age
Sep 17, 2026
Merged

wjones127 merged 3 commits into
lance-format:mainfrom
wjones127:ci-dependency-min-age

Conversation

@wjones127

@wjones127 wjones127 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Aikido endpoint protection blocks developers from pulling crates.io versions that are too young, so a dependency bump that looks fine to whoever opened the PR can leave everyone else unable to build. Today nothing catches that until it reaches someone's machine, and the failure they see is an opaque cargo resolution error rather than "this version is too new".

This PR moves the check into CI. ci/check_dependency_age.py dates every crates.io version in Cargo.lock, python/Cargo.lock and java/lance-jni/Cargo.lock, and fails if any of them is under 48 hours old. It runs as a new step in the existing cargo-deny job.

Cargo's own registry.global-min-publish-age setting does not work for this. It only constrains resolution — versions already written into a lockfile are grandfathered in and never looked at again, so it cannot tell us whether what we committed is too new.

Publish times come from the pubtime field of the sparse index, looked up in three tiers: cargo's index cache under $CARGO_HOME/registry/index/*/.cache first, which dates all three lockfiles in about two seconds with no network at all; then index.crates.io; then the crates.io API. The last tier matters more than it looks — Aikido filters the index it serves to developer machines, so on a laptop the index simply omits the offending version, and without the API fallback the script would report "cannot be dated" for precisely the versions it exists to flag.

A version that no source can date fails rather than passing silently. A few very old versions predate the index pubtime backfill; those are reported and allowed.

Escape hatch

Dependabot security updates ignore cooldown by design, so a security fix can arrive minutes after publication. ci/dependency-age-allowlist.toml exempts a specific crate and version, with a required reason so every exemption is reviewable in the diff. Entries go inert once the version passes 48 hours, so pruning them is housekeeping rather than urgent.

An entry here only unblocks CI. The version has to be allowlisted in Aikido too, or developers stay blocked locally — the file says so at the top.

Also changed

Dependabot gets a matching two-day cooldown on the cargo ecosystem so grouped weekly bumps stop proposing versions CI would reject. python/Cargo.lock and java/lance-jni/Cargo.lock are added to the paths filter in rust.yml, since otherwise a bump confined to one of those would not trigger the job that checks it.

Testing

The check fails on main right now, which demonstrates it works rather than indicating a problem with the PR. #9236 bumped opendal to 0.59.2 about six hours after that version was published; it clears the 48h floor around 02:40 UTC on 2026-09-17, after which CI here should go green on its own. Verified locally that adding the opendal family to the allowlist takes the run from 18 violations to a clean exit.

Not included

The uv, pip and maven ecosystems get no cooldown, and there is no equivalent age check for Python or Java dependencies. The same argument applies to them, but Aikido's Rust path is what is biting today.

🤖 Generated with Claude Code

Cargo's `registry.global-min-publish-age` only constrains resolution, so
versions already pinned in a Cargo.lock are grandfathered in and never
re-checked. It cannot answer whether what we committed is too new.

Add ci/check_dependency_age.py, which dates every crates.io version in the
three lockfiles from the sparse index `pubtime` field and fails on anything
under 48h old. Give dependabot a matching two-day cooldown so grouped bumps
do not land versions CI will then reject.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added A-ci CI / build workflows ci Github Action or Test issues labels Sep 16, 2026
Dependabot security updates ignore cooldown by design, so a security fix can
propose a version that is only minutes old. Without an escape hatch that PR
sits red until the version ages out, which is backwards for exactly the
updates that should land fastest.

Add ci/dependency-age-allowlist.toml. Each entry names a crate, a version and
a reason, and the reason is required so every exemption is reviewable in the
diff. Entries go inert once the version passes the minimum age.

The same floor is enforced on developer machines by Aikido endpoint
protection, so an entry here must be paired with an Aikido allowlist entry;
on its own it only unblocks CI.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@wjones127

Copy link
Copy Markdown
Contributor Author

This check will succeed in 6 hours. It's a new rule and I don't want to have to revert and republish #9236

@wjones127
wjones127 marked this pull request as ready for review September 16, 2026 21:06
lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot added K-approved Latest Gatekeeper recommendation permits acceptance. K-risk Latest Gatekeeper recommendation includes a non-blocking risk. labels Sep 16, 2026
@lance-gatekeeper lance-gatekeeper Bot removed K-approved Latest Gatekeeper recommendation permits acceptance. K-risk Latest Gatekeeper recommendation includes a non-blocking risk. labels Sep 17, 2026

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Gate recommendation: approve with a non-blocking risk.

The merge from main leaves the assessed patch unchanged, and the dependency-age scan now succeeds against the updated lockfiles; the earlier OpenDAL age window elapsed as expected.

The escape hatch remains an operational two-system contract: CI cannot verify that the matching Aikido exception was installed, so an exemption merged out of order can still leave developer builds blocked. The allowlist documents the required ordering; maintainers should confirm the Aikido change before accepting an exemption.

@lance-gatekeeper lance-gatekeeper Bot added K-approved Latest Gatekeeper recommendation permits acceptance. K-risk Latest Gatekeeper recommendation includes a non-blocking risk. labels Sep 17, 2026
@wjones127
wjones127 merged commit fea83fe into lance-format:main Sep 17, 2026
22 of 24 checks passed
@wjones127
wjones127 deleted the ci-dependency-min-age branch September 17, 2026 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-ci CI / build workflows ci Github Action or Test issues K-approved Latest Gatekeeper recommendation permits acceptance. K-risk Latest Gatekeeper recommendation includes a non-blocking risk.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants