Skip to content

Add concurrency-safe stale capture cleanup - #398

Closed
milyin wants to merge 1 commit into
fix-201from
cleanup-397
Closed

Add concurrency-safe stale capture cleanup#398
milyin wants to merge 1 commit into
fix-201from
cleanup-397

Conversation

@milyin

@milyin milyin commented Aug 13, 2026

Copy link
Copy Markdown
Owner

Summary

Adds an explicit, scoped cargo prebindgen clean operation for the stale prebindgen payloads tracked by #397.

This PR is stacked on #396 (fix-201). #396 bounds JSONL growth inside one Cargo build output; this PR handles the remaining accumulation across multiple Cargo build hashes.

  • discovers legacy build/<package>-<hash>/out/prebindgen and Cargo's new build/<package>/<hash>/out/prebindgen layout across profiles and target triples
  • removes only capture directories whose crate/features metadata and two state slots validate
  • supports --dry-run, --target-dir, repeatable --build-dir, and --manifest-path
  • leaves Cargo-owned <package>-<hash> directories and every unrelated artifact intact

Build and concurrency safety

Cleanup is explicit, never automatic from build.rs.

Before mutation it non-blockingly acquires every discovered profile lock, in Cargo's order: current .cargo-build-lock first, then compatibility .cargo-lock. If any profile is active, all locks are released and no capture is removed. This covers current shared/fine-grained locks, current exclusive locks, and Cargo 1.85's profile lock. Detected Linux NFS mounts are refused because Cargo skips these locks there.

Each producer has two alternating state files outside out/prebindgen, both tracked as rustc input dependencies. Cleanup durably advances the inactive slot before atomically renaming the capture to a tombstone. Therefore:

  • a later selection of that exact feature/target/toolchain unit is Dirty and regenerates the capture
  • an interrupted state write retains one valid slot
  • an interrupted directory removal leaves a recognized tombstone for the next cleanup
  • a warm build after regeneration remains Fresh

Legacy captures created before this protocol are skipped; a one-time ordinary cargo clean is documented for those.

Leakage test

Using one isolated target directory and example-flat with default, unstable, internal, and all features:

  • before this PR: 4 retained build hashes, each with a 4-file capture set (about 79 KiB total in this small fixture)
  • after cargo prebindgen clean: 0 out/prebindgen directories
  • selecting unstable again: exactly that producer became Dirty because its state slot changed, recreated one valid capture, then remained Fresh and byte-stable on the next build

Verification

  • cargo +1.85.0 test -p prebindgen -p prebindgen-proc-macro --all-features --no-fail-fast
  • cargo test --all --all-features --no-fail-fast
  • cargo clippy --all-targets --all-features -- --deny warnings
  • repository CI rustfmt configuration
  • strict prebindgen rustdoc warnings check
  • cargo check --target aarch64-unknown-linux-gnu -p prebindgen --bin cargo-prebindgen
  • ./examples/regen-check.sh (all committed bindings byte-identical)
  • package file-list check (the Cargo subcommand and protocol are included)
  • downstream example-cbindgen build
  • synthetic legacy/new/cross-target layouts, malformed state, symlinks, dry-run, tombstone recovery, and both lock generations
  • live Cargo build contention: cleanup exited before mutation, the build completed normally, cleanup succeeded afterward
  • two simultaneous cleaners: one removed the captures and the other safely observed zero
  • 8 rapid clean → state-dirty rebuild → fresh rebuild cycles, all successful

Closes #397

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant