Fix three cursor-expiry repair gaps (eviction history, overwrite-only gaps, restore memory) - #25
Merged
Conversation
…nly gaps, restore memory - A fold now remembers, on its cursor, that it saw its bucket evict current values. Turning max_age / discard:old off later (or recreating the bucket) no longer makes the key listing look trustworthy, so Auto keeps restoring instead of deleting keys that aged out. The memo is carried in artifacts. - On a max_age-only bucket, a resume whose gap was all overwrites (nothing young enough to have aged out) resumes from the first retained revision instead of fail-stopping for want of a newer artifact. Uses the cursor message's server timestamp and the server's clock, with half of max_age as headroom. - The restore no longer holds every live key name: rewrites spill to scratch, only delete candidates stay in memory, and the listing is streamed past them (and re-checked against retention, which the model checker showed is needed once the listing moves after the diff). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0152kQKDdP8XRhoeqisJYpWr
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
max_age/discard: oldoff (or recreate the bucket), andAutorelisted and deleted keys that had aged out while eviction was on, though the local copies held the only remaining record of them. A fold now remembers, on its cursor, that it saw eviction (once the log is past its first revision), and never trusts the listing again. The flag rides in exports, and a cursor without it keeps the old encoding byte for byte. Retention is read at every start, every expiry, and every 60 s while a relist is still possible.max_age-only bucket, a cursor whose gap is all overwrites looked expired and needed an export newer than the node. A flagged cursor now keeps its message's server timestamp. If the cursor's message is at most half ofmax_ageold by NATS's clock (tsin stream info), nothing after it can have aged out, so the watch resumes from the first retained revision.discard: oldlimits and per-message TTLs still fail-stop.KvReader::for_each_key, a provided method). The model checker caught that moving the listing after the diff needs a retention recheck, so that's added and pinned (mutationNoListingRecheck).All API changes are additive (provided trait method, builder method, pub
WatchCursor::to_bytes/from_bytes), so this fits a patch release. On-disk caveat: a fold on an evicting bucket that gets the new flag can't be read by 0.8.0 (it refuses loudly, it doesn't misread). Folds on buckets that never evict are unchanged.Test plan
cargo clippy --all-targets --all-features -D warnings,cargo doccargo test --lib --tests --features fjall,rocksdb,transport. Two live eviction tests flaked once under machine memory pressure, then passed 3/3 standalone.eviction_turned_off_repair_steps_are_correct; mutationsNoEvictionMemoandNoListingRecheckare each caughteviction_turned_off_never_trusts_the_listing_again,supersession_only_restart_resumes_on_a_max_age_bucket(flips the old pinned fail-stop),supersession_gap_still_expires_when_eviction_is_not_only_by_age🤖 Generated with Claude Code
https://claude.ai/code/session_0152kQKDdP8XRhoeqisJYpWr