Repository navigation
Retire the "fold is a cache of NATS" model the store layer still assumed - #22
Merged
Merged
Conversation
slipstream's design is a bounded log: folds are the replicas of record and NATS holds only the retained tail. The store layer and its docs still assumed the older model — "NATS is the source of truth, the snapshot is a cache" — and every cursor-expiry bug fixed in the previous PR was a component running on it. This removes the remaining instances. Behavior: - A delete_with_version tombstone (an empty-value Put) now reaches watchers, and so folds, as a Delete carrying the tombstone's revision. The fold held it as a present-but-empty key while get/scan/keys and the repairs' key listings treated it as deleted, so the key-listing repair deleted it and then re-listed it. entry() still exposes the raw tombstone for CAS. - watch_applied refuses ExpiryRepair::Restore/Auto without a store (a restore needs a fold to restore into; such a consumer silently held NATS's retained view), and warns when a cursor-less start with no repair armed re-lists a bucket that has already evicted current values. Docs: the store invariants are now fold + tail = truth, cursor-after-apply, and replica of record. The durability rationale and recovery runbooks no longer say "delete the snapshot and replay NATS" (on a bounded log that loses everything evicted); a lost or corrupt fold is rebuilt by importing an artifact. Tests: live tombstone-as-delete through the plain watch and the multi-prefix resume consumer (made race-free: attach the resume consumer before anything supersedes its cursor; 100/100 under parallel load); store-less restore refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0152kQKDdP8XRhoeqisJYpWr
The resume check sees only the stream's first retained revision, so a node whose post-cursor messages were all superseded (every key rewritten while it was down) looks expired though nothing was evicted. On an evicting bucket with no artifact ahead of the node, the restore refuses and the watch fail-stops: safe, but unavailable until the next export round. Pinned live and documented as a failure mode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0152kQKDdP8XRhoeqisJYpWr
jaredLunde
force-pushed
the
jared/replica-of-record
branch
from
October 1, 2026 23:14
22ec039 to
d7c4932
Compare
Merged
jaredLunde
added a commit
that referenced
this pull request
Oct 2, 2026
Cursor-expiry repair for bounded logs (#21) and the replica-of-record cleanup (#22). Breaking: ExpiryRepair replaces the reader argument (the old Option<reader> still compiles), Relist is refused on buckets that evict current values, Restore/Auto require a store, artifact manifests move to schema 2 (upgrade importers before exporters), delete_with_version tombstones reach watchers as deletes, and the floor guard reports CursorExpired. Also moves the lockfile off yanked fjall/lsm-tree 3.1.4 to 3.1.10 (and chacha20, spin), so the suite runs on what dependents resolve, and refreshes the README install snippets to 0.8. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0152kQKDdP8XRhoeqisJYpWr
jaredLunde
added a commit
that referenced
this pull request
Oct 2, 2026
Cursor-expiry repair for bounded logs (#21) and the replica-of-record cleanup (#22). Breaking: ExpiryRepair replaces the reader argument (the old Option<reader> still compiles), Relist is refused on buckets that evict current values, Restore/Auto require a store, artifact manifests move to schema 2 (upgrade importers before exporters), delete_with_version tombstones reach watchers as deletes, and the floor guard reports CursorExpired. Also moves the lockfile off yanked fjall/lsm-tree 3.1.4 to 3.1.10 (and chacha20, spin), so the suite runs on what dependents resolve, and refreshes the README install snippets to 0.8. Claude-Session: https://claude.ai/code/session_0152kQKDdP8XRhoeqisJYpWr Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
slipstream's design is a bounded log: NATS keeps only the retained tail, and folds are the replicas of record. The store layer and its docs still assumed an older model, "NATS is the source of truth, the snapshot is a cache". Every cursor-expiry bug fixed in #21 was a component running on that model:
on_applied's cursor was treated as durable.This PR removes the remaining instances, so the code and docs state one model.
Behavior changes
delete_with_versiontombstones reach watchers, and so folds, as deletes. A tombstone is an empty-value write, and it now arrives as aDeletecarrying its own revision. Previously the fold kept the key as present-but-empty whileget/scan/keysand the repairs' key listings treated it as deleted, so the key-listing repair deleted it and its re-list put it straight back.entry()still exposes the raw tombstone for CAS callers (the export lease's takeover depends on it).watch_appliedrefusesExpiryRepair::Restore/Autowithout a store. A restore needs a fold to restore into; such a consumer silently held only NATS's retained view.ExpiryRepair::Noneremains the explicit way to accept that view.Docs
New edge, pinned and documented, not fixed
The resume check is conservative: it sees only the stream's first retained revision. When every message after a node's cursor has been superseded (every key rewritten while the node was down), the cursor looks expired even though nothing was evicted. On an evicting bucket with no artifact ahead of the node, the watch fail-stops until the next export round. That's safe but unavailable, and a small hot bucket (heartbeat-style) hits it on a short restart.
tests/eviction.rs::supersession_only_restart_fail_stops_without_a_newer_artifactpins it against a live server. Telling supersession from eviction needs information the stream doesn't expose today (e.g. the cursor's message timestamp againstmax_age); that's a follow-up design.Tests
🤖 Generated with Claude Code
https://claude.ai/code/session_0152kQKDdP8XRhoeqisJYpWr