v0.8.2

Added

• Added explicit Tree::vacuum and DB::vacuum maintenance APIs. gc
  still performs logical reachability reclamation; vacuum follows it with a
  checkpoint and physically trims trailing reusable slots from blobs.dat,
  read.idx, and value.seg.
• Added store-space observability for physical allocated bytes, tail
  reclaimable slots/bytes, reusable middle slots, and vacuum relocation
  counters.
• Added GC observability for current orphan backlog, cumulative physical
  reclamation, and the deferred count from the most recent full reachability
  sweep. The latter is intentionally not reset by exact FIFO reclaim.

Changed

• Breaking stats API hardening. TreeStats, DBStats, and OpenStats are
  now #[non_exhaustive]; TreeStats/DBStats add GC lifecycle fields and
  OpenStats adds DB epoch-recovery duration. External code that constructed
  these returned telemetry structs or destructured them without .. must use
  field access/non-exhaustive matching instead. This one-time break makes
  future telemetry additions semver-compatible without compatibility shims.
• GC Prometheus names follow their actual contracts:
  holt_bm_gc_reclaimed_total is the cumulative full-sweep plus exact-reclaim
  counter, while holt_bm_gc_last_full_sweep_deferred_count is a gauge for
  the most recently completed full reachability sweep only.

Fixed

• Made snapshot copy-on-write ownership durable through persisted leases and
  epochs, preserving snapshot-shared frames across checkpoints and reopen
  recovery.
• Fenced physical GC against readers, writers, checkpoints, and reopen
  recovery. Dropping-tree and structural-reclaim work is now persisted so a
  crash cannot leak or resurrect blobs.
• Enforced child-before-parent checkpoint durability and bounded FIFO
  reclamation for crash-safe structural cleanup.
• Fixed hot-update churn reaching spillover with only reclaimable dead bytes or
  tombstoned leaves by compacting locally before migration.
• Checkpoint retries now recapture a current snapshot after stale-snapshot
  errors instead of reusing stale state.
• File-backed stores now persist a reduced manifest high-water mark and
  truncate packed accelerator/data files after tail-slot reclamation. This
  prevents long-running delete/compact workloads from staying pinned to their
  historical slot high-water mark after the tail is durably free.
• File-backed vacuum now compacts live high-water slots into lower reusable
  holes before tail truncation, carrying read.idx and value.seg
  accelerator slots with the authoritative blob slot. This reduces packed-file
  high-water bloat after delete-heavy workloads instead of relying on future
  writes to reuse middle holes.
• After slot compaction, file-backed vacuum hole-punches any remaining reusable
  middle slots on Linux, returning physical filesystem blocks without changing
  live GUID mappings.
• Background checkpointing now opportunistically auto-vacuums when
  tail-reclaimable space crosses the configured threshold.
• WAL group-commit flushing no longer issues duplicate fsyncs when a sync
  target is visible before the corresponding committed ring records are
  readable by the flusher. The concurrent durability regression now checks the
  stable invariant (syncs <= appends) instead of assuming scheduler-dependent
  batching.

Upgrade notes

• Forward upgrades from 0.7.3 retain the authoritative manifest v6, WAL v3,
  blob-frame, DB catalog, and checkpoint-image formats. The new read.idx and
  value.seg files are rebuildable accelerators rather than recovery truth.
• In-place downgrade from 0.8.2 to 0.7.3 is not supported. Version 0.8.2 can
  write a WAL batch encoding that 0.7.3 does not recognize, and downgrading
  would also restore the older snapshot-reclamation behavior fixed here. Use a
  logical checkpoint image and a fresh directory if rollback is required.

Validation

• cargo fmt --all --check
• cargo fmt --manifest-path tools/soak/Cargo.toml --check
• git diff --check
• cargo clippy --workspace --all-features --all-targets --locked -- -D warnings
• cargo clippy --manifest-path tools/soak/Cargo.toml --locked -- -D warnings
• cargo test --workspace --lib --tests --examples --locked
• cargo test --workspace --doc --locked
• RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps --locked
• cargo package --locked
• NoKV compatibility probe: 0.7.3 data-directory reopen, continued writes,
  checkpoint/reopen, and 0.7.3 checkpoint-image restore under 0.8.2.

v0.8.1

Fixed

• Hardened corrupted on-disk root pointers on read-side entry paths. Lookup,
  routed indexed reads, and blob-topology scans now return NodeCorrupt
  instead of tripping a debug assertion when a persisted root child pointer is
  the null sentinel.
• Added system fault regressions for manifest corruption, authoritative blob
  root corruption, read-index corruption fallback, value-segment corruption
  fallback, store-directory removal, permission denial, and injected ENOSPC
  checkpoint write failure.

Changed

• Expanded fuzz coverage for metadata-shaped delimiters (/, :, |, #,
  @, \) and public Tree::is_prefix_empty.
• Added a dedicated read_index_model fuzz target that exercises multi-blob
  spillover, checkpoint, reopen, constrained buffer pools, indexed positive and
  negative reads, prefix liveness, delimiter scans, stale read-index fallback,
  and repeated checkpoint/reopen cycles.
• Extended the soak harness to mix public prefix-empty checks and metadata
  delimiter scans in both single-tree and DB modes.
• Extended the Verus model with delimiter-tagged component-summary invariants
  and prefix-liveness tri-state safety.

Validation

• cargo fmt --all --check
• cargo fmt --manifest-path tools/soak/Cargo.toml --check
• git diff --check
• cargo clippy --workspace --all-features --all-targets --locked -- -D warnings
• cargo clippy --manifest-path tools/soak/Cargo.toml --locked -- -D warnings
• cargo test --workspace --locked
• cargo +nightly fuzz run atomic_model -- -runs=64
• cargo +nightly fuzz run db_model -- -runs=48
• cargo +nightly fuzz run read_index_model -- -runs=8
• VERUS=/Users/guochengsong/.local/bin/verus ./verified/verify.sh
• cargo run --manifest-path tools/soak/Cargo.toml --locked -- --mode normal --dir target/holt-soak-local-normal --reset --duration-secs 2 --keys 1000 --ops 4000 --threads 2 --buffer-pool 16 --wal-sync false
• cargo run --manifest-path tools/soak/Cargo.toml --locked -- --mode db-normal --dir target/holt-soak-local-db --reset --duration-secs 2 --keys 1000 --ops 4000 --threads 2 --buffer-pool 16 --wal-sync false

v0.8.0

Added

• Added checkpoint-built read indexes and value segments as the named
  acceleration layer for page-granular indexed reads. read.idx stores the
  routable directory, filters, component summaries, crossings, and inline small
  values; value.seg stores larger value payloads referenced by the index.
  Both files are rebuildable accelerators and never the source of truth.
• Added read-index component summaries for delimiter/prefix rollups, allowing
  some list(..., delimiter="/") paths to emit CommonPrefix entries without
  pinning every child blob.

Changed

• Breaking observability/API cleanup. Public stats, metrics, and benchmark
  output now use read_index_*, read_page_*, and value_segment_* names
  instead of the older cold_* / sidecar terminology.
• Breaking accelerator file rename. File-backed stores now use read.idx
  and value.seg instead of cold.idx and cold.val. These files are
  advisory and can be regenerated from blobs.dat; committed data, WAL, and
  manifest recovery remain authoritative.
• Renamed the internal cold_read module to read_index and aligned walker,
  buffer-manager, blob-store, tests, README, and metrics terminology around the
  indexed-read model.

Validation

• cargo check --workspace --all-features --locked
• cargo clippy --workspace --all-features --all-targets --locked -- -D warnings
• cargo test --workspace --all-features --locked
• cargo test --test wal_tree_integration --all-features --locked

v0.7.3

Fixed

• Fixed read-only view / snapshot captures allocating persistent blob
  slots. Snapshot roots are now ephemeral in-memory frames; repeated scans or
  views no longer grow blobs.dat unless concurrent live writes actually fork
  shared frames.
• Added a regression test that repeatedly captures read views on a file-backed
  tree and verifies checkpointing does not increase the backing store size.

v0.7.2

Fixed

• Fixed nightly DB/crash soak failures where stale cross-blob routes could reach
  a delete-fenced blob. DB no longer runs GUID-only background auto-merge;
  DB-wide merge stays rooted in live trees through explicit compaction. The
  route cache is now restricted to root-child crossings, and walkers restart
  from the root when they encounter a delete-fenced child instead of treating it
  as NotFound.
• DB::view now uses the same fenced snapshot capture path as Tree::view,
  so multi-tree views cannot capture parent/child topology from mixed write
  generations.
• Merge eligibility now rejects snapshot-shared child blobs, preventing
  maintenance from deleting a blob still referenced by a live snapshot.

v0.7.1

Fixed

• Durability: an acknowledged write could be lost after a crash. The
  checkpoint's WAL-truncate gate (maybe_truncate) only checked the
  BufferManager dirty/flushing/pending counters, not the store's own deferred
  durability (needs_flush). The I/O worker retires a written-through blob
  right after the pwrite but before the data fsync + manifest-delta persist,
  so the WAL could be truncated while a just-written blob's new slot mapping
  was still only in the in-memory manifest — leaving a crashed reopen with the
  acknowledged record in neither the WAL nor manifest.log. The gate now also
  waits on needs_flush(), mirroring the existing run_round early-skip
  guard. Surfaced by the nightly crash-soak; 0.7.0's lazy routing compaction
  amplified the exposure by re-writing the root blob every round.
• Durability: a torn WAL tail is now truncated on reopen. Previously the
  writer reopened with O_APPEND over the torn bytes, turning a partial tail
  record into a mid-log torn record that a later replay would stop at,
  silently stranding every acknowledged record written after it. replay_wal
  now truncates the WAL to the last complete record on open — standard WAL
  recovery; the torn record was never acknowledged (the crash preceded its
  fdatasync), so nothing durable is lost.

v0.5.5

Fixed

• FileBlobStore::open now takes an exclusive flock(2) on
  <data_dir>/store.lock and holds it for the lifetime of the
  instance. Two live instances on one data directory previously
  replayed manifest.log into the same next_slot, assigned the
  same slot to different blob GUIDs, and appended conflicting set
  deltas — permanently poisoning the manifest (every later open
  failed with FileBlobStore::Manifest::duplicate slot) while the
  colliding frames overwrote each other in blobs.dat. Since 0.5.0
  even read-only snapshots persist frozen root frames, so the
  overlap window of a plain handover (store = reopen(path)) was
  enough to trip this. Open now waits up to 5 s for the previous
  instance to finish dropping, so handover reopen serializes; a
  genuinely concurrent second opener fails with a clear
  WouldBlock error instead of corrupting the store. Same-process
  double-opens are caught too (flock is per open-file-description),
  and the kernel releases the lock if the holder crashes.

v0.7.0

Added

• BlobStore::read_blobs — a batched full-frame read on the public trait.
  The default loops over read_blob; stores override it for device
  parallelism (Linux io_uring submits one ring batch, the pread store
  fans the reads across worker threads). Used by the cold-scan read-ahead.
• Page-granular cold reads. A point lookup on routed (write-cold) data fetches
  only the header page, the blob's routing region, and the one leaf page its
  descent reaches (~18 KB mean) instead of pinning the whole 512 KB frame
  (~27× less cold I/O). The routing region is built at compaction.
• Per-blob bloom filter at the tail of the routing region (read for free with
  it): cold negative lookups answer NotFound without a leaf-page read.
  No false negatives. Additive on disk (bloom_len == 0 = no bloom).
• Bounded resident routing cache: routing regions for hot blobs are held in a
  byte-bounded cache so repeat cold reads skip the routing-region read.
• Cold-scan I/O read-ahead: range scans prefetch upcoming child blobs through
  pin_scan_many → batched read_blobs, reading them at the device's natural
  queue depth instead of one serial round-trip each.

Changed

• Breaking — on-disk format. Manifest format v4 → v6 (the blob header now
  records the per-blob routing-region geometry). Older manifests are not
  migrated — the loader rejects any non-v6 manifest, so a store written by
  0.6.x cannot be opened by this release (and a v6 store cannot be opened by
  0.6.x). Pre-1.0 with no production deployments; recreate the store on upgrade.
• Compaction builds (and the read path validates) the routing region + bloom;
  structural write-path mutations de-route a blob, and write-cold blobs are
  re-routed lazily by maintenance.

Removed

• Removed the cold.idx cold-read sidecar — the in-blob routing region is now
  the sole cold-read path.
• Removed the docs/design/ working notes. Rationale for shipped features lives
  in commit messages; rationale for rejected paths (io_uring WAL rewrite, the
  two blob-fill fixes) lives in git history.

v0.6.0

Added

• Added a shared in-memory WAL byte ring as the only append path. Foreground
  writers reserve byte ranges concurrently, copy encoded records directly into
  the ring, and a single flusher drains committed byte prefixes to the WAL file.
• Added loom coverage and crash-soak validation for the WAL ring's
  reserve/publish/flush ordering, including multi-publisher gap-safety checks.

Changed

• Flattened leaf storage and child addressing for the persistent ART layout:
  small records can stay inline, child body offsets are stored directly, and
  inner-node child scans use compact u16 addressing with SIMD fast paths.
• Reworked the WAL group-commit plumbing around ring backpressure instead of a
  per-record channel/worker handoff, reducing the concurrent durable write
  bottleneck while preserving the existing WAL record format and replay reader.
• Tightened journal validation so empty or over-capacity records are rejected
  before reservation instead of relying on debug-only assertions.

Removed

• Removed the legacy WAL channel backend and its transitional design documents.
  The ring-backed journal is now the only implementation.
• Removed rejected experiment notes that were no longer part of the supported
  architecture.

Validation

• cargo test --workspace --all-features --locked
• cargo clippy --workspace --all-features --all-targets --locked -- -D warnings
• RUSTFLAGS="--cfg loom" cargo test -p holt --lib journal::ring::loom --locked

v0.5.4

Removed

• Removed the external-log state-machine surface from holt core:
  Durability::StateMachine, DB::commit_durable,
  Tree::commit_durable, durable_applied_index, DB::scatter,
  DB::scatter_independent, and the file-store DurableManifest
  trailer.
• Checkpoint images are now pure DB archive/transfer images. They
  contain family key/value data and no longer carry an external
  applied_index.
• Atomic DB/Tree batches always use the exclusive mutation gate again;
  holt no longer has a StateMachine-only relaxed batch mode.