Prevent dead node accumulation in eviction queue

[krleonid]

Summary

• Skip re-insertion into the eviction queue when a block's existing entry is still valid, preventing dead node accumulation
• Track queue insertion need via an atomic needs_eviction_queue_insert flag on BlockMemory
• Fix TryGetBlockMemory to only check seq_num (not full CanUnload) so pinned blocks are retained during Purge

Problem

In production with 70K attached databases, the eviction queue accumulated 224M entries with 163M dead nodes (73%). Every 4096th unpin triggered a Purge that scanned millions of entries, blocking queries for ~5 seconds.

Root cause: every pin/unpin cycle unconditionally re-inserted the block into the queue, creating a dead node for the previous entry. Hot blocks (read frequently, never evicted) generated dead nodes linearly with access count.

Fix

• needs_eviction_queue_insert flag: set true on load from disk (no queue entry exists), cleared after insertion, re-set if eviction consumes entry without unloading
• AddToEvictionQueue: early-return when flag is false (entry still valid)
• IterateUnloadableBlocks: when eviction dequeues a valid entry but can't evict (block pinned), mark block for re-insertion on next unpin
• TryGetBlockMemory: only check seq_num match, not CanUnload() — pinned blocks are alive (not dead) and should be re-enqueued during Purge

Results (local benchmark)

Same workload (20-column table, repeated reads):

Metric	Before	After	Improvement
total_insertions	1,933	85	23x fewer
dead_nodes	1,853	5	370x fewer
dead_pct	95.9%	5.9%	16x better

Test plan

• All buffer tests pass (34 test cases, 4552 assertions)
• All storage tests pass (116 test cases, 53879 assertions)
• All eviction tests pass (10 test cases, 4095 assertions)
• Deploy to production and verify eviction queue stays bounded

🤖 Generated with Claude Code