Lock-free fast path for buffer Pin/Unpin under concurrent readers

[krleonid]

Problem

When multiple connections concurrently scan the same table (common in our workload), every segment access requires Pin/Unpin of its block handle. The current implementation acquires a per-block mutex for both operations, even when the block is already loaded and has active readers — a state where no transition can occur.

For wide tables (250 columns, ~9K segments), this creates severe mutex contention. A query that takes 95ms single-threaded degrades to 1.2s with 20 concurrent connections, all serializing on the same block locks.

Production evidence (from mdb-engine-shadow logs):

• CPU_TIME: 0.227s vs LATENCY: 1.216s — ~1s unaccounted gap
• BLOCKED_THREAD_TIME: 0, TOTAL_BYTES_READ: 0 — all in memory, no I/O wait
• The gap comes from threads stalling on mutex acquisition under cgroup CPU scheduling

Solution

Add an optimistic lock-free fast path using atomic operations:

Pin fast path: If state == BLOCK_LOADED and readers > 0, atomically increment readers via CAS. Safe because readers > 0 prevents any concurrent unload (CanUnload() checks this).

Unpin fast path: Atomically decrement readers. If result > 0, return immediately — no state transition possible. Only fall back to mutex when readers hits 0 (eviction queue / unload logic needed).

Changes

• block_handle.hpp: Add TryIncrementReadersIfPositive() (CAS loop) and DecrementReadersAtomic() (fetch_sub)
• standard_buffer_manager.cpp: Add fast path before mutex acquisition in Pin() and Unpin()

Benchmark Results

Test: SELECT * FROM 250-column table WHERE itemId IN (1000 ids) — 95K rows, 9284 segments, all data in buffer pool (128GB memory, no eviction).

Connections	Before (avg)	After (avg)	Improvement
1	95ms	92ms	~same
3	138ms	139ms	~same
10	562ms	193ms	66% faster
20	1177ms	497ms	58% faster

The optimization is zero-cost at low concurrency and eliminates the scalability cliff at high concurrency.

Safety Argument

The fast path is safe due to the following invariant:

1. Pin: readers > 0 → block cannot be unloaded (enforced by CanUnload()). Incrementing from positive is always safe — the block remains loaded throughout.
2. Unpin: Decrementing from N>1 to N-1>0 → no state transition occurs. Only the transition to 0 triggers eviction queue / unload logic, which uses the mutex path.
3. Memory ordering: acquire on successful CAS (Pin) ensures subsequent reads of the buffer pointer see the loaded data. release on fetch_sub (Unpin) ensures all writes to the buffer are visible before the reader count decreases.

Test plan

• Benchmark: single connection — no regression
• Benchmark: 3/10/20 concurrent connections — significant improvement
• Full release build compiles and links cleanly
• Run DuckDB test suite (test/sql/storage/ tests)
• Stress test: concurrent Pin/Unpin with eviction (readers hitting 0 under load)

🤖 Generated with Claude Code