topic 00: kBigPrime is not prime, and the residue is what matters

[AviAvni]

reading-rocksdb-db-bench.md said db_bench scatters hot keys "by multiplying by a large prime", pointing at line 7119 of tools/db_bench_tool.cc. It is not a prime: 0x5bd1e995 = 13 × 199 × 457 × 1303, the mixing constant m of 32-bit MurmurHash2, which the same tree carries at util/murmurhash.cc:97 and :153. This corrects the claim and adds the demo that establishes it.

kbigprime_demo.py reimplements GetRandomKey (7103-7121 at facebook/rocksdb@7c80a5a) and swaps the constant on 7117. Stdlib only, seeded, no arguments, ~15 s.

Measured

--num=1000000 --read_random_exp_range=10, 2M draws, modelling 35 keys per 4 KiB block (16 B key + 100 B value — defaults at db_bench_tool.cc:388, :337, include/rocksdb/table.h:400).

The multiply costs 13.9× the block-cache working set for identical traffic. The 6,589 keys carrying 50% of requests sit in 277 blocks unmapped and 3,855 mapped. The ceiling on that ratio is the 35 keys per block: unmapped, 35 adjacent hot keys ride in on one block read; mapped, each costs its own. That is FAST'20 §7.1's "extremely large number of block reads" as a number — and the paper's complaint about db_bench is that 7119 does it deliberately.

What makes a multiplier work is M mod FLAGS_num, not its size. The multiply is a stride walk, so only the residue matters. The larger, genuinely prime 1000000007 is ≡ 7 and lands hot keys 5 to a block: 1,615 blocks against kBigPrime's 3,855. Coprimality is separately necessary — 65536 shares 2^6 with 10^6 and loses 98% of the key space at this --num.

multiplier	gcd	M mod num	distinct	span%	ws blocks
1 (control)	1	1	426,075	0.12%	277
65,536	64	65,536	15,625	99.85%	2,845
1,000,000,007	1	7	426,075	0.83%	1,615
2,654,435,761	1	435,761	426,075	99.88%	6,580
1,540,483,477	1	483,477	426,075	99.86%	3,855

Among residues that do cover the key space, this one is poor. Its stride walk's smallest gap falls to 16 keys — under the 35 per block — from 1,394 hot keys on, so 79% of neighbouring hot keys share a block: 3,765 blocks where a uniform random scatter of the same 6,589 keys gives ~5,885. It de-localizes worse than chance. The prime nearest 2^32/φ holds out to 9,023 hot keys and beats it. Crossovers are measured by walking the stride and cross-checked against the continued-fraction convergent denominator, which they land two points after, as they should. The honest general statement: no fixed multiplier keeps hot keys out of shared blocks forever — the golden ratio only maximises how long.

--num=13000000 collapses it. 13 divides the constant, so every key drawn is a multiple of 13, 12/13 of the database is never read, and db_bench prints no warning. Line 7118's 64-bit overflow does not rescue it: the product only wraps above ~12 billion keys.

Checks

• python3 tools/check-reading-depth.py passes on the guide.
• mdbook build renders the new paragraph.
• Script exits 0 and every number it prints is computed by it.

🤖 Generated with Claude Code