zu2: map_settled makes one VMA per log page, so the mapping count grows with the database

[tamnd]

Log::remap_settled maps each settled 4 MiB page with its own mmap and a null address hint, so the kernel picks an address for each one. Nothing ever unmaps a page above the compaction floor: evict_behind steps over a mapped victim by design since #759, and the only unmap paths are reclaim_to, which clears the slots under a punched range, and the drop.

So with map_settled on, the number of mappings the process holds rises with the size of the log and never comes down.

Measured on server1 rather than argued from the manual page. A small C program that maps a file a page at a time the way remap_settled does, counting the lines of /proc/self/maps either side:

64 pages mapped one at a time: 23 map lines before, 87 after, 64 added
512 pages mapped one at a time: 23 map lines before, 535 after, 512 added

One VMA a page, no merging. The merge does not happen because the kernel hands out addresses top down while the log maps ascending file offsets, so the addresses run the opposite way to the offsets and two neighbours in the file are never neighbours in the address space.

Two costs, and the second is the one that matters:

• Every page fault walks the VMA tree, and the tree is now O(pages) instead of O(1). This is the fault that Every resident page is anonymous memory, so zu2 holds 65x more of it than lmdb and the kernel cannot reclaim any of it #757 traded a pread for, so it is exactly the path the option is supposed to make cheaper.
• vm.max_map_count is 65530 by default, which is 256 GiB of log. Past that mmap refuses, and it refuses for everything else in the process too, not only for this. server1 has it at 1048576, so no run of record has hit it, and a host with the default would.

The fix is to reserve the span once and put the pages into it, so the addresses run the way the offsets do. Measured the same way:

512 pages into a reservation: 23 before, 24 reserved, 24 filled, 535 with every other one dropped

One anonymous PROT_NONE MAP_NORESERVE reservation, then each page mmaped over it with MAP_FIXED, and 512 mappings collapse into one VMA. MAP_NORESERVE and PROT_NONE so the reservation is address space and not memory, which is free on a 64 bit host.

The last number in that line is the honest caveat and it is why this needs care rather than a patch. Drop every other page back to PROT_NONE and the single VMA splits into 512 again, so the collapse holds only while the mapped region is a contiguous run. That happens to be what remap_settled produces today, since remap_from is a monotonic cursor and eviction steps over mapped pages rather than taking them, but it is a property of the current policy rather than of the design, and anything that starts unmapping in the middle gets the old behaviour back without a test noticing.

So this issue is three pieces of work:

• Reserve max_pages of address space at open when map_settled is on, and map into it with MAP_FIXED.
• A test that asserts the mapping count, which on Linux means reading /proc/self/maps and on macOS means skipping. It is the only way this stays fixed.
• Decide what a page below the mapped run means. reclaim_to already drops slots under the punched range, and those are at the bottom of the run rather than in the middle of it, so the run stays contiguous; that wants stating in the code rather than being true by accident.

Windows has no map_read at all, so none of this applies there.

[tamnd]

Implemented in d13f725 on zu2-planes.

The log takes the address space once at open, PROT_NONE, anonymous and MAP_NORESERVE, sized at the span cap plus two pages. A settled page goes in at page % reserved_pages with MAP_FIXED and comes back out as PROT_NONE rather than through munmap, because a munmap would take the range out of the reservation and let the next unrelated mmap anywhere in the process land in the middle of it.

On the third piece of work above, deciding what a page below the mapped run means. The ring is the span cap plus two, so two live pages can never want the same address, but that is an invariant living in another function and what it costs to be wrong is a MAP_FIXED over a mapping a reader is inside. So map_page checks the one page that could collide, the one reserved_pages below, and refuses if it is still a live mapping. A refusal costs a page that stays on the heap and nothing else. The check should never fire and the point of it is that it does not have to be argued about.

A refused reservation, map_settled off and a platform with no mmap all land in the same place, a null hint and a munmap, which is what the code did before this.

tests/mapping.rs is new. It is deliberately not behind a crate level cfg: a test file behind one is not type checked on the machine it is written on, which is how a Linux only test goes stale without anybody noticing. The region counts come from /proc/self/maps and are skipped where there is nowhere to ask, and the rest runs everywhere, including that the records read back through the mappings and that ten opens and closes give the regions back.

What is not done, and I would rather say it than let the commit imply otherwise: the region halves of both tests have not run on Linux. Every host of record is mid rival benchmark and a cargo build is load I am not putting on a published number. s3-768.sh is queued on server3 behind the sqlite workload E retake and the #757 sweep, and it runs the test twice, once as committed and once with file::reserve forced to refuse, because a regression test nobody has watched fail asserts whatever the code happens to do. Numbers when it lands.

[tamnd]

Second box done at 3eb773c. reclaim_to and evict_behind both drop a mapped page from the bottom of the run, from..upto being a prefix by construction in one and the head boundary walking up in the other, and neither of them said so anywhere. Now they do, with the caveat from the server1 measurement attached: 512 pages collapse to one region, and dropping every other one splits it back into 512, so the collapse is a property of the drop policy rather than of the reservation. A read bit or any per page policy takes the region count back and has to say so.

The test on Linux is still the open box. s3-768.sh is on server3 as pid 2476224 with its log at /root/bench/map768.log, gated behind the sqlite E retake at a million records and one thread, which is still going. It fetches and resets to origin/zu2-planes before it builds, so it runs the current branch and not the one it was written against.

[tamnd]

One more hole in the region, found while reading remap_settled for
#769's test and filed as #776.

The reservation only stays one region if every page that leaves it
leaves as PROT_NONE. release_page and Drop both dispatch on
within_reservation, but the lost race path in remap_settled, where a
mapping is made and then thrown away because an evictor took the slot
before it could be published, called plain crate::file::unmap. That
returns the range to the process, so the reservation is three regions
instead of one and an unrelated mmap can land in the gap.

Fixed in edb47aa. Db::remap_lost now counts the path so a test can say
it was reached, and a run with map_settled on under a four page memory
bound reached it nine times in a quarter of a second, so this was a live
hole and not a corner.

The region assertions here are still unmeasured on Linux. The queued
server3 run of tests/mapping.rs now covers three tests rather than
two, since it resets to origin/zu2-planes before it builds.