Question: why MAP_PRIVATE for first BRANCH instead of userfaultfd for everything?

[Bug-Keeper]

Reading the design — MAP_PRIVATE mmap of memory.bin as the CoW primitive, then optionally userfaultfd (v0.3 scaffolding per docs/design/userfaultfd.md) for live branching. The current code lands on MemoryBackend::File as default and MemoryBackend::Userfault is bail-only (crates/forkd-vmm/src/lib.rs:1035-1040).

Two related questions for the design discussion:

1. What's holding you back from defaulting Userfault for spawn, not just BRANCH? The prewarm flag (prewarm_scratch_dir) is explicitly there to compensate for the cold-cache penalty of MAP_PRIVATE — which userfaultfd dodges entirely because pages are served on demand. If the handler is implemented, would there be any reason to keep File as the default at all?

2. Why a per-child uffd handler socket rather than a daemon-wide one? The Userfault { handler_sock: PathBuf } shape in MemoryBackend (lib.rs:284-289) implies one socket per child. A single handler with a routing layer could amortize the metadata bookkeeping across all children of a tag — particularly relevant for the 100-children fanout numbers in the README. I'd expect the design doc to address this trade-off but couldn't find it.

I'm asking because I'm thinking about contributing to the uffd path for v0.3 and want to make sure I don't go in the wrong direction.

[WaylandYang]

Good questions — and the short answer probably saves you a wrong turn: the MemoryBackend::Userfault arm you're reading is v0.3 scaffolding that was never wired into the restore path. It's referenced in exactly two places — the enum variant and a bail! arm in restore_many_with — so setting it just errors out. Don't build on it.

The live-fork primitive that actually shipped is the mode="live" BRANCH path (forkd-uffd's WpBranch): it arms UFFDIO_WRITEPROTECT on the running source's guest memory and asynchronously copies dirty pages. It's built on top of the MemoryBackend::MemfdShared backing — a per-child MAP_SHARED memfd ("shared": true) served by the vendored Firecracker fork, which the controller also mmaps so it can observe the guest's writes. That's the code to contribute to — note MemfdShared is the memory backing, the primitive is WpBranch.

1. Why not default uffd for spawn: spawn stays File deliberately. At fan-out, mmap(memory.bin, MAP_PRIVATE) across N children gives kernel-managed CoW page-cache sharing — clean pages are shared across children, only dirtied pages are copied per-child. A naive UFFD spawn backend would serve every faulted page via UFFDIO_COPY, which copies, so each of N children ends up with a private copy of every page it touches — kernel-CoW is gone, and you lose the memory-efficient fan-out for a negligible pause-window win. It also wouldn't shrink the pause-window: the bytes-to-move are unchanged, the work just relocates (see docs/design/userfaultfd.md:36-48). UFFD's payoff is capturing a running VM's writes (BRANCH), which is exactly where the live path uses it.

2. Per-child vs daemon-wide handler: the per-child handler_sock is part of that un-wired v0.3 shape, so don't anchor on it. You're right that the design doc doesn't weigh one-socket-per-child against a shared/daemon-wide handler or discuss amortizing handler setup cost — it's a genuinely open question and a good follow-up to raise.

Happy to point you at the live-BRANCH (WpBranch) code if you want to dig in.