zu2: re-adding an edge that is already there costs a live log record, so a MERGE shaped workload grows the log without bound

[tamnd]

Session::edge appends a record for every call, and compact::keep_edge decides an add record is live by asking whether the adjacency has the edge:

if !session.neighbours(Direction::Out, src, |n| n.binary_search(&dst).is_ok()) {
    return Ok(false);
}
session.append_untracked(record::KIND_EDGE, payload)?;

Nothing asks whether the adjacency got the edge from this record or from the two hundred below it, so all of them are live and every pass copies all of them forward. The live set is then a record per edge operation rather than per edge, over a graph that is not growing at all.

That is not an exotic workload, it is the normal one. Every MERGE shaped write re-asserts an edge that is usually already there, and so does any loader that is run twice.

crates/zu2/tests/edgedup.rs measures it. One edge, then the same edge again a hundred thousand times, compacting throughout so the span is what a pass could not get rid of rather than what has not been looked at:

span after one edge 128, after a hundred thousand re-adds 4800152, migrated 0

48 bytes an operation that no pass will ever reclaim. It also has a second face, which is where this was actually noticed: tests/graphracy.rs with four threads writing edges into 1024 nodes ends in LogFull { span: 19, max: 16 } with under 5 MB of live padding, because the rest of the log is duplicate add records.

The fix on the write side is that a write which changes nothing does not go on the log. Under the node's edge order lock, an add of an edge that is there and a remove of one that is not are both no-ops in memory, so they can return without appending. With that, the same test reads 128 bytes after a hundred thousand re-adds.

What has to be true when this is closed:

• a no-op edge write does not append
• tests/edgedup.rs, which fails on the old behaviour with 4800152 against 128
• the pass drops duplicates it meets in a log written by a build that did not have this, since the write side fix only stops new ones. Every copy a pass makes goes to the tail, above everything in the region it is reading, so remembering which pairs it has already copied in this pass and dropping later records for them is sound
• what the check costs an edge write, measured on a host. It is a binary search over a sorted slice against an append, so the expectation is that it does not show, but the expectation is not a number

[tamnd]

Write side fixed at a964620 on zu2-planes.

span after one edge 128, after a hundred thousand re-adds 4800152   before
span after one edge 128, after a hundred thousand re-adds 128       after

The check sits under the node's edge order lock and inside the epoch, so it and the append are one step against another writer on the same node, and reading the neighbours cannot meet a block a grow is retiring.

tests/graphracy.rs goes back to max_pages: 16 with it, which is where it started before the duplicates pushed it over: twelve runs of 200000 operations a thread across four threads, green, where 16 pages used to end in LogFull { span: 19, max: 16 } with under 5 MB of live padding.

Suite 32 binaries, 196 passed 0 failed.

The pass side stays open and it only matters for a file written by a build without this, since nothing writes a duplicate any more. The shape it wants: every copy a pass makes goes to the tail, above everything in the region it is reading, so a set of the pairs already copied in this pass makes a later record for the same pair redundant, and a replay that applies the tail copy last ends in the same state. It needs a bound on the set, because a region is a quarter of the span and one entry per 48 byte record is not something to size off the log.