mesh relay: a message can die inside a dense cluster when no fan-out ever picks the one bridge node (mesh_forwarding flake, ~1.5%) #510

Found while smoke-testing v0.28. mesh_forwarding::everyone_hearing_everyone_does_not_multiply_the_traffic fails intermittently:

• it failed in CI on fix(examples): one identity per install, Wi-Fi Direct without Bluetooth, and peers that stay listed #508 (a demo-only change) and on an earlier unrelated run (36897622339);
• locally it fails 3 times in 200 runs at main c8a5d123.

It looks like real behaviour rather than a harness problem, so I'm filing it instead of patching the test.

What fails

The assert after run_until_quiet(24): far's inbox is empty. The network went quiet without delivering.

Trace of a failing run

Transmissions of the message, captured by temporarily printing net.transmissions when the inbox is empty:

n0 -> n4, n0 -> n1, n0 -> n2
n1 -> n2, n1 -> n4, n1 -> n5
n2 -> n3, n2 -> n5, n2 -> n1
n4 -> n1, n4 -> n3, n4 -> n2

• n6, the only device with a link to far, is never chosen by any fan-out.
• n3 and n5 receive copies, but stand down because they have already heard duplicates.
• Result: the frame never leaves the cluster.

Every node mints a fresh identity per run, so addresses, message ids and the hash-seeded jitter and target selection differ each time. That is why the failure is probabilistic.

Why it matters beyond the test

In the test, time never advances to the ACK timeout. On a device, the sender would retry with the same message id. Every cluster member already holds that id in its dedup cache, so (as the_only_device_that_can_reach_the_recipient_does_not_stand_down notes for its own case) the retries can't rescue it. Some fraction of multi-hop sends through a dense, fully connected cluster would then be lost permanently rather than delayed.

Possible directions

These are the owner's call; I haven't changed the relay:

• A forwarder doesn't stand down while it has a neighbour that no copy it heard was addressed to. n3 and n5 both border n6.
• Prefer, in fan-out selection, neighbours not yet covered by the copies heard.
• Let a sender's retransmission past the ACK timeout re-enter forwarding once (a new relay generation), instead of being suppressed as a duplicate everywhere.

Reproduce

cargo test -q -p offline-protocol --test mesh_forwarding --no-run
BIN=$(ls -t target/debug/deps/mesh_forwarding-* | grep -v '\.d$' | head -1)
fails=0; for i in $(seq 1 200); do $BIN everyone_hearing_everyone_does_not_multiply_the_traffic -q >/dev/null 2>&1 || fails=$((fails+1)); done; echo $fails