Refuse a channel change while a pending dataset is in place - #272
Conversation
agners
left a comment
There was a problem hiding this comment.
Nice catch!
However, I wonder if simply bumping the active timestamp and write another pending dataset is a good idea: Since any pending dataset needs time to propagate through the network, what happens if a device doesn't receives that newer update in time (before delay expires)? 🤔 The device would then switch to the channel from the first pending dataset, and never receive the second update.
Feeding in this concern into Claude Code in the ot-br-posix repo says:
What happens when you overwrite a pending dataset
A second MGMT_PENDING_SET needs a newer Pending Timestamp; once the leader accepts it, it propagates like any dataset. On each device that receives it, the stored pending dataset is replaced and the delay timer restarts from the new dataset's Delay Timer TLV (
PendingDatasetManager::StartDelayTimer,src/core/meshcop/dataset_manager.cpp:939). So for a device that got update 2 in time, channel change 1 simply never happens.Propagation is eventually-consistent, though: the leader has no barrier that confirms every device holds the latest pending dataset before the old delay fires. Routers usually learn of it within seconds, but a sleepy child only learns at its next poll/child-update. If dataset 2 arrives on the leader, say, 10 s before dataset 1's delay expires, any device that doesn't hear about it in those 10 s will run PendingDatasetManager::HandleDelayTimer (dataset_manager.cpp:955), promote dataset 1 to active, and hop to channel A — while the rest of the network later hops to channel B. You now have two partitions on two different channels, both with valid (same-key) datasets, differing only in Active Timestamp (T1 on channel A < T2 on channel B).
How the stragglers recover
Partitions on different channels can't merge through normal MLE advertisements — cross-channel healing goes exclusively through MLE Announce:
- AnnounceSender (src/core/thread/announce_sender.cpp, enabled by default via OPENTHREAD_CONFIG_ANNOUNCE_SENDER_ENABLE 1): every router-eligible, rx-on device runs a trickle-driven cycle that transmits Announce messages across all channels in the supported mask. So routers of the channel-B partition will
periodically announce (channel B, PAN ID, Active Timestamp T2) on channel A too.- A device on channel A that hears an Announce with a newer active timestamp does an announce-attach: it switches to the announced channel/PAN ID and attaches there (Mle::AnnounceHandler, src/core/thread/mle.cpp:5873 and StartAnnounceAttach at mle.cpp:5929). If the attach fails it reverts to its previous channel,
so it isn't a one-way jump.- After a successful announce-attach, the device announces back on its previous channel (InformPreviousChannel, mle.cpp:5975), which helps pull along other stragglers on channel A.
- Conversely, if a channel-B device hears a stale Announce (timestamp T1), it responds with its own newer Announce (kSendAnnouceBack, mle.cpp:5866), correcting the stale side directly.
So in a stock build the channel-A partition converges back onto channel B, typically within minutes (the AnnounceSender trickle interval is ~11 min per sweep, faster with jittered per-channel steps).
When devices can genuinely get stuck
- AnnounceSender compiled out. Some vendor builds disable it to save airtime/power. Then nobody ever announces on channel A, and a self-sufficient stray partition there (it has routers, a leader, a valid dataset) will happily live on the wrong channel indefinitely. Manual recovery: MGMT_ANNOUNCE_BEGIN (commissioner announce-begin sweep) from the healthy partition.
- Non-monotonic active timestamps. All of the healing above keys off "newer Active Timestamp wins." If dataset 2's Active Timestamp isn't strictly greater than dataset 1's, HandleDelayTimer won't even replace the active dataset (unless the key changed), and Announce arbitration becomes ambiguous. Whatever issues the channel changes must strictly increment the active timestamp each time.
- Sleepy end devices are the most likely to miss dataset 2 in the first place (poll interval > remaining delay). They do recover — a detached SED keeps its receiver on while trying to attach and can process Announces — but the outage can be long if announce sweeps are slow.
Sooo, I guess it is relatively safe due to the recovery features, but I wonder if we want to take the chance.... Changing a channel consecutively within 5 minutes seems not a typical use case. So I think I'd prefer to take the safe route and simply reject another update if a pending dataset is present. Thoughts 🤔 ?
set_channel() builds its pending dataset from the active dataset alone, stepping that timestamp by one second. A Thread mesh ignores a pending dataset that is not newer than the one it already holds, while the border router accepts the write all the same, so a channel change issued while another pending dataset was still waiting out its delay -- a previous channel change, or a network migration -- was silently dropped by the mesh with nothing telling the caller. Superseding the in-flight dataset with a newer stamp would be the other option, but it races the delay timer: a device that misses the newer dataset before its timer expires applies the superseded one and has to find the network again through MLE Announce. Refusing is the honest answer; the caller can retry once the pending delay has passed. Assisted-By: Claude Fable 5
d7aedb0 to
2e35374
Compare
|
@agners agreed, reworked to reject when another dataset is already pending. |
set_channel() builds its pending dataset from the active dataset alone,
stepping that timestamp by one second. A Thread mesh ignores a pending
dataset that is not newer than the one it already holds, while the
border router accepts the write all the same, so a channel change issued
while another pending dataset was still waiting out its delay -- a
previous channel change, or a network migration -- was silently dropped
by the mesh with nothing telling the caller.
Superseding the in-flight dataset with a newer stamp would be the other
option, but it races the delay timer: a device that misses the newer
dataset before its timer expires applies the superseded one and has to
find the network again through MLE Announce. Refusing is the honest
answer; the caller can retry once the pending delay has passed.
Came up in review of: home-assistant/core#178291
AI use: Written with AI assistance -- the commit carries an Assisted-By trailer. I've reviewed and understand all of it, and will be answering questions myself.