Add an OTBR action to migrate the whole Thread network - #178291
Add an OTBR action to migrate the whole Thread network#178291LorbusChris wants to merge 3 commits into
Conversation
|
Hey there @home-assistant/core, mind taking a look at this pull request as it has been labeled with an integration ( Code owner commandsCode owners of
|
|
Hey there @home-assistant/core, mind taking a look at this pull request as it has been labeled with an integration ( Code owner commandsCode owners of
|
Check requirementsChecked at commit
📦 python-otbr-api: 2.10.0 → 2.11.0
|
There was a problem hiding this comment.
Pull request overview
Adds an OTBR action to migrate an entire Thread network using a pending operational dataset.
Changes:
- Implements migration validation, timestamping, serialization, and dataset-store updates.
- Adds action metadata, translations, icons, and tests.
- Bumps
python-otbr-apito 2.11.0.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
homeassistant/components/otbr/__init__.py |
Registers the new action. |
homeassistant/components/otbr/icons.json |
Adds the action icon. |
homeassistant/components/otbr/manifest.json |
Updates the OTBR dependency. |
homeassistant/components/otbr/services.py |
Implements network migration. |
homeassistant/components/otbr/services.yaml |
Defines action fields. |
homeassistant/components/otbr/strings.json |
Adds action and error text. |
homeassistant/components/otbr/util.py |
Adds pending-dataset support and locking. |
homeassistant/components/thread/manifest.json |
Synchronizes the dependency version. |
requirements_all.txt |
Updates generated requirements. |
tests/components/otbr/test_services.py |
Tests migration behavior and validation. |
Suppressed comments (4)
homeassistant/components/otbr/services.py:283
- Register this action with
service.async_register_admin_service. Migrating an entire Thread network changes security credentials and network configuration, but direct registration currently lets non-admin users invoke it.
hass.services.async_register(
homeassistant/components/otbr/services.py:263
- Fetch the router identifiers before scheduling the pending dataset. Either API call can fail after line 249 has already committed the migration, causing the action to report failure while leaving the dataset store and preferred pointer on the old credentials.
preferred_border_agent_id=(await data.get_border_agent_id()).hex(),
preferred_extended_address=(await data.get_extended_address()).hex(),
homeassistant/components/otbr/services.py:264
- Defer publishing the target dataset until the pending delay has expired. This update runs immediately after the PUT while the mesh still uses the source credentials for up to an hour; same-network key rotation even replaces the stored preferred dataset in place, so commissioning during the delay receives credentials that cannot yet join the network.
await async_add_dataset(
call.hass,
DOMAIN,
tlv_parser.encode_tlv(pending),
preferred_border_agent_id=(await data.get_border_agent_id()).hex(),
preferred_extended_address=(await data.get_extended_address()).hex(),
)
homeassistant/components/otbr/services.py:195
- Check the in-flight pending dataset before returning this no-op response. After scheduling A→B, a request to migrate back to the still-active A matches
activehere and returns success without superseding B, so the network migrates to B anyway when the timer expires.
if _same_network_settings(active, target):
return {"status": "already_on_network"}
3890422 to
f685406
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Suppressed comments (3)
homeassistant/components/otbr/services.yaml:1
- Document the new user-facing action before merge. The integration documentation needs to explain the required complete dataset, delay, preferred-network default, and whole-network impact, and the PR description currently has no documentation PR linked.
migrate_network:
homeassistant/components/otbr/services.py:271
- Fetch both router identifiers before sending the pending dataset. Either GET can fail after the PUT succeeds, in which case the action reports an error even though the mesh is already migrating and the dataset store was never updated.
preferred_border_agent_id=(await data.get_border_agent_id()).hex(),
preferred_extended_address=(await data.get_extended_address()).hex(),
homeassistant/components/otbr/services.py:230
- Include the dataset store's matching target timestamp when computing
newest.DatasetStore.async_addrejects a same-extended-PAN-ID dataset whose timestamp is older than the stored one (homeassistant/components/thread/dataset_store.py:296-329), so an explicitly supplied target can migrate successfully but fail to replace different stored credentials, after which the preferred pointer is moved to those wrong credentials.
newest = max(
_timestamp_seconds(active, MeshcopTLVType.ACTIVETIMESTAMP),
_timestamp_seconds(target, MeshcopTLVType.ACTIVETIMESTAMP),
)
Check requirementsChecked at commit
📦 python-otbr-api: 2.10.0 → 2.11.0
|
f685406 to
195bf35
Compare
Check requirementsChecked at commit
📦 python-otbr-api: 2.10.0 → 2.11.0
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.
Suppressed comments (3)
homeassistant/components/otbr/services.py:166
- Resolve the preferred target while holding the migration lock. A queued no-dataset call can capture the old preferred dataset before another migration repoints it, then incorrectly report
already_on_networkwhile that first migration is still moving the router away.
dataset = await _target_dataset(call)
homeassistant/components/otbr/services.py:270
- Fetch the router identifiers before submitting the pending dataset. If either follow-up API call fails after the PUT succeeds, the action reports failure and skips the dataset-store/preferred updates even though the mesh is already migrating.
preferred_border_agent_id=(await data.get_border_agent_id()).hex(),
preferred_extended_address=(await data.get_extended_address()).hex(),
tests/components/otbr/test_services.py:299
- Add a successful no-dataset test that verifies the preferred Thread dataset is selected and sent. The current test covers only the
no_preferred_dataseterror path, leaving the advertised default-target behavior unverified.
async def test_no_preferred_dataset(
742c067 to
b2aecab
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 19 out of 19 changed files in this pull request and generated no new comments.
Suppressed comments (2)
homeassistant/components/otbr/services.py:324
- Persist the per-source timestamp watermark across Home Assistant restarts. This
hass.dataentry is lost on restart, so a second border router on the same mesh can again read the old active dataset and no pending dataset, issue the same timestamp as a pre-restart migration, and have one migration silently ignored; store durable per-source state (and add a restart regression test).
issued = call.hass.data.setdefault(ISSUED_TIMESTAMPS_KEY, {})
source_xpan_item = active.get(MeshcopTLVType.EXTPANID)
source_xpan = str(source_xpan_item).lower() if source_xpan_item else None
if source_xpan is not None:
newest = max(newest, issued.get(source_xpan, (0, 0)))
homeassistant/components/otbr/util.py:50
- Update the PR description to state that dataset mutations are serialized globally, not per router. This lock covers every config entry (including unrelated routers), which contradicts the current concurrency description and has different operational behavior.
It covers every config entry, and is acquired by callers rather than by
the OTBRData methods, so a sequence that reads the router's state and
The watermark is only a backstop. What actually orders the datasets survives a |
b2aecab to
6b5f974
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.
Suppressed comments (1)
homeassistant/components/otbr/services.py:360
- Make this preferred-dataset check atomic with the pending write and store update.
thread/add_dataset_tlvdoes not take the OTBR lock, so it can store rotated credentials while the later PUT is awaited; if that update is newer than this snapshot but below the newly issued timestamp (for example, newer only by ticks), the add at line 378 overwrites it and returnsSTORED, silently restoring stale credentials in both the mesh and store. Use a store revision/CAS or shared transaction that detects any target change through the PUT.
preferred = await async_get_preferred_dataset(call.hass)
if preferred is not None and bytes.fromhex(preferred) != dataset:
6b5f974 to
bbb972e
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.
Suppressed comments (1)
homeassistant/components/otbr/services.py:360
- Coordinate preferred-dataset writes with this transaction instead of relying on this one-time recheck.
issued.async_set()immediately below performs an executor-backed disk write, so Thread's websocket can change the preferred pointer or replace its TLV after this comparison but before the pending PUT; the action then migrates to the stale default, and a tick-newer credential update can even be overwritten by the whole-second stamp. Keep a shared lock or versioned comparison valid through the router write.
preferred = await async_get_preferred_dataset(call.hass)
if preferred is not None and bytes.fromhex(preferred) != dataset:
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.
Suppressed comments (2)
homeassistant/components/otbr/services.py:195
- Check every pinned router on the source mesh before allowing the migration. Returning the first match means a second matching router with a different pinned channel is never examined; likewise, the caller skips this helper entirely when the selected router is pinned, so the pending dataset can still move another shared radio off its required channel.
if other_active.get(MeshcopTLVType.EXTPANID) == source_xpan:
return pinned
homeassistant/components/otbr/services.py:374
- Coordinate Thread dataset-store updates through the final router write. The preferred-dataset check is followed by awaited storage and HTTP operations, while
thread/add_dataset_tlvdoes not take this lock; an update landing after the check can therefore be outstamped by this stale snapshot and silently overwritten in both the mesh and store, which the laterDISCARDEDresult cannot detect when this generated timestamp is higher.
if source_xpan is not None:
await issued.async_set(source_xpan, (seconds, 0))
await data.set_pending_dataset_tlvs(
bytes.fromhex(tlv_parser.encode_tlv(pending))
)
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.
Suppressed comments (2)
homeassistant/components/otbr/services.py:397
- Repoint the target before awaiting repair updates.
update_issues()can yield while the newly stored target is still non-preferred, allowingthread/delete_datasetto remove it; repointing then becomes a no-op and this action reports success while retaining the abandoned source as preferred.
await update_issues(call.hass, data, migrated_tlvs)
homeassistant/components/otbr/services.py:378
- Coordinate preferred-dataset writes through this transaction before sending the pending dataset. The verification at line 369 is followed by the disk-I/O await in
issued.async_set(), sothread/add_dataset_tlvcan replace the preferred credentials during that await and this line will still migrate the mesh using the stale snapshot.
if source_xpan is not None:
await issued.async_set(source_xpan, (seconds, 0))
await data.set_pending_dataset_tlvs(
|
cc @lboue -- this change does not just enable moving devices to and from an external Matter NIM TBR, it also enables Thread credential rotation for the HA-managed OTBR, i.e. moving all connected Thread devices to a new dataset. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.
Suppressed comments (1)
homeassistant/components/otbr/services.py:181
- Include configured but unloaded OTBR entries in the pinned-router safety check.
async_loaded_entries()skips a multiprotocol router whose setup failed or was unloaded, while unloading this integration does not stop the external router; it can therefore remain on this mesh, receive the pending dataset, and be moved off its radio's pinned channel. Query all configured entries and either verify each pinned router's source network or refuse when it cannot be verified.
other: OTBRConfigEntry
for other in hass.config_entries.async_loaded_entries(DOMAIN):
if other.entry_id == entry.entry_id:
continue
pinned = await get_allowed_channel(hass, other.data["url"])
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.
Suppressed comments (2)
homeassistant/components/otbr/services.py:370
- Revalidate the preferred dataset immediately before starting the router write. This guard runs before
issued.async_set(), whose disk save awaits; a concurrent Thread store update can therefore replace the preferred credentials during that await, after the guard has passed but before anything reaches the router, and the stale snapshot can still be stamped over that update.
# now would put credentials on the mesh that Home Assistant has
# already superseded -- and stamped newer, so the newer ones would
homeassistant/components/otbr/services.py:157
- Set the target as preferred when no preference exists even if the source dataset is absent. A router can be provisioned externally or its source entry can be removed from the store; requiring
source_idthen leaves the preference unset after migration, so the action's default target remains unavailable despite the documented behavior.
# With no source entry -- a router re-provisioned by another controller
# runs a network the store never saw -- the promotion still applies:
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.
Suppressed comments (3)
homeassistant/components/otbr/services.py:290
- Always validate the other routers on the source mesh, even when the selected router is pinned. This branch skips
_pinned_channel_of_another_router()whenever the selected entry has an allowed channel, so a second multiprotocol router on the same mesh can still be moved off its own pinned channel.
# network, so the migration proceeds and supersedes it.
if _same_network_settings(active, target) and (
in_flight is None or _same_network_settings(in_flight, target)
):
return {"status": "already_on_network"}
# A different radio (like Zigbee in multiprotocol setups) may pin
homeassistant/components/otbr/websocket_api.py:226
- Include Thread dataset-store mutations in this serialization.
thread/add_dataset_tlvdoes not acquire this lock, so it can replace the selected entry after line 227 while these awaited router calls run; line 254 then installs the stale credentials even though the store now holds a newer dataset.
# Held from the first read: the dataset read here is what gets written
# below, so a concurrent writer must not replace it in between -- a key
# rotation landing in that window would push the superseded credentials.
async with async_get_dataset_lock(hass):
homeassistant/components/otbr/services.py:376
- Keep the preferred-dataset validation atomic through the pending write. After this check,
await issued.async_set(...)yields before the router update, sothread/add_dataset_tlvcan still replace the preferred credentials in that gap and the stale snapshot is then sent to the whole mesh.
delay * 1000
)
# Fetched before the write: a failure here must abort the action
# before the mesh starts migrating, not after.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.
Suppressed comments (3)
homeassistant/components/otbr/services.py:338
- Include the stored source network timestamp in the migration floor. Another router can already have imported a newer revision of the source mesh while this router still reports an older active dataset; checking only the target entry then emits an older pending timestamp that the mesh ignores while the action reports success.
store = await async_get_store(call.hass)
target_xpan = str(target[MeshcopTLVType.EXTPANID]).lower()
for entry_ in store.datasets.values():
if entry_.extended_pan_id.lower() == target_xpan:
newest = max(
homeassistant/components/otbr/util.py:51
- Record timestamps issued by channel changes in this watermark as well. The global lock only serializes calls; after router A schedules a channel change, router B on the same mesh can still report the old active dataset and no pending dataset, so a migration through B reuses A's timestamp and one successful write is ignored.
Kept on disk, not only in memory: a pending dataset takes its delay to
reach every router on the mesh, and a restart inside that window must not
let the next migration hand out the stamp again.
homeassistant/components/otbr/services.py:155
- Check the source and target IDs independently so same-network credential rotation can select the target as preferred. With
elif, equal extended PAN IDs only setsource_id, leavingtarget_idunset and defeating the documented no-preference promotion.
elif entry.extended_pan_id.lower() == target_extended_pan_id.lower():
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.
Suppressed comments (2)
homeassistant/components/otbr/silabs_multiprotocol.py:76
- Make channel changes participate in the per-mesh timestamp watermark instead of relying only on this lock. The lock is released before the pending dataset has propagated to other routers, so a later channel change through another router can still see no pending dataset and an old active timestamp;
python-otbr-apithen stampsactive + 1, and the mesh may silently ignore it as older than or equal to the migration already issued. Reject writes during that propagation window or extend the shared timestamp mechanism toset_channel.
async with async_get_dataset_lock(hass):
await data.set_channel(channel, delay)
homeassistant/components/otbr/websocket_api.py:302
- Make channel changes participate in the per-mesh timestamp watermark instead of relying only on this lock. Once the lock is released, the pending dataset may not yet be visible on another router on the same mesh; a channel change through that router can therefore stamp its stale active timestamp plus one and report success even though the mesh ignores it as older than or equal to the earlier migration. Reject writes during propagation or extend the shared timestamp mechanism to
set_channel.
async with async_get_dataset_lock(hass):
try:
await data.set_channel(channel)
By "rejoin a border router" you mean the reset border router available in the frontend? I don't really like this approach, it exposes the user to TLV. Actions are meant to automate things, is migrating OTBR networks really something we want to commonly automate on? 🤔 I don't really see this as a use case. Maybe we should distill down to a typical use-case (merge two networks Home Assistant is aware of), then offer a UI feature with confirmation dialog and clear information on what the risks are of this operation (e.g. I wonder how do other border routers react to that?). If you want access to these features for development or in-depth testing maybe some development OTBR app similar to the Supervisor API explorer would be better. |
Joining a border router to another Thread network meant replacing its active dataset with Thread down, which moves only the router and strands every device on the network it leaves. Thread has a graceful way to do this that the integration only used for channel changes: a pending operational dataset, disseminated by the router to every device, applied by all of them together when its delay expires. The new otbr.migrate_network action generalises that mechanism to a change of network. The target dataset - given as TLVs, or the preferred Thread network by default - must describe the network completely: the router merges a pending dataset over a base it chooses, so any field left out would come from that base and the mesh would migrate onto settings nobody picked. The dataset is re-stamped newer than the network being left - a not-newer pending dataset is silently ignored by the mesh - and otherwise taken verbatim. Rotating credentials on the same network is the same operation, so it works too; only a fully identical dataset is a no-op. A migration is refused outright while a pending dataset is already in flight, and the user is told so. Superseding one is never safe to do implicitly: the replacement races the delay timer on every device already holding the old dataset, so a late replacement can split the mesh, and it would silently undo whatever the in-flight dataset was doing, such as a channel change the user never heard about. The check runs before anything is recorded, and the library's own guard on the write - If-None-Match on border routers that honor it - backstops the race where a pending dataset appears between the check and the write; that refusal surfaces as the same error. A migration off a channel that another radio pins in multiprotocol setups is refused, the same way selecting a different network already is. The border router is picked with a config entry selector, needed only when several are set up. The Thread dataset store learns the re-stamped dataset, bound to the router like setup binds datasets, and when the network being left was the preferred one, the preferred pointer moves along - everything handing out Thread credentials starts there, and the action's own default would otherwise migrate a router back onto the abandoned network. Migrations are serialised per router: two racing calls would compute identical timestamps and the loser would be ignored by the mesh while its credentials sat in the store. The TLV construction and the pending-dataset write live in python-otbr-api 2.11.0 (set_pending_dataset_tlvs, Timestamp.from_values, DelayTimer.from_milliseconds); the integration validates, compares and stamps. Assisted-By: Claude Fable 5
otbr.migrate_network hands the border router a pending dataset and then records what the network will run. That record can be dropped: the Thread dataset store keeps the newest dataset per network, so another writer storing newer credentials for the same network while the router is being written to wins, and the action would still report a plain success. The store now says which happened, so report the case where Home Assistant's copy of the credentials is not the dataset the mesh is switching to. The mesh cannot be called back, so the repair issues and the preferred dataset are still brought up to date first -- they describe which network is being adopted rather than with which credentials, and the preferred pointer is what this action's own default target reads. Assisted-By: Claude Fable 5
A migration is handed to one border router, but the pending dataset it sends reaches every router on the mesh, and Home Assistant may be managing several of them. Three things then go wrong for the others: A router sharing its radio with Zigbee is pinned to a channel, which the migration only respected when started through that router. Ask the other routers on the same network whether they are pinned before choosing. A pinned router that cannot be read refuses the migration rather than being skipped: its REST API being down says nothing about its radio, which may still be on the mesh and would follow the pending dataset off the shared channel. A second migration of the same mesh -- another border router on it, moving to a different network -- can still read the old active dataset and no pending one, and would pick the same timestamp as the first. Stamp above what this integration has already issued for that mesh, keyed by extended PAN ID so one busy network does not exhaust the rest. That record is written to disk before the router is, since a restart inside the delay window would otherwise forget what is still in flight. The target taken from the preferred dataset can be replaced while the router is being read. Sending that snapshot would put credentials on the mesh that Home Assistant has already superseded and stamped newer, so the newer ones would lose. Nothing has been written yet at that point, so refuse and let the caller try again. Assisted-By: Claude Fable 5
|
Re-join in the sense that if a dataset changed, the OTBR needs to be re-joined -- the wording here isn't great. Regarding use-cases: The OTBR dataset migration is also the mechanism to change credentials for an existing network, so credential rotation is definitely one of the use-cases. |
Draft until home-assistant-libs/python-otbr-api#269 is merged and released -- the manifest pins
python-otbr-api==2.11.0, which doesn't exist on PyPI yet. Everything else is ready for review.#178306 and #179098 have merged and this branch is rebased on them; nothing else is bundled.
Proposed change
Adds an
otbr.migrate_networkaction that moves a border router and every device onits Thread network onto another network. So far the only way to rejoin a border router
was to replace its active dataset with Thread down, which moves only the router and
strands all the devices. Thread already has a graceful mechanism for this, and the
integration already uses it for channel changes: a pending operational dataset that the
router distributes and all devices apply together after a delay. This generalizes that
to a full network change. Thread stays up and nothing needs to be commissioned again.
A few things worth noting:
complete: channel, channel mask, PAN ID, extended PAN ID, mesh-local prefix, network
name, network key, PSKC and security policy. OTBR merges a pending dataset over a
base it picks itself (the in-flight pending dataset, or a freshly generated random
network), so missing fields would get filled in with settings nobody chose. Partial
datasets are refused with a message that says what's missing.
silently ignores pending datasets that aren't newer. Everything else is sent
as-is; the tests check the PUT body matches the input exactly.
and the user is told so. Superseding one implicitly is never safe: the
replacement races the delay timer on every device that already holds the old
dataset, so a late replacement can split the mesh, and it would silently undo
whatever the in-flight dataset was doing. The check happens both here (with a
clear message) and in python-otbr-api's
set_pending_dataset_tlvs(), whichbackstops the race where a pending dataset appears between the read and the
write (via
If-None-Match: *on border routers with[rest] honor If-None-Match on the dataset PUT endpoints openthread/ot-br-posix#3552).
identical dataset is a no-op.
selecting a different network is today.
the preferred network, the preferred dataset moves along. Everything that hands out
Thread credentials starts from the preferred dataset, so leaving it on the old
network would keep sharing credentials nothing uses anymore -- and this action's own
default (no dataset given) would migrate a router right back to it.
of them can sit on the same mesh. Otherwise concurrent writes end up with the
same timestamp and the mesh just drops one of them. The channel change paths and
the config flow take the same lock, since they write datasets too.
are in python-otbr-api 2.11.0
(Add pending-dataset TLV write support home-assistant-libs/python-otbr-api#269). The integration only validates, compares
and stamps, and goes through OTBRData like all the other router calls.
Dependency: python-otbr-api 2.10.0 → 2.11.0 — the full diff is home-assistant-libs/python-otbr-api#269 (that PR would be the entire release as things stand today).
AI use: Written with AI assistance -- the commit carries an
Assisted-Bytrailer. I've reviewed and understand all of it, and I'll be answeringquestions myself.
Type of change
Additional information
Checklist
ruff format homeassistant tests)If user exposed functionality or configuration variables are added/changed:
If the code communicates with devices, web services, or third-party tools:
Updated and included derived files by running:
python3 -m script.hassfest.requirements_all.txt.Updated by running
python3 -m script.gen_requirements_all.To help with the load of incoming pull requests: