feat(orchestrator): sleeping-node interview via PendingQueue (closes #203)

CLAUDE.md

@@ -166,7 +166,7 @@ Remaining S2 phases — live encapsulate/decapsulate routing + SPAN runtime sync
 
 `src/node-values/NodeValues` is the durable per-node **value cache** (#213): the last-known dynamic values a node has reported (on/off, level, battery %, sensor readings, config params, …), each with the unix-seconds time it was recorded. It backstops the node-info view (#45) so it can show "last known value + when" without a live read — essential for sleeping nodes. Deliberately separate from its neighbours so each store stays single-concern: node-registry holds slow-changing **identity** (its "static info only" contract), node-metadata holds human **labels**, and this holds the high-churn **values** (keeping the identity table out of the write path). A `value_id` string names the logical value with a discriminator for multi-instance CCs (`battery`, `binary_switch`, `config:3`, `sensor:1`, `setpoint:1`). SQLite `node_values` table in the shared `nodes.db` on its own connection, keyed by `(home_id, node_id, value_id)`. Follows the NodeMetadata/PendingQueue split: a testable `NodeValues::Store` (injectable clock for deterministic timestamps; two instances against one file model a restart) + a `NodeValues::instance()` singleton wired via `StorageConfig`. The bus **recorder** (`src/node-values/Recorder.cpp`, constructor priority 204) feeds the cache: it subscribes to the typed CC report events (BinarySwitch / MultilevelSwitch / Battery / SensorMultilevel / SensorBinary / Configuration / ThermostatMode / ThermostatSetpoint) the cc-translator publishes, renders each into a `value_id` + display string, calls `record(...)`, and publishes the retained-stream `NodeValueChanged{nodeId, valueId, value}` so a UI can live-update; it binds the cache's home from the retained `DongleInfo`. The D-Bus surface is `GetNodeValues(y) → a(sst)` (custom method → `emitGetNodeValues`, returning `getAll`) + the `NodeValueChanged` signal, so the terminal reads the cache and live-updates (MANUAL §16d).
 
-`src/orchestrator/` holds bus-only post-event state machines (constructor priority 204; own no thread, react synchronously to bus events, talk to the rest of the daemon over `MessageBus` only). `WakeUpOrchestrator` subscribes to `WakeUpNotification`, drains `PendingQueue` for that node, replays each payload as a `SendDataCommand`, then issues `SendWakeUpNoMoreInformationCommand` so the node sleeps again ASAP (battery), and publishes `WakeUpCycleComplete`. `InclusionOrchestrator` subscribes to the high-level `NodeIncluded` event (published by ProtocolThread at terminal inclusion, carrying the node's CC list), sets the lifeline (Association group 1 → controller, gated on the `[behavior] auto_lifeline` toggle + the node supporting Association + a known controller id from the retained `DongleInfo`), applies the `effectivePolicy` from PolicyRegister gated on the node's supported CCs, and publishes `InclusionLifelineSet` / `InclusionPolicyApplied` / `InclusionComplete` progress events. The auto-lifeline logic used to live inline in ProtocolThread; it moved here. The progress events leave a clean seam for a future SecurityOrchestrator (#26/#27) to gate on before the policy step. `SecurityBootstrapOrchestrator` (#167) is the first such security reactor — it runs the S0 inclusion key-exchange handshake on `NodeIncluded` (see the `security/s0` description above). `SecurityS2BootstrapOrchestrator` (#187) is its S2 counterpart — it runs the CC `0x9F` KEX handshake (see the `security/s2` description above). `InterviewOrchestrator` (#203) auto-drives the post-inclusion node interview — an ordered `ManufacturerSpecific → Version → [Multi Channel endpoints if 0x60] → [Z-Wave+ Info if 0x5E]` Get sequence (publishing the existing `Get*Command` events, advancing on each matching typed report) ending in `NodeInterviewComplete`. It realises the `SecurityBootstrap → Interview` ordering: a node whose NIF advertises S0/S2 is interviewed only after it reports secure (`NodeSecurityStatus`), so the Gets ride the encrypted channel; a non-secure node is interviewed straight off `NodeIncluded`. The `ManufacturerSpecificReport` it triggers feeds `PolicyRegister::noteDeviceIdentity`. Wiring `NodeInterviewComplete` to gate the `InclusionOrchestrator` policy step, persisting the gathered identity into node-registry, and sleeping-node interview via `PendingQueue` are later #203 layers.
+`src/orchestrator/` holds bus-only post-event state machines (constructor priority 204; own no thread, react synchronously to bus events, talk to the rest of the daemon over `MessageBus` only). `WakeUpOrchestrator` subscribes to `WakeUpNotification`, drains `PendingQueue` for that node, replays each payload as a `SendDataCommand`, then issues `SendWakeUpNoMoreInformationCommand` so the node sleeps again ASAP (battery), and publishes `WakeUpCycleComplete`. `InclusionOrchestrator` subscribes to the high-level `NodeIncluded` event (published by ProtocolThread at terminal inclusion, carrying the node's CC list), sets the lifeline (Association group 1 → controller, gated on the `[behavior] auto_lifeline` toggle + the node supporting Association + a known controller id from the retained `DongleInfo`), applies the `effectivePolicy` from PolicyRegister gated on the node's supported CCs, and publishes `InclusionLifelineSet` / `InclusionPolicyApplied` / `InclusionComplete` progress events. The auto-lifeline logic used to live inline in ProtocolThread; it moved here. The progress events leave a clean seam for a future SecurityOrchestrator (#26/#27) to gate on before the policy step. `SecurityBootstrapOrchestrator` (#167) is the first such security reactor — it runs the S0 inclusion key-exchange handshake on `NodeIncluded` (see the `security/s0` description above). `SecurityS2BootstrapOrchestrator` (#187) is its S2 counterpart — it runs the CC `0x9F` KEX handshake (see the `security/s2` description above). `InterviewOrchestrator` (#203) auto-drives the post-inclusion node interview — an ordered `ManufacturerSpecific → Version → [Multi Channel endpoints if 0x60] → [Z-Wave+ Info if 0x5E]` Get sequence (publishing the existing `Get*Command` events, advancing on each matching typed report) ending in `NodeInterviewComplete`. It realises the `SecurityBootstrap → Interview` ordering: a node whose NIF advertises S0/S2 is interviewed only after it reports secure (`NodeSecurityStatus`), so the Gets ride the encrypted channel; a non-secure node is interviewed straight off `NodeIncluded`. The `ManufacturerSpecificReport` it triggers feeds `PolicyRegister::noteDeviceIdentity`. A **sleeping node** (NIF advertises Wake Up CC `0x84`) is awake only briefly after a `WAKE_UP_NOTIFICATION`, so rather than sending the Gets live, the orchestrator enqueues the whole `{COMMAND_CLASS, *_GET}` sequence into `PendingQueue` up front; `WakeUpOrchestrator` drains it on the next wake-up and the reports flow back during that window, driving completion exactly as for an always-on node (the Gets are all drained in one wake window — an unanswered Get is not retried, an MVP limit). With `NodeInterviewComplete` now gating the `InclusionOrchestrator` policy step and the gathered identity + capabilities persisted into node-registry (schema v4/v5), #203 is complete bar hardware verification (#189).
 
 `src/message-bus/MessageBus` is the in-process publish/subscribe bus and the **only** coupling between the dongle monitor and the protocol thread: the monitor publishes `DongleStatus` events (connected with TTY path / disconnected) and the protocol thread subscribes to them, blocking on its own condition variable until a path arrives. The bus also carries `ApplicationCommand` events — unsolicited Command Class frames received from nodes via `FUNC_ID_APPLICATION_COMMAND_HANDLER` (0x04), published by the protocol thread and consumed by the D-Bus backend (which re-emits them as raw + typed D-Bus signals). State events (`DongleStatus`) are retained — a late subscriber sees the latest value on subscribe. Transient events (`ApplicationCommand`) are not retained. Implementation is a thin wrapper around the header-only **eventpp** library (resolved via `find_package(eventpp REQUIRED)` — expected from system / cross-sysroot, not vendored); eventpp does not appear in the public header so the backing library is swappable. Dispatch is synchronous under the bus mutex, which is **recursive**: a handler may `publish()` or `subscribe()` reentrantly and the nested call runs as a depth-first dispatch on the same thread before the outer handler resumes (this is how the cc-translator republishes typed events from an `ApplicationCommand` handler, and how the orchestrators issue commands from notification handlers). Avoid unbounded recursion (a handler re-publishing the event it handles).
 

TODO.md

@@ -62,12 +62,12 @@ Implementation order (each shippable independently):
 
 **The full inclusion + wake-up orchestration plan is now complete (steps 1–7).**
 
-- [ ] [Node interview (introspection) orchestrator](https://github.com/Assar63/zwaved/issues/203) — bus-only orchestrator (prio 204) that auto-drives the post-inclusion node interview (ManufacturerSpecific → Version → Multi Channel endpoints → Z-Wave+), since nothing does today. Built in layers:
+- [x] [Node interview (introspection) orchestrator](https://github.com/Assar63/zwaved/issues/203) — **done** (bar hardware verification #189). Bus-only orchestrator (prio 204) that auto-drives the post-inclusion node interview (ManufacturerSpecific → Version → Multi Channel endpoints → Z-Wave+), since nothing did before. Built in layers:
   - [x] the orchestrator — `src/orchestrator/InterviewOrchestrator.cpp`: ordered Get sequence advancing on the typed reports, emits `NodeInterviewComplete`; secure nodes deferred until `NodeSecurityStatus` (SecurityBootstrap → Interview); feeds `PolicyRegister::noteDeviceIdentity`. 3 bus-driven tests.
   - [x] gate the `InclusionOrchestrator` policy step on `NodeInterviewComplete` — lifeline still on `NodeIncluded`, policy (effectivePolicy + apply + `InclusionPolicyApplied`/`InclusionComplete`) now on interview completion (device defaults match); supported-CCs cached between the two events. Order is now SecurityBootstrap → Interview → Policy.
   - [x] persist the device-identity triple (mfr / product type / product id) into node-registry (schema v4, `setDeviceIdentity` driven by `ManufacturerSpecificReport`) so it survives restarts
   - [x] persist the remaining gathered capabilities (Version triple, Multi Channel endpoints, Z-Wave+ role/icons) into node-registry (schema v5, `setVersionInfo` / `setEndpointInfo` / `setZWavePlusInfo` driven by the typed report events) so the console detail pane + #45 can show them
-  - [ ] sleeping-node interview via `PendingQueue` (enqueue the Gets, run on next wake-up) Closes the gap where `PolicyRegister`'s device-level defaults never match (the `(mfr, type, product)` triple is unknown at inclusion time) and `InclusionOrchestrator` applies policy before device identity is known. Emits `NodeInterviewComplete{nodeId}`, which becomes the correct trigger for the policy step — making the post-inclusion reactor order explicit: **SecurityBootstrap → Interview → InclusionPolicy**. Sleeping nodes interview via `PendingQueue` on next wake-up. Unit-testable like the other orchestrators. **Refined design:** the interview writes *identity/capabilities* into node-registry (+ a `last_interviewed` timestamp); *live values* go to a separate value cache (below); #45 composes the two + node-metadata.
+  - [x] sleeping-node interview via `PendingQueue` — a node advertising Wake Up (0x84) has its whole `{CC, GET}` interview sequence enqueued up front; `WakeUpOrchestrator` drains it on the next wake-up and the reports drive completion (one wake window; unanswered Gets not retried, an MVP limit). Closes the gap where `PolicyRegister`'s device-level defaults never match (the `(mfr, type, product)` triple is unknown at inclusion time) and `InclusionOrchestrator` applies policy before device identity is known. Emits `NodeInterviewComplete{nodeId}`, which becomes the correct trigger for the policy step — making the post-inclusion reactor order explicit: **SecurityBootstrap → Interview → InclusionPolicy**. Sleeping nodes interview via `PendingQueue` on next wake-up. Unit-testable like the other orchestrators. **Refined design:** the interview writes *identity/capabilities* into node-registry (+ a `last_interviewed` timestamp); *live values* go to a separate value cache (below); #45 composes the two + node-metadata.
 - [x] [Per-node value cache (last-known values + timestamps)](https://github.com/Assar63/zwaved/issues/213) — **done** (store + recorder + D-Bus accessor). A dedicated store (sibling to node-registry/node-metadata, `Store` + `instance()` split) keyed by `(home, node, value_id)` → `{value, updated_at}`, fed by a bus subscriber on the typed CC report events. Keeps node-registry identity-focused (no high-churn value writes) while giving #45 "last-known + when". Read by #45; refreshed on-request only (awake → live read, asleep → enqueue via PendingQueue). Built in layers:
   - [x] store — `src/node-values/NodeValues` (`Store` + `instance()`, SQLite `node_values`, injectable clock; 4 tests)
   - [x] recorder — `src/node-values/Recorder.cpp` maps the typed report events (BinarySwitch/Multilevel/Battery/SensorMultilevel/SensorBinary/Configuration/ThermostatMode/ThermostatSetpoint) → `record(node, value_id, rendered)`; publishes `NodeValueChanged`; binds home from `DongleInfo`. 2 bus-driven tests.

src/orchestrator/InterviewOrchestrator.cpp

@@ -13,13 +13,22 @@
 // Gets ride the encrypted channel; a non-secure node is interviewed straight
 // off NodeIncluded.
 //
+// Sleeping nodes (#203): a battery node advertises Wake Up (CC 0x84) in its
+// NIF and is awake only briefly after a WAKE_UP_NOTIFICATION. Sending the Gets
+// live would race the node back to sleep, so for such a node the whole Get
+// sequence is enqueued into PendingQueue up front; WakeUpOrchestrator drains it
+// on the next wake-up and the reports flow back during that window, driving
+// advance()/completion exactly as for an always-on node.
+//
 // MVP limits (like the other orchestrators): no inactivity timeout — a node
-// that never answers a Get leaves its session parked; sleeping-node interview
-// via PendingQueue and the InclusionOrchestrator policy-step re-trigger on
-// NodeInterviewComplete are later #203 layers. Hardware-verified in #189.
+// that never answers a Get leaves its session parked. For a sleeping node the
+// Gets are all drained in one wake window (PendingQueue::drain pops the lot);
+// any Get the node doesn't answer before sleeping again is not retried.
+// Hardware-verified in #189.
 
 #include "../logger/Logger.hpp"
 #include "../message-bus/MessageBus.hpp"
+#include "../pending-queue/PendingQueue.hpp"
 #include "../zwaved.h"  // IWYU pragma: keep — CONFIG_ORCHESTRATOR_PRIO
 
 #include <algorithm>
@@ -30,10 +39,20 @@
 #include <utility>
 #include <vector>
 
+// Generated CC constants — the interview Gets are the trivial
+// {COMMAND_CLASS, *_GET} byte pairs each codec's encodeGet() produces; we
+// build them from the same manifest constants so a sleeping node's queued
+// payloads match exactly what ProtocolThread would have sent live.
+#include <application/ManufacturerSpecific.gen.hpp>  // IWYU pragma: keep
+#include <application/MultiChannel.gen.hpp>          // IWYU pragma: keep
+#include <application/NodeVersion.gen.hpp>           // IWYU pragma: keep
+#include <application/ZWavePlusInfo.gen.hpp>         // IWYU pragma: keep
+
 namespace
 {
 constexpr std::uint8_t CC_MULTI_CHANNEL  = 0x60;
 constexpr std::uint8_t CC_ZWAVEPLUS_INFO = 0x5E;
+constexpr std::uint8_t CC_WAKE_UP        = 0x84;  // sleeping/battery node marker
 constexpr std::uint8_t CC_SECURITY_0     = 0x98;
 constexpr std::uint8_t CC_SECURITY_2     = 0x9F;
 constexpr std::uint8_t NO_CALLBACK       = 0x00;
@@ -52,8 +71,28 @@ struct Session
     std::vector<Step> steps;
     std::size_t index = 0;
     bool started      = false;  // false while deferred awaiting security bootstrap
+    bool viaQueue     = false;  // sleeping node: Gets enqueued to PendingQueue, drained on wake-up
 };
 
+// The byte payload for a step's Get — the {COMMAND_CLASS, *_GET} pair each
+// codec's encodeGet() produces. Used only for the sleeping-node queue path;
+// the live path goes through the Get*Command events (ProtocolThread encodes).
+auto payloadForStep(Step step) -> std::vector<std::uint8_t>
+{
+    switch (step)
+    {
+    case Step::ManufacturerSpecific:
+        return {ManufacturerSpecific::COMMAND_CLASS, ManufacturerSpecific::MANUFACTURER_SPECIFIC_GET};
+    case Step::Version:
+        return {NodeVersion::COMMAND_CLASS, NodeVersion::VERSION_GET};
+    case Step::Endpoints:
+        return {MultiChannel::COMMAND_CLASS, MultiChannel::MULTI_CHANNEL_ENDPOINT_GET};
+    case Step::ZWavePlus:
+        return {ZWavePlusInfo::COMMAND_CLASS, ZWavePlusInfo::ZWAVEPLUS_INFO_GET};
+    }
+    return {};
+}
+
 // NOLINTBEGIN(misc-non-private-member-variables-in-classes): file-local singleton, public members read like a struct
 struct State
 {
@@ -103,6 +142,19 @@ auto beginInterview(std::uint8_t nodeId, Session& session) -> void
 {
     session.started = true;
     session.index   = 0;
+    if (session.viaQueue)
+    {
+        // Sleeping node: enqueue every Get instead of sending live.
+        // WakeUpOrchestrator drains them on the next WAKE_UP_NOTIFICATION;
+        // the reports drive advance()/completion as usual.
+        Logger::info("[interview] node " + std::to_string(nodeId) + " — sleeping; queueing " +
+                     std::to_string(session.steps.size()) + " interview Get(s) for next wake-up");
+        for (const auto step : session.steps)
+        {
+            PendingQueue::instance().enqueue(nodeId, payloadForStep(step), PendingQueue::PRIORITY_NORMAL);
+        }
+        return;
+    }
     Logger::info("[interview] node " + std::to_string(nodeId) + " — starting interview");
     sendStep(nodeId, session.steps.at(0));
 }
@@ -128,7 +180,12 @@ auto advance(std::uint8_t nodeId, Step completed) -> void
         state().sessions.erase(iter);
         return;
     }
-    sendStep(nodeId, session.steps.at(session.index));
+    // A sleeping node's Gets were all enqueued up front; only the live path
+    // sends the next Get on each report.
+    if (!session.viaQueue)
+    {
+        sendStep(nodeId, session.steps.at(session.index));
+    }
 }
 
 auto onNodeIncluded(const MessageBus::NodeIncluded& event) -> void
@@ -147,6 +204,10 @@ auto onNodeIncluded(const MessageBus::NodeIncluded& event) -> void
         session.steps.push_back(Step::ZWavePlus);
     }
 
+    // A node advertising Wake Up (0x84) is a battery node, awake only briefly
+    // after a wake-up — interview it through the queue rather than live.
+    session.viaQueue = has(CC_WAKE_UP);
+
     const bool secure              = has(CC_SECURITY_0) || has(CC_SECURITY_2);
     state().sessions[event.nodeId] = std::move(session);
     if (secure)

tests/CMakeLists.txt

@@ -1214,19 +1214,22 @@ gtest_discover_tests(InclusionOrchestrator_test)
 
 # ---- InterviewOrchestrator ------------------------------------------
 # Bus-driven: NodeIncluded + typed reports → the Get sequence + completion.
-# Bus-only orchestrator (no stores), so it links just the bus + Logger.
+# The sleeping-node path (#203) enqueues the Gets into PendingQueue, so the
+# test links PendingQueue + sqlite alongside the bus + Logger.
 add_executable(InterviewOrchestrator_test
     InterviewOrchestrator_test.cpp
     "${CMAKE_SOURCE_DIR}/src/orchestrator/InterviewOrchestrator.cpp"
+    "${CMAKE_SOURCE_DIR}/src/pending-queue/PendingQueue.cpp"
     "${CMAKE_SOURCE_DIR}/src/logger/Logger.cpp"
 )
 target_include_directories(InterviewOrchestrator_test PRIVATE
+    "${CMAKE_SOURCE_DIR}/src/pending-queue"
     "${CMAKE_SOURCE_DIR}/src/message-bus"
     "${ZWAVED_GENERATED_DIR}"
 )
 add_dependencies(InterviewOrchestrator_test zwaved_codegen)
 target_link_libraries(InterviewOrchestrator_test PRIVATE
-    GTest::gtest GTest::gtest_main zwaved_bus)
+    GTest::gtest GTest::gtest_main PkgConfig::SQLITE3 zwaved_bus)
 zwaved_test_target(InterviewOrchestrator_test)
 gtest_discover_tests(InterviewOrchestrator_test)