diff --git a/doc/smart-switch/pmon/images/dpu-reboot-seq.mmd b/doc/smart-switch/pmon/images/dpu-reboot-seq.mmd new file mode 100644 index 00000000000..a3dee89fd89 --- /dev/null +++ b/doc/smart-switch/pmon/images/dpu-reboot-seq.mmd @@ -0,0 +1,56 @@ +%%{init: { + "theme": "default", + "themeVariables": { + "actorBkg": "#eaeaea", + "actorBorder": "#666", + "actorTextColor": "#333", + "actorLineColor": "#999", + "signalColor": "#333", + "signalTextColor": "#333", + "labelBoxBkgColor": "#eaeaea", + "labelBoxBorderColor": "#666", + "labelTextColor": "#333", + "loopTextColor": "#333", + "activationBkgColor": "#f4f4f4", + "activationBorderColor": "#666", + "noteBkgColor": "#fff5ad", + "noteBorderColor": "#aaaa33", + "noteTextColor": "#333" + } +}}%% + +sequenceDiagram + actor User + participant HostService as NPU: host-service + participant StateDB as NPU: stateDB + participant DpuChassisd as DPU: chassisd + participant ChassisStateDB as NPU: chassisStateDB + participant Chassisd as NPU: chassisd + participant PlatformAPI as NPU: platform API + + User->>HostService: Input: PowerOn or Reboot config or CLI + HostService->>HostService: determine-reboot-cause + HostService->>HostService: update-reboot-cause + activate HostService + HostService->>StateDB: update-reboot-cause + HostService->>ChassisStateDB: update-reboot-cause + HostService->>HostService: end processing + deactivate HostService + + DpuChassisd->>ChassisStateDB: publish current boot_id when reachable + ChassisStateDB-->>Chassisd: boot_id appears or changes + activate Chassisd + Chassisd->>Chassisd: compare with persisted boot_id + + alt boot_id differs or no valid baseline exists + Chassisd->>PlatformAPI: get-reboot-cause + activate PlatformAPI + PlatformAPI-->>Chassisd: reboot-cause + deactivate PlatformAPI + Chassisd->>Chassisd: persist history record with boot_id + Chassisd->>ChassisStateDB: refresh REBOOT_CAUSE entries + else boot_id is unchanged + Chassisd->>Chassisd: no new record + end + + deactivate Chassisd diff --git a/doc/smart-switch/pmon/images/dpu-reboot-seq.svg b/doc/smart-switch/pmon/images/dpu-reboot-seq.svg index 8182562f926..17759bc53e6 100644 --- a/doc/smart-switch/pmon/images/dpu-reboot-seq.svg +++ b/doc/smart-switch/pmon/images/dpu-reboot-seq.svg @@ -1 +1 @@ -NPU: Platform codeNPU: platform APINPU: chassisdNPU: chassisStateDBNPU: stateDBNPU: host-serviceNPU: Platform codeNPU: platform APINPU: chassisdNPU: chassisStateDBNPU: stateDBNPU: host-serviceloop[moduel_db_update]UserInput: PowerOn or Reboot config or clidetermine-reboot-causeupdate-reboot-causeupdate-reboot-causeupdate-reboot-causeend processingoper-status out of offlineget-reboot-causextract-dpu-reboot-causedpu-reboot-causereboot-causepersist-dpu-reboot-causeupdate-dpu-reboot-causeUser \ No newline at end of file +NPU: platform APINPU: chassisdNPU: chassisStateDBDPU: chassisdNPU: stateDBNPU: host-serviceNPU: platform APINPU: chassisdNPU: chassisStateDBDPU: chassisdNPU: stateDBNPU: host-servicealt[boot_id differs or no valid baseline exists][boot_id is unchanged]UserInput: PowerOn or Reboot config or CLIdetermine-reboot-causeupdate-reboot-causeupdate-reboot-causeupdate-reboot-causeend processingpublish current boot_id when reachableboot_id appears or changescompare with persisted boot_idget-reboot-causereboot-causepersist history record with boot_idrefresh REBOOT_CAUSE entriesno new recordUser \ No newline at end of file diff --git a/doc/smart-switch/pmon/smartswitch-pmon.md b/doc/smart-switch/pmon/smartswitch-pmon.md index b26e5d021be..d53760e8e9b 100644 --- a/doc/smart-switch/pmon/smartswitch-pmon.md +++ b/doc/smart-switch/pmon/smartswitch-pmon.md @@ -8,6 +8,7 @@ | 0.4 | 06/06/2024 | Ramesh Raghupathy | Added schema for DPU health-info and called out phase:1 and phase:2 activities for DPU health-info. Added key suffix to module reboot-cause to avoid key conflicts | | 0.5 | 04/30/2025 | Gagan Punathil Ellath | Added Post Startup and Pre shutdown sections for DPU | | 0.6 | 06/12/2025 | Gagan Punathil Ellath | DPU State Management Implementation | +| 0.7 | 06/11/2026 | Charles Tsai | Updated DPU reboot-cause and midplane-down reason | ## Definitions / Abbreviations @@ -614,9 +615,10 @@ The smartswitch needs to know the reboot cause for the NPU and the DPUs. ``` #### DPU Reboot Cause * The smartswitch needs to know the reboot cause for all the DPUs. -* The NPU hardware should be capable of providing the DPU reboot-cause even when the DPUs are dead. +* The reboot-cause can be captured by the platforms at the appropriate time. The reboot-cause should be reported only when the midplane of DPU transitions to online by calling the `get_reboot_cause` API. +* Each DPU SONiC publishes a `boot_id` (a fresh UUID generated per boot from `/proc/sys/kernel/random/boot_id`) into its `DPU_STATE` entry in CHASSIS_STATE_DB. The NPU chassisd compares the reported `boot_id` against the last `boot_id` it persisted; when they differ, the NPU chassisd calls `get_reboot_cause()`, writes a history record containing the cause and the `boot_id`, and then refreshes the `REBOOT_CAUSE` entries in CHASSIS_STATE_DB from those records. A changed `boot_id` can only become visible on the NPU once the DPU is reachable and able to write into the NPU's CHASSIS_STATE_DB, so the reported reboot-cause is naturally aligned with the midplane coming online. * The get_reboot_cause will return the current reboot-cause of the module. -* For persistent storage of the DPU reboot-cause and reboot-cause-history files use the existing mechanism and host storage path under "/host/reboot-cause/module/dpux". +* Persistent DPU reboot-cause records are stored under `/host/reboot-cause/module/dpux/history/`, and each record includes the `boot_id`. The `previous-reboot-cause.json` symlink identifies the latest record and provides the persisted `boot_id` baseline. Unlike the NPU workflow, DPU capture does not use `module/dpux/reboot-cause.txt`. * The storage and retrieval of the reboot-cause of the Switch and PDUs are shown in the sequence diagram

@@ -625,12 +627,9 @@ The smartswitch needs to know the reboot cause for the NPU and the DPUs. * The switch boots up. Determines the NPU reboot cause. * Processes the previously stored NPU and DPU reboot-cause files and history files. * A maximum of ten reboot-cause history entries per dpu will be persisted just like the npu. -* Updates the NPU reboot-cause into the StateDB and the DPU reboot-cause into the ChassisStateDB. +* The reboot-cause service (`process-reboot-cause`) updates the NPU reboot-cause in StateDB and rebuilds the DPU `REBOOT_CAUSE|DPUx|