Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion docs/src/design/orchestrator/orchestrator-overview.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Orchestrator State Machine
# Orchestrator

The orchestrator is the eRoT's boot-sequence controller. It walks the platform
trust chain — verifying each component's firmware and releasing it from reset in
Expand Down Expand Up @@ -28,6 +28,10 @@ avoid the two drifting apart.
- [**Platform Architecture**](./orchestrator-platform.md): The platform half around
the core — surrounding services, capability contracts, the board device table,
and the fail-safe rules at the responsibility boundary.
- [**Runtime**](./orchestrator-runtime.md): How the runtime loop
gathers hardware interrupts, IPC channel messages, and watchdog deadlines into
the core's event stream, and carries the resulting effects and decisions back
out.

## Design Principles

Expand All @@ -48,6 +52,22 @@ hiding them as implicit state changes. See the
The platform supplies the trust chain (component ids, kinds, and required/optional
policy) and the recovery-retry cap at startup.

## Board composition

*Board composition* (or *system composition*) is the per-target choice — described
declaratively in a [`system.json5`](../../architecture.md) file, assembled by
Pigweed at build time — of how the platform's functions are split across
processes and which resources each process owns: hardware register blocks and the
kernel [interrupt objects and IPC channels](../pw-kernel-ipc.md) built on top of
them. It is separate from the
board-supplied *policy* above: policy is *what* to verify (the trust chain);
composition is *how* the surrounding services are wired. The orchestrator core
and its platform-agnostic crates are the same across every composition — a driver
may own a GPIO bank and forward boot-progress over a channel in one image, while
the orchestrator holds the pins directly in another. That choice changes which
inbound sources the [runtime](./orchestrator-runtime.md) sees and who owns each
device, but never the core's states or the loop that serves them.

## Relationship to CSA Architecture

The state machine is a direct implementation of the boot sequence described in
Expand Down
111 changes: 111 additions & 0 deletions docs/src/design/orchestrator/orchestrator-runtime.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# Runtime

The **runtime** is the orchestrator's event loop: the single task that sits
between two collaborators — the pure [state machine](./orchestrator-machine.md)
(the *core*) that decides, and the `PlatformDriver` that executes — turning
outside happenings (interrupts, timeouts, IPC messages) into the `Event`s the
core consumes, and handing the `Effect`s it emits to the driver. It holds no
policy of its own — every decision stays in the core; the runtime only moves
information across the process boundary.

It gathers three possible inbound sources — **hardware interrupts**
(boot-progress lines the orchestrator owns directly), **IPC channels**, and
**watchdog deadlines** (timeouts) — into the one event stream the core
supervises, and carries the core's decision back out. Which sources are present
is a [board-composition](./orchestrator-overview.md#board-composition) choice:
boot-progress is a hardware interrupt only when the orchestrator owns the pins;
otherwise a monitor forwards it as an IPC message.

The unifying idea is the kernel **wait group**: interrupts and IPC are not
separate mechanisms but interchangeable *members* of one group, and a watchdog
deadline is that same wait's *timeout* — so all three collapse into the return
of one `object_wait(handle, signal_mask, deadline)`. A *signal* here is just a
named bit on a waitable object, not a source in its own right: a latched IRQ bit
on an interrupt object, or `READABLE` / `USER` on a channel (a service raises
`Signals::USER` on a client channel to notify without a reply). The loop is
written once against "a member that signaled, or the deadline that lapsed,"
never against a specific source; what differs between sources is only the
*decoder* that turns each into an `Event`.

Two neighboring pages carry the supporting detail:
[Platform Architecture](./orchestrator-platform.md) names the services and the
responsibility boundary, and [pw_kernel IPC](../pw-kernel-ipc.md) gives the
concrete channel syscalls. The worked, compiling reference is the QEMU
integration test at `target/ast10x0/tests/orchestrator/runtime/main.rs`.

## The single wait point

The runtime is a single-threaded loop parked in one place: a kernel
`object_wait` over a **wait group**. Every inbound source is registered once as a
*member* of that group (`wait_group_add`), and the wait returns whichever member
signaled. Each member resolves to at most one `Event`:

- **Boot-progress signals** — a component reaching a checkpoint raises a
boot-progress signal. Depending on the board's composition it arrives as an
interrupt object the orchestrator holds directly or as a message a monitor
forwards over a channel; either way the loop maps it to
[`Event::ComponentReady`] (an `Active` component's iRoT-verified readiness)
or [`Event::Booted`] (a `Passive` component's liveness).
- **Watchdog deadlines** — the timer is not a separate task. `BootWatchdogs`
(`services/orchestrator/server`) folds all armed boot windows and the commit
window into a *single deadline* passed straight to `object_wait`. When the
wait returns `DeadlineExceeded`, `poll_expired()` yields the mapped
[`Event::Timeout`] / [`Event::CommitTimeout`].
- **IPC channel messages** — an update agent, management path, peer service, or
a boot-progress-forwarding monitor holds a channel *initiator*; the runtime
holds the *handler*. A readable channel is another object the loop waits on;
its message decodes to an `Event` — an [`Event::UpdateRequest`] to answer, or
a forwarded [`Event::Booted`] / [`Event::ComponentReady`] notification.

The three share one `object_wait`, so a slow image hash on one path cannot
delay a boot window on another — this is the "never block" rule of the
[Platform Architecture](./orchestrator-platform.md#responsibility-scope) made
concrete: the loop only ever blocks at the wait, and only until the *nearest*
of any signal, any channel, or the nearest deadline.

**Members are uniform; only the decode differs.** A wait-group member is an
object watched for a signal bit — a latched IRQ bit on an *interrupt object*, or
`READABLE` / `USER` on a *channel* — and the loop treats them identically: wait,
see which member signaled, run that member's decode, dispatch the resulting
`Event`. Nothing above the decode step knows which kind a member is. That
uniformity pushes two questions *below* the runtime layer, where they belong:

- **Who owns the underlying hardware** — does the orchestrator own the GPIO bank
and hold the boot-progress interrupt object itself, or does a monitor/GPIO
server own the pins and forward boot-progress over a channel? — is a
[board-composition](./orchestrator-overview.md#board-composition) choice made
in `system.json5`. Either shape is just one member of the group; the loop is
byte-for-byte the same.
- **What a member means** is the per-member decode.

```mermaid
flowchart LR
subgraph SRC["Inbound sources (SIG/CH are wait-group members; TMR is the wait's timeout; boot-progress rides CH if a monitor owns the pins)"]
SIG["Boot-progress signal<br/>(directly-owned interrupt object, latched)"]
TMR["Watchdog deadline<br/>(BootWatchdogs → one Instant)"]
CH["IPC channel<br/>(handler endpoint, READABLE)"]
end

WAIT["object_wait(signals, deadline)<br/>the single park point"]

subgraph MAP["Inbound adapters (source → Event)"]
SMAP["signal → ComponentReady / Booted"]
TMAP["poll_expired → Timeout / CommitTimeout"]
CMAP["channel_read → decode → UpdateRequest / ..."]
end

CORE["Orchestrator::dispatch<br/>(pure reducer)"]

OUT["Effects → PlatformDriver<br/>+ response → channel_respond"]

SIG --> WAIT --> SMAP --> CORE
TMR --> WAIT --> TMAP --> CORE
CH --> WAIT --> CMAP --> CORE
CORE --> OUT
```

[`Event::ComponentReady`]: ./orchestrator-machine.md
[`Event::Booted`]: ./orchestrator-machine.md
[`Event::Timeout`]: ./orchestrator-machine.md
[`Event::CommitTimeout`]: ./orchestrator-machine.md
[`Event::UpdateRequest`]: ./orchestrator-machine.md
24 changes: 24 additions & 0 deletions services/orchestrator/update-api/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Licensed under the Apache-2.0 license
# SPDX-License-Identifier: Apache-2.0

load("@rules_rust//rust:defs.bzl", "rust_library", "rust_test")

rust_library(
name = "orchestrator_update_api",
srcs = [
"src/lib.rs",
"src/status.rs",
"src/traits.rs",
"src/wire.rs",
],
crate_name = "openprot_orchestrator_update_api",
edition = "2024",
visibility = ["//visibility:public"],
deps = ["@rust_crates//:zerocopy"],
)

# Host tests: build on the host platform, no kernel/QEMU.
rust_test(
name = "orchestrator_update_api_test",
crate = ":orchestrator_update_api",
)
37 changes: 37 additions & 0 deletions services/orchestrator/update-api/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# orchestrator-update-api

The update intake seam between the orchestrator and an update source (the
PLDM firmware-device service in `services/pldm` today, a self-update path
later). Contract only: the `UpdateIntake` trait a source calls, the
`IntakeStatus` phases the orchestrator answers with, and the `wire` encoding
that carries both over a kernel channel. No transport, no policy, no state,
so both processes depend on it and it builds and tests on the host.

The source is the channel's initiator and the orchestrator its handler, and
there is no channel the other way. The orchestrator never waits on the
source, and every request it answers is one bounded step, so a wedged update
source cannot delay a boot window or a recovery. Everything the orchestrator
has to say comes back as the answer to a request the source made: each
response carries the current phase, and the source reads it with `poll` as
often as its own protocol needs.

One update, from the source's side:

1. `offer(target, total)` reserves the staging region.
2. `write(offset, bytes)` fills it, at most 512 bytes per call, in any order.
3. `complete()` declares the candidate complete, which is what starts the
update.
4. `poll()` until `Activated` or `Failed`.

What backs the staging region is the board's choice. Where the eRoT sits in
the target's flash path there is no room to hold a copy, so a write goes
straight into that device's inactive slot and the eRoT authenticates by
reading the slot back. The phase order follows from that and is not fixed:
see the `IntakeStatus` docs before assuming one.

`abort()` drops the job from any phase. Activation needs no call: the state
machine activates on its own verdict once the candidate authenticates.

Run the tests with:

bazel test //services/orchestrator/update-api:orchestrator_update_api_test
64 changes: 64 additions & 0 deletions services/orchestrator/update-api/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
// Licensed under the Apache-2.0 license
// SPDX-License-Identifier: Apache-2.0

//! The update intake seam between the orchestrator and an update source.
//!
//! An update source is the protocol frontend that takes a candidate image
//! from outside the platform: the PLDM firmware-device service
//! (`services/pldm`) today, self-update later. This crate is the contract
//! and nothing else, so both processes depend on it and it builds on the
//! host: the [`UpdateIntake`] trait a source calls, the [`IntakeStatus`]
//! phases the orchestrator answers with, and the [`wire`] encoding between
//! them.
//!
//! The source is the kernel channel's initiator and the orchestrator its
//! handler, with no channel the other way. So the orchestrator never waits
//! on the source, and handling a request is one bounded step: no request
//! means "wait until the state machine decides". Everything the
//! orchestrator decides therefore comes back as the phase on a response,
//! read as often as the source needs ([`UpdateIntake::poll`]).
//!
//! Two consequences, both easy to undo by accident:
//!
//! - `Effect::ReportUpdateDeferred` and `Effect::ReportUpdateAborted` latch
//! a phase, they do not send. The source collects them as
//! [`FailureCause::Deferred`] / [`FailureCause::Superseded`].
//! - Polling need not be timer-driven: the handler can raise
//! `Signals::USER` on the source's channel end
//! (`syscall::object_set_peer_user_signal`, as `services/i2c` does) when
//! the phase changes. That is a nudge carrying no data, and the syscall
//! does not block on the peer.
//!
//! What backs the staging region is the board's choice and never the
//! source's. On a passive device the eRoT sits in the flash path, so the
//! region is that device's inactive slot and a write goes straight into it;
//! the eRoT then authenticates by reading the slot back. A board with its
//! own staging flash copies first instead. Either way the source offers,
//! writes at offsets, and completes.
//!
//! This is the `api` layer of the pattern `services/i2c` and
//! `services/mctp` established. The orchestrator-side dispatch and the
//! source-side [`UpdateIntake`] impl over `channel_transact` are separate
//! crates.

#![cfg_attr(not(test), no_std)]
#![forbid(unsafe_code)]
#![warn(missing_docs)]

pub mod status;
pub mod traits;
pub mod wire;

pub use status::{FailureCause, IntakeStatus, Progress, Reject};
pub use traits::{IntakeError, UpdateIntake};
pub use wire::{Request, Response, UpdateOp};

/// Which managed device a candidate is for: the device's index in the
/// board's device table, which is what the driver validates an offer
/// against.
///
/// The orchestrator's `ComponentId` stays inside the orchestrator. A source
/// names a target, and the state machine never sees it at all: it only
/// decides that an update runs.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct TargetId(pub u8);
117 changes: 117 additions & 0 deletions services/orchestrator/update-api/src/status.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
// Licensed under the Apache-2.0 license
// SPDX-License-Identifier: Apache-2.0

//! What the orchestrator answers: the phase the source polls
//! ([`IntakeStatus`]), plus one failure vocabulary for a request refused on
//! the spot ([`Reject`]) and one for a job that ran and failed
//! ([`FailureCause`]).

/// Bytes moved so far out of the total the offer declared.
///
/// The wire form of `StageProgress::Transferring`: `written` is monotonic
/// and may hold still across polls, so a source watching for a stall keys on
/// the value, not on the poll count.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Progress {
/// Bytes moved so far, at or below `total`.
pub written: u64,
/// Total payload bytes, from the accepted offer.
pub total: u64,
}

/// The phase of the update in flight.
///
/// The source maps it onto what its own protocol owes its peer: a PLDM
/// firmware device turns [`Receiving`](Self::Receiving) into a transfer in
/// progress, [`Authenticating`](Self::Authenticating) into `VerifyPending`,
/// [`Staging`](Self::Staging) into apply progress, and
/// [`Failed`](Self::Failed) into the matching result code.
///
/// The order is not fixed and a source must not assume one. A board that
/// writes through to the target's inactive slot authenticates by reading
/// that slot back, so it runs [`Receiving`](Self::Receiving),
/// [`Authenticating`](Self::Authenticating), [`Activated`](Self::Activated).
/// A target whose gate is a signed manifest is authenticated before any byte
/// moves, which swaps the first two.
///
/// Intentionally exhaustive: adding a phase is a breaking change, so every
/// source handles each one.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum IntakeStatus {
/// No job: none offered, or the last one collected by a fresh offer.
Idle,
/// The source is writing the candidate into the staging region, which
/// on a write-through board is the target's inactive slot. `written`
/// counts what the orchestrator took, so a source that lost track can
/// resume.
Receiving(Progress),
/// The orchestrator is authenticating the complete candidate. No
/// counters: authentication is one verdict, not a transfer.
Authenticating,
/// The candidate authenticated; the orchestrator is pushing it to the
/// target device, one polled step at a time.
///
/// Only a board that stages into its own region ever reports this. Under
/// write-through there is nothing left to push, so the phase goes
/// straight from [`Authenticating`](Self::Authenticating) to
/// [`Activated`](Self::Activated).
Staging(Progress),
/// The staged image is the device's boot candidate, tentatively. The
/// commit gate is orchestrator policy, so this is the last phase a
/// successful update shows the source.
Activated,
/// The job ended without activating. Terminal: it holds until the next
/// accepted offer, so a source that polls late still learns the outcome.
Failed(FailureCause),
}

/// Why a job that started did not activate.
///
/// Coarse on purpose, mirroring `UpdateError`: the source reports an outcome
/// and retries or does not, and the orchestrator logs the concrete fault
/// while it is still in scope.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum FailureCause {
/// The candidate failed authentication. Not retriable with these bytes.
Authentication,
/// The state machine was mid-walk, mid-recovery, or mid-update when the
/// candidate completed, so it refused the update. The candidate is
/// discarded, not judged: offering it again later may succeed.
Deferred,
/// Recovery preempted the update after it started. Retriable like
/// [`Deferred`](Self::Deferred), different cause: the platform took the
/// update away, not the timing.
Superseded,
/// The device or the staging region failed the transfer, or activation
/// failed. Offering again may succeed.
Device,
}

/// Why the orchestrator refused a request outright.
///
/// A [`Reject`] answers the request itself and never leaves a job
/// half-started: nothing was written, no phase changed. Contrast
/// [`FailureCause`], which is a job that ran.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Reject {
/// The offer named a target the board's device table does not declare.
UnknownTarget,
/// An offer while a job is in flight. That job is untouched: a second
/// source cannot write into a region that is mid-authenticate.
Busy,
/// A write or complete with no accepted offer, or after the job left
/// [`IntakeStatus::Receiving`].
NoJob,
/// A write range outside the offered payload, or an offer larger than
/// the staging region.
OutOfRange,
/// An offer of zero bytes, or a complete before every offered byte was
/// written. There is no candidate, so authentication must not run.
Incomplete,
/// The staging write failed. Possibly transient.
Storage,
/// The request did not decode (see [`wire::WireError`](crate::wire::WireError)).
/// Two sides built from this crate never see it; the request side is
/// another process and is treated as untrusted.
Malformed,
}
Loading
Loading