Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 78 additions & 8 deletions CLAUDE.md

Large diffs are not rendered by default.

12 changes: 11 additions & 1 deletion docs/design/fleet.md
Original file line number Diff line number Diff line change
Expand Up @@ -978,7 +978,8 @@ M7 (security hardening) : cross-cutting, folds into each; can start after M0
basemaps are read from **site bundles on the fleet box** through routes M4
keeps while replacing where the bytes come from; and the **operator token is
one credential on one path**, not the auth story M7 owes — the read routes and
the broker are still open on the tailnet.
the broker were still open on the tailnet. (The read routes are closed since;
the broker is not.)

### v1 — second robot enrolled, plus shared infrastructure

Expand Down Expand Up @@ -1058,6 +1059,15 @@ M7 (security hardening) : cross-cutting, folds into each; can start after M0
a robot can't read another robot's command topic. *Depends on:* M0; folds into
each milestone as it lands.

**Landed so far:** operator auth on every `/v1` route (one gate in front of
routing, four stated carve-outs), the Tailscale access policy as a committed
file with its own `tests` block, and `pixi install --locked` in provisioning.
**Still owed:** the broker half — per-robot and per-operator broker
credentials and the ACL that keeps them apart — which is where the second
acceptance criterion ("a robot can't read another robot's command topic")
lives, and which waits on M6. Package signing waits on M5, when there is a
package to sign.

### Cross-cutting — adopting the open specifications

Not a milestone of its own: it re-shapes M1's control plane and M3's dispatch
Expand Down
85 changes: 53 additions & 32 deletions docs/fleet/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,19 +75,26 @@ Tags do not exist until an owner is declared for them, and `--advertise-tags` on
a machine fails with "requested tags are invalid or not permitted" if you skip
this:

```jsonc
{
"tagOwners": {
"tag:robot": ["autogroup:admin"],
"tag:fleet": ["autogroup:admin"],
"tag:inference": ["autogroup:admin"],
},
// The default policy already allows every device to reach every other, which
// is what M0 wants. M7 replaces this with per-tag rules (operators reach
// robots; robots reach the broker and their own inference box; robots cannot
// reach each other).
}
```
The policy this repo ships is
[`mote_bringup/tailscale/policy.hujson`](../../mote_bringup/tailscale/policy.hujson):
the three tags above, plus the rules that say who may open a socket to what.
Tailscale keeps the policy in its console rather than in a repo, so **the file is
the source of truth and the console is a copy** — edit here first, then paste:

1. <https://login.tailscale.com/admin/acls/file>
2. paste the file, **Preview** to see what changes, then **Save**.

Two things it does that the default "everyone reaches everyone" policy does not.
Operators reach a robot's SSH and Foxglove ports and nothing else on it; a robot
reaches the fleet server's API and broker and its inference box's two wire ports,
and **no robot can reach another robot** — there is no robot-to-robot anything in
v1, so the absence of a rule is the design. The file carries a `tests` block
asserting exactly that, and Tailscale evaluates it on every save and refuses a
policy that fails one, so the rule is checked by the thing enforcing it.

Do this before joining any machine: `--advertise-tags` on a tag the policy does
not declare fails with "requested tags are invalid or not permitted", which is
the first thing to check if joining a robot fails.

**4. Mint an auth key per robot.** Admin console → **Settings → Keys → Generate
auth key**. For a robot:
Expand All @@ -109,7 +116,7 @@ opens a browser to authenticate you.
(180 days by default) and needs a human to re-authenticate it, which for a robot
means it silently drops off the tailnet one day months from now. Tagged devices
do not expire. That failure mode is the practical argument for `tag:robot`,
ahead of anything M7 does with it.
ahead of anything the access policy does with it.

### 1b. Joining machines

Expand Down Expand Up @@ -145,9 +152,11 @@ any tagged role are mutually exclusive and the script refuses the combination.
For one operator at one site, leave the dev machine an untagged workstation that
happens to run Mosquitto and the inference servers: nothing functional depends on
the tag (robots reach it by MagicDNS either way, and `inference_host` is just a
name). What you defer is M7's ACLs — a rule keyed on your user's device rather
than `tag:fleet`/`tag:inference`. Tag it when a second person or a second machine
appears and the roles want to outlive your account.
name). What you defer is the policy's rules — they are keyed on
`tag:fleet`/`tag:inference`, so an untagged dev box is reachable under
`autogroup:member` rather than under the rule written for its role. Tag it when a
second person or a second machine appears and the roles want to outlive your
account.

**Verify off-LAN** (the M0 acceptance test) — from a device on a *different*
network, e.g. a laptop tethered to a phone:
Expand Down Expand Up @@ -483,14 +492,18 @@ fleet box's MagicDNS name (`fleet-box`), not `localhost` — it is handed out
verbatim in every enrollment answer. It defaults to the box's hostname. Under
compose it is `BROKER_HOST` in `.env`, and the stack refuses to start without it.

**Security, plainly:** the broker is anonymous and the API's *read* routes are
unauthenticated. Dispatch is not — it needs an operator token (§8) — but that is
one credential on one path, not an auth story. It is proportionate only because
the tailnet is the boundary: WireGuard authenticates, and nothing here is
exposed to the internet. Do not put either on a network the robots are not
already trusted on. Per-robot broker credentials and operator auth everywhere
are M7; the shape of what changes is in
[`control-plane.md`](control-plane.md#security-posture-and-what-m7-changes).
**Security, plainly:** every `/v1` route on the API needs an operator token
(§8), checked by one gate in front of routing rather than by each handler. Four
things are open and each for a reason: `/healthz`, the static UI, enrollment
(which carries its own token), and the two robot-facing map routes, which carry
no credential because robots have none to carry yet. The **broker is still
anonymous**, so the dashboard's read path and `fleetctl watch` connect to it
without one. The tailnet is the outer boundary — WireGuard authenticates, the
policy in §1a says who may reach which port, and nothing here is exposed to the
internet. Do not put the broker on a network the robots are not already trusted
on. Per-robot and per-operator broker credentials are still to come; the shape of
what changes is in
[`control-plane.md`](control-plane.md#security-posture-and-what-is-still-owed).

---

Expand Down Expand Up @@ -571,6 +584,11 @@ over the network; the **name on it is what the audit log records**, which is why
an unnamed one is refused. `fleetctl operator list|revoke` are the other two
verbs, and the route contract is [`fleet-api.md`](fleet-api.md).

**`MOTE_FLEET_TOKEN` is now needed for every `fleetctl` verb that talks to the
API**, not only `dispatch` and `audit`: the roster, the maps and the registry are
operator-only too. Without it they answer `401` and say so. `watch` is the
exception, because it reads the broker rather than the API.

**A mission is a capability and a typed input**, not a sentence: the first
argument is a capability key and the rest are `key=value` pairs (or one `{...}`
JSON object when a value is not a string). What keys a robot offers, and what
Expand Down Expand Up @@ -654,8 +672,8 @@ mote-01 Scout (home)
off publishes `online: false` through its Last Will and reads `offline`, while
`unknown` means nothing has ever been heard from it — or that the fleet server
is not connected to the broker, which both commands say outright when it is so.
The detail view needs the operator token; the roster does not. What it shows of
a mission is the **last** status, because one is all the broker retains; `watch`
Both need the operator token, like every API route. What the detail view shows
of a mission is the **last** status, because one is all the broker retains; `watch`
and `dispatch` are what show every transition, and they keep the broker for
exactly that reason. The route is
[`fleet-api.md`](fleet-api.md#get-v1robotsrobot_id), and it is the same answer
Expand Down Expand Up @@ -684,11 +702,14 @@ no request/response loop, and no service between the broker and the browser.
That is the read path in [`fleet.md`](../design/fleet.md) Q5, and it is why the
broker needs the WebSocket listener from §6.

**Paste an operator token to dispatch.** Without one the page is read-only,
which is a perfectly good wall display. The token is kept in the browser's local
storage and sent to the fleet API as a bearer credential; the page holds **no
broker credential that can publish**, and its MQTT client implements no PUBLISH
packet at all.
**Paste an operator token to see anything at all.** `/v1/config` — what the
page is built out of — is operator-only like every other route, so there is no
read-only mode: the dashboard is signed in, or it is asking to be, and a wall
display whose token was revoked shows the gate rather than a stale fleet. Pasting
a token starts everything without a reload. The token is kept in the browser's
local storage and sent to the fleet API as a bearer credential; the page holds
**no broker credential that can publish**, and its MQTT client implements no
PUBLISH packet at all.

**Each state is said once**, by the strongest idiom the page has. The roster
row's dot and its state column already read `ok`, so the line under them speaks
Expand Down
40 changes: 24 additions & 16 deletions docs/fleet/control-plane.md
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,8 @@ broker that lost its retained state with its volume, repairs itself.
fleet box is reached — MagicDNS name, tailnet address or localhost.

`sha256` is checked by the puller before anything is staged. It is not a
security boundary (the tailnet is that until M7); it is there because a
security boundary (the tailnet is that, while the broker is anonymous); it is
there because a
transfer that silently truncated would otherwise become a map, and a wrong map
is worse than no map.

Expand Down Expand Up @@ -482,25 +483,32 @@ robots enrolling at once get eight distinct ids.

---

## Security posture (and what M7 changes)
## Security posture (and what is still owed)

M1 is proportionate to the M0 substrate and no further. Stated plainly so it is
not mistaken for a finished story:
Stated plainly so it is not mistaken for a finished story:

- **The broker is anonymous.** Any client that can reach it may publish or
subscribe anywhere in the tree. WireGuard is the authentication boundary;
nothing here is reachable from the public internet.
- **The fleet API has no auth** on its read routes. Enrollment tokens and, since
M3, operator tokens are the only credentials in the system.
- **The fleet API is not.** Every `/v1` route needs an operator token, checked by
one gate in front of routing ([`fleet-api.md`](fleet-api.md)); `/healthz`, the
static UI, enrollment and the two robot-facing map routes are the carve-outs,
each for a stated reason.
- **The tailnet has rules.** `mote_bringup/tailscale/policy.hujson` is the
committed access policy: operators reach the fleet box and a robot's SSH and
Foxglove ports, robots reach the fleet server and their inference box, and no
robot reaches another. Its `tests` block asserts that last one, and Tailscale
refuses to save a policy that fails it.
- **Dispatch is mediated, as of M3.** M1's `fleetctl` published straight to the
broker; now it and the dashboard both POST to `/v1/robots/<id>/dispatch`,
which authorizes an operator token and writes an audit row before publishing
([`fleet-api.md`](fleet-api.md)). As this section promised, **the topic tree
did not change** — only who publishes to it. The browser holds no broker
credential that can publish; making that structural on the broker side, with a
subscribe-only credential, is still M7's.

M7 adds per-robot broker credentials (username = `robot_id`, publish confined to
its own prefix), operator auth on the API, and the Tailscale ACLs that stop
robots reaching each other. Until then: do not put the broker or the API on a
network the robots are not already trusted on.
which authorizes an operator token and writes an audit row before publishing.
As this section promised, **the topic tree did not change** — only who
publishes to it. The browser holds no broker credential that can publish;
making that structural on the broker side, with a subscribe-only credential,
waits on the broker having credentials at all.

What is still owed is the broker half: per-robot credentials (username =
`robot_id`, publish confined to its own prefix), a subscribe-only operator
credential for the browser, and the ACL that keeps the three principals apart.
Until then, do not put the broker on a network the robots are not already
trusted on.
72 changes: 44 additions & 28 deletions docs/fleet/fleet-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,13 +53,20 @@ Status codes are part of the contract: a client may switch on them.

## Authentication

**Every `/v1` route needs an operator token except those named below.** The
check is one gate in front of route dispatch, not a line in each handler, so a
route added later is authenticated by default and has to opt out in a place a
reviewer reads. An anonymous caller is refused *before* the route table is
consulted for existence: an unknown path answers `401`, never a `404` that would
say which routes are real.

| Route | Credential |
|---|---|
| `POST /v1/enroll` | an **enrollment token** in the body (single-use by default) |
| `POST /v1/robots/<id>/dispatch`, `GET /v1/audit`, `GET /v1/robots/<id>` | an **operator token** as `Authorization: Bearer <token>` |
| `POST …/revisions/<rev>/promote` | an **operator token** |
| `POST …/revisions/<rev>` (map upload) | none, but the `robot_id` must be enrolled — see [the registry](#the-map-registry-m4) |
| everything else | none — see the security note below |
| everything under `/v1` | an **operator token** as `Authorization: Bearer <token>` |
| `GET /healthz` | none — a liveness probe that needs a secret is one nobody wires up |
| the static UI (`/`, `/*.mjs`, …) | none — the page has to load in order to ask for a token, and holds no fleet data until it has one |
| `POST /v1/enroll` | an **enrollment token** in the body (single-use by default) — a robot is not an operator, and an unattended first boot has no human behind it |
| `POST …/revisions/<rev>` (map upload) and `GET …/revisions/<rev>/bundle.tar.gz` (pull) | none, but an upload's `robot_id` must be enrolled — see [the registry](#the-map-registry-m4) |

Operator tokens are minted on the fleet box, against the registry file, never
over the network:
Expand All @@ -76,15 +83,22 @@ is refused. Revocation keeps the row: who *had* access is part of the record.
Bearer header only — never a query parameter, which would put the credential in
every access log between here and the browser.

**Security posture, plainly.** Most read routes are unauthenticated and the
broker is anonymous, exactly as M1 left them. M3 adds a credential on the
*write* path and a record of who used it, which is the milestone's brief; since
then two reads have been gated as well: the audit log, which nothing else
serves, and one robot's live state, whose payloads the anonymous broker also
carries until M7. It is proportionate only while the tailnet is the boundary. M7 adds operator
auth on the rest of the read routes, per-robot broker credentials, and the
Tailscale ACLs. Until then, do not expose this port to a network the robots are
not already trusted on.
**The two robot routes are the carve-out that costs something, and it is
deliberate.** M4's rule is that uploading is not publishing: a candidate changes
nothing about any floor until an operator promotes it, so the upload is bounded,
audited and inert, and the pull serves only what an operator has already
promoted. The alternative today is a credential robots do not have — issuing one
at enrollment is its own piece of work, and until it lands these two routes are
what the tailnet protects rather than what the API does.

**Security posture, plainly.** The API needs a credential everywhere; the broker
is still anonymous, so the dashboard's MQTT read path is open to anything on the
tailnet that can reach port 9001. The write path is not: the browser's MQTT
client implements no PUBLISH packet, and every write to `mission/command` goes
through `POST …/dispatch` here. The outer boundary is the tailnet
(`mote_bringup/tailscale/policy.hujson`), which is what keeps this port off the
public internet. Do not expose it to a network the robots are not already
trusted on.

---

Expand Down Expand Up @@ -210,14 +224,9 @@ mission, which is a property of asking rather than of listening.
fleet server is repopulated by the broker within about a second of connecting;
a stored copy could only ever be the staler answer.

**The operator token hides nothing yet.** This route carries the coordinates —
a pose says where in a building a robot is, the mission status what it was told
to do there — so it takes the credential M7 will require of every read. Until
M7, the same payloads are on the anonymous broker and every `robot_id` is in the
anonymous roster, whose `presence` column shows nothing a broker subscriber
cannot already see. The token is checked *before* the robot is looked up, so an
unauthenticated request gets `401` whatever id it names; that stops ids leaking
through this route once M7 gates the roster, and not before.
**The operator token hides less than it looks.** The gate takes it before the
robot is looked up, so an unauthenticated request gets `401` whatever id it
names. The same payloads are on the broker, which is still anonymous.

### `POST /v1/robots/<robot_id>/dispatch`

Expand Down Expand Up @@ -570,8 +579,8 @@ grey rectangle, which is what a mapping run that never got going looks like.
**Why this route has no credential.** Everything it can do is inert: a candidate
changes no floor, is bounded in size and count, and is recorded in the audit log
against the robot that sent it. The write that *does* change something —
promote — is the operator's. M7 replaces the `robot_id` check with a per-robot
credential.
promote — is the operator's. Replacing the `robot_id` check with a per-robot
credential waits on robots having one to present.

### `POST /v1/sites/<site>/floors/<floor>/revisions/<rev>/promote`

Expand Down Expand Up @@ -646,8 +655,8 @@ with nothing published at all. It stays under a `/v1/maps`-shaped path and never
over `/v1/zones`, because it is served beside a basemap and that is what the two
prefixes divide.

All three are reads, so like every other read route they take no operator token;
M7 changes that for all of them at once.
All three are reads, and like every other `/v1` route they need an operator
token.

### `POST /v1/sites/<site>/floors/<floor>/zones`

Expand Down Expand Up @@ -731,8 +740,15 @@ The dashboard holds an operator token for this API and **no broker credential at
all that can publish**. The read path connects to the broker's WebSocket
listener with a client that implements no PUBLISH packet
([`ui/mqtt.mjs`](../../mote_fleet/server/ui/mqtt.mjs)) — the split is enforced by
omission, not by intention. M7 makes that structural on the broker side too,
with a subscribe-only credential.
omission, not by intention. A subscribe-only broker credential would make it
structural on the broker's side too, and waits on the broker having credentials
at all.

**The token is what the page is built out of.** `/v1/config` is operator-only
like every other route, so there is no read-only mode to fall back to: the
dashboard has two states, signed in or asking to be. Pasting a token starts
everything without a reload, and a token that stops working puts the page back at
the gate rather than into a half-state showing stale rows.

**Reviewing a candidate is all GETs.** The review pane reads a revision's
`map.json`, `map.png` and `zones.json`; the two writes beside them are the
Expand Down
Loading