From c4aeef38e085a95ddb6685722ff3956150b9b4c3 Mon Sep 17 00:00:00 2001 From: Stephen Brouhard Date: Thu, 2 Jul 2026 22:52:30 -0400 Subject: [PATCH 1/2] docs: add codex app backend protocol --- README.md | 3 + docs/architecture.md | 1 + docs/codex-app-backend.md | 194 ++++++++++++++++++++++++++++++++++++++ docs/configuration.md | 1 + 4 files changed, 199 insertions(+) create mode 100644 docs/codex-app-backend.md diff --git a/README.md b/README.md index 1bb8c961..185f4f74 100644 --- a/README.md +++ b/README.md @@ -119,6 +119,8 @@ When that profile file exists, crewmate and scout spawns must pass the resolved Secondmate launch can use a separate local `config/secondmate-harness`, whose first non-empty, non-comment line is parsed as ` [] []` to durably pin that secondmate's launch profile. The runtime session-provider backend is selected from explicit `--backend`, `FM_BACKEND`, local `config/backend`, runtime auto-detection from `$TMUX` or `HERDR_ENV=1`, then the hard `tmux` default. `tmux` is the verified reference backend, and `herdr` is experimental. +A proposed Codex App visible-thread backend protocol is documented separately, +but it is not a selectable upstream runtime backend yet. Secondmate homes inherit the primary's declared local config, including `config/crew-dispatch.json`, `config/crew-harness`, and `config/backlog-backend`, at launch, during the locked session-start bootstrap step, or during an explicit `bin/fm-config-push.sh` run, so their own crewmates, dispatch profiles, and backlog backend use the primary settings. When a routed request goes to a secondmate, firstmate marks it so the answer returns through status or a document pointer; direct typing into that secondmate window stays conversational. A presence-gated sub-supervisor (`/afk`) can self-handle routine events and batch only what matters while you step away. @@ -151,6 +153,7 @@ Agent-only reference skills live under `.agents/skills/` and are loaded by first - [docs/architecture.md](docs/architecture.md) - how the crew, supervision, worktrees, secondmates, and project modes work. - [docs/configuration.md](docs/configuration.md) - environment variables, `FM_HOME`, runtime backend selection, optional X mode, the files you set, and harness support. - [docs/herdr-backend.md](docs/herdr-backend.md) - experimental herdr backend verification notes and known gaps. +- [docs/codex-app-backend.md](docs/codex-app-backend.md) - proposed Codex App visible-thread backend protocol and future smoke expectations. - [docs/scripts.md](docs/scripts.md) - the `bin/` toolbelt reference. - [`AGENTS.md`](AGENTS.md) - firstmate's full operating manual for the orchestrator agent. - [CONTRIBUTING.md](CONTRIBUTING.md) - how to contribute, including the dev/test commands. diff --git a/docs/architecture.md b/docs/architecture.md index 9a233eb5..d13cc46b 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -40,6 +40,7 @@ Its tmux supervisor injection path shares the same submit core used by the tmux The runtime backend is the session-provider layer below firstmate's scripts. It owns task endpoint creation, bounded capture, text/key sends, current-path reads for spawn-time worktree discovery, live-window fallback lookup, and endpoint teardown. `bin/fm-backend.sh` centralizes backend selection, `state/.meta` helpers, selector resolution, and operation dispatch; `bin/backends/tmux.sh` is the verified reference adapter, and `bin/backends/herdr.sh` (P2) is an experimental second adapter. +The proposed Codex App visible-thread backend is documented as a protocol in [`docs/codex-app-backend.md`](codex-app-backend.md), but upstream does not implement a selectable `codex-app` adapter yet. New spawns select a backend from `--backend`, then `FM_BACKEND`, then local `config/backend`, then runtime auto-detection from `$TMUX` or `HERDR_ENV=1`, then default `tmux`. Runtime auto-detection is innermost-first: `$TMUX` wins over `HERDR_ENV=1` when firstmate is inside tmux nested in herdr, auto-detected herdr prints a one-time opt-out notice, and auto-detected tmux stays silent. Unknown backend names fail loudly. diff --git a/docs/codex-app-backend.md b/docs/codex-app-backend.md new file mode 100644 index 00000000..491d626a --- /dev/null +++ b/docs/codex-app-backend.md @@ -0,0 +1,194 @@ +# Codex App visible-thread backend protocol + +This document defines the intended protocol for a future `codex-app` runtime +backend. It is a design contract, not an implementation guide for a backend +that exists today. At the time this document is added, upstream firstmate +implements `tmux` and experimental `herdr` runtime backends only. + +The goal of `codex-app` is to make a crewmate a real Codex Desktop thread that +the captain can see, read, steer, and archive from the app sidebar while +firstmate keeps its normal shell-owned supervision state. + +## Boundary: shell state versus Codex Desktop tools + +Firstmate shell scripts own local orchestration state: + +- task ids, briefs, and status-file paths +- `state/.meta` records +- project delivery mode and yolo posture +- git worktree identity and branch checks +- no-mistakes branch/run hygiene +- PR polling, landed-work checks, and teardown safety + +Codex Desktop owns visible thread actions: + +- create or fork a thread +- send a message to a thread +- read a thread +- set a thread title or pinned state +- hand off or continue an existing thread +- archive a completed thread + +Shell helpers must not pretend that a headless transport is a visible Codex App +thread. If an operation requires a visible thread, the shell side should prepare +or validate local state and then report the host-tool action that must happen in +Codex Desktop. A future bridge may automate those host-tool calls, but that +bridge must still preserve the boundary: shell records state; Codex Desktop owns +thread visibility and thread mutations. + +## Identity alignment + +A Codex App task has three identities that must stay aligned: + +- Firstmate task identity: `id`, `fm-` title, `state/.meta`, and + `state/.status`. +- Codex App identity: visible `thread_id`, the app-created worktree path or + pending worktree handle, and the saved project/worktree context used by + Codex Desktop. +- Git/no-mistakes identity: the actual git worktree where edits and commits + occur, the task branch, and the no-mistakes run state for that branch. + +The crewmate must run git commands, tests, commits, pushes, and no-mistakes from +the Codex App-created worktree, not from the saved project checkout that merely +identifies the repository. Losing that alignment creates a dangerous false +success: the visible thread may look active while no-mistakes is checking a +different branch or firstmate is supervising a different worktree. + +At minimum, visible-thread metadata needs to identify: + +- `backend=codex-app` +- `window=` or another selector that resolves to the visible thread +- `thread_id=` once Codex Desktop has created the thread +- `codex_app_thread_state=` as `pending`, `visible`, or `archived` +- `worktree=` when the app-created worktree path is known +- a pending worktree handle when Codex Desktop has started worktree creation but + has not yet surfaced a final worktree path +- the usual `project=`, `harness=`, `kind=`, `mode=`, `yolo=`, `tasktmp=`, and + PR/no-mistakes fields used by the rest of firstmate + +## Lifecycle states + +### Pending + +`pending` means firstmate has prepared local state but the visible thread is not +fully recorded yet. This can happen when spawn preparation has written a brief +and meta scaffold, or when Codex Desktop has begun creating an app-owned +worktree but has not returned a usable thread id and worktree path. + +Allowed shell actions in `pending` are narrow: print the next required Codex +Desktop action, record a returned thread id or pending worktree handle, refuse +normal send/read/teardown operations that require a visible thread, and allow +safe cleanup of failed preparation if no work has started. + +### Visible + +`visible` means `state/.meta` has a Codex App thread id and enough worktree +identity for firstmate to reconcile the task after restart. The thread should be +discoverable through Codex Desktop thread listing, readable through the app, and +steerable through app thread messaging. + +In this state, firstmate may supervise the task through status files and +no-mistakes state, but live thread reads and sends still happen through Codex +Desktop host tools or a verified bridge to those tools. A visible task may be a +ship task, scout task, adopted existing thread, or handoff/fork from an earlier +thread. + +### Archived + +`archived` means the visible thread has been archived in Codex Desktop after the +task reached a safe completion state. Archive is not deletion. Firstmate must +still preserve the landed PR, local merge evidence, scout report, backlog entry, +and any durable records needed for future audit. + +Teardown for Codex App tasks should require archive evidence before removing the +local task state. Ship teardown must also prove work is landed using the same +git and PR rules as other backends. Scout teardown may proceed once the report +exists and the thread archive is recorded. + +## Spawn and live thread creation are separate + +Spawn preparation and live thread creation are intentionally separate steps. +`fm-spawn` can choose an id, write a brief, create or reserve local metadata, set +the delivery mode, and establish no-mistakes branch expectations without +impersonating Codex Desktop. It cannot, by itself, guarantee a sidebar-visible +Codex App thread unless it is running with a verified host-tool bridge. + +The split prevents two failure modes: + +- A shell-only helper completing a headless conversation and falsely satisfying + the "visible backend" contract. +- A manually created visible thread that lacks Firstmate metadata, status-file + reporting, worktree identity, or no-mistakes branch alignment. + +The protocol is therefore prepare, create/fork in Codex Desktop, record the +returned app identity, then supervise. Adoption follows the same rule in reverse: +read the existing visible thread identity from Codex Desktop, then write the +missing Firstmate metadata before treating it as managed work. + +## Host-tool operations + +A future Codex App backend should treat these host-tool capabilities as the live +surface, whether exposed directly to the agent or through a verified bridge: + +- `list_projects` to find the saved project whose path matches the target repo +- `create_thread` for project-scoped task spawn +- `fork_thread` when the captain explicitly wants to inherit completed context + from an existing thread +- `send_message_to_thread` to deliver the crewmate brief and later steering +- `read_thread` to inspect progress or reconcile after restart +- `set_thread_title` to set the visible `fm-` title +- `set_thread_pinned` when pinning active work is part of the operator flow +- `set_thread_archived` to archive completed work before teardown +- `handoff_thread` only for host-supported handoff flows where ownership or + continuation semantics differ from a normal send + +Project task spawn and current-thread context fork are different flows. Project +spawn should create a task in the intended project/worktree context. Forking the +current firstmate conversation copies conversation context and may put the task +in the wrong repository context unless the host tool explicitly binds it to the +target project. + +## No-mistakes hygiene + +Codex App visible-thread work must keep no-mistakes tied to the worker's actual +git worktree and branch. The backend protocol should make the crewmate prove its +working directory before editing and before starting the gate: + +```sh +pwd -P +git rev-parse --show-toplevel +git branch --show-current +git log --oneline --max-count=3 +``` + +If no-mistakes reports that there is no previous run for the branch, or if a PR +is opened from an unexpected branch, firstmate should check for crossed +worktree/branch bookkeeping before treating the result as a normal validation +failure. + +## Future smoke expectations + +These are acceptance expectations for a future implementation. They are not +claims about the current upstream tree. + +A non-interactive contract test can verify local ledger behavior: pending, +visible, and archived meta transitions; duplicate task/thread refusal; refusal +to treat shell-only send/read as a visible host-tool action; and teardown gates +that require archive plus landed-work evidence. + +A live Codex Desktop smoke should prove: + +- project-scoped `create_thread` produces a sidebar-visible `fm-` thread +- the thread appears in `list_threads` +- `read_thread` can inspect the task after creation and after restart +- `send_message_to_thread` can steer the task +- the crewmate writes the expected status-file events from the app-created + worktree +- no-mistakes runs on the same branch and worktree recorded in metadata +- `set_thread_archived` archives completed work without losing PR/report + evidence +- tmux and herdr regression tests still pass unchanged + +Until those smokes exist and pass, documentation and code should describe +`codex-app` as a proposed visible-thread backend protocol rather than a +selectable upstream backend. diff --git a/docs/configuration.md b/docs/configuration.md index 5649c7a6..6ba059d8 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -24,6 +24,7 @@ New spawns choose the backend in this order: explicit `fm-spawn.sh --backend Date: Thu, 2 Jul 2026 23:21:37 -0400 Subject: [PATCH 2/2] no-mistakes(document): Document Codex backend proposal --- AGENTS.md | 4 ++-- CONTRIBUTING.md | 3 ++- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index d657a01b..2b974fde 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -77,7 +77,7 @@ config/crew-harness crewmate harness override; LOCAL, gitignored; absent or "de config/crew-dispatch.json optional crewmate dispatch profiles; LOCAL, gitignored; firstmate-maintained but human-editable natural-language rules that choose a per-task harness/model/effort profile (section 4). Inherited by secondmate homes config/secondmate-harness harness the PRIMARY uses to launch SECONDMATE agents, optionally followed by a model and effort token on the same line (" [] []"; section 4); LOCAL, gitignored; absent or "default" harness falls back to config/crew-harness then firstmate's own. The primary's own setting; NOT inherited into secondmate homes (secondmates do not spawn secondmates) config/backlog-backend backlog backend override; LOCAL, gitignored; absent or "tasks-axi" = default tasks-axi backend, "manual" = force hand-editing; inherited by secondmate homes (section 10) -config/backend runtime session-provider backend override for new tasks; LOCAL, gitignored; absent = falls through to runtime auto-detection (the runtime firstmate itself is executing inside), then tmux; tmux is the verified reference backend, herdr is a second, experimental backend (docs/herdr-backend.md); not inherited into secondmate homes +config/backend runtime session-provider backend override for new tasks; LOCAL, gitignored; absent = falls through to runtime auto-detection (the runtime firstmate itself is executing inside), then tmux; tmux is the verified reference backend, herdr is a second, experimental backend (docs/herdr-backend.md); the proposed Codex App visible-thread protocol is documented in docs/codex-app-backend.md but `codex-app` is not a selectable backend yet; not inherited into secondmate homes config/x-mode.env generated X-mode watcher cadence; LOCAL, gitignored; source before arming watcher when present data/ personal fleet records; LOCAL, gitignored as a whole backlog.md task queue, dependencies, history @@ -500,7 +500,7 @@ Dispatch several tasks in one call by passing `id=repo` pairs instead of a singl If one pair fails, the rest still run and the batch exits non-zero. When `config/crew-dispatch.json` exists, include a shared `--harness` for every crewmate or scout batch after consulting the dispatch rules. -The script resolves the harness (`fm-harness.sh crew` for crewmate/scout tasks only when `config/crew-dispatch.json` is absent, `fm-harness.sh secondmate` for `kind=secondmate`; section 4), resolves the runtime backend (`--backend`, then `FM_BACKEND`, then `config/backend`, then runtime auto-detection - the runtime firstmate itself is executing inside, from `$TMUX`/`HERDR_ENV=1` markers, nesting resolved innermost-first - then `tmux`; an auto-detected herdr spawn prints a loud stderr notice, auto-detected tmux stays silent), owns the verified launch templates, resolves the project's delivery mode (`fm-project-mode.sh`) for ship/scout tasks, and records `harness=`, `model=`, `effort=`, `kind=`, `mode=`, and `yolo=` in the task's meta; only a non-default runtime backend is recorded as `backend=` because absent means tmux. +The script resolves the harness (`fm-harness.sh crew` for crewmate/scout tasks only when `config/crew-dispatch.json` is absent, `fm-harness.sh secondmate` for `kind=secondmate`; section 4), resolves the runtime backend (`--backend`, then `FM_BACKEND`, then `config/backend`, then runtime auto-detection - the runtime firstmate itself is executing inside, from `$TMUX`/`HERDR_ENV=1` markers, nesting resolved innermost-first - then `tmux`; an auto-detected herdr spawn prints a loud stderr notice, auto-detected tmux stays silent; `codex-app` remains proposal-only until docs/codex-app-backend.md is implemented), owns the verified launch templates, resolves the project's delivery mode (`fm-project-mode.sh`) for ship/scout tasks, and records `harness=`, `model=`, `effort=`, `kind=`, `mode=`, and `yolo=` in the task's meta; only a non-default runtime backend is recorded as `backend=` because absent means tmux. A non-flag third argument containing whitespace is treated as a raw launch command (only for verifying new adapters). When `config/crew-dispatch.json` exists, the script refuses crewmate or scout launches without an explicit harness because firstmate must have already resolved the profile choice at intake. When `--model` or `--effort` is omitted, the corresponding meta value is `default` and no launch flag is passed for that axis, except that a `kind=secondmate` spawn can fill the omitted axis from the optional tokens in `config/secondmate-harness`. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 2de74545..f45ce0c3 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -39,13 +39,14 @@ See the [no-mistakes quick start](https://kunchenguid.github.io/no-mistakes/star The root `.tasks.toml` is tracked `tasks-axi` config for `data/backlog.md`; compatible `tasks-axi` is the default backend for routine backlog mutations. A local `config/backlog-backend=manual` opt-out forces hand-editing and stays gitignored. A local `config/backend` file explicitly overrides runtime auto-detection for new task endpoints and stays gitignored; accepted values are `tmux` and experimental `herdr`. + The proposed `codex-app` visible-thread backend is documented in `docs/codex-app-backend.md`, but is not a selectable backend value yet. It does not make `data/` tracked. - Helper scripts in `bin/` are plain bash. Each starts with a usage header comment; keep it accurate when you change behavior. Test scripts and helpers in `tests/` are plain bash too. `shellcheck bin/*.sh bin/backends/*.sh tests/*.sh` must pass, and CI enforces it. - Changes to harness adapters (detection in `bin/fm-harness.sh`, launch and hook mechanics in `bin/fm-spawn.sh`, busy signatures in `bin/fm-watch.sh` and `bin/fm-tmux-lib.sh`, cleanup in `bin/fm-teardown.sh`, and facts in `.agents/skills/harness-adapters/SKILL.md`) must be verified empirically against the real harness, never written from documentation alone. -- Changes to runtime session backends (`bin/fm-backend.sh`, `bin/backends/`, and the scripts that dispatch through them) need empirical adapter notes in the relevant docs, following `docs/herdr-backend.md` for non-tmux backends. +- Changes to runtime session backends (`bin/fm-backend.sh`, `bin/backends/`, and the scripts that dispatch through them) need empirical adapter notes in the relevant docs, following `docs/herdr-backend.md` for implemented non-tmux backends and `docs/codex-app-backend.md` for future Codex App visible-thread protocol work. - In Markdown, put each full sentence on its own line. ## Development