Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/architecture/FORGE_PLATFORM_ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,14 @@ Detailed decisions are recorded in the [ADRs](adr/README.md). The [ownership mat

The [Governed Engineering Knowledge Learning Loop](KNOWLEDGE_LEARNING_LOOP.md) records the independently owned Knowledge Base lifecycle and its additive cross-product integration boundary.

## Installed-server deployment and topology bootstrap

The accepted target deployment is defined by [ADR-0006](adr/ADR-0006-server-deployment-and-discovery.md) and the shared [instance discovery and pairing contract](INSTANCE_DISCOVERY_AND_PAIRING_CONTRACT.md). Forge Server, EP Server and Workspace Server are headless, independently installed services with separate central runtime roots, product-owned SQL/files and versioned HTTP APIs above their application services. They may share a host but not a database or direct SQL path.

Forge Platform composes verified artifacts, service lifecycle and topology bootstrap for EP-only, Forge+EP, Workspace+EP, all-server and remote-peer configurations. It calls product APIs to create product-owned bindings; it never writes product databases. The installer is not a required runtime dependency after installation.

Discovery yields an unauthenticated candidate only. Pairing verifies stable identity and creates a pinned authenticated binding; a later discovery result cannot silently replace it. Workspace Client↔Workspace Server, EP Agent↔EP Server and server-peer credentials remain separate even on localhost.

## First-class component model

| Repository | First-class authority / published component |
Expand Down
52 changes: 52 additions & 0 deletions docs/architecture/INSTANCE_DISCOVERY_AND_PAIRING_CONTRACT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Instance discovery and pairing contract v1

**Status:** Canonical target contract. Product implementation and qualification are separately governed.

This contract is shared by Forge Server, Engineering Platform (EP) Server and Workspace Server. It gives the installer and product-owned clients one safe way to find a candidate, prove which instance it is, and create an explicit binding. It does not create a shared runtime, database, authorization domain, or service registry authority.

## Descriptor and discovery

Every installed server owns a random, stable, opaque `instance_id`. The ID is created with its installation identity, survives ordinary restart/update and is included in product-owned backup/restore semantics. It is never derived from a hostname, IP address, checkout, project or mDNS name. A replacement or a deliberately reissued identity is a different instance.

`forge-platform.instance-descriptor/v1` is the public candidate document:

```json
{
"schema": "forge-platform.instance-descriptor/v1",
"product": "forge-server | ep-server | workspace-server",
"instance_id": "opaque-stable-id",
"api": [{"version": "v1", "endpoint": "https://host.example/api/v1"}],
"capabilities": ["health", "pairing", "..."],
"identity_fingerprint": "public-key-or-certificate-fingerprint",
"expires_at": "RFC3339 timestamp"
}
```

Implementations may add non-sensitive display metadata, but descriptors and discovery records contain **no** bearer credential, pairing secret, project or repository data, account data, queue/run state, path, log, or private topology. Unknown required fields and unsupported schema/API versions fail closed.

LAN discovery may advertise a short-lived descriptor through DNS-SD/mDNS (for example `_forge-platform._tcp`, with product, descriptor version, instance ID, API versions and a descriptor endpoint/fingerprint in TXT data). It is only a candidate locator. A configured HTTPS/unicast endpoint, private-overlay or tailnet endpoint is an equal bootstrap route and is required where multicast is unavailable. Discovery never implies reachability, authenticity, authorization or permission to bind.

## Authenticated pairing and durable binding

An operator or an already authorized product principal starts a pairing flow. The initiating side verifies the endpoint's authenticated identity fingerprint, product, stable ID, supported API/capabilities and requested scope. The target performs its own authentication/authorization and creates a short-lived, single-use pairing ceremony. On completion each owning product persists its own binding through its application service/API: peer product, `instance_id`, verified fingerprint, endpoint set, negotiated versions/capabilities, scope, creation/rotation evidence and revocation state.

Discovery is deliberately not a pairing API. Neither the Forge Platform installer nor a peer writes directly to another product database. An existing binding is pinned to its recorded instance ID and fingerprint: rediscovery of a different candidate, changed fingerprint or changed endpoint identity fails closed and requires an explicit inspect/re-pair/rebind operation. Endpoint rotation for the same identity follows the product's authenticated binding update protocol; it is never an automatic switch to a newly found instance.

Trust domains remain separate:

| Relationship | Trust owner | It must not become |
| --- | --- | --- |
| Workspace Client ↔ Workspace Server | Workspace user/session | server-peer or EP Agent trust |
| EP Project Agent ↔ EP Server | EP host/agent | Workspace Client or Forge peer trust |
| Forge/EP/Workspace Server peers | respective product peer bindings | a user session or Agent credential |

Co-location and loopback do not remove authentication, pairing or authorization.

## Server-operational invariants

Each server is a headless, independently restartable installed service with a product-owned runtime storage root outside source/Git checkouts. Its versioned HTTP API is a transport adapter over interface-neutral application services; direct cross-product SQL and shared databases are prohibited. On macOS the product publishes a launchd service contract; Forge Platform orchestrates its lifecycle but does not become its runtime authority. Product storage contains its own SQL database and product-owned files, artifacts, logs, backups and cache, with permissions, retention, migration and recovery defined by the product owner.

## Delivery sequencing

The first Forge → EP → Forge autonomy canary needs only the minimum seams: installed Forge/EP service storage and stable identity, their versioned authenticated HTTP contract, an explicitly configured/pinned EP binding, and restart-safe recovery. It does **not** require Workspace UI, LAN discovery or the universal installer to be complete. Network discovery, full pairing UX, all topology combinations, Workspace Client distribution and installer productization are post-canary capabilities; they must preserve these seams.

16 changes: 16 additions & 0 deletions docs/architecture/adr/ADR-0006-server-deployment-and-discovery.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# ADR-0006 — Installed servers, explicit discovery, and pairing

**Status:** Accepted

## Decision

Forge Server, EP Server and Workspace Server are independently installable, headless services. Each has a product-owned central runtime-storage root outside Git/source checkouts, a SQL database plus owned files/artifacts/logs/backups/cache, a stable installation identity, a versioned HTTP API over application services and a macOS launchd service contract. They may be co-located or distributed, but never share a database or use direct cross-product SQL.

Servers use the shared [instance discovery and pairing contract](../INSTANCE_DISCOVERY_AND_PAIRING_CONTRACT.md): DNS-SD/mDNS is a candidate locator; configured/unicast/tailnet endpoints are equivalent bootstrap routes; explicit authenticated pairing creates product-owned durable bindings. Discovery is not authorization and an existing binding cannot silently move to a new discovered instance.

Forge Platform owns component choice, verified artifact acquisition, installation/update/repair/uninstall, lifecycle choreography and discovery-driven topology bootstrap. It invokes each product's public API to pair/bind and never writes a product database. Its runtime is not needed after installation. Workspace Client and EP Project Agent remain independently installable local roles with separate client/session, agent/server and server-peer trust domains.

## Consequences

The installer can compose EP-only, Forge+EP, Workspace+EP, all-server and remote-peer deployments without inventing a shared control plane. Products must define their own storage migration/relocation, backup/recovery and pairing credentials. The first Forge→EP→Forge canary remains bounded to Forge/EP minimum service seams; Workspace UI and full installer/discovery productization are not predecessors.

1 change: 1 addition & 0 deletions docs/architecture/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@
| [ADR-0003](ADR-0003-deployment-and-trust-boundaries.md) | Deployment roles and explicit trust boundaries |
| [ADR-0004](ADR-0004-universal-installer-artifact-composition.md) | Universal installer and independently published artifact composition |
| [ADR-0005](ADR-0005-governed-engineering-learning-loop.md) | Governed engineering learning loop and knowledge authority boundaries |
| [ADR-0006](ADR-0006-server-deployment-and-discovery.md) | Installed server deployment, discovery, pairing, and lifecycle boundaries |
4 changes: 2 additions & 2 deletions docs/roadmap/MVP_1_0.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ The following need later ADRs or versioned product contracts; they do not block
| W2 `MVP-WORK-001` Shared Workspace interaction | Workspace Server and Workspace Client | **Current:** Workspace boundary is canonical, but product integration maturity is not established here. **Target:** a user can view server-authoritative project/run state and issue permitted control intent without the client becoming authority or directly executing work. | `MVP-PROJ-001`; `MVP-EXEC-001`; `MVP-FORGE-001`. | Multi-client state/permission tests; remote-action and local-Agent boundary tests. | Yes |
| W3 `MVP-OPS-001` Operations, evidence, and honest usage | EP primarily; Workspace projects; Forge Platform qualifies composition | **Current:** historical EP material documents bounded status, logs, Prompt History, receipts, and Codex usage semantics. **Target:** qualified views for health, version, queue/run state, Agent availability, logs/evidence, and provider/model/host usage; unavailable measures remain explicitly unavailable. | W2 integration; provider telemetry contract. | Redaction/privacy tests; evidence and availability tests; provider attribution fixtures; no fabricated cost/token assertions. | Yes |
| W3 `MVP-REC-001` Persistence, backup, and recovery | EP and product storage contracts; Forge Platform validates deployment lifecycle | **Current:** clean-store and transactional-recovery principles exist in historical EP evidence. **Target:** durable supported-store lifecycle, backup/restore and upgrade recovery for the installed composition, with one writable authority. | `MVP-EXEC-001`; product storage migration contract. | Restart/interruption, backup/restore, upgrade/rollback-or-fail-safe evidence; forensic legacy-store disposition. | Yes |
| W4 `MVP-INST-001` macOS installation lifecycle | Forge Platform, consuming product artifacts | **Current:** no universal installer implementation. **Target:** clean-machine macOS Complete, Server/headless, Developer Workstation, and Custom role flows; verification, credentials, explicit service registration, receipts, upgrade, repair, and uninstall. | W0–W3; `MVP-COMP-001`. | Clean-install and repeatability tests; upgrade/repair/uninstall tests; least-privilege/security review; receipts and diagnostics. | Yes |
| W4 `MVP-INST-001` macOS installation lifecycle | Forge Platform, consuming product artifacts | **Current:** no universal installer implementation. **Target:** clean-machine macOS EP-only, Forge+EP, Workspace+EP, all-server, remote-peer, Developer Workstation and Custom flows; verified artifacts, explicit launchd registration, discovery-driven topology bootstrap, authenticated product-API pairing/binding, receipts, upgrade, repair and uninstall. The installer never writes product databases and is not a runtime dependency after install. | W0–W3; `MVP-COMP-001`; ADR-0006/instance contract. | Clean-install and repeatability tests; binding no-silent-retarget negatives; upgrade/repair/uninstall tests; least-privilege/security review; receipts and diagnostics. | Yes |
| W4 `MVP-MIG-001` Historical EP transition qualification | EP owns product migration; Forge Platform owns composed-install disposition | **Current:** DJConnect records clean-slate extraction and forensic-retention decisions; it is not a production migration authorization. **Target:** a qualified legacy-to-standalone clean-install/re-registration runbook: fresh official Schema 41 CENTRAL database, no legacy database migration, retained legacy store as immutable forensic evidence only, and no legacy runtime authority. | `MVP-REC-001`; `MVP-INST-001`; EP-owned contract. | Executed non-production clean-host qualification; legacy-store fingerprint/integrity evidence; fresh-store and new-registration evidence; recovery evidence; migration decision record; documented operator path. | Yes |
| B8C `MULTI_PROJECT_CONSOLE_QUALIFIED` First real multi-project qualification | Engineering Platform Server/Operations Console and Project Agent; Forge Platform records composed evidence | **Target:** immediately after B8, attach `djconnect` and the `engineering-platform` development checkout as two first-class projects to the same CENTRAL/Agent. The latter uses its portable `.engineering-platform/repository.json`; it remains a development/source project while the installed EP artifact remains runtime authority. | B8 pairing and first DJConnect attachment; installed EP artifacts. | Project-selector/pulldown, `project_id` scoping, project-isolated queue/runs/reports/Prompt History/evidence/status, installation-wide versus project-scoped views, refresh/deep-link/selection behavior, and fail-closed cross-project-leakage negatives. | Yes for B9 / `STANDALONE_EP_VERIFIED`; it does not execute EP against its own source checkout. |
| B8D `CANONICAL_SUBMISSION_INGRESS_READY` Canonical submission service and multi-adapter ingress | Engineering Platform Server | **Target:** installed CENTRAL exposes one authenticated HTTP JSON consumer ingress for project adapters, Forge and Workspace; CLI submission and legacy file Inbox input are adapters to that same Server-owned submission/admission service. | `MULTI_PROJECT_CONSOLE_QUALIFIED`; installed EP Server; project-identity/scope contract. | One versioned submission model; required project/repository scope; producer provenance; idempotency; common validation/normalization/persistence/admission/preflight/queue/Prompt History/telemetry/retry-recovery behavior; route-equivalence and fail-closed negative tests. | Yes for B9; it introduces no second lifecycle domain and does not itself execute engineering work. |
Expand All @@ -66,7 +66,7 @@ The following need later ADRs or versioned product contracts; they do not block
| P2 `POST-VERIFY-PAR-002` Repository lock/lease admission | Engineering Platform | **Target:** EP uses one durable repository lock/lease per mutating repository as the resource-exclusion boundary. A lane targeting multiple repositories atomically obtains every required lease before mutation; distinct repositories remain independently lockable. | `POST-VERIFY-PAR-001`; versioned EP admission/lease contract. | Contention, lease renewal/expiry/recovery, multi-repository acquisition, no-double-writer, and dependency-versus-resource-exclusion qualification. | No — post-verification only. |
| P3 `POST-VERIFY-PAR-003` Agent execution slots and capacity | Engineering Platform Project Agent and Server | **Target:** an Agent serving `0..N` repositories per Host/OS-user context advertises bounded capacity such as `max_parallel_executions`/slots; EP matches host/capability, capacity, dependencies, and leases, then applies admission, backpressure, and fairness. | `POST-VERIFY-PAR-002`; versioned Agent capability contract. | Capacity advertisement/reconciliation, eligible-host matching, saturation/backpressure, fairness, Agent disconnect, and no-self-admission qualification. | No — post-verification only. |
| P4 `MULTI_REPOSITORY_PARALLEL_EXECUTION_VERIFIED` Installed-Codex multi-repository parallel qualification | Engineering Platform; Forge/Workspace as contract consumers; Forge Platform qualifies only the supported composition | **Target:** a qualified installed-Codex run performs independent mutating lanes in parallel across different repositories while Forge-provided dependencies still sequence dependent work. EP remains execution/admission/durable-state authority; Workspace remains presentation/permitted control only. | P1–P3; qualified installed artifacts and test repositories. | Retained end-to-end qualification bundle proving parallel independent execution, one mutating lane per repository, dependency sequencing, capacity/backpressure, failure recovery, isolated evidence/validation/finalization, and authority-boundary negative tests. | No — it follows `STANDALONE_EP_VERIFIED` and does not delay the B8/B9 critical path. |
| P5 `POST-DIST-001` Distributed/team execution | EP, Workspace, Forge; Forge Platform deployment qualification | Multi-host capacity, assignment, disconnect handling, and team fleet operations after a single supported topology and bounded multi-repository parallel execution qualify. | W4 stable composition; `MULTI_REPOSITORY_PARALLEL_EXECUTION_VERIFIED`; deferred discovery/scheduling contracts. | Multi-host recovery and concurrency qualification. | No |
| P5 `POST-DIST-001` Distributed/team execution | EP, Workspace, Forge; Forge Platform deployment qualification | Multi-host capacity, assignment, disconnect handling and team fleet operations after a single supported topology and bounded multi-repository parallel execution qualify. The common v1 descriptor supports LAN DNS-SD/mDNS and configured/unicast/tailnet candidate discovery; authenticated pairing/binding, not discovery, permits peers. | W4 stable composition; `MULTI_REPOSITORY_PARALLEL_EXECUTION_VERIFIED`; ADR-0006/instance contract. | Multi-host recovery, concurrency, discovery, pairing and no-silent-rebind qualification. | No |

## Critical path

Expand Down