diff --git a/docs/architecture/FORGE_PLATFORM_ARCHITECTURE.md b/docs/architecture/FORGE_PLATFORM_ARCHITECTURE.md index 8875c0e..f990c88 100644 --- a/docs/architecture/FORGE_PLATFORM_ARCHITECTURE.md +++ b/docs/architecture/FORGE_PLATFORM_ARCHITECTURE.md @@ -10,6 +10,14 @@ Detailed decisions are recorded in the [ADRs](adr/README.md). The [ownership mat The [Governed Engineering Knowledge Learning Loop](KNOWLEDGE_LEARNING_LOOP.md) records the independently owned Knowledge Base lifecycle and its additive cross-product integration boundary. +## Installed-server deployment and topology bootstrap + +The accepted target deployment is defined by [ADR-0006](adr/ADR-0006-server-deployment-and-discovery.md) and the shared [instance discovery and pairing contract](INSTANCE_DISCOVERY_AND_PAIRING_CONTRACT.md). Forge Server, EP Server and Workspace Server are headless, independently installed services with separate central runtime roots, product-owned SQL/files and versioned HTTP APIs above their application services. They may share a host but not a database or direct SQL path. + +Forge Platform composes verified artifacts, service lifecycle and topology bootstrap for EP-only, Forge+EP, Workspace+EP, all-server and remote-peer configurations. It calls product APIs to create product-owned bindings; it never writes product databases. The installer is not a required runtime dependency after installation. + +Discovery yields an unauthenticated candidate only. Pairing verifies stable identity and creates a pinned authenticated binding; a later discovery result cannot silently replace it. Workspace Client↔Workspace Server, EP Agent↔EP Server and server-peer credentials remain separate even on localhost. + ## First-class component model | Repository | First-class authority / published component | diff --git a/docs/architecture/INSTANCE_DISCOVERY_AND_PAIRING_CONTRACT.md b/docs/architecture/INSTANCE_DISCOVERY_AND_PAIRING_CONTRACT.md new file mode 100644 index 0000000..9b6d15b --- /dev/null +++ b/docs/architecture/INSTANCE_DISCOVERY_AND_PAIRING_CONTRACT.md @@ -0,0 +1,52 @@ +# Instance discovery and pairing contract v1 + +**Status:** Canonical target contract. Product implementation and qualification are separately governed. + +This contract is shared by Forge Server, Engineering Platform (EP) Server and Workspace Server. It gives the installer and product-owned clients one safe way to find a candidate, prove which instance it is, and create an explicit binding. It does not create a shared runtime, database, authorization domain, or service registry authority. + +## Descriptor and discovery + +Every installed server owns a random, stable, opaque `instance_id`. The ID is created with its installation identity, survives ordinary restart/update and is included in product-owned backup/restore semantics. It is never derived from a hostname, IP address, checkout, project or mDNS name. A replacement or a deliberately reissued identity is a different instance. + +`forge-platform.instance-descriptor/v1` is the public candidate document: + +```json +{ + "schema": "forge-platform.instance-descriptor/v1", + "product": "forge-server | ep-server | workspace-server", + "instance_id": "opaque-stable-id", + "api": [{"version": "v1", "endpoint": "https://host.example/api/v1"}], + "capabilities": ["health", "pairing", "..."], + "identity_fingerprint": "public-key-or-certificate-fingerprint", + "expires_at": "RFC3339 timestamp" +} +``` + +Implementations may add non-sensitive display metadata, but descriptors and discovery records contain **no** bearer credential, pairing secret, project or repository data, account data, queue/run state, path, log, or private topology. Unknown required fields and unsupported schema/API versions fail closed. + +LAN discovery may advertise a short-lived descriptor through DNS-SD/mDNS (for example `_forge-platform._tcp`, with product, descriptor version, instance ID, API versions and a descriptor endpoint/fingerprint in TXT data). It is only a candidate locator. A configured HTTPS/unicast endpoint, private-overlay or tailnet endpoint is an equal bootstrap route and is required where multicast is unavailable. Discovery never implies reachability, authenticity, authorization or permission to bind. + +## Authenticated pairing and durable binding + +An operator or an already authorized product principal starts a pairing flow. The initiating side verifies the endpoint's authenticated identity fingerprint, product, stable ID, supported API/capabilities and requested scope. The target performs its own authentication/authorization and creates a short-lived, single-use pairing ceremony. On completion each owning product persists its own binding through its application service/API: peer product, `instance_id`, verified fingerprint, endpoint set, negotiated versions/capabilities, scope, creation/rotation evidence and revocation state. + +Discovery is deliberately not a pairing API. Neither the Forge Platform installer nor a peer writes directly to another product database. An existing binding is pinned to its recorded instance ID and fingerprint: rediscovery of a different candidate, changed fingerprint or changed endpoint identity fails closed and requires an explicit inspect/re-pair/rebind operation. Endpoint rotation for the same identity follows the product's authenticated binding update protocol; it is never an automatic switch to a newly found instance. + +Trust domains remain separate: + +| Relationship | Trust owner | It must not become | +| --- | --- | --- | +| Workspace Client ↔ Workspace Server | Workspace user/session | server-peer or EP Agent trust | +| EP Project Agent ↔ EP Server | EP host/agent | Workspace Client or Forge peer trust | +| Forge/EP/Workspace Server peers | respective product peer bindings | a user session or Agent credential | + +Co-location and loopback do not remove authentication, pairing or authorization. + +## Server-operational invariants + +Each server is a headless, independently restartable installed service with a product-owned runtime storage root outside source/Git checkouts. Its versioned HTTP API is a transport adapter over interface-neutral application services; direct cross-product SQL and shared databases are prohibited. On macOS the product publishes a launchd service contract; Forge Platform orchestrates its lifecycle but does not become its runtime authority. Product storage contains its own SQL database and product-owned files, artifacts, logs, backups and cache, with permissions, retention, migration and recovery defined by the product owner. + +## Delivery sequencing + +The first Forge → EP → Forge autonomy canary needs only the minimum seams: installed Forge/EP service storage and stable identity, their versioned authenticated HTTP contract, an explicitly configured/pinned EP binding, and restart-safe recovery. It does **not** require Workspace UI, LAN discovery or the universal installer to be complete. Network discovery, full pairing UX, all topology combinations, Workspace Client distribution and installer productization are post-canary capabilities; they must preserve these seams. + diff --git a/docs/architecture/adr/ADR-0006-server-deployment-and-discovery.md b/docs/architecture/adr/ADR-0006-server-deployment-and-discovery.md new file mode 100644 index 0000000..c0732de --- /dev/null +++ b/docs/architecture/adr/ADR-0006-server-deployment-and-discovery.md @@ -0,0 +1,16 @@ +# ADR-0006 — Installed servers, explicit discovery, and pairing + +**Status:** Accepted + +## Decision + +Forge Server, EP Server and Workspace Server are independently installable, headless services. Each has a product-owned central runtime-storage root outside Git/source checkouts, a SQL database plus owned files/artifacts/logs/backups/cache, a stable installation identity, a versioned HTTP API over application services and a macOS launchd service contract. They may be co-located or distributed, but never share a database or use direct cross-product SQL. + +Servers use the shared [instance discovery and pairing contract](../INSTANCE_DISCOVERY_AND_PAIRING_CONTRACT.md): DNS-SD/mDNS is a candidate locator; configured/unicast/tailnet endpoints are equivalent bootstrap routes; explicit authenticated pairing creates product-owned durable bindings. Discovery is not authorization and an existing binding cannot silently move to a new discovered instance. + +Forge Platform owns component choice, verified artifact acquisition, installation/update/repair/uninstall, lifecycle choreography and discovery-driven topology bootstrap. It invokes each product's public API to pair/bind and never writes a product database. Its runtime is not needed after installation. Workspace Client and EP Project Agent remain independently installable local roles with separate client/session, agent/server and server-peer trust domains. + +## Consequences + +The installer can compose EP-only, Forge+EP, Workspace+EP, all-server and remote-peer deployments without inventing a shared control plane. Products must define their own storage migration/relocation, backup/recovery and pairing credentials. The first Forge→EP→Forge canary remains bounded to Forge/EP minimum service seams; Workspace UI and full installer/discovery productization are not predecessors. + diff --git a/docs/architecture/adr/README.md b/docs/architecture/adr/README.md index 619d31f..cfbb714 100644 --- a/docs/architecture/adr/README.md +++ b/docs/architecture/adr/README.md @@ -7,3 +7,4 @@ | [ADR-0003](ADR-0003-deployment-and-trust-boundaries.md) | Deployment roles and explicit trust boundaries | | [ADR-0004](ADR-0004-universal-installer-artifact-composition.md) | Universal installer and independently published artifact composition | | [ADR-0005](ADR-0005-governed-engineering-learning-loop.md) | Governed engineering learning loop and knowledge authority boundaries | +| [ADR-0006](ADR-0006-server-deployment-and-discovery.md) | Installed server deployment, discovery, pairing, and lifecycle boundaries | diff --git a/docs/roadmap/MVP_1_0.md b/docs/roadmap/MVP_1_0.md index 282d241..918f5e7 100644 --- a/docs/roadmap/MVP_1_0.md +++ b/docs/roadmap/MVP_1_0.md @@ -53,7 +53,7 @@ The following need later ADRs or versioned product contracts; they do not block | W2 `MVP-WORK-001` Shared Workspace interaction | Workspace Server and Workspace Client | **Current:** Workspace boundary is canonical, but product integration maturity is not established here. **Target:** a user can view server-authoritative project/run state and issue permitted control intent without the client becoming authority or directly executing work. | `MVP-PROJ-001`; `MVP-EXEC-001`; `MVP-FORGE-001`. | Multi-client state/permission tests; remote-action and local-Agent boundary tests. | Yes | | W3 `MVP-OPS-001` Operations, evidence, and honest usage | EP primarily; Workspace projects; Forge Platform qualifies composition | **Current:** historical EP material documents bounded status, logs, Prompt History, receipts, and Codex usage semantics. **Target:** qualified views for health, version, queue/run state, Agent availability, logs/evidence, and provider/model/host usage; unavailable measures remain explicitly unavailable. | W2 integration; provider telemetry contract. | Redaction/privacy tests; evidence and availability tests; provider attribution fixtures; no fabricated cost/token assertions. | Yes | | W3 `MVP-REC-001` Persistence, backup, and recovery | EP and product storage contracts; Forge Platform validates deployment lifecycle | **Current:** clean-store and transactional-recovery principles exist in historical EP evidence. **Target:** durable supported-store lifecycle, backup/restore and upgrade recovery for the installed composition, with one writable authority. | `MVP-EXEC-001`; product storage migration contract. | Restart/interruption, backup/restore, upgrade/rollback-or-fail-safe evidence; forensic legacy-store disposition. | Yes | -| W4 `MVP-INST-001` macOS installation lifecycle | Forge Platform, consuming product artifacts | **Current:** no universal installer implementation. **Target:** clean-machine macOS Complete, Server/headless, Developer Workstation, and Custom role flows; verification, credentials, explicit service registration, receipts, upgrade, repair, and uninstall. | W0–W3; `MVP-COMP-001`. | Clean-install and repeatability tests; upgrade/repair/uninstall tests; least-privilege/security review; receipts and diagnostics. | Yes | +| W4 `MVP-INST-001` macOS installation lifecycle | Forge Platform, consuming product artifacts | **Current:** no universal installer implementation. **Target:** clean-machine macOS EP-only, Forge+EP, Workspace+EP, all-server, remote-peer, Developer Workstation and Custom flows; verified artifacts, explicit launchd registration, discovery-driven topology bootstrap, authenticated product-API pairing/binding, receipts, upgrade, repair and uninstall. The installer never writes product databases and is not a runtime dependency after install. | W0–W3; `MVP-COMP-001`; ADR-0006/instance contract. | Clean-install and repeatability tests; binding no-silent-retarget negatives; upgrade/repair/uninstall tests; least-privilege/security review; receipts and diagnostics. | Yes | | W4 `MVP-MIG-001` Historical EP transition qualification | EP owns product migration; Forge Platform owns composed-install disposition | **Current:** DJConnect records clean-slate extraction and forensic-retention decisions; it is not a production migration authorization. **Target:** a qualified legacy-to-standalone clean-install/re-registration runbook: fresh official Schema 41 CENTRAL database, no legacy database migration, retained legacy store as immutable forensic evidence only, and no legacy runtime authority. | `MVP-REC-001`; `MVP-INST-001`; EP-owned contract. | Executed non-production clean-host qualification; legacy-store fingerprint/integrity evidence; fresh-store and new-registration evidence; recovery evidence; migration decision record; documented operator path. | Yes | | B8C `MULTI_PROJECT_CONSOLE_QUALIFIED` First real multi-project qualification | Engineering Platform Server/Operations Console and Project Agent; Forge Platform records composed evidence | **Target:** immediately after B8, attach `djconnect` and the `engineering-platform` development checkout as two first-class projects to the same CENTRAL/Agent. The latter uses its portable `.engineering-platform/repository.json`; it remains a development/source project while the installed EP artifact remains runtime authority. | B8 pairing and first DJConnect attachment; installed EP artifacts. | Project-selector/pulldown, `project_id` scoping, project-isolated queue/runs/reports/Prompt History/evidence/status, installation-wide versus project-scoped views, refresh/deep-link/selection behavior, and fail-closed cross-project-leakage negatives. | Yes for B9 / `STANDALONE_EP_VERIFIED`; it does not execute EP against its own source checkout. | | B8D `CANONICAL_SUBMISSION_INGRESS_READY` Canonical submission service and multi-adapter ingress | Engineering Platform Server | **Target:** installed CENTRAL exposes one authenticated HTTP JSON consumer ingress for project adapters, Forge and Workspace; CLI submission and legacy file Inbox input are adapters to that same Server-owned submission/admission service. | `MULTI_PROJECT_CONSOLE_QUALIFIED`; installed EP Server; project-identity/scope contract. | One versioned submission model; required project/repository scope; producer provenance; idempotency; common validation/normalization/persistence/admission/preflight/queue/Prompt History/telemetry/retry-recovery behavior; route-equivalence and fail-closed negative tests. | Yes for B9; it introduces no second lifecycle domain and does not itself execute engineering work. | @@ -66,7 +66,7 @@ The following need later ADRs or versioned product contracts; they do not block | P2 `POST-VERIFY-PAR-002` Repository lock/lease admission | Engineering Platform | **Target:** EP uses one durable repository lock/lease per mutating repository as the resource-exclusion boundary. A lane targeting multiple repositories atomically obtains every required lease before mutation; distinct repositories remain independently lockable. | `POST-VERIFY-PAR-001`; versioned EP admission/lease contract. | Contention, lease renewal/expiry/recovery, multi-repository acquisition, no-double-writer, and dependency-versus-resource-exclusion qualification. | No — post-verification only. | | P3 `POST-VERIFY-PAR-003` Agent execution slots and capacity | Engineering Platform Project Agent and Server | **Target:** an Agent serving `0..N` repositories per Host/OS-user context advertises bounded capacity such as `max_parallel_executions`/slots; EP matches host/capability, capacity, dependencies, and leases, then applies admission, backpressure, and fairness. | `POST-VERIFY-PAR-002`; versioned Agent capability contract. | Capacity advertisement/reconciliation, eligible-host matching, saturation/backpressure, fairness, Agent disconnect, and no-self-admission qualification. | No — post-verification only. | | P4 `MULTI_REPOSITORY_PARALLEL_EXECUTION_VERIFIED` Installed-Codex multi-repository parallel qualification | Engineering Platform; Forge/Workspace as contract consumers; Forge Platform qualifies only the supported composition | **Target:** a qualified installed-Codex run performs independent mutating lanes in parallel across different repositories while Forge-provided dependencies still sequence dependent work. EP remains execution/admission/durable-state authority; Workspace remains presentation/permitted control only. | P1–P3; qualified installed artifacts and test repositories. | Retained end-to-end qualification bundle proving parallel independent execution, one mutating lane per repository, dependency sequencing, capacity/backpressure, failure recovery, isolated evidence/validation/finalization, and authority-boundary negative tests. | No — it follows `STANDALONE_EP_VERIFIED` and does not delay the B8/B9 critical path. | -| P5 `POST-DIST-001` Distributed/team execution | EP, Workspace, Forge; Forge Platform deployment qualification | Multi-host capacity, assignment, disconnect handling, and team fleet operations after a single supported topology and bounded multi-repository parallel execution qualify. | W4 stable composition; `MULTI_REPOSITORY_PARALLEL_EXECUTION_VERIFIED`; deferred discovery/scheduling contracts. | Multi-host recovery and concurrency qualification. | No | +| P5 `POST-DIST-001` Distributed/team execution | EP, Workspace, Forge; Forge Platform deployment qualification | Multi-host capacity, assignment, disconnect handling and team fleet operations after a single supported topology and bounded multi-repository parallel execution qualify. The common v1 descriptor supports LAN DNS-SD/mDNS and configured/unicast/tailnet candidate discovery; authenticated pairing/binding, not discovery, permits peers. | W4 stable composition; `MULTI_REPOSITORY_PARALLEL_EXECUTION_VERIFIED`; ADR-0006/instance contract. | Multi-host recovery, concurrency, discovery, pairing and no-silent-rebind qualification. | No | ## Critical path