docs(#6407): adopt deterministic entity context staging

docs/ADRs/0017-credential-isolation-for-sandboxed-agents.md

@@ -18,6 +18,9 @@ Date: 2026-04-01
 
 Accepted (credential delivery tiers extended by [ADR 0025](0025-provider-credential-delivery-for-sandboxed-agents.md))
 
+The default prefetch model is standardized for handled entity content by
+[ADR 0107](0107-deterministic-filtered-entity-context-staging.md).
+
 ## Context
 
 When sandboxed agents need to perform operations requiring credentials (e.g. reading or writing GitHub issues), the credential must be kept away from the agent process. A compromised agent with access to a credential can exfiltrate it — once the credential leaves the sandbox, the attacker can use it without any sandbox constraints.

docs/ADRs/0107-deterministic-filtered-entity-context-staging.md

@@ -0,0 +1,95 @@
+---
+title: "107. Deterministic filtered entity-context staging"
+status: Accepted
+relates_to:
+  - agent-architecture
+  - security-threat-model
+topics:
+  - entity-context
+  - security
+  - harness
+  - token-cost
+---
+
+# 107. Deterministic filtered entity-context staging
+
+Date: 2026-09-07
+
+## Status
+
+Accepted
+
+## Context
+
+[Issue #6407](https://github.com/fullsend-ai/fullsend/issues/6407) identifies
+that agents and harness scripts repeatedly fetch the issue or change proposal
+they are handling, including comments, reviews, diffs, checks, and logs. Those
+tool calls spend tokens, make runs depend on runtime network access, and give
+each consumer a different view when the entity changes during a run.
+
+Forge content is also untrusted input. Fetching it directly from inside the
+sandbox bypasses the deterministic point where Fullsend can bound, normalize,
+redact, and label content before it reaches an agent. This decision generalizes
+the preferred prefetch model from
+[ADR 0017](0017-credential-isolation-for-sandboxed-agents.md) to every issue and
+change-proposal run.
+
+## Decision
+
+Before the harness pre-script, `fullsend run` uses `forge.Client` to assemble
+one immutable snapshot of the handled entity. The runner applies a mandatory,
+deterministic content pipeline: size limits, Unicode safety normalization,
+secret and sensitive-data redaction, and injection scanning. It fails closed
+when required data cannot be fetched or safely represented. Filtered content
+is the only copy exposed to scripts and the agent; the manifest records every
+truncation, replacement, finding, and fetch error without retaining rejected
+content.
+
+The snapshot is written outside the repository clone. Host-side pre- and
+post-scripts receive `FULLSEND_CONTEXT_DIR` pointing to
+an access-restricted temporary directory outside the retained run-output
+tree; inside the sandbox the same variable points to
+`/sandbox/workspace/context`. Fullsend uploads that directory after sandbox
+creation and before repository/runtime execution. Consumers therefore use the
+same variable and relative paths on both sides of the sandbox boundary, and
+the context cannot be staged or committed accidentally with repository files.
+
+The exact tree, schemas, canonical serialization, stable record-key derivation,
+filter statuses, and compatibility rules are the versioned
+[entity-context v1 specification](../normative/entity-context/v1/README.md).
+Content records and mutable observation state are separate: resolving or
+reordering a thread changes its state/index files, never an unchanged comment
+or review body. No runner-clock timestamp enters the staged tree. Given the
+same forge state and filter version, implementations produce the same paths
+and bytes; breaking that guarantee requires a new major specification.
+
+Each comment and review is a self-contained attributed record whose filename
+sorts chronologically. The initial body uses the same record format and sorts
+first. Whole-conversation assembly is a glob concatenation; per-thread order
+files contain paths to the same records rather than copied content. New replies
+append without rewriting an unchanged prefix, preserving prompt-cache
+eligibility when runtimes concatenate records before mutable state.
+
+The pre-script may inspect the host snapshot and skip the run. It cannot mutate
+the agent's view: Fullsend verifies the manifest digests before upload and
+restores or rejects changed files. The sandbox copy is read-only to the agent.
+Agent prompts should concatenate conversation records first and place mutable
+state after that stable prefix; `summary.md` remains the navigation aid for
+selective reads. Runtime forge reads are an explicit fallback for data outside
+the snapshot, not the default way to obtain it.
+
+The host snapshot uses a mode-`0700` directory and mode-`0600` files. Fullsend
+removes the sandbox copy after its last sandbox consumer and the host copy after
+the post-script, on success, failure, skip, or handled cancellation; startup
+also scavenges orphaned context directories after abnormal termination. Context
+is excluded from retained run artifacts by construction. Diagnostics may retain
+only bounded counts, digests, and filtering findings, never bodies, diffs, or
+logs.
+
+## Consequences
+
+- Agents start with a consistent, filtered view of entity content and need fewer forge tool calls and prompt tokens.
+- Pre-scripts, agents, validation, and post-scripts share one versioned relative-path contract without putting generated input in Git.
+- Snapshot assembly adds startup latency and ephemeral storage, bounded by per-entry and total-size limits.
+- A snapshot can become stale during a run, so outputs that mutate forge state must still validate relevant revisions in deterministic post-processing.
+- Forge adapters must expose the snapshot inputs through `forge.Client`; platform-specific gaps are explicit manifest errors rather than silent omissions.

docs/architecture.md

@@ -170,6 +170,11 @@ repo baseline and overrides)
   (contract: [`docs/normative/prescript-output/v1`](normative/prescript-output/v1/README.md)),
   replacing the inline workflow pre-checks and their scaffold script copies
   ([ADR 0072](ADRs/0072-pre-script-output-protocol.md)).
+- Deterministic entity context: before the pre-script, the runner fetches one
+  bounded entity snapshot through `forge.Client`, filters untrusted content,
+  and stages a versioned per-record file tree outside the repository. Scripts
+  and the sandbox use `FULLSEND_CONTEXT_DIR` with the same relative paths
+  ([ADR 0107](ADRs/0107-deterministic-filtered-entity-context-staging.md)).
 - CEL-guarded overlays: an `overlays:` list of CEL-guarded config
   overlays generalizes the `forge:` block, letting harness authors
   condition scripts, skills, env vars, and other fields on any event

docs/normative/entity-context/v1/README.md

@@ -0,0 +1,171 @@
+# Entity context v1
+
+This specification defines the deterministic, filtered entity snapshot adopted
+by [ADR 0107](../../../ADRs/0107-deterministic-filtered-entity-context-staging.md).
+It is the contract between forge adapters, `fullsend run`, harness scripts, and
+agent runtimes.
+
+## Tree
+
+Entries that do not apply to an entity are omitted. JSON documents use their
+linked schemas.
+
+```text
+context/
+├── index.json
+├── summary.md
+├── entity/metadata.json
+├── records/<order-key>-<record-key>.md
+├── threads/<thread-key>.order
+├── changes/
+│   ├── diff.patch
+│   └── commits.json
+├── checks/<record-key>/
+│   ├── metadata.json
+│   └── log.txt
+└── state/
+    └── threads.json
+```
+
+`index.json` conforms to
+[`index.schema.json`](index.schema.json) and enumerates every other staged file.
+`entity/metadata.json`, `changes/commits.json`, check metadata, and
+`state/threads.json` conform respectively to
+[`entity.schema.json`](entity.schema.json),
+[`commits.schema.json`](commits.schema.json),
+[`check.schema.json`](check.schema.json), and
+[`thread-state.schema.json`](thread-state.schema.json). `summary.md` is a
+bounded navigation view generated only from the manifest and state documents;
+it must not duplicate record bodies or logs. Files under `threads/` contain
+only ordered relative paths to records.
+
+## Stable records and mutable state
+
+A record key is the lowercase hexadecimal SHA-256 of these UTF-8 strings joined
+by a single NUL byte, with no trailing NUL:
+
+```text
+forge identifier, canonical repository identifier, record kind, forge record ID
+```
+
+The forge record ID is the platform's immutable opaque ID, not a mutable URL,
+ordinal, database row position, or display number. Record kinds are `comment`,
+`review`, `check`, `thread`, and `entity`. This derivation makes paths safe and
+stable without requiring consumers to parse forge-specific IDs.
+
+Comment and review Markdown files are self-contained records with this exact
+UTF-8 layout; header values are canonical JSON strings (or `null`) on one line:
+
+```text
+Fullsend-Record: "comment"
+Source-ID: "opaque-forge-id"
+Source-URL: "https://forge.example/..."
+Author-ID: "opaque-actor-id"
+Author: "forge-login"
+Created-At: "2026-09-08T10:15:30Z"
+
+Filtered Markdown body.
+```
+
+`Fullsend-Record` is `"comment"` or `"review"`. The header order and blank
+line are fixed. `Author-ID` and `Author` may be `null` when the forge withholds
+or has deleted the actor. All header strings are filtered before JSON-string
+serialization. The file contains no update time, ordering, thread membership,
+resolution, outdated, or minimized state; those properties belong in
+`index.json` or `state/threads.json`. Consequently, resolving a thread or
+inserting an earlier record must not rename or rewrite an unchanged record.
+Changing its body or attribution fields changes that record's bytes and digest.
+
+The initial issue or change-proposal body uses the same layout with
+`Fullsend-Record: "entity"`, the entity's stable ID and URL, and its author
+attribution and creation time. This makes it the first self-contained turn.
+
+## Filename order and prompt caching
+
+An ordinary record filename is
+`records/<order-key>-<record-key>.md`. `<order-key>` is its source `created_at`
+normalized to UTC as `YYYYMMDDTHHMMSSnnnnnnnnnZ`, with exactly nine fractional
+second digits and no punctuation other than `T` and `Z`. The entity-body record
+uses the reserved key `00000000T000000000000000Z`, so it always sorts first.
+Creation time is immutable forge data; edits do not rename a record.
+
+Because all path components are restricted to these ASCII forms,
+`LC_ALL=C cat records/*.md` concatenates the entire conversation in canonical
+order without an intermediate file. Each `threads/<thread-key>.order` contains
+the relative record path for each thread member followed by LF, in forge thread
+order. From the context root, `xargs cat < threads/<thread-key>.order`
+concatenates one thread; paths contain no whitespace or shell metacharacters.
+
+A normal later reply adds one lexically later file and appends one path to its
+thread order file, leaving all earlier record bytes and the whole-conversation
+prefix unchanged for prompt-cache reuse. A backfilled earlier record, edit,
+deletion, or attribution change necessarily invalidates the assembled context
+from the first affected record onward. Consumers place mutable state after the
+record concatenation and never infer resolution from record content.
+
+Check status is observation state in `checks/<record-key>/metadata.json`; its
+log file contains only filtered log bytes. A growing or replaced forge log is
+changed content and may change `log.txt`. A new check attempt has a new forge
+record ID and therefore a new record key.
+
+## Canonical bytes
+
+JSON is UTF-8 serialized with the JSON Canonicalization Scheme (RFC 8785), with
+no byte-order mark or trailing newline. Arrays use the order defined below;
+objects use RFC 8785 member ordering.
+
+Text bodies, patches, and logs are UTF-8 after the v1 filter pipeline, use LF
+line endings, have no byte-order mark, and end in exactly one LF. The pipeline
+applies size bounds, Unicode safety normalization, secret/sensitive-data
+redaction, and injection scanning in that order. `filter.status` is:
+
+All attacker-controlled strings in JSON metadata pass through the same pipeline
+before canonical serialization.
+
+- `unchanged`: emitted bytes equal normalized source bytes;
+- `modified`: one or more replacements or redactions were applied;
+- `truncated`: a size bound removed source bytes, whether or not other filters also changed them;
+- `rejected`: no content file is emitted because the source could not be represented safely.
+
+Every emitted file has a manifest `sha256` over its emitted bytes. A rejected
+source has a record but no content path or file entry.
+Findings contain codes and counts, not rejected source text. Filters and bounds
+are identified by `filter_version`; changing emitted bytes for the same input
+requires a new filter version. Removing or reinterpreting a status requires v2.
+
+## Ordering and determinism
+
+Manifest record arrays and record filenames are sorted by source `created_at`,
+then by the record key's ASCII byte order. The reserved entity-body order key
+sorts before them. Thread arrays use thread creation time and then thread ID;
+`comment_ids` and `.order` lines preserve forge thread order. Commit arrays
+preserve forge history order. Other arrays state their ordering in their owning
+schema before being added to v1.
+
+`generated_at` or another runner-clock value is forbidden anywhere under the
+context root. Acquisition timing belongs in run telemetry outside the staged
+tree. Source-provided timestamps, entity update time, PR head SHA, and check
+attempt IDs are permitted because they describe forge state. With identical
+forge responses, size bounds, and `filter_version`, the complete tree has
+identical paths and bytes.
+
+## Lifecycle and access
+
+The host tree is created outside both the repository and retained run-output
+tree with directory mode `0700` and file mode `0600`. The sandbox copy is
+read-only. Fullsend removes the sandbox copy after the runtime's last use and
+the host copy after the post-script on every controlled exit, including skip,
+failure, and cancellation. Fullsend also scavenges abandoned context trees on
+startup after an unclean termination.
+
+Artifact collectors must exclude entity-context trees. Retained diagnostics
+may contain bounded record counts, content digests, filter codes/counts, and
+cleanup errors, but never entity bodies, comment/review bodies, diffs, or logs.
+
+## Compatibility
+
+Consumers must reject an unsupported `schema_version` or `filter_version`; they
+must ignore unknown object properties within v1. Adding an optional record kind
+or property is compatible. Changing existing path derivation, canonical bytes,
+required fields, field meaning, or ordering requires
+`docs/normative/entity-context/v2/` and a superseding ADR.

docs/normative/entity-context/v1/check.schema.json

@@ -0,0 +1,18 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://fullsend.sh/schemas/entity-context/v1/check.schema.json",
+  "title": "Fullsend entity-context check metadata v1",
+  "type": "object",
+  "required": ["id", "name", "status", "attempt"],
+  "properties": {
+    "id": { "type": "string", "minLength": 1 },
+    "name": { "type": "string" },
+    "status": { "type": "string", "minLength": 1 },
+    "conclusion": { "type": ["string", "null"] },
+    "attempt": { "type": "integer", "minimum": 1 },
+    "source_url": { "type": "string", "format": "uri" },
+    "started_at": { "type": "string", "format": "date-time" },
+    "completed_at": { "type": ["string", "null"], "format": "date-time" }
+  },
+  "additionalProperties": false
+}

docs/normative/entity-context/v1/commits.schema.json

@@ -0,0 +1,25 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://fullsend.sh/schemas/entity-context/v1/commits.schema.json",
+  "title": "Fullsend entity-context commits v1",
+  "type": "object",
+  "required": ["commits"],
+  "properties": {
+    "commits": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": ["id", "sha", "subject", "committed_at"],
+        "properties": {
+          "id": { "type": "string", "minLength": 1 },
+          "sha": { "type": "string", "pattern": "^[0-9a-fA-F]+$" },
+          "subject": { "type": "string" },
+          "author": { "type": "string" },
+          "committed_at": { "type": "string", "format": "date-time" }
+        },
+        "additionalProperties": false
+      }
+    }
+  },
+  "additionalProperties": false
+}

docs/normative/entity-context/v1/entity.schema.json

@@ -0,0 +1,28 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://fullsend.sh/schemas/entity-context/v1/entity.schema.json",
+  "title": "Fullsend entity metadata v1",
+  "type": "object",
+  "required": ["kind", "id", "number", "title", "state", "source_url", "author_id", "author", "created_at", "updated_at", "labels"],
+  "properties": {
+    "kind": { "enum": ["issue", "change_proposal"] },
+    "id": { "type": "string", "minLength": 1 },
+    "number": { "type": ["integer", "string"] },
+    "title": { "type": "string" },
+    "state": { "type": "string", "minLength": 1 },
+    "source_url": { "type": "string", "format": "uri" },
+    "author_id": { "type": ["string", "null"] },
+    "author": { "type": ["string", "null"] },
+    "created_at": { "type": "string", "format": "date-time" },
+    "updated_at": { "type": "string", "format": "date-time" },
+    "labels": {
+      "type": "array",
+      "items": { "type": "string" },
+      "uniqueItems": true
+    },
+    "head_sha": { "type": "string", "pattern": "^[0-9a-fA-F]+$" },
+    "head_ref": { "type": "string" },
+    "base_ref": { "type": "string" }
+  },
+  "additionalProperties": false
+}