diff --git a/.kiro/specs/compaction-continuity-preservation/design-review.md b/.kiro/specs/compaction-continuity-preservation/design-review.md new file mode 100644 index 00000000..8d32952c --- /dev/null +++ b/.kiro/specs/compaction-continuity-preservation/design-review.md @@ -0,0 +1,482 @@ +# Brownfield Design Validation + +## Verdict + +**GO after correction loops.** The design fits current Go-LIP ownership boundaries and reuses existing routing, auxiliary execution, generation pins, B2BUA, secure-session, extension and billing authorities. + +The initial brownfield validation identified four architectural corrections before task generation: final-release detector ordering, private child A-leg semantics, useful late-result retention, and explicit prerequisite/fail-closed composition when the #312 detector surface is unavailable. A later CodeRabbit review identified additional specification-level correctness gaps around field-level billing evidence, preservation callback failure handling, capsule self-validation, decision conflict identity, completion-only pre-open billing/coalescing, parent-branch ownership, reinjection commit identity, and the disabled-by-default example. Those gaps are now incorporated into the requirements, design, and task plan. + +No provider-specific implementation, second transcript database, second billing path, or generic workflow engine is required. + +## Validation Inputs + +Reviewed against current `main` and steering constraints, including: + +- `.kiro/specs/compaction-event-detection/{requirements,research,design,tasks}.md`; +- `pkg/lipsdk/auxiliary/client.go`; +- `internal/core/auxreq/client.go`; +- `pkg/lipsdk/genpin/pin.go`; +- `pkg/lipapi/{call,items}.go`; +- `pkg/lipsdk/feature/bundle.go` and `internal/featurebundle/merge_surface.go`; +- `internal/core/extensions` / `internal/infra/runtimebundle/build_extension.go`; +- `internal/infra/runtimebundle/{process_services,shared_mutable}.go`; +- `pkg/lipsdk/session/{view,opener}.go`; +- `internal/core/runtime/executor_prepare_secure.go`; +- `internal/core/runtime/billing_leg.go`; +- `internal/core/state/{partition,mem}.go`; +- `internal/infra/billingcompose/identity.go`; +- `.kiro/steering/{structure,testing,routing-and-orchestration}.md`. + +## Finding V1 — Final detector observation must still see the actually released event + +### Initial concern + +The first design ordering said: + +```text +selected event + -> detector derives/commits completion + -> preservation mutates verified plaintext carrier + -> observer dispatch + -> client +``` + +The #312 invariant is stronger: `ResponseReleased` represents the canonical event actually released after final mutation/gating. Calling committed detector observation before preservation means it technically observes a pre-final form. + +### Correction + +Add a pure `PreviewResponse` (or equivalent shared matcher preview) analogous to request preview: + +```text +selected event + -> detector PreviewResponse (pure, no state/event emission) + -> preservation BeforeResponseRelease (bounded join/mutation) + -> detector ResponseReleased(final event) (commits completion) + -> metadata compaction observer dispatch + -> client release +``` + +`PreviewResponse` may expose the active transaction/rule candidate required to find the preservation job, but it cannot mark completion. This keeps the detector non-mutating and truthful to the final-release contract. + +### Status + +**PASS after correction.** + +## Finding V2 — Detached child needs its own private A-leg, not no A-leg and not the parent A-leg + +### Initial concern + +Normal runtime authority/billing/attempt mechanics assume a logical A-leg. Reusing the parent A-leg would risk route-override/session-state coupling. Running completely without an A-leg would require a second execution path and weaken current B2BUA/request-authority assumptions. + +### Correction + +Detached auxiliary execution creates/touches a **private child A-leg** for the child canonical call using existing B2BUA lifecycle/store semantics. Parent IDs stay explicit lineage only: + +```text +primary SessionID / A-leg A123 + | + +-- auxiliary lineage parent_a_leg=A123 + | + +-- child A-leg AUX456 + +-- B-leg 1 extractor backend/model + +-- B-leg 2 failover if needed +``` + +Rules: + +- child A-leg gets no primary secure-session ownership/resume semantics; +- parent route override is never read as child authority; +- child gets normal request authority/BillingCallID/B-leg accounting; +- child A-leg/attempts remain distinguishable as internal auxiliary workload; +- client-visible primary transcript/turn/activity remains unchanged. + +### Status + +**PASS after correction.** + +## Finding V3 — A short raw-result TTL can lose a valid background extraction before the next turn + +### Initial concern + +If the strict-compaction response barrier times out, the semantic job may finish later. If its raw result expires after only a short scheduler TTL and the next user turn arrives much later, the capsule never receives that valid result. + +### Correction + +Pending-job result retention must be useful for the branch, while remaining bounded: + +- raw output is still size/count bounded and never logged; +- while a BranchState references `PendingJobID`, the scheduler result may be retained up to the configured pending-continuity TTL, not an arbitrary tiny default; +- first eligible `Await` parses/validates/merges and immediately `Forget`s raw output; +- branch/job expiry clears both sides coherently; +- no result lives longer than the configured bounded continuity/source retention window; +- implementation may later add a typed completion-consumer optimization, but v1 does not need arbitrary callbacks in the scheduler. + +### Status + +**PASS after correction.** + +## Finding V4 — Feature composition must fail clearly when prerequisite preservation/detector services are absent + +### Initial concern + +#312 is not runtime code on the original validation baseline. A feature plugin silently receiving disabled compaction services would appear enabled while doing nothing, which is misleading and hard to operate. + +### Correction + +When `compaction-continuity` is enabled: + +- generation/process composition verifies the prerequisite detector preview/commit capability, branch coordinator and BackgroundAux client exist; +- missing prerequisite service is a startup/generation compile error with a clear message; +- feature disabled keeps zero/no-op behavior and does not require semantic extractor configuration; +- implementation tasks are chronologically ordered so #312 runtime lands first. + +This is configuration fail-closed, not request traffic fail-closed. + +### Status + +**PASS after correction.** + +## Finding V5 — Process-owned branch coordination is justified despite feature-state guidance + +### Concern + +The extension platform generally prefers plugin state through `lipsdk/state.Store` rather than per-feature core state. However current Store `Get`/`Put` has no atomic CAS, and a per-generation plugin mutex cannot protect overlapping old/new generation callbacks and background workers. + +### Decision + +Keep the process-owned **narrow BranchCoordinator** as a synchronization facade and use process `ExtensionState` as its serialized backing where practical. The coordinator owns only: + +- parent branch-key lock/serialization; +- revision/high-watermark/job/preview-intent/injection metadata; +- bounded entry TTL/eviction; +- opaque capsule/source blobs. + +It does **not** own plan semantics, extractor prompts, provider clients, billing, generic state transactions, or arbitrary tasks. + +This is smaller and safer than adding a generic transactional plugin-store API solely for this feature. + +### Status + +**PASS as designed.** + +## Finding V6 — Background auxiliary scheduler is a reusable primitive but must remain narrow + +### Concern + +Adding an async service can become a generic workflow engine. + +### Decision + +The scheduler accepts only canonical auxiliary model collection requests and returns bounded collection jobs. It cannot schedule arbitrary functions, timers, cron, durable tasks, webhooks, or user-defined callbacks. ProcessServices owns it; generation snapshots receive non-owning adapters that capture the current runner/pin at submission. + +Completion-only pre-open identity is held as a non-billable BranchCoordinator preview intent, not as a scheduler job. Only a committed post-Open transaction may create a new billable BackgroundAux submission. + +### Status + +**PASS with architecture tests.** + +## Finding V7 — Detached mode must not become a client bypass around secure session + +### Concern + +An exported/internal session mode that suppresses secure-session BeginTurn could be abused if a frontend could set it. + +### Decision + +Detached mode is carried only on the trusted auxiliary SDK/internal request object after frontend decode; no `lipapi.Call` JSON/wire field or header maps to it. Architecture/tests prove frontend inputs cannot request detached mode. Feature plugins are already trusted in-process extensions; the standard continuity feature is the initial consumer. + +### Status + +**PASS with security tests.** + +## Finding V8 — Billing requirements must include the current independent-leg evidence contract + +### Validation + +Billing account identity is principal-scope-derived, so the detached child can preserve the originating account while receiving an independent BillingCallID and private child A-leg/B-legs. Normal Executor submission can therefore reuse existing exposure, terminal usage, settlement, and provider COGS authorities. + +The runtime contract is stricter than merely assigning a distinct BillingCallID: + +- independent B-leg persistence requires an authoritative positive `AttemptSeq`; records with `AttemptSeq <= 0` are rejected rather than assigned synthetic order; +- final billing evidence carries the existing token/usage quantities and presence bits, cost evidence/presence, `Source`, `Authority`, and `DedupeKey`; +- retry/failover legs must satisfy the same field-level contract independently. + +The continuity spec therefore requires these invariants for every auxiliary/failover B-leg and requires RED coverage for rejected non-positive AttemptSeq and invalid accounting evidence. Workload/role classification remains content-free and must not implicitly change pricing. + +### Status + +**PASS after field-level billing correction; RED tests required.** + +## Finding V9 — Primary protocol usage separation is achievable + +The child is an independent internal call and is never encoded into the primary frontend response. Existing protocol usage remains scoped to the primary canonical stream. Account-level durable usage/billing includes child records separately. + +### Status + +**PASS; regression tests required.** + +## Finding V10 — Opaque compaction fallback is mandatory, not optional polish + +`CompactionItem` exposes encrypted/opaque state rather than universal plaintext summary. The result-augmentation path therefore cannot be the only preservation mechanism. + +Reinjection on the first eligible post-compaction request is the universal safe fallback and must be implemented/tested even if one or more plaintext carriers are supported. + +### Status + +**PASS after design explicitly treats reinjection as mandatory fallback.** + +## Finding V11 — Preserver callback errors must be explicitly fail-open and mutation-safe + +### Review concern + +`BeforeRequest`, `RequestOpened`, and `BeforeResponseRelease` return `error`. Without a composition rule, a preservation-only timeout/state/sanitizer error could escape into primary handling and make native compaction unusable. `BeforeRequest`/response-finalization can also mutate canonical objects before returning an error. + +### Correction + +- core dispatch isolates callback panic/error and records content-free preservation failure rather than propagating it as primary model/provider failure; +- `BeforeRequest` and `BeforeResponseRelease` run through a bounded transactional clone/undo helper; +- callback error, panic, or post-mutation canonical validation failure restores the exact pre-preservation `Call`/`Event`; +- `RequestOpened` failure cannot undo the already-opened primary request and only affects preservation-local state; +- preservation failure never changes routing, failover or no-retry-after-output authority. + +This remains a focused mutation rollback seam, not a general transaction framework. + +### Status + +**PASS after correction; callback error/panic/rollback tests required.** + +## Finding V12 — Serialized capsule must carry branch binding and canonical content digest + +### Review concern + +The requirement already called for branch identity and content digest, but the capsule JSON example kept them only outside the serialized envelope. Such bytes could not self-validate their scope/integrity after registry transfer, reinjection, recovery or reload. + +### Correction + +Capsule V1 now carries: + +- `branch_binding`: a stable content-free digest derived from the authoritative **parent** BranchKey; +- `content_digest`: SHA-256 over the canonical versioned envelope excluding only the digest field itself. + +The digest scope includes schema version, revision, source high-watermark, branch binding and semantic payload. Registry consume, merge, reinjection/projection, recovery and reload reuse validate binding + digest before use. Raw account/session identifiers need not be exported to the extractor model. + +### Status + +**PASS after correction; canonical encoding and mismatch tests required.** + +## Finding V13 — Decision conflict identity cannot rely on extractor-generated fact IDs + +### Review concern + +A semantically conflicting decision can arrive under a fresh semantic fact ID, so ID reuse alone cannot ensure that later explicit corrections supersede older active choices. + +### Correction + +Decision facts now separate: + +- stable fact `id` for provenance/history; +- normalized `conflict_key` for the semantic decision slot; +- optional validated `supersedes` references for explicit cross-slot corrections. + +At most one decision may remain active per conflict key. A newer decision for an occupied conflict key deterministically supersedes the older active decision under the existing authority/revision precedence. Unknown/cross-branch `supersedes` references are rejected. + +### Status + +**PASS after correction; contradictory-active-decision tests required.** + +## Finding V14 — Completion-only extraction cannot submit fresh billable work before primary Open + +### Review concern + +The earlier completion-only flow called `SubmitCollect` before the first post-compaction primary B-leg opened. If that Open failed, the detached child could still produce real provider usage, contradicting failed-Open zero-billing semantics. The earlier coalescing rule also required a transaction that did not yet exist. + +### Correction + +Before Open, completion-only flow may only: + +1. use detector pure preview to derive a stable boundary/fingerprint; +2. derive/coalesce a **non-billable** preview intent from parent branch + preview boundary + target source revision; +3. merge deterministic state; +4. await a matching job that was already submitted by an earlier successfully opened request; +5. inject already-ready/deterministic continuity if needed. + +Only after successful primary Open does detector state commit, the preview intent bind to the committed transaction, and a fresh billable BackgroundAux job become eligible for submission. A failed Open creates no new child BillingCallID/B-leg/provider work. + +This intentionally accepts that semantic information first discoverable at this boundary may improve later turns rather than justify premature provider billing. + +### Status + +**PASS after correction; failed-Open zero-child-work and preview-intent binding tests required.** + +## Finding V15 — Continuity state belongs to the parent branch, never the detached child A-leg + +### Review concern + +The child must have a private A-leg for execution, but if a worker derives BranchCoordinator identity from that A-leg, late results update an auxiliary branch and cannot protect the primary conversation. + +### Correction + +The authoritative parent BranchKey is captured before auxiliary submission/child A-leg creation and stored as a content-free branch binding with pending job state. The parent key is used for: + +- preview intent binding; +- `Await` result ownership checks; +- capsule merge/revision; +- pending injection/reinjection; +- late-result and reload coordination. + +The child A-leg remains only child execution/routing/billing authority and lineage. Tests must use different parent and child A-leg IDs. + +### Status + +**PASS after correction; distinct-parent/child concurrency/reload tests required.** + +## Finding V16 — Reinjection identity and commit point must be boundary-scoped + +### Review concern + +A revision-only `LastInjectedRevision` can wrongly suppress the same capsule revision after a second opaque compaction. Advancing a durable watermark at insertion time is also too early: canonical validation, primary Open, or final release can still fail, causing a later retry to skip required continuity. + +### Correction + +Reinjection now uses a compound identity: + +```text +(parent branch binding, compaction boundary/transaction, capsule revision) +``` + +- the same revision may be injected again for a later distinct compaction boundary; +- a call-local ephemeral marker prevents duplicate insertion during one request's internal retry/failover lifecycle; +- callback/validation failure restores the pre-injection call and leaves pending state; +- failed primary Open leaves pending state; +- branch-level `LastReleasedInjection` advances and matching pending state clears only after successful final client release; +- an aborted/no-release turn therefore retries continuity on a later eligible request. + +### Status + +**PASS after correction; same-revision/two-boundary and failure-then-retry tests required.** + +## Finding V17 — Configuration example must not contradict disabled-by-default behavior + +### Review concern + +The example had `enabled: true` while the security requirement says the new egress/billed feature is disabled by default. Copying the example could unintentionally enable remote extraction. + +### Correction + +The illustrative configuration is now `enabled: false` and explicitly described as disabled unless the operator opts in. The remote route remains illustrative only. + +### Status + +**PASS after correction.** + +## Validation Checklist + +| Check | Result | +|---|---| +| one compaction-recognition authority | PASS | +| detector observer remains content-free/non-mutating | PASS | +| actual final event observed by committed detector | PASS after V1 correction | +| strict extraction not billed before primary Open | PASS | +| completion-only fresh extraction not billed before primary Open | PASS after V14 correction | +| completion-only preview identity binds to committed transaction | PASS after V14 correction | +| real background execution with submit-time generation ownership | PASS | +| worker process-owned across generation reload | PASS | +| no unbounded goroutine/queue | PASS | +| off-primary-session semantics | PASS after V2 correction | +| private child A-leg never becomes continuity branch key | PASS after V15 correction | +| independent extractor selector | PASS | +| parent route override cannot hijack child | PASS | +| same user/account billed by default | PASS | +| separate auxiliary BillingCallID/B-legs | PASS | +| auxiliary/failover AttemptSeq > 0 | PASS after V8 correction | +| auxiliary usage/cost/Source/Authority/DedupeKey evidence pinned | PASS after V8 correction | +| primary protocol usage unchanged | PASS | +| preserver callback failure is fail-open | PASS after V11 correction | +| failed preservation mutation rolls back canonical object | PASS after V11 correction | +| capsule carries parent branch binding and canonical digest | PASS after V12 correction | +| decision conflicts use deterministic conflict identity | PASS after V13 correction | +| late job result remains useful | PASS after V3 correction | +| reinjection dedupe is branch/boundary/revision scoped | PASS after V16 correction | +| reinjection watermark commits only after final client release | PASS after V16 correction | +| encrypted/opaque content immutable | PASS | +| mandatory reinjection fallback | PASS | +| shipped example remains disabled by default | PASS after V17 correction | +| no second transcript DB | PASS | +| no provider-specific core branches | PASS | +| no generic workflow engine | PASS | +| prerequisite absence visible at startup | PASS after V4 correction | +| generation reload semantics explicit | PASS | +| restart durability honest | PASS | +| TDD/race/goleak coverage feasible offline | PASS | + +## Design-to-Requirement Trace + +| Design decision/component | Primary requirements | +|---|---| +| shared detector preview + prerequisite | 1.1–1.13, 4.10 | +| separate Preserver FeatureBundle surface + rollback | 1.2–1.4, 7.14, 7.16, 10.4–10.5, 11.12–11.13 | +| Continuity Capsule + branch digest + decision precedence | 2.1–2.17, 8.1, 8.4, 8.11–8.12 | +| deterministic carrier catalog | 3.1–3.7 | +| sanitizer/source window | 3.8–3.16, 9.3–9.10 | +| BackgroundAux scheduler + committed coalescing | 4.1–4.15, 11.2, 11.10–11.11 | +| detached child + parent branch ownership | 4.16, 5.5–5.15, 8.1–8.3, 11.4–11.5, 11.16 | +| explicit extractor route | 5.1–5.5, 10.1–10.2 | +| normal user billing + field-level B-leg evidence | 6.1–6.14, 10.7–10.8, 11.6–11.7 | +| strict and completion-only compaction flows | 7.1–7.16, 11.3 | +| boundary-scoped authority-aware reinjection | 7.7–7.16, 8.5, 11.17 | +| branch coordinator / preview intents / watermarks | 4.10, 4.16, 8.1–8.12, 11.8, 11.10–11.11, 11.16–11.17 | +| reload/restart behavior | 8.9–8.15, 10.8–10.10 | +| trusted policy/privacy | 9.1–9.13 | +| failure/observability | 10.1–10.13 | +| architecture/testing gates | 11.1–11.17 | + +## Simplification Review + +The chosen design adds two infrastructure capabilities only because existing seams cannot safely satisfy the requirements: + +1. **BackgroundAux scheduler** — required because current Aux.Collect is synchronous and post-lease spawn is unsafe. +2. **BranchCoordinator** — required because process ExtensionState lacks atomic revision update and feature-instance locks do not span generations. + +The CodeRabbit corrections do **not** justify further infrastructure. Preview intents, injection watermarks, parent-branch bindings and preservation rollback remain narrow state/contracts inside those existing seams; they do not require a generic transaction service, workflow scheduler or durable queue. + +Everything else is additive use/refactoring of existing authorities. In particular, do **not** add: + +- a second Executor/provider client; +- generic async function/task APIs; +- durable queue/database; +- event bus for preservation; +- general memory/RAG subsystem; +- new per-provider compaction adapters; +- new billing journal/rater; +- a generic transactional state platform; +- a second summary-model pass. + +## Implementation Risks to Pin With Tests + +1. `KindAsync` retained too late -> deterministic post-lease submission RED test. +2. child accidentally resumes parent secure session -> transcript/turn/activity RED tests. +3. child inherits parent route override -> explicit different-route RED test. +4. parent/child BillingCallID collision -> billing RED test. +5. auxiliary `AttemptSeq <= 0` or invalid final evidence silently loses accounting -> field-level billing RED tests. +6. primary protocol usage includes child tokens -> frontend usage RED test. +7. detector commits completion before preservation final event -> ResponsePreview/ResponseReleased ordering test. +8. preserver error/panic aborts native traffic or leaves partial mutation -> rollback/fail-open RED tests. +9. capsule branch binding/digest mismatch is consumed -> transfer/reload/reinjection integrity RED tests. +10. semantic new ID allows contradictory active decision -> conflict-key/supersedes RED tests. +11. completion-only pre-open path submits provider work -> failed-Open zero-child-billing RED test. +12. preview intent fails to bind/coalesce after Open -> retry/failover duplicate-submission RED test. +13. child A-leg is used as continuity key -> distinct parent/child late-result RED test. +14. revision-only injection suppresses later opaque boundary -> same-revision/two-boundary RED test. +15. injection watermark advances before validation/Open/final release -> failure-then-retry RED test. +16. opaque `EncryptedContent` altered -> exact bytes RED test. +17. job result expires before next turn -> pending-result retention/expiry RED test. +18. stale worker overwrites user correction -> revision race RED test. +19. generation reload drops job/coordinator -> retained-generation RED test. +20. queue saturation creates goroutine fallback -> goroutine-count/goleak RED test. +21. disable/reload drops already-submitted billing settlement -> terminal accounting RED test. + +## Final Gate + +After the original V1–V4 correction loop and the later V8/V11–V17 CodeRabbit correction pass, the specification is **GO** for TDD implementation. Requirements, design, validation, and tasks now agree on the critical lifecycle rules: parent-branch state ownership, no fresh billable completion-only child before successful primary Open, current independent-leg billing evidence, fail-open transactional preservation callbacks, self-validating capsule bytes, deterministic decision conflict replacement, and boundary-scoped reinjection committed only after final client release. + +The implementation should establish detector/preservation/background/detached/billing/branch contracts with RED tests before semantic extractor prompting and final compaction integration. If implementation requires pre-open billable child work, child-keyed continuity state, a second billing path, a generic workflow/transaction engine, or opaque provider mutation to pass, the design must be re-scoped rather than weakening these invariants. diff --git a/.kiro/specs/compaction-continuity-preservation/design.md b/.kiro/specs/compaction-continuity-preservation/design.md new file mode 100644 index 00000000..be8160be --- /dev/null +++ b/.kiro/specs/compaction-continuity-preservation/design.md @@ -0,0 +1,928 @@ +# Design Document + +## Overview + +Implement issue #344 as a bounded compaction-continuity feature that preserves decision state without replacing coding-agent compaction or creating a parallel inference/billing stack. + +The design has seven cooperating pieces: + +1. the `compaction-event-detection` detector/rule authority, implemented first and refactored only enough to provide pure request/response previews; +2. a separate additive `pkg/lipsdk/compaction.Preserver` content-bearing extension surface; +3. a process-owned bounded background auxiliary collector under `internal/core/auxreq`; +4. a typed detached auxiliary execution mode that preserves principal/billing identity while using a private child A-leg and no primary session-turn effects; +5. a process-owned branch coordinator serializing revision/job/injection state across runtime generations; +6. an official `internal/plugins/features/compactioncontinuity` implementation for capsule schema, deterministic plan carriers, sanitizer, extractor prompt/schema, merge and injection; +7. existing routing, B2BUA, usage/metering and BillingCallID accounting reused as execution authorities. + +The semantic extractor runs independently from the primary coding turn and may use a completely different model/provider. It is nevertheless a normal canonical Go-LIP child call, so actual extractor usage is billed to the originating authenticated user/account by default. + +## Goals + +- Preserve accepted/current plan state and explicit user product/architecture decisions across repeated lossy compactions. +- Make structured planning state deterministic and LLM-free where possible. +- Run semantic extraction off-session/background with bounded process ownership. +- Allow a separately configured extractor route independent of the primary session route. +- Attribute auxiliary usage/cost to the originating user through existing billing authorities. +- Prevent the first post-compaction turn from outrunning already-ready or deterministically recoverable continuity state without submitting fresh billable provider work before its primary Open. +- Keep provider-native encrypted/opaque compaction content byte-identical. +- Keep the feature bounded, fail-open and content-safe in observability. + +## Non-Goals + +- general long-term/RAG memory; +- storing every conversation fact; +- replacing native/agent compaction; +- a general durable job/workflow framework; +- a second transcript database; +- a second financial ledger/rating engine; +- direct provider clients for the extractor; +- mutating `compaction.Observer`; +- provider-specific continuity branches in core; +- claiming restart durability without an authorized durable source. + +## Dependency and Composition Gate + +`.kiro/specs/compaction-event-detection/` is a hard runtime prerequisite. + +```text +compaction-event-detection runtime + | + +--> pure request/response previews from the same matcher/fingerprint authority + | + v +compaction-continuity-preservation +``` + +When the continuity feature is **enabled**, generation/process composition verifies that detector preview/commit services, process BranchCoordinator and BackgroundAux are available. Missing prerequisites fail generation/startup composition with a clear error. When the feature is disabled, these requirements are no-op/compatible and no extractor configuration is required. + +Continuity must never recreate the #312 signature matrix as a fallback. + +## Existing Architecture Reused + +- `lipapi.Call`, canonical messages/items/tools and normalized walkers provide provider-neutral source data. +- prerequisite `internal/core/compactiondetect` owns compaction evidence, rule IDs, A-leg transactions and history heuristic. +- `FeatureBundle` / single merge surface / request snapshot provide additive extension composition. +- `internal/core/auxreq` delegates child calls through the normal Executor, clones principal/scope, marks internal origin and suppresses plugins. +- `pkg/lipsdk/genpin.KindAsync` already models request-spawned asynchronous generation ownership. +- `ProcessServices` owns process lifetime and process `ExtensionState` across generation reloads. +- secure-session recording already provides optional durable transcript data when explicitly enabled. +- billing account identity derives from authenticated principal scope; independent child calls receive independent BillingCallIDs. + +## Target Architecture + +```text + PRIMARY / COMPACTION FLOW PROCESS-OWNED AUXILIARY PLANE + + canonical request + | + v + detector PreviewRequest (pure) + | + +--> Preserver.BeforeRequest + | pending capsule/job? + | bounded Await only for previously submitted work + +--> canonical reinjection (transactional helper) + | + normal prepare / route / billing / Open + | + +-- Open succeeds + | + +--> detector RequestOpened (commit) + +--> Preserver.RequestOpened + | deterministic harvest / sanitizer + +--> BackgroundAux.SubmitCollect --------+ + | + primary stream continues v + bounded workers + | + detached child call + | + private child A-leg + | + independent selector + | + normal B-legs/billing + | + strict JSON result + | + bounded result registry + | + final selected event + | + detector PreviewResponse (pure) + | + Preserver.BeforeResponseRelease + |---- bounded Await --------------------------------------+ + |---- validate/merge capsule + |---- safe plaintext? mechanical augment + |---- opaque/late? pending reinjection + | + detector ResponseReleased(final event) (commit) + | + metadata-only compaction observers + | + client release + | + successful release commits branch-level reinjection watermark +``` + +## D1. Pure Preview Versus Committed Detector State + +The prerequisite detector remains the one recognition authority. + +Add pure internal/public-to-core preview shapes (exact names may differ): + +```go +type RequestPreview struct { + Evidence Evidence + RuleID string + Kind PreviewKind // none | start_candidate | completion_candidate + TransactionID string // only when safely derivable from existing active state + BoundaryFingerprint string // stable detector-owned preview identity when no transaction exists +} + +type ResponsePreview struct { + Evidence Evidence + RuleID string + Kind PreviewKind // none | completion_candidate + TransactionID string +} + +func (d *Detector) PreviewRequest(meta RequestMeta, call lipapi.Call) RequestPreview +func (d *Detector) PreviewResponse(meta ResponseMeta, ev lipapi.Event) ResponsePreview +``` + +Preview properties: + +- same matcher/fingerprint logic as committed detection; +- no transaction/fingerprint mutation; +- no lifecycle event emission; +- no observer dispatch; +- request preview cannot establish a billable semantic-extraction start; +- completion-only preview can expose a stable boundary fingerprint for a non-billable intent before a committed transaction exists; +- response preview lets preservation identify the matching job before final response mutation. + +Committed boundaries remain: + +- `RequestOpened` only after successful upstream Open; +- `ResponseReleased` only after all permitted preservation finalization, on the exact event sent to the client. + +## D2. Preservation Extension Contract + +Extend the `pkg/lipsdk/compaction` package introduced by #312 with a distinct preservation surface. + +Conceptual contract: + +```go +type PreservationMeta struct { + TraceID string + SessionID string + ALegID string + BLegID string + AttemptSeq int + TransactionID string + RuleID string + Evidence Evidence +} + +type Services struct { + State ContinuityState + BackgroundAux auxiliary.BackgroundClient +} + +type Preserver interface { + ID() string + BeforeRequest(context.Context, *lipapi.Call, RequestPreview, PreservationMeta, Services) error + RequestOpened(context.Context, lipapi.Call, []Event, PreservationMeta, Services) error + BeforeResponseRelease(context.Context, *lipapi.Event, ResponsePreview, PreservationMeta, Services) error +} +``` + +Semantics: + +- `BeforeRequest`: pre-open pending reinjection / completion-only barrier; may mutate only through the continuity injection helper. +- `RequestOpened`: successful-open source commit and background job scheduling; current primary request has already been sent upstream. +- `BeforeResponseRelease`: pure-preview-guided bounded join and verified result-side augmentation before committed `ResponseReleased`. + +The interface remains `error`-returning, but **preserver errors are not primary-traffic errors**. Core dispatch owns the following composition rule: + +1. invoke each preserver behind panic isolation; +2. on callback error/panic, emit only content-free preservation diagnostics and continue native traffic; +3. `BeforeRequest` and `BeforeResponseRelease` mutation runs against a transactional helper/snapshot so callback error, panic, or post-mutation canonical validation restores the exact pre-preservation `Call`/`Event` before continuing; +4. `RequestOpened` failure cannot undo the already-opened primary request and therefore only records preservation failure/cleans feature-local pending state; +5. preservation failure never becomes route/failover/no-retry authority and never masquerades as a provider failure. + +A practical implementation can clone the bounded canonical mutation target before dispatch or make the continuity helper return an undo/commit handle; do not add a general transaction framework. + +`FeatureBundle` gains `CompactionPreservers []compaction.Preserver`; the single merge surface concatenates in registration order and the runtime snapshot exposes a frozen defensive copy. + +No request/response content is added to `compaction.Event` and `Observer` remains unchanged. + +## D3. Process-Owned Background Auxiliary Collector + +### Additive SDK capability + +Keep synchronous `auxiliary.Client` source-compatible. Add a narrow interface: + +```go +type JobID string + +type SubmitOptions struct { + CoalesceKey string + Timeout time.Duration +} + +type BackgroundClient interface { + SubmitCollect(ctx context.Context, req Request, opts SubmitOptions) (JobID, error) + Await(ctx context.Context, id JobID) (lipapi.Collected, error) + Forget(id JobID) +} +``` + +`DisabledBackgroundClient` returns explicit not-configured errors. + +### Scheduler implementation + +Create focused `internal/core/auxreq` background collection support: + +```text +BackgroundScheduler + bounded worker count + bounded queue + coalescing map + bounded result registry + process root context + closeOnce + WaitGroup +``` + +It accepts only canonical auxiliary model collection jobs, never arbitrary Go callbacks/tasks/timers. + +The scheduler only sees **committed billable job identities**. A completion-only preview that has no committed transaction is represented in BranchCoordinator as a non-billable `PreviewIntentKey`; it is not inserted into the scheduler coalescing map and does not retain a generation pin. After successful primary Open, the preview intent binds to the committed transaction and produces the normal scheduler key: + +```text +coalesce_key = H(parent_branch_binding | committed_transaction_id | target_source_revision) +``` + +An empty transaction ID is invalid for a new billable continuity submission. + +### Submit-time ownership transfer + +`SubmitCollect` synchronously: + +1. validates/clones the child request; +2. resolves the current generation `ExecutorRunner` from the request snapshot cell; +3. obtains `genpin.Retainer` and `Retain(KindAsync)` while spawn authority is live; +4. clones required principal/scope/correlation attribution plus the **captured parent continuity branch binding** as opaque lineage; +5. reserves/coalesces the committed job; +6. enqueues or fails atomically; +7. releases a newly retained pin immediately on failed handoff. + +A worker never attempts a later generation retain. The captured pin is released exactly once after terminal child collection/cancel. + +The captured parent branch binding is not execution authority for the child. It exists so feature logic can associate pending results with the primary continuity branch; the private child A-leg created later must never replace it. + +### Worker context + +Worker context derives from the scheduler/process root and copied attribution, not from parent request cancellation. A per-job timeout bounds inference. + +Preserve principal/scope, internal origin, parent correlation, captured parent continuity branch binding and captured generation. Do not preserve primary resume/session authority or primary A-leg route authority. + +### Result retention + +Raw `lipapi.Collected` is count/byte bounded and never logged. A result referenced by a BranchState `PendingJobID` remains available for the configured bounded pending-continuity retention window, not merely a tiny cache TTL. First successful consume validates/merges then calls `Forget` immediately. Branch/job expiry clears both sides coherently. No result outlives the configured bounded continuity/source retention horizon. + +## D4. Detached Auxiliary Execution and Private Child A-Leg + +Add trusted auxiliary execution policy (exact type location may differ): + +```go +type SessionMode uint8 +const ( + SessionModeNormal SessionMode = iota + SessionModeDetached +) +``` + +This is carried on the trusted auxiliary request object after frontend decode; no wire field/header maps to it. + +Detached child flow: + +```text +parent continuity BranchKey = Session/A-leg A123 (captured before submit) + | + +---- opaque lineage only --------------------------------+ + | +originating principal/scope | + | | + v | +private auxiliary logical call | + | | + +--> create/touch private child A-leg AUX456 | + | parent_a_leg=A123 only in lineage | + | | + +--> normal route selector / request authority / BillingCallID + | + +--> one or more child B-legs + | + +--> normal terminal usage/billing + +continuity Await/merge/revision/reinjection always use captured parent BranchKey, +never AUX456. +``` + +Detached semantics: + +- preserve authenticated principal/scope for security and billing; +- parent SessionID/A-leg/trace are correlation only for child execution but remain the separately captured authority for continuity-state lookup; +- do not call primary secure-session BeginTurn or create primary TurnID; +- do not mutate primary transcript/activity/turn counts; +- do not propagate primary resume authority; +- do not fetch/apply primary A-leg route override; +- create/use a **private child A-leg** through existing B2BUA lifecycle/store semantics; +- keep child A-leg/B-legs private/internal but fully usable by existing request authority/billing machinery; +- no client-visible session/history/header effects. + +Implementation should factor shared Executor prepare logic rather than clone an entire second executor. + +## D5. Independent Extractor Routing + +Feature config requires either explicit `extractor.route` or explicit `route_policy: inherit`. + +The plugin builds a canonical child `Call.Route.Selector`; normal routing owns parsing, aliases, safe composition policy, health, failover and B-leg opening. The private child A-leg has no parent route override state, so the primary A-leg override cannot hijack the extractor. + +Child attribution: + +```text +Role = compaction_continuity_extractor +Visibility = private +Origin = internal auxiliary +``` + +Tools are absent/disabled; tool choice is none/legal equivalent. + +Submission captures route/timeouts/token bounds/failure policy immutably. Reload changes only future jobs. + +## D6. Originating-User Billing and Workload Classification + +Detached execution retains original authenticated scope, so existing billing identity resolves the same account. + +Every extraction has: + +- independent BillingCallID; +- private child A-leg and independent B-leg attempts; +- positive authoritative `AttemptSeq` on every independently persisted child/failover B-leg; +- normal usage/concurrency authority; +- normal credit screen/exposure admission when authoritative billing applies; +- normal terminal usage/customer settlement/provider COGS; +- final billing evidence with the existing usage quantities/presence and cost evidence/presence plus valid `Source`, `Authority`, and `DedupeKey` for each auxiliary B-leg/failover record. + +The existing independent-leg billing boundary rejects `AttemptSeq <= 0`; continuity must not treat a distinct BillingCallID as sufficient if the per-leg record is invalid or would be rejected. Tests therefore pin field-level evidence, not only call identity. + +No special charging function or money ledger is added. + +Project a bounded content-free workload classification into existing accounting/metering/diagnostics, preferably from auxiliary lineage: + +```text +workload_class = auxiliary +aux_role = compaction_continuity_extractor +``` + +This classification does not implicitly change rates. + +Primary frontend protocol usage stays primary-call-only. Account/operator totals include child records and can group auxiliary cost. + +Pre-submit rejection means no provider usage and preservation fail-open. Once provider work is submitted, actual usage remains accountable even if the result is invalid/late/stale. + +## D7. Process-Owned Branch Coordinator + +A per-generation feature mutex cannot protect overlapping generations/workers. Current process `ExtensionState` is reload-stable but has no atomic compare-and-swap. + +Create a small process-owned `internal/core/compactioncontinuity` coordinator. It is a synchronization/state facade, not semantic memory logic. + +Responsibilities: + +- serialize updates per authoritative **parent** branch key; +- use process `ExtensionState` as serialized backing where practical; +- revision-checked load/update; +- non-billable preview intent -> committed transaction binding; +- pending job/injection watermarks; +- bounded max entries/TTL and lazy cleanup; +- opaque capsule/source blobs only. + +Conceptual branch key: + +```go +type BranchKey struct { + AuthoritativeSessionID string + ALegID string // parent primary A-leg; never detached child A-leg + PrincipalPartition string // used when no secure SessionID +} +``` + +Client session hints never authorize lookup. The feature captures this key before any auxiliary child A-leg is created and derives a stable content-free `BranchBinding = SHA256(domain || canonical(BranchKey))` for serialized capsule/job identity. Raw principal/session identifiers need not be sent to the extractor model. + +Conceptual state: + +```go +type PreviewIntent struct { + Key string // H(branch binding | detector preview boundary/fingerprint | target revision) + TargetSourceRevision uint64 +} + +type InjectionTarget struct { + BoundaryKey string + CapsuleRevision uint64 +} + +type InjectionWatermark struct { + BranchBinding string + BoundaryKey string + CapsuleRevision uint64 +} + +type BranchState struct { + Revision uint64 + CapsuleJSON json.RawMessage + CapsuleDigest [32]byte + SourceHighWatermark string + SanitizedSourceJSON json.RawMessage + PendingPreviewIntent *PreviewIntent + PendingJobID auxiliary.JobID + PendingJobTargetRevision uint64 + PendingJobBranchBinding string + PendingInjection *InjectionTarget + LastReleasedInjection *InjectionWatermark + LastCompactionTransaction string + UpdatedAt time.Time +} +``` + +`PendingJobBranchBinding` must match the branch state that owns the JobID before Await/merge. Child A-leg IDs are never accepted as a replacement key. + +Core coordinator never interprets plan/decision facts. Worker/provider work runs outside coordinator locks. + +## D8. Continuity Capsule V1 + +Feature-owned serialized envelope: + +```json +{ + "schema_version": 1, + "revision": 12, + "source_high_watermark": "...", + "branch_binding": "sha256:...", + "content_digest": "sha256:...", + "plan": { + "status": "accepted", + "source": "structured|user_acceptance|semantic", + "steps": [ + {"id":"...","text":"...","status":"pending","source_ref":"..."} + ] + }, + "decisions": [ + { + "id":"...", + "conflict_key":"architecture.billing.mode", + "supersedes":[], + "statement":"...", + "status":"active", + "authority":"user_explicit|user_acceptance|semantic", + "rationale":"...", + "source_ref":"..." + } + ], + "constraints": [], + "rejected_alternatives": [], + "open_questions": [] +} +``` + +### Envelope binding and digest + +- `branch_binding` is the content-free digest derived from the authoritative parent `BranchKey`; the private child A-leg never contributes to it. +- `content_digest` is `SHA256(domain || canonical_json(envelope_without_content_digest))`. The digest scope includes schema version, revision, high-watermark, branch binding and the complete semantic payload; only the digest field itself is omitted to avoid recursion. +- Canonical JSON rules (object-key order, number/string representation, no insignificant whitespace) are fixed by feature tests. The implementation may use a typed canonical encoder rather than generic map serialization. +- Registry consume, merge, reinjection/projection, authorized recovery and generation-reload reuse first validate branch binding + digest. A mismatch is fail-open discard, never cross-branch adoption. +- Model-facing extractor input/projection may omit raw branch identifiers and may omit the branch binding itself after local validation; self-binding is a storage/transfer integrity contract, not a reason to expose account/session identity remotely. + +### Decision identity and merge precedence + +Merge precedence: + +1. later explicit user correction/decision; +2. later explicit user acceptance/selection; +3. authoritative deterministic structured-plan update; +4. validated semantic inference; +5. older capsule state. + +Each decision has two identities with different purposes: + +- `id`: stable fact identity/provenance across revisions; +- `conflict_key`: stable normalized semantic slot describing what choice the decision governs. + +Extractor-generated IDs alone never decide conflicts. Validation/merge enforces: + +- at most one `active` decision per `conflict_key`; +- a new active decision for an existing conflict key deterministically supersedes the older active decision according to precedence/revision; +- a correction whose semantic slot cannot safely reuse the prior conflict key must carry validated `supersedes` references to known active decision IDs; unknown/cross-branch references are rejected; +- non-conflicting decisions retain independent conflict keys and stable IDs; +- semantic output cannot resurrect a superseded fact against newer explicit intent. + +Deterministic carrier facts use deterministic normalized IDs. Semantic IDs are accepted only after schema/source/conflict validation. + +Bounded pruning preserves active decisions/constraints and pending/in-progress plan steps first, then useful rationale/current rejections, then condenses completed/superseded history. Prune whole facts and revalidate/re-digest; never truncate JSON syntactically. + +## D9. Deterministic Plan Carrier Catalog + +Place structured carrier parsing in `internal/plugins/features/compactioncontinuity`, not provider adapters/core detector identity. + +Initial versioned families, after pinning actual canonical fixtures: + +```text +codex.update_plan.v1 +opencode.todo.v1 +cline.task_progress.v1 +``` + +Rules match canonical tool/item shapes, not agent brand. Each needs positive/near-miss tests. There is no generic “markdown checklist means accepted plan” rule. + +## D10. Sanitized Extraction Source + +Build a bounded structured envelope from effective canonical calls rather than raw wire bodies. + +Priority: + +1. user decision/constraint text; +2. assistant plan/proposal/clarification needed to interpret later user replies; +3. recognized structured plan carriers; +4. prior locally validated capsule semantic payload. + +Default drop/truncate: + +- ordinary tool results; +- command/compiler logs; +- large source/file/code dumps; +- images/video/binary/file blobs; +- unnecessary reasoning payloads; +- unrelated external content. + +Any exceptional included tool/external material is tagged untrusted and delimited. + +Every **successfully opened primary request** may refresh the process sanitized source/high-watermark. Failed pre-open calls never become committed source state. + +Source priority is: current/effective canonical baseline -> process sanitized window -> optional existing authorized secure-session transcript via a narrow reader. No durable transcript read is required on ordinary hot path. + +## D11. Semantic Extractor Call + +One canonical child call contains: + +- configured independent route; +- fixed extraction instructions; +- prior validated capsule semantic payload/base revision; +- deterministic plan facts; +- sanitized delta; +- no tools; +- bounded max output; +- continuity plugin suppressed; +- detached mode/private child A-leg; +- role `compaction_continuity_extractor`. + +Expected output is one strict JSON object such as: + +```json +{ + "schema_version": 1, + "base_revision": 11, + "facts": [], + "plan_updates": [], + "decision_updates": [ + { + "id": "...", + "conflict_key": "architecture.billing.mode", + "supersedes": ["decision-old"] + } + ], + "remove_or_supersede": [] +} +``` + +Validate before merge: + +- exactly one JSON object; +- exact schema version/base revision handling; +- byte/depth/item/string/count limits; +- exact enums; +- no unknown authority escalation; +- no raw tool/blob fields; +- valid/allowed source refs where used; +- decision conflict keys normalized/bounded and `supersedes` references limited to known decisions in the same validated parent capsule. + +Malformed output is discarded fail-open. There is normally no second LLM pass. + +## D12. Semantic Eligibility Gate + +The local gate saves cost; it does not decide semantic truth. + +Candidate signals may include structured plan change, substantial assistant plan followed by affirmative/corrective user text, explicit user choice/constraint/correction language, absent/stale capsule around planning markers, or new decision-relevant user turns after high-watermark. + +Generic words alone never trigger. If deterministic state fully satisfies the configured preservation categories, skip the semantic job. + +## D13. Strict Remote-Compaction Flow + +### Pre-open + +- `PreviewRequest` may identify a strict start candidate. +- preservation may prepare source/check old pending reinjection. +- **do not submit a strict-start semantic job yet**; a failed Open must not bill extraction. + +### Successful primary Open + +1. detector `RequestOpened` commits start/transaction; +2. preservation commits sanitized source; +3. deterministic carrier extraction updates/re-digests capsule; +4. eligibility decides whether semantic work adds value; +5. derive committed coalescing identity `H(parent_branch_binding | transaction_id | target_source_revision)`; +6. `BackgroundAux.SubmitCollect` submits one coalesced job if needed and BranchState stores the JobID against the captured parent branch; +7. primary compaction stream proceeds concurrently. + +### Final selected event + +1. detector `PreviewResponse` identifies potential completion/transaction without committing; +2. preservation resolves the matching parent-branch JobID and optionally `Await`s it for the bounded barrier; +3. ready strict output is validated and revision-merged/re-digested into the same parent branch; +4. verified plaintext carrier may receive deterministic continuity projection; +5. unsafe opaque/late paths set `PendingInjection{BoundaryKey, CapsuleRevision}`; +6. detector `ResponseReleased` receives the **post-preservation final event** and commits completion; +7. metadata-only observers dispatch; +8. final event releases to client. + +A barrier timeout is fail-open. A late result remains available while referenced by bounded pending state and can be consumed by the next eligible turn. + +## D14. Completion-Only / Local-Compaction Flow + +If the first evidence is the next rewritten request, v1 deliberately avoids fresh billable semantic work before the primary request is known to have opened. + +### Before primary Open + +1. `PreviewRequest` identifies a completion candidate from the shared history matcher/fingerprint state without committing it. +2. capture the authoritative parent BranchKey/branch binding and derive `PreviewIntentKey = H(parent_branch_binding | preview_boundary_fingerprint | target_source_revision)`. +3. store/coalesce only that **non-billable preview intent** in BranchCoordinator; do not call `BackgroundAux.SubmitCollect`, do not retain `KindAsync`, and do not create a child A-leg yet. +4. load/validate the prior capsule and merge deterministic carrier delta. +5. if a matching semantic job was already submitted by an earlier successfully opened request (for example a late strict job), optionally `Await` that existing JobID up to the bounded barrier; otherwise there is no pre-open semantic wait. +6. inject the best already-ready/deterministic capsule transactionally if required, then continue normal primary Open fail-open. + +### After successful primary Open + +7. detector `RequestOpened` commits the completion-only transaction. +8. bind the PreviewIntentKey to the committed transaction and commit source/high-watermark/deterministic state for this successfully opened request. +9. if semantic eligibility still requires additional information, derive the normal committed coalescing key and submit one background job now. +10. that newly submitted job may improve subsequent turns/continuity state, but it is **not allowed to retroactively justify pre-open billing** for this request. + +If primary Open fails, discard/expire the preview intent under bounded cleanup and produce **zero new billable continuity child work**. The extractor remains off-session/background after successful submission; v1 accepts that completion-only semantic information discovered only at this boundary may not be available for the same request, because preserving billing/lifecycle correctness is stronger than paying before Open. + +## D15. Canonical Reinjection + +One provider-neutral helper applies a deterministic text projection of a locally branch/digest-validated capsule. + +- message-authoritative call -> legal proxy-owned developer/system instruction/message representation; +- item-authoritative call -> legal canonical message item; +- never populate legacy Messages/Instructions alongside item authority in violation of `Call.Validate`. + +Projection format is versioned/delimited and says it is prior continuation state, not a new user request. Internal branch binding/digest metadata need not be emitted into the model-facing text after local validation. + +### Boundary-scoped deduplication + +Revision alone is not an injection identity. Use: + +```go +type InjectionWatermark struct { + BranchBinding string + BoundaryKey string // committed transaction when available; detector preview boundary key while awaiting bind + CapsuleRevision uint64 +} +``` + +Rules: + +1. `PendingInjection` records the boundary + revision requiring carry-forward. +2. a successive distinct compaction boundary may inject the same capsule revision again; this is required when no new facts were learned between two opaque compactions. +3. within one primary call, a call-local ephemeral marker prevents the helper from inserting the same continuity block twice during internal retry/failover preparation; this marker is not the durable branch watermark. +4. canonical insertion must pass `Call.Validate`. Callback error/panic/validation failure restores the exact pre-injection call and leaves `PendingInjection` unchanged. +5. primary Open failure leaves `PendingInjection` unchanged. +6. `LastReleasedInjection` advances and matching `PendingInjection` clears **only after the primary turn reaches successful final client release**. If the turn aborts before release, a later eligible request may inject again. +7. after completion-only Open binds a preview boundary to a committed transaction, the branch state rewrites the pending boundary key consistently before final release bookkeeping. + +Projection is rechecked against injection budget after serialization. + +## D16. Plaintext Result Augmentation Boundary + +`ItemKindCompaction` is not assumed text. Maintain an allowlisted capability/matcher for verified mutable plaintext continuation carriers only. + +Never modify: + +- `CompactionItem.EncryptedContent`; +- `CompactionItem.Opaque`; +- provider signatures/opaque reasoning; +- unknown extension blobs. + +Tests compare those bytes exactly with feature on/off. + +Reinjection is mandatory universal fallback; response augmentation is only a safe-path optimization/stronger immediate carry-through. + +## D17. Per-Session Policy + +Global feature config is standard opaque feature YAML. Trusted session resolution: + +```text +operator hard maxima / safety policy + > trusted per-session value + > global feature default +``` + +Allowed trusted session controls may include enabled, preserved categories, approved extractor selector, and tighter limits. Client headers/session metadata do not directly set these values and no new unauthenticated control header is introduced. + +## D18. Configuration + +Indicative only; implementation tests pin defaults. The example intentionally remains disabled so copying it cannot silently enable remote data egress or extra billed inference: + +```yaml +plugins: + features: + - id: compaction-continuity + enabled: false + config: + preserve: + plan: true + user_decisions: true + constraints: true + rationale: true + rejected_alternatives: true + extractor: + route: "openai-responses:small-model" + timeout: 8s + max_input_tokens: 12000 + max_output_tokens: 2000 + max_concurrency: 2 + queue_capacity: 16 + barrier_timeout: 2s + pending_result_ttl: 2h + max_capsule_tokens: 2500 + source_ttl: 2h + failure_mode: fail_open +``` + +Values are examples, not normative defaults. + +Validation: + +- enabled feature requires prerequisite detector preview/commit + BranchCoordinator + BackgroundAux; +- semantic mode requires explicit route or explicit inherit; +- finite positive queue/concurrency/source/result/capsule bounds; +- pending result TTL must be consistent with branch/source usefulness and remains hard bounded; +- trusted session overrides cannot exceed global maxima without explicit authorization; +- invalid enabled config fails generation/startup composition instead of selecting an expensive model silently. + +## D19. Failure Semantics + +Default request-time behavior is fail-open. + +| Failure | Behavior | +|---|---| +| prerequisite missing while feature enabled | generation/startup compile error | +| no decision candidate | no semantic job | +| queue full | deterministic capsule/native flow continue | +| generation retain fails | no job; fail-open | +| child billing/admission denied | no bypass; deterministic/native flow continue | +| extractor backend/timeout fails | child normal recovery then fail-open | +| invalid schema | discard result; actual submitted usage still accountable | +| stale result | discard/forget; no state regression | +| barrier timeout | continue native flow; pending result may be consumed later while bounded/useful | +| capsule branch-binding/digest mismatch | discard capsule/result for this branch; native flow continues | +| capsule over budget | whole-fact deterministic pruning then re-digest | +| preserver callback error/panic | record content-free stage failure; do not propagate to primary traffic | +| failed/invalid BeforeRequest mutation | restore pre-preservation `Call`; keep pending reinjection; native flow continues | +| failed/invalid BeforeResponseRelease mutation | restore pre-preservation `Event`; committed detector sees restored final event | +| completion-only primary Open fails | discard/expire non-billable preview intent; no new child submission/billing | +| state/coordinator unavailable | skip preservation mutation; native traffic continues | +| process shutdown | stop admission; cancel/join worker; normal submitted usage/accounting remains terminally owned | + +## D20. Privacy and Security + +- disabled by default; +- remote extractor is explicit data egress; +- existing redaction/secret policy precedes egress; +- source is bounded/sanitized, not full raw session by default; +- source content is untrusted quoted data; +- no tools and preservation plugin suppressed; +- prompt/output/capsule absent from normal logs/metrics; +- branch binding is a content-free one-way binding and raw parent BranchKey identifiers need not be sent to the model; +- pending raw result exists only in bounded process memory; +- transcript reads preserve principal/session/workspace authorization; +- no implicit durable transcript capture; +- detached mode is trusted auxiliary metadata unavailable to frontends. + +## D21. Observability + +Content-free metrics/diagnostics: + +- compaction preview/start/completion by evidence/rule; +- structured carrier hits by rule ID; +- semantic eligibility skip reasons; +- preview intents created/bound/expired without provider submission; +- background jobs submitted/coalesced/saturated/canceled/completed/failed/stale; +- queue depth/in-flight; +- extractor route/backend/model identifiers where policy allows; +- extractor tokens/cache/cost through existing usage/billing; +- auxiliary accounting rejection by invalid attempt/evidence contract; +- capsule revision/serialized size/fact counts/digest-validation outcome; +- barrier wait duration/timeouts; +- augmentation/reinjection counts and pending/committed watermark outcomes; +- preserver callback fail-open outcomes by stage; +- stale/revision/conflict-key conflicts; +- opaque-carrier no-mutation count. + +Logs use hashes/IDs/counts only. + +## D22. Lifetime and Concurrency Invariants + +1. ProcessServices owns BackgroundScheduler and BranchCoordinator. +2. Generation snapshots hold non-owning clients only. +3. Each successfully submitted background job owns exactly one captured `KindAsync` pin; non-billable preview intents own none. +4. Pin releases exactly once on every terminal/cancel/handoff-failure path. +5. A worker never obtains a different generation after submit. +6. Parent BranchKey/branch binding is captured before child A-leg creation and remains the only continuity-state key for that job. +7. BranchCoordinator serializes revision/job/injection state across overlapping generations. +8. No external model/provider call executes while a branch-coordinator lock is held. +9. No observer/plugin callback runs under scheduler/coordinator internal locks. +10. Job/result/branch/preview-intent maps have hard bounds/TTL and no unbounded cleanup goroutine pattern. +11. Reload changes future jobs only; already-submitted work remains accountable and terminally owned. +12. Branch-level reinjection watermark commits only after successful final client release; validation/Open/aborted-release failures retain pending state. + +## D23. Restart and Durable Resume + +V1 guarantees continuity across compaction and immutable generation reload within one process. It does not create durable capsule/job storage. + +An optional future/implementation recovery adapter may reconstruct from an already-enabled authorized secure-session transcript. It must be read-only, bounded and authorization-preserving. Reconstructed capsules receive the current authoritative branch binding and a freshly validated canonical digest before use. Without that source, missing process capsule after restart is explicit fail-open state. + +No durable job queue or full transcript store is added here. + +## D24. Expected Change Surface + +Primary packages: + +- `pkg/lipsdk/compaction` — preview-related preservation contract beside prerequisite observer; +- `pkg/lipsdk/feature` — additive preserver slice; +- `pkg/lipsdk/auxiliary` — additive BackgroundClient/job types and trusted detached policy carrier; +- `internal/featurebundle` / `internal/core/extensions` — merge/snapshot/preservation stages and fail-open transactional mutation dispatch; +- `internal/core/compactiondetect` — pure request/response previews sharing matcher authority; +- `internal/core/auxreq` — bounded scheduler, submit-time pin handoff, detached child adapter; +- `internal/core/compactioncontinuity` — narrow BranchCoordinator/preview-intent/injection-watermark state; +- `internal/core/runtime` — request preview/open, final-response ordering, detached prepare/private child A-leg wiring, successful-release watermark commit; +- `internal/infra/runtimebundle` — process ownership and generation snapshot clients; +- `internal/plugins/features/compactioncontinuity` — config/capsule/carriers/sanitizer/eligibility/extractor/merge/injection; +- existing metering/billing/report projections only for content-free auxiliary workload classification; +- focused testkit/architecture fixtures. + +Individual provider/backend and frontend protocol packages should not need compaction-continuity-specific branches. + +## Testing Strategy + +TDD order: + +1. RED capsule branch-binding/digest, decision conflict-key/supersession, merge/pruning + carrier fixtures. +2. RED sanitizer/eligibility/extractor-schema tests. +3. RED BackgroundAux submit-time pin/committed coalescing/saturation/result-lifetime/shutdown tests plus non-billable preview-intent binding tests. +4. RED detached-session/private-child-A-leg tests proving parent BranchKey remains continuity authority. +5. RED separate-route + user-billing/BillingCallID/per-B-leg AttemptSeq/evidence/protocol-usage tests. +6. RED detector request/response preview and final-release ordering tests. +7. RED preserver callback error/panic rollback and canonical validation fail-open tests. +8. RED strict/remote and completion-only compaction barriers including failed-Open zero-child-billing. +9. RED boundary-scoped reinjection tests: same revision across two compactions, validation/Open/release failure then retry, plaintext/opaque paths. +10. RED three-plus repeated-compaction/concurrency/generation-reload tests. +11. GREEN minimal implementation in that order. +12. repository quality/race/architecture/simplification gates. + +No external model credentials are required; deterministic fake/local backends return fixture JSON. + +## Design Invariants + +1. One compaction recognition authority; no duplicate signature matrix. +2. Metadata observer remains content-free/non-mutating. +3. Committed detector sees the actual final released event after permitted preservation mutation. +4. Preserver callback failure is feature-local/fail-open; failed mutation restores the prior canonical object. +5. Extractor is off primary session but billed to originating user by default. +6. No new billable semantic child is submitted before successful primary Open, including completion-only/local discovery. +7. Extractor route is explicit/independent unless explicit inherit. +8. Background ownership is captured before request spawn right ends. +9. ProcessServices owns worker/coordinator lifetime. +10. Detached child uses a private auxiliary A-leg and never creates a primary secure-session turn; continuity state remains keyed to the captured parent branch. +11. Normal routing/B2BUA/usage/billing own child execution; every persisted auxiliary B-leg has valid AttemptSeq/final evidence and there is no direct provider path. +12. Primary protocol usage excludes child usage; account totals include it. +13. Continuity capsule is bounded, revisioned, parent-branch-bound, digest-validated and reload-safe. +14. Contradictory active decisions cannot coexist solely because semantic extraction emitted new IDs; conflict keys/supersedes define deterministic replacement. +15. Opaque/encrypted compaction bytes are immutable. +16. First post-compaction request may await only already-submitted semantic work before Open; fresh semantic work waits for successful Open. +17. Reinjection dedupe is branch/boundary/revision-scoped and commits only after final client release. +18. Pending late results remain useful only within a bounded branch retention window. +19. Configuration examples/default behavior remain disabled until explicit operator opt-in. +20. No second transcript DB, money ledger, generic workflow engine or redundant summary LLM pass. diff --git a/.kiro/specs/compaction-continuity-preservation/gap-analysis.md b/.kiro/specs/compaction-continuity-preservation/gap-analysis.md new file mode 100644 index 00000000..d9faef46 --- /dev/null +++ b/.kiro/specs/compaction-continuity-preservation/gap-analysis.md @@ -0,0 +1,176 @@ +# Brownfield Requirements Gap Analysis + +## Result + +**PASS after requirements and design-validation corrections.** The feature can be implemented without a new provider client, second transcript database, second billing path, or general workflow runtime. The correction loop made the real brownfield constraints explicit: #312 is a prerequisite; preservation mutation is separate from metadata observation; background auxiliary work needs submit-time generation ownership and ProcessServices lifetime; detached extraction needs a private child A-leg; late results must remain useful within a bounded branch window; and opaque/encrypted compaction state is reinjection-only. + +## Existing Brownfield Facts + +- `compaction-event-detection` is currently a merged **specification**, not runtime code on `main`. It defines process-owned A-leg detector state, one versioned rule matrix, metadata-only `compaction.Observer`, start only after successful upstream `Open`, and committed response observation at the final release seam. +- Current `pkg/lipsdk/auxiliary.Client` is synchronous. `Collect` delegates to `Stream` and the normal runtime Executor. +- `internal/core/auxreq.Client` already clones authenticated principal/scope, marks internal origin, supports plugin suppression and retains `genpin.KindAsync` when auxiliary execution starts. +- `genpin.Retainer` makes spawn rights explicit; attempting to retain after the request lease ends fails closed. +- Current auxiliary `Role`/`Visibility` are lineage metadata only. They do not suppress secure-session BeginTurn/turn transcript/activity or parent route-override authority. +- `lipapi.Call.Route.Selector` already gives a child canonical independent route that normal core routing can parse/execute. +- Billing account identity is principal-scope-derived. Preserving principal scope lets an independent child reuse current usage/billing authorities and obtain its own BillingCallID. +- `ProcessServices` owns process lifetime. `ExtensionState` survives immutable generation reload, while feature lifecycles are generation-composed. +- `ExtensionState` is process-local/in-memory by default and `ScopeSession` partitions by authoritative SessionID when available; A-leg/branch must remain an additional continuity key. +- Secure-session transcript storage already exists when explicitly enabled; it is the only appropriate durable historical recovery source in v1. +- `lipapi.CompactionItem` carries `EncryptedContent` and opaque provider data rather than a universal plaintext summary field. +- Current `FeatureBundle` has no content-bearing compaction-preservation surface; #312 proposes only non-mutating compaction observers. + +## Gaps and Corrections + +### G1 — #312 runtime capability is absent + +**Gap:** #344 could otherwise silently duplicate compaction signatures or pretend lifecycle events already exist. + +**Correction:** implementation order makes `compaction-event-detection` runtime a hard prerequisite. Enabled continuity composition fails clearly if preview/commit capability is absent. Disabled continuity remains compatible/no-op. + +### G2 — `compaction.Observer` cannot carry preservation content/mutation + +**Gap:** observer events intentionally contain no canonical request/response body and return no replacement decision. + +**Correction:** add a distinct `compaction.Preserver`-style slice, separately merged/frozen in FeatureBundle/runtime snapshot. Observer remains unchanged. + +### G3 — pre-open protection must not commit detector truth + +**Gap:** first post-compaction request may need continuity before B-leg Open, while detector start/completion commitment is intentionally tied to successful Open/final release. + +**Correction:** factor pure request preview from the same matcher/fingerprint authority. Preview can guide a barrier but cannot mutate detector state, emit lifecycle events, or start a strict billable extraction merely from an unopened signature. + +### G4 — response preservation must not make `ResponseReleased` observe a pre-final event + +**Gap:** an initial sequence of committed detector completion -> preservation mutation would violate #312's “event actually released” meaning. + +**Correction:** add pure response preview and use final ordering: + +```text +selected event + -> PreviewResponse (pure) + -> preservation finalization / safe plaintext mutation + -> ResponseReleased(final event) (commit) + -> metadata observers + -> client +``` + +No ordinary response hook runs after preservation finalization. + +### G5 — synchronous `Aux.Collect` is not a safe background worker + +**Gap:** `go Aux.Collect(parentCtx)` can start after spawn authority is gone, inherit cancellation, leak across shutdown, and lacks bounded queue/result ownership. + +**Correction:** add a narrow process-owned BackgroundAux collector. `SubmitCollect` synchronously resolves the current runner, retains `KindAsync`, clones safe attribution and transfers ownership to a bounded scheduler. Workers use scheduler-rooted deadlines. Await/Forget operate on bounded job IDs/results. No arbitrary callbacks/functions/tasks are accepted. + +### G6 — worker lifetime cannot be generation-only + +**Gap:** generation-scoped feature lifecycle may retire while a job remains in flight. + +**Correction:** ProcessServices owns scheduler and BranchCoordinator. Generation snapshots hold non-owning adapters. Each job retains exactly the generation it needs. + +### G7 — detached child cannot reuse primary secure-session turn semantics + +**Gap:** normal Executor preparation would otherwise enter primary BeginTurn/transcript/activity/route-override paths. + +**Correction:** add trusted internal detached auxiliary mode. It preserves principal/scope but suppresses primary secure-session turn effects and parent route authority. + +### G8 — detached child still needs normal B2BUA/request authority + +**Gap:** making the child completely session/A-leg-less would force a second execution/billing path; reusing the parent A-leg would contaminate primary authority. + +**Correction:** detached child creates/touches a **private child A-leg** via existing B2BUA semantics. Parent A-leg is lineage only. The child then gets ordinary request authority, private B-legs, separate BillingCallID, usage and provider COGS. + +### G9 — extractor route must remain independent + +**Gap:** parent A-leg route override could otherwise hijack the extractor model. + +**Correction:** child uses explicit configured `Call.Route.Selector` (or explicit inherit policy). Detached private A-leg has no inherited parent override. Normal router remains authoritative; no provider client bypass. + +### G10 — accounting needs workload distinction, not a new money path + +**Gap:** principal inheritance solves account attribution but operators/users need to separate continuity overhead from primary inference. + +**Correction:** project bounded content-free auxiliary workload/role (`compaction_continuity_extractor`) into existing metering/billing/report correlation. Child has separate BillingCallID/B-legs; primary protocol usage remains unchanged; account totals include child usage. + +### G11 — `CompactionItem` cannot be treated as plaintext summary + +**Gap:** mechanical append into `EncryptedContent`/opaque state can corrupt provider replay/continuation semantics. + +**Correction:** result augmentation is allowlisted only for verified mutable plaintext carriers. Encrypted/opaque bytes are immutable. Mandatory safe fallback is proxy-owned first-post-compaction reinjection. + +### G12 — process state is reload-safe, not restart-durable + +**Gap:** ExtensionState is useful for v1 but cannot support a restart-survival claim. + +**Correction:** v1 promises process/generation continuity only. Authorized existing secure-session transcript may be used through a narrow bounded reader to reconstruct after restart; otherwise missing capsule is fail-open. No second durable transcript/job/state platform. + +### G13 — session partition alone is not a branch identity + +**Gap:** `ScopeSession` can group multiple A-leg branches/forks. + +**Correction:** continuity key is authoritative SessionID partition plus explicit A-leg/branch. Without secure SessionID, principal-isolated proxy-owned A-leg is authority. Client hints never select another branch. + +### G14 — per-generation mutex cannot protect capsule revisions + +**Gap:** Store Get/Put has no CAS and a feature-instance lock disappears on reload. + +**Correction:** add a small process-owned BranchCoordinator that serializes revision/high-watermark/job/injection updates while using ExtensionState as serialized backing where practical. It treats capsule/source blobs opaquely and is not a generic transactional state framework. + +### G15 — raw late result must remain useful without becoming durable memory + +**Gap:** if the completion barrier times out and result TTL is too short, a valid extraction can disappear before the next turn. + +**Correction:** while BranchState references a PendingJobID, bounded raw result retention remains useful up to the configured pending continuity window; first consumption parses/merges and Forget deletes raw output. Branch/job expiry clears both coherently. No result outlives bounded continuity/source retention. + +### G16 — durable transcript access must stay behind authorization boundary + +**Gap:** feature code importing secure-session Bun/store internals would break layering/security. + +**Correction:** optional restart/historical recovery uses a narrow authorized read adapter. Ordinary compaction uses current canonical baseline/process sanitized window first. + +### G17 — config reload/disable cannot erase submitted billing obligations + +**Gap:** disabling feature after provider submission cannot retroactively make the child free or orphan its generation. + +**Correction:** jobs use immutable submission-time config/generation and complete/cancel/settle through captured authorities. New config only affects future jobs; disable stops new submissions. + +## Brownfield Compatibility Matrix + +| Existing subsystem | Required treatment | +|---|---| +| #312 `compactiondetect` | prerequisite; shared request/response preview + committed state; no duplicate matrix | +| `compaction.Observer` | unchanged metadata-only/non-mutating contract | +| FeatureBundle/snapshot | additive separately frozen preservation-interceptor slice | +| synchronous `auxiliary.Client` | retained source-compatible | +| new BackgroundAux | additive narrow model-collection scheduler only | +| `genpin` | `KindAsync` retained synchronously at submit | +| ProcessServices | owns scheduler + BranchCoordinator | +| ExtensionState | process backing for bounded serializable branch state | +| secure session | primary turn/transcript/activity untouched by detached child | +| B2BUA | private child A-leg/B-legs; parent IDs lineage only | +| routing | explicit child selector; parent override not inherited implicitly | +| usage/billing | existing authorities; separate child BillingCallID and workload class | +| frontend protocol usage | remains primary-call-only | +| secure transcript | optional authorized recovery only; no second DB | +| compaction opaque/encrypted fields | exact byte preservation | +| provider/frontends | no continuity-specific core/adapter branches | +| generation reload | process state survives; jobs use captured generation/config | + +## Corrected Invariants + +1. One compaction recognition authority; previews never commit lifecycle truth. +2. `compaction.Observer` remains metadata-only and preservation mutation is separate. +3. Committed `ResponseReleased` sees the actual final event after permitted preservation mutation. +4. Strict semantic extraction is not submitted before successful primary Open. +5. Background jobs acquire generation ownership before request spawn authority ends. +6. ProcessServices owns workers/branch synchronization across reload. +7. Detached extractor uses a private child A-leg, not the parent A-leg and not a second provider path. +8. Same authenticated user/account is billed by default; primary protocol usage stays separate. +9. Parent route override cannot silently change extractor route. +10. Opaque/encrypted compaction state is immutable; reinjection is universal fallback. +11. Branch state is bounded/revisioned/SessionID+A-leg scoped and reload-safe, not falsely restart-durable. +12. No second transcript DB, financial ledger, generic task framework, or second summary LLM pass. + +## Final Gate + +All identified requirements gaps and design-validation corrections are reflected in final `requirements.md` and `design.md`. The brownfield gate is **PASS** for TDD task generation. diff --git a/.kiro/specs/compaction-continuity-preservation/requirements.md b/.kiro/specs/compaction-continuity-preservation/requirements.md new file mode 100644 index 00000000..1c75e94d --- /dev/null +++ b/.kiro/specs/compaction-continuity-preservation/requirements.md @@ -0,0 +1,216 @@ +# Requirements Document + +## Introduction + +Go-LIP shall preserve the decision state that matters for continuing a long coding-agent session across lossy context-compaction events. The feature is not general memory and is not a replacement compactor. Its purpose is narrower: retain the latest accepted plan, explicit user product/architecture decisions, constraints, useful rationale, meaningful rejected alternatives, current plan progress, and unresolved next actions when ordinary compaction would otherwise discard them. + +The preserved state shall be represented as a bounded, versioned **Continuity Capsule** rather than an ever-growing natural-language transcript. Structured planning state already exposed by an agent shall be harvested deterministically. A separately configured auxiliary LLM may be used only for genuinely semantic extraction such as identifying conversational acceptance, user decisions, rationale, or supersession that cannot be recovered safely from structured carriers. + +Auxiliary semantic extraction is explicitly **off the primary agent session** and runs as independent background work. It may use a completely different model/provider/route from the main coding session. It is nevertheless real additional model usage: by default its usage and cost belong to the same authenticated user/account that caused the extraction and must flow through normal Go-LIP admission, usage, metering, customer billing, and provider-cost accounting. + +This specification depends on the compaction-recognition authority defined by the existing `compaction-event-detection` specification. That detector remains metadata-only and non-mutating. Continuity preservation adds a separate content-bearing preservation/interception capability and shall not turn the detector observer into a request/response mutation surface. + +## Boundary Context + +- **In scope:** continuity-capsule schema/merge semantics; deterministic structured-plan harvesting; bounded semantic-extraction eligibility; sanitized extraction input; process-owned background auxiliary execution; independent extractor routing; originating-user billing; detached auxiliary session semantics; compaction-boundary synchronization; verified plaintext augmentation and post-compaction reinjection; repeated compactions; authoritative session/A-leg scoping; trusted session overrides; privacy, observability, and TDD gates. +- **Dependency:** implementation requires the detector/rule authority specified by `compaction-event-detection`. If that runtime capability has not landed, it is a prerequisite rather than a reason to duplicate compaction signatures in this feature. +- **Existing authorities preserved:** core routing/selector parsing; B2BUA lineage; secure-session authority; generation pinning; `ProcessServices`; auxiliary execution; usage/concurrency authority; BillingCallID-based post-usage billing; canonical stream/retry commitment; extension merge/runtime snapshots. +- **Out of scope:** general RAG/long-term memory; retaining every conversation fact; replacing an agent compactor; a new provider client; rewriting encrypted/opaque native compaction blobs; hidden system-account billing by default; general-purpose durable job orchestration; general agent identity inference; provider-specific branching in core; a second full-transcript database. + +## Requirement 1: Shared Compaction Recognition and Separate Preservation Contract + +1.1. Continuity preservation shall reuse the protocol/signature/history recognition authority defined by `compaction-event-detection`; it shall not maintain a second independent compaction-signature matrix. +1.2. The metadata-only `compaction.Observer` contract shall remain non-mutating, content-free, and fail-open. +1.3. Go-LIP may add an additive content-bearing preservation/interception contract, but it shall be distinct from `compaction.Observer` and from ordinary response hooks. +1.4. `FeatureBundle`, the single feature merge surface, and the frozen request-runtime snapshot shall expose preservation interceptors separately and defensively copy/freeze them using existing extension conventions. +1.5. Preservation shall use the same authoritative A-leg and compaction transaction identity as the detector when available. +1.6. A strict start candidate shall not cause a billable semantic-extraction job if the compaction-looking request never successfully opens upstream. +1.7. When a recognized compaction request successfully opens upstream, preservation may start one extraction job for that logical transaction while primary compaction work continues independently. +1.8. Completion-only/history evidence shall never invent a historical start, but it may trigger extraction/reinjection required for a compaction only observable after installation. +1.9. Any pre-open request preview needed to protect the first post-compaction turn shall be a pure/non-committing view over the same matcher/fingerprint authority; preview shall not emit lifecycle events, advance detector state, or establish a billable semantic-extraction trigger. +1.10. Response-side preservation shall use a pure/non-committing response preview over the same detector authority to identify a potential completion before mutation; preview shall not mark a transaction complete or dispatch observers. +1.11. Final-release ordering shall be: selected canonical event -> pure response preview -> separate preservation finalization -> committed `ResponseReleased` on the resulting final event -> metadata-observer dispatch -> client release. No ordinary response hook may run after preservation finalization. +1.12. Provider/frontend wire DTOs and provider SDK types shall not enter core continuity logic. +1.13. Generic words such as `plan`, `summarize`, `continue`, or `compact` shall not by themselves establish a compaction boundary or accepted plan. + +## Requirement 2: Versioned Continuity Capsule and Decision Semantics + +2.1. Continuity state shall use a versioned schema containing at least schema version, monotonic revision, source/high-watermark metadata, branch identity, and content digest. +2.2. The capsule shall represent the latest accepted/current plan separately from user decisions and constraints. +2.3. Plan steps shall support at least pending, in-progress, completed, and removed/superseded semantics. +2.4. User decisions shall support at least active, superseded, and rejected semantics. +2.5. Retained facts shall carry bounded provenance sufficient to distinguish explicit user text, user acceptance/correction, deterministic structured-plan state, and semantic extractor inference without retaining the full transcript. +2.6. Later explicit user intent shall supersede conflicting older active intent; contradictory decisions shall not remain simultaneously active merely because both occurred historically. +2.7. Assistant brainstorming/proposals shall not become authoritative user decisions unless later user evidence accepts/selects/instructs execution of them. +2.8. A structured current plan exposed authoritatively by the harness may be retained as current plan state without requiring a conversational acceptance sentence. +2.9. Explicitly rejected alternatives may be retained when they constrain future work; they shall not reappear as active absent later user reversal. +2.10. Useful rationale/trade-offs may be retained with the associated decision and shall follow that decision's active/superseded state. +2.11. Ambiguous semantic evidence shall be omitted or represented as provisional/non-authoritative, never promoted to explicit user intent. +2.12. Merge behavior shall be deterministic for `previous capsule + validated delta`: duplicate facts coalesce, statuses do not regress, and stale revisions cannot overwrite newer state. +2.13. Capsule size shall be bounded by configured byte/token-equivalent limits; overflow shall prioritize active decisions/constraints, pending/in-progress plan steps, useful rationale, then historical/completed material. +2.14. Completed steps and superseded history may be condensed/dropped under retention policy while active decisions and unresolved work remain until superseded/removed or the branch ends. +2.15. The capsule is continuity state, not an audit transcript, and shall not contain arbitrary logs, file dumps, binaries, credentials, or unrelated tool output. +2.16. The serialized capsule envelope shall be self-binding and self-validating: it shall carry a stable content-free branch binding plus a digest computed over a canonical representation of the complete versioned envelope excluding only the digest field itself. The branch binding and digest shall be checked before registry transfer consumption, merge, reinjection/projection, recovery, or generation-reload reuse; raw branch/account identifiers need not be exposed to the extractor model merely to satisfy this invariant. +2.17. Every active decision shall have deterministic conflict identity independent of its extractor-generated fact ID. The implementation shall use a stable normalized conflict key and may additionally use validated `supersedes` references; at most one decision may remain active for one conflict key, and a correction that targets an existing active decision shall deactivate/supersede that decision rather than coexist under a new ID. + +## Requirement 3: Deterministic-First Extraction and Bounded Source Preparation + +3.1. Before invoking an extractor LLM, the feature shall harvest supported machine-readable planning state mechanically from canonical calls/items/tool data. +3.2. Initial deterministic carrier coverage shall include versioned equivalents of Codex `update_plan`, OpenCode todo state, Cline-style task-progress/checklist state where observable, and other stable structured plan carriers established during implementation research. +3.3. Carrier matching shall use canonical item/tool shapes rather than provider DTOs; each rule shall have a stable versioned ID and positive/near-miss fixtures. +3.4. Carrier rules identify structured plan semantics, not agent identity; compaction-family identity remains owned by the shared detector. +3.5. Structured state that can be normalized deterministically shall not be sent to an LLM merely to rediscover the same facts. +3.6. Cheap local eligibility heuristics shall decide whether semantic extraction is likely to add information; they may suppress a model call but shall not declare arbitrary prose to be accepted user intent. +3.7. A compaction with no relevant plan/decision/constraint candidates and no stale/missing capsule need shall perform zero semantic-extractor calls. +3.8. Extractor input shall include the previous capsule plus only bounded new decision-relevant context needed to advance its source high-watermark. +3.9. User messages receive highest source priority; assistant plan/proposal/clarification content may be retained only as needed to interpret user acceptance/correction. +3.10. Ordinary tool outputs, shell/compiler logs, large file/code dumps, images/video/binary payloads, and unrelated external content shall be dropped or heavily truncated by default. +3.11. Structured plan/TODO tool calls and small required results may survive sanitization. +3.12. Untrusted tool-result/external text shall be excluded from semantic decision extraction by default; any included content remains untrusted data rather than extractor instructions. +3.13. Source preparation shall be incremental where possible so repeated-compaction cost follows new relevant context plus the bounded prior capsule, not total session age. +3.14. The feature may keep a bounded process-local sanitized source window for completion-only/local compactions, but it shall not become an unbounded or durable shadow transcript. +3.15. When existing secure-session transcript capture is enabled and authorized historical recovery is needed, it shall be accessed through a narrow read/source adapter rather than by importing secure-session store/Bun internals into the feature. +3.16. When transcript capture is disabled, the feature shall not enable durable full-transcript capture solely for continuity extraction. + +## Requirement 4: Off-Session Background Auxiliary Execution + +4.1. Semantic extraction shall execute as a proxy-created auxiliary/internal LLM invocation, not as a visible user/assistant turn in the primary agent conversation. +4.2. Extractor output shall be consumed by continuity logic and never surfaced as an assistant message. +4.3. The child shall have its own execution/B-leg lifecycle; parent session/A-leg/trace values are non-authoritative correlation/accounting lineage only. +4.4. Background extraction shall run behind a bounded independent asynchronous worker boundary. A bounded in-process goroutine/worker pool is acceptable; an external worker may be supported later without changing semantics. +4.5. The implementation shall enforce explicit maximum concurrent jobs and queue capacity and shall not spawn an unbounded goroutine per candidate/event. +4.6. Background scheduling shall be process-owned through `ProcessServices` or an equivalent existing process ownership seam, not solely by a generation-scoped feature lifecycle. +4.7. Submission shall synchronously resolve/capture the executable generation and retain `genpin.KindAsync` before returning; a delayed worker shall not attempt a new spawn after the originating request lease is gone. +4.8. Worker execution shall use an independent worker-owned context/deadline carrying only cloned required principal/scope/correlation values, not the canceled parent request context as its lifetime root. +4.9. The process-owned scheduler shall expose a narrow background-auxiliary collection contract with bounded job IDs/results and await/forget behavior; it shall not become a generic arbitrary task engine. +4.10. Equivalent submitted jobs shall coalesce by authoritative **parent continuity branch + committed compaction transaction + target source revision**. When a completion-only pure preview exists before any transaction is committed, the feature shall derive a stable non-billable preview intent key from the authoritative parent branch, detector-owned preview boundary/fingerprint, and target source revision; an empty transaction ID shall never be used as a billable submission key. After successful primary Open, that preview intent shall bind to the committed transaction before any new `SubmitCollect`, and retries/failover shall reuse the bound identity. +4.11. Queue saturation, unavailable worker, shutdown, or failed generation retention shall follow preservation failure policy and shall not fall back to an unbounded goroutine/direct provider call. +4.12. Process shutdown shall stop admission, cancel/join worker execution, release generation pins exactly once, and satisfy repository goleak/race requirements. +4.13. Job results shall be revision-checked before merge; stale jobs cannot overwrite newer intent/capsule revisions. +4.14. Completed raw auxiliary results shall be bounded by count/bytes/TTL, never logged, and forgotten immediately after validation/merge. While a BranchState still references a pending useful JobID, its result retention shall be long enough for the configured bounded continuity/pending window rather than an arbitrary short cache TTL; branch/job expiry shall clear both sides coherently. +4.15. Disabling/reloading the feature shall prevent new jobs according to the new configuration but shall not erase accounting obligations or leak execution for provider work already submitted. +4.16. The authoritative parent continuity BranchKey shall be captured before detached child execution begins and stored with pending job state. Await, result validation/merge, capsule revision, pending injection, and reinjection shall use that parent key; the private child A-leg exists only for auxiliary execution/routing/billing and shall never become the continuity-state branch key. + +## Requirement 5: Independently Configurable Route and Detached Session Semantics + +5.1. The extractor route/model shall be independently configurable from the main session route/model and may use a completely different provider/model. +5.2. Changing the main session model, alias, or A-leg route override shall not implicitly change the extractor route. +5.3. Extractor routing shall still use normal Go-LIP canonical selector parsing, routing, capability, failover, admission, and attempt machinery; no direct provider client may be opened by the feature. +5.4. The child shall use an explicit configured `Call.Route.Selector` or an explicit `inherit` policy. Missing/invalid route shall not silently inherit the main model. +5.5. Parent A-leg ID shall be correlation only and shall not become child route-override authority. +5.6. The child shall be stamped with stable content-free role/origin equivalent to `compaction_continuity_extractor` / internal auxiliary. +5.7. Tools shall be disabled; the child shall not execute workspace tools, MCP actions, shell commands, or other side effects. +5.8. Extractor output shall be bounded and schema-oriented rather than unconstrained conversational output. +5.9. The preservation plugin shall suppress itself on the child; existing auxiliary-depth recursion limits remain effective. +5.10. Core auxiliary execution shall provide a typed detached-session mode for this workload. It shall preserve authenticated principal/scope and parent correlation while suppressing primary secure-session BeginTurn/turn transcript/last-activity effects. +5.11. Detached execution shall create/use a **private child A-leg** for the auxiliary logical call using existing B2BUA lifecycle semantics; it shall neither be sessionless nor reuse the primary A-leg as execution authority. +5.12. The private child A-leg may own normal B-leg attempts/request authority/billing correlation, while `ParentALegID` remains explicit lineage only. +5.13. Detached execution shall not mutate the primary A-leg route override, primary session turn count, primary session transcript, or client-visible session history. +5.14. Detached-session authority shall be trusted internal execution metadata and shall not be encoded as provider-visible opaque call content or settable from frontend wire fields. +5.15. Child A-leg/attempt lineage shall remain clearly distinguishable as internal auxiliary workload from the user's primary conversational turn. + +## Requirement 6: Originating-User Billing, Usage, and Cost Attribution + +6.1. Every semantic extractor invocation is real additional model usage and shall be billable auxiliary inference unless a future explicit operator-funded policy says otherwise. +6.2. By default the child shall inherit the originating authenticated principal/scope so the same customer/account identity is resolved. +6.3. The child shall pass through normal usage/concurrency authority and, where enabled, normal credit screen, route/quote, operational-exposure admission, terminal usage append, customer settlement, and provider-cost processing. +6.4. The auxiliary call shall receive its own BillingCallID and normal B-leg records; it shall not share the primary call's BillingCallID. +6.5. Actual extractor retries/failover B-legs shall be accounted under that auxiliary BillingCallID using existing per-leg semantics. +6.6. User/account aggregate usage/cost totals shall include extractor usage even though the call is not a visible primary turn. +6.7. Primary protocol-visible response usage shall not be inflated by extractor tokens; auxiliary and primary protocol usage remain separate execution records. +6.8. Existing metering/billing/diagnostics shall receive a bounded content-free workload/origin classification identifying continuity extraction, without creating a second money ledger or rating engine. +6.9. Provider COGS shall remain attributed to the actual extractor B-legs/provider/model selected by the extractor route. +6.10. If credit/admission rejects the child before upstream submission, semantic extraction shall skip/fail-open by default; billing policy shall not be bypassed and an unrelated system account shall not be charged. +6.11. If upstream extractor work was submitted, resulting usage remains billable/accountable even if the result is late, invalid, or discarded as stale. +6.12. Enabling the feature shall be documented as potentially generating extra billed inference beyond visible primary turns. +6.13. Operator-funded/system-funded extraction is outside the first implementation and, if later added, must be explicit opt-in accounting policy. +6.14. Every auxiliary B-leg and failover leg that reaches independent terminal accounting shall carry a positive authoritative `AttemptSeq` and valid final billing evidence under the existing billing contract: usage quantities/presence, cost evidence/presence, `Source`, `Authority`, and `DedupeKey`. A record with `AttemptSeq <= 0` or otherwise invalid accounting identity/evidence shall not be treated as successfully accounted merely because the auxiliary BillingCallID is distinct. + +## Requirement 7: Compaction Integration, Barriers, Augmentation, and Reinjection + +7.1. A **new billable semantic-extraction job**, whether triggered by strict start or completion-only/local evidence, shall start no earlier than successful primary B-leg Open. Pre-open completion-only handling may use deterministic state, await a previously submitted matching job, or create a non-billable preview intent, but shall not submit fresh provider work. +7.2. After submission, extraction shall run concurrently with primary compaction work where possible rather than serially blocking the main B-leg for its full inference duration. +7.3. Preservation may await a background job only at a narrow bounded barrier where its result is needed to protect a compaction result or first eligible post-compaction turn. +7.4. Barrier timeout obeys configured failure policy; default fail-open continues native compaction/continuation rather than deadlocking. +7.5. A completed validated capsule may be mechanically added to a compaction result only for a verified mutable plaintext continuation-summary carrier. +7.6. `CompactionItem.EncryptedContent`, opaque provider blobs, signatures, encrypted state, and unknown native compaction payloads shall remain byte-identical. +7.7. If result-side augmentation is unsafe/unavailable, the capsule shall be marked for proxy-owned reinjection on the first eligible post-compaction request. +7.8. Before that first B-leg opens, preservation shall use an already-ready capsule or await a matching job that was submitted by an earlier successfully opened request up to the configured barrier; it shall not create new billable semantic provider work solely to satisfy the pre-open barrier. +7.9. Reinjection shall be canonical-authority-aware: legacy/message-authoritative and item-authoritative calls shall receive a valid bounded proxy-owned instruction/message representation without violating `Call.Validate` authority rules. +7.10. The continuity block shall be versioned/delimited/bounded and distinguishable from user text; it shall state that facts are prior continuation state, not a new user request. +7.11. Reinjection deduplication shall use a compound watermark containing authoritative branch binding, compaction boundary/transaction identity, and capsule revision. The same revision may therefore be reinjected for a later distinct compaction boundary, while one boundary/revision is not duplicated by retries. +7.12. Completion-only/local/history compaction first recognized on the post-compaction request may prepare a non-billable preview intent and deterministic capsule before Open. Only after that request successfully opens and detector state commits may the intent bind to the committed transaction and submit one coalesced semantic job; a failed Open produces no billable child job. +7.13. If deterministic extraction already supplies the necessary capsule, no semantic model job/barrier is required. +7.14. Preservation errors shall not change route selection, failover, no-retry-after-output, or output commitment except for the explicitly bounded pre-output barrier and canonical continuity injection. +7.15. No second LLM pass shall normally rewrite/improve the native summary after one validated semantic extractor result already exists. +7.16. `BeforeRequest`, `RequestOpened`, and `BeforeResponseRelease` errors/panics shall be recorded as preservation failures and shall not propagate into primary request/compaction/response handling. Any canonical request/event mutation attempted by preservation shall be transactional at the preservation seam: if the callback fails, panics, or leaves an invalid canonical object, the runtime shall restore the pre-preservation object and continue native traffic fail-open. + +## Requirement 8: Repeated Compactions, Branch Scope, Concurrency, Reload, and Restart + +8.1. Continuity state shall be keyed by authoritative SessionID when available plus the **parent primary A-leg/branch identity** captured before any detached child A-leg is created; client session hints and child auxiliary A-leg IDs alone shall never select another branch's capsule. +8.2. Without secure-session authority, principal-isolated proxy-owned A-leg authority shall be used rather than arbitrary client hints. +8.3. `ScopeSession` may provide the session partition, but the feature key shall additionally include A-leg/branch identity to prevent branch aliasing. +8.4. Capsule updates shall use monotonic revision/compare-and-merge semantics so concurrent turns/jobs cannot overwrite newer explicit intent. +8.5. Duplicate lifecycle signals and B-leg retry/failover shall be idempotent for job submission and merge; reinjection idempotency shall be scoped to its compound branch/boundary/revision watermark rather than revision alone. +8.6. New unrelated sessions/A-legs start without inherited state. +8.7. Reset/branch replacement shall retire/cancel pending old-branch work under bounded cleanup and shall not leak decisions into the new branch. +8.8. Fork/clone inheritance, if supported, shall be explicit copy-on-fork from a known parent revision; no explicit parent relationship means no inheritance. +8.9. Capsule/source/job state and the background scheduler shall be process-owned so they survive immutable generation replacement/config reload. +8.10. In-flight jobs use immutable route/budget/config and generation captured at submission; later reload affects newly submitted jobs only. +8.11. Three or more successive compactions shall merge `previous capsule + new relevant delta -> new capsule`, not recursively trust only the prior lossy summary. +8.12. Repeated compaction shall not cause monotonic duplicate capsule/prompt growth. +8.13. First implementation restart durability shall be honest: process-local state is not claimed durable across process restart. +8.14. When an authorized durable secure-session transcript already exists, missing capsule state may be reconstructed through the narrow transcript-source adapter; otherwise resume/restart fails open with no hidden capture or cross-session borrowing. +8.15. This spec shall not add a generic durable feature-state/job framework merely to make the capsule restart-durable. + +## Requirement 9: Privacy, Security, and Trusted Session Policy + +9.1. The feature shall be disabled by default and requires explicit operator enablement. +9.2. A remote extractor route is a new data-egress path and shall be documented/configured as such. +9.3. Applicable redaction/secret policy shall run before extractor egress; raw credentials/secrets shall not be exported merely because they appeared in the session. +9.4. Transcript/source text shall be framed as untrusted data. Embedded user/tool/external instructions cannot override fixed extractor task/schema/system policy. +9.5. The child has no tools and no side-effect capability beyond the configured model request. +9.6. Extractor prompt/output/capsule contents shall not appear in normal logs/metrics/diagnostics by default. +9.7. Content-free IDs, revisions, rule/carrier IDs, counts, sizes, latency, token usage, route/backend/model identifiers, and outcomes may be observed. +9.8. Existing principal/workspace/tenant authorization applies to any transcript/source read. +9.9. Transcript-disabled sessions shall not silently acquire a durable full transcript. +9.10. A bounded sanitized process-local source window is allowed but must be branch-isolated and TTL/size bounded. +9.11. Per-session enable/disable or extractor-route override shall come only from trusted proxy-owned policy/session state; unauthenticated client headers/metadata cannot self-enable egress or choose the billed route. +9.12. Trusted per-session overrides shall remain within operator-defined maxima unless separately authorized. +9.13. Auxiliary lineage/accounting metadata shall not contain raw prompt excerpts or capsule content. + +## Requirement 10: Configuration, Failure Policy, and Observability + +10.1. Global feature config shall support at least enablement, preserved categories, extractor route, timeout/input/output limits, worker concurrency/queue bounds, barrier timeout, capsule/source/result bounds/TTL, and failure mode. +10.2. Semantic extraction requires an explicit route or explicit inherit policy; no accidental main-model default is allowed. +10.3. Enabling the feature shall fail generation/startup composition clearly if the prerequisite compaction detector preview/commit service, process branch coordinator, or BackgroundAux capability is unavailable; disabled feature mode shall retain no-op compatibility. +10.4. Default request-time failure mode shall be fail-open for model traffic: extractor timeout/error/saturation/invalid output/state failure **and preservation callback error/panic** do not make native compaction unusable. +10.5. Fail-open preserves already-valid deterministic capsule state, never injects malformed/partial extractor output, and restores the pre-preservation canonical request/event if a preservation mutation fails validation or the callback itself fails before the mutation is committed. +10.6. Extractor output shall be validated against a strict versioned schema with byte/depth/count limits before merge/injection. +10.7. Observability shall count compaction candidates; deterministic hits; semantic jobs submitted/coalesced/skipped/saturated; latency/outcome; extractor input/output tokens; capsule revision/size/fact counts; barrier waits/timeouts; augmentation/reinjection; stale/conflict rejection. +10.8. Billing/usage observability shall expose auxiliary continuity cost through existing accounting stores rather than duplicating financial truth. +10.9. Config reload shall not mutate active job route/budgets mid-flight. +10.10. Disabling the feature prevents new jobs while allowing bounded cleanup/accounting of already-submitted work. +10.11. No failure path may retry indefinitely, spin, block shutdown indefinitely, or fall back to an unconfigured provider/model. +10.12. Background raw-result retention shall be bounded and useful only while referenced/pending under the configured continuity window; normalized capsule state replaces raw result after consumption. +10.13. Content-free preservation-failure diagnostics shall identify callback stage and outcome without logging prompt/capsule content; callback failure shall not be mistaken for primary model/provider failure or influence retry authority. + +## Requirement 11: TDD, Architecture, and Non-Interference Gates + +11.1. Implementation shall begin with RED tests for capsule merge/supersession, structured carriers, eligibility/sanitization, background lifetime, detached session behavior, independent routing, billing attribution, barriers, repeated compactions, and opaque payload protection. +11.2. Worker packages shall use deterministic scheduling controls plus `goleak`/race tests for saturation, cancellation, shutdown, duplicate submission, stale completion, and barrier races. +11.3. Tests shall prove no billable semantic extraction job is submitted when either a strict compaction-looking request **or a completion-only first post-compaction request** fails before upstream Open; completion-only pre-open intent creation/deduplication must remain non-billable. +11.4. Tests shall prove the child uses a private auxiliary A-leg, is absent from primary secure-session transcript/turn count/last activity, and does not mutate primary A-leg route-override/session state. +11.5. Tests shall prove extractor routing can differ from primary routing and that primary A-leg overrides do not rewrite the child selector. +11.6. Billing tests shall prove separate auxiliary BillingCallID/B-leg usage, originating account attribution, aggregate inclusion, continuity workload classification, unchanged primary protocol usage, positive `AttemptSeq` on every auxiliary/failover B-leg, and valid usage/cost/`Source`/`Authority`/`DedupeKey` evidence under the existing terminal-accounting contract. Coverage shall include rejection of `AttemptSeq <= 0` rather than silently treating such a record as accounted. +11.7. Tests shall cover pre-submit billing rejection, submitted-but-invalid result, submitted-but-stale result, retry/failover cost, and fail-open behavior. +11.8. Tests shall cover at least three successive compactions with deterministic decision conflict keys/validated supersession, plan progress, dedupe, capsule branch-binding/digest validation, and bounded growth. +11.9. Tests shall prove encrypted/opaque compaction content is byte-identical with preservation enabled. +11.10. Tests shall prove response preview does not commit lifecycle state and committed `ResponseReleased` sees the exact final event after any permitted plaintext preservation mutation. +11.11. Tests shall prove process generation reload cannot orphan jobs/state and that job submission captured `KindAsync` ownership before the request spawn right ended. +11.12. Architecture gates shall prevent provider/frontend DTO imports, direct provider clients, generic service locators/task runtimes, a second full-transcript store, feature-owned money ledgers, or generation-scoped ownership as the sole worker lifetime. +11.13. The implementation shall preserve canonical call/event validity, output commitment/no-retry-after-output, B2BUA lineage, secure-session ownership, generation pinning, and current billing settlement authorities. +11.14. Focused/repository tests shall run without external model credentials; live extractor tests are optional environment-gated evidence. +11.15. Final review shall remove duplicate rule catalogs, unnecessary persistence/framework layers, unbounded queues/goroutines, provider-specific core branches, and any redundant second LLM summary-rewrite pass. +11.16. Tests shall use distinct parent and child A-leg IDs and prove pending jobs, Await, merge, revision, late-result handling, and reinjection remain attached to the captured parent continuity BranchKey across concurrency and reload. +11.17. Reinjection tests shall cover two successive compaction boundaries with the same capsule revision, canonical validation failure, failed primary Open, aborted/no client release, then retry. The compound watermark shall advance only after successful final client release; pending injection shall survive earlier failures, and call-local retry/failover shall not duplicate insertion within one primary attempt lifecycle. diff --git a/.kiro/specs/compaction-continuity-preservation/research.md b/.kiro/specs/compaction-continuity-preservation/research.md new file mode 100644 index 00000000..77b4b3a2 --- /dev/null +++ b/.kiro/specs/compaction-continuity-preservation/research.md @@ -0,0 +1,460 @@ +# Research & Design Decisions + +## Summary + +Issue #344 is valid, but the safest implementation is not “run another summarizer and append whatever it returns.” Go-LIP already has most of the right architectural pieces: canonical requests/items, secure-session/B2BUA authority, process-owned extension state, auxiliary execution with principal propagation, generation pins, and BillingCallID-scoped billing. The missing pieces are narrow: + +1. the `compaction-event-detection` runtime capability must land and remain the single compaction-recognition authority; +2. preservation needs a **separate content-bearing interceptor** rather than mutating the metadata-only observer contract; +3. true background auxiliary execution needs a process-owned bounded scheduler that captures generation ownership at submit time; +4. the child needs typed **detached session semantics** so it is financially attributable to the user without becoming another primary conversation turn; +5. continuity must use a bounded structured capsule and only patch verified plaintext carriers; opaque/encrypted compaction state is reinjection-only. + +The resulting design is a parallel-first pipeline: deterministic plan harvest and source preparation are local; one independently routed semantic extractor job may run concurrently with the agent's own compaction; the main path only waits at a bounded preservation barrier when a result is actually required. + +## External Precedent + +The feature follows patterns already used by mature coding-agent harnesses: + +| Project | Relevant behavior | Design implication | +|---|---|---| +| Pi | default compaction summary explicitly carries goal, constraints/preferences, progress, key decisions, next steps, and critical context | decision state deserves first-class preservation rather than generic prose only | +| Pi | `session_before_compact` can replace/customize compaction and examples use a cheaper secondary model | pre-/around-compaction interception and independent model choice are established patterns | +| OpenAI Codex | compact prompt asks for progress/key decisions, user preferences/constraints, next steps, critical continuation data | preservation categories align with real agent continuation needs | +| OpenAI Codex | `update_plan` exposes structured plan state | deterministic plan harvest should precede semantic LLM inference | +| OpenCode | pre-summary compaction hook and rolling structured session checkpoint | structured cumulative state is preferable to repeatedly summarizing prior summaries | +| OpenCode | todo state is structured and appears in compaction/session representations | machine-readable plan progress can survive without LLM rediscovery | +| Codex issue #14347 | repeated compaction can lose historical decisions | repeated compactions require cumulative merge semantics, not recursive summary trust | + +References: + +- https://github.com/earendil-works/pi/blob/main/packages/coding-agent/src/core/compaction/compaction.ts +- https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/extensions.md +- https://github.com/earendil-works/pi/blob/main/packages/coding-agent/examples/extensions/custom-compaction.ts +- https://github.com/openai/codex/blob/main/codex-rs/prompts/templates/compact/prompt.md +- https://github.com/openai/codex/blob/main/codex-rs/core/src/tools/handlers/plan.rs +- https://github.com/anomalyco/opencode/blob/dev/packages/web/src/content/docs/plugins.mdx +- https://github.com/anomalyco/opencode/blob/dev/specs/v2/session.md +- https://github.com/anomalyco/opencode/blob/dev/packages/sdk/js/src/gen/types.gen.ts +- https://github.com/openai/codex/issues/14347 + +## Go-LIP Brownfield Findings + +### Compaction detection is designed but not implemented yet + +`.kiro/specs/compaction-event-detection/` defines: + +- canonical protocol/signature/history recognition; +- a process-owned detector keyed by authoritative A-leg; +- strict start only after successful upstream `Open`; +- completion at the final canonical release seam; +- a metadata-only, non-mutating `pkg/lipsdk/compaction.Observer`. + +There is no runtime `pkg/lipsdk/compaction` package on current `main`. Continuity preservation therefore has a hard implementation-order dependency on that spec. The correct response is to make #312 a prerequisite, not to recreate its signature table in #344. + +### Existing auxiliary execution already preserves user identity + +`pkg/lipsdk/auxiliary.Request` carries role, visibility, parent lineage, disabled plugin IDs, and a canonical child call. `internal/core/auxreq.Client`: + +- increments auxiliary depth; +- suppresses requested plugins; +- clones the parent principal/scope and marks `scope.OriginInternal`; +- creates an auxiliary trace ID and lineage extension; +- delegates to the ordinary runtime Executor. + +This is exactly the path we want for routing, attempts, usage, billing and provider calls. A direct HTTP/provider client would duplicate too many authorities. + +### But current auxiliary execution is synchronous + +`Aux.Collect` calls `Stream` and collects immediately. The client retains `genpin.KindAsync` when the child call starts. `genpin.Retainer` explicitly says a post-lease spawn attempt fails closed. + +Therefore this is unsafe: + +```text +request hook + -> go func() { Aux.Collect(parentCtx, ...) } // wrong +request returns / generation lease ends + -> goroutine starts later and tries to retain/execute +``` + +The fix is a submit-time handoff: resolve the current executor and retain `KindAsync` synchronously, then enqueue immutable child work to a process-owned scheduler. The worker owns its own context/deadline and releases the pin exactly once after terminal collection. + +### Worker ownership belongs to ProcessServices + +`ProcessServices` already owns process-lifetime mutable services, worker-like subsystems, stores and closers. `ExtensionState` is also process-owned and survives generation replacement. A feature `Lifecycle` alone is generation-composed and is not the right sole owner for work that may overlap a hot reload. + +A narrow `auxreq` background collector/scheduler under ProcessServices matches current ownership architecture and keeps the feature plugin away from Executor/provider internals. + +### Current auxiliary execution needs a detached-session policy + +The child delegates through the ordinary Executor. The primary prepare path performs secure-session BeginTurn, transcript/activity recording, A-leg route-override snapshotting and other user-turn semantics. + +`auxiliary.Request.Visibility` is currently only lineage metadata; it does not suppress those primary session effects. A continuity extractor must therefore use a typed internal execution/session mode that says: + +- preserve authenticated principal/scope; +- preserve parent IDs only as correlation; +- do not begin/append a primary secure-session turn; +- do not update primary last activity/turn count/transcript; +- do not apply the primary A-leg routing override as child route authority. + +This is execution metadata, not provider-visible prompt content. + +### Independent routing fits the canonical child call + +`lipapi.Call.Route.Selector` already expresses route intent. The extractor can set an operator-configured selector on its child call and then use the normal core planner, aliases, failover, capability checks and admission. + +This allows, for example: + +```text +primary: anthropic:claude-frontier-coding +extractor: openai-responses:small-fast-model +``` + +without any provider-specific code. Parent route overrides remain primary-session authority only. An explicit `inherit` option can be supported, but accidental inheritance is a bad default because it defeats the cost-control purpose of a dedicated extractor route. + +### Existing billing naturally supports originating-user attribution + +The authoritative billing identity adapter derives the account from `scope.PrincipalID`. Auxiliary execution already clones scope from the parent. If detached mode preserves principal/scope, the child can pass through the normal Executor and receive: + +- normal usage/concurrency admission; +- its own BillingCallID; +- its own B-leg attempts/failover; +- normal operational exposure and post-usage customer settlement; +- normal provider COGS processing. + +No new money path is necessary. The missing detail is classification: accounting/diagnostics should project a bounded `compaction_continuity_extractor` workload/origin so users/operators can distinguish auxiliary continuity cost from primary inference. That is metadata, not a new ledger/rating authority. + +Primary response protocol usage must remain the primary call's usage only. Account totals include both independent calls. + +### Process ExtensionState is useful but not restart-durable + +`ProcessServices.ExtensionState` is an in-memory process-owned store. `ScopeSession` partitions by `SessionView.PartitionKey()`, which chooses authoritative SessionID when available. This is suitable for: + +- current capsule revision; +- high-watermarks; +- bounded sanitized source window; +- pending background job ID; +- pending/injected revision watermarks. + +The feature key must additionally include authoritative A-leg/branch identity because session partition alone is not a branch key. + +This state survives generation reload, but not process restart. The spec should say so. When secure-session transcript capture is already enabled, a narrow authorized transcript reader may reconstruct missing capsule state. Otherwise restart/resume is fail-open; this feature does not justify a second durable transcript or general durable plugin-state platform. + +### Native compaction is often opaque + +`lipapi.CompactionItem` carries: + +- `EncapsulatedID`; +- `Dialect` / `Implementor`; +- `EncryptedContent`; +- `Opaque` provider data. + +There is no universal plaintext “summary” property. Editing encrypted/opaque state could break replay or provider protocol semantics. + +Therefore: + +- result-side augmentation is allowed only when the path exposes a verified mutable plaintext summary/continuation carrier; +- otherwise the validated capsule remains proxy-owned state and is injected into the first eligible post-compaction request; +- opaque/encrypted content is an exact-preservation invariant. + +## Decision D1 — Treat #312 as a hard prerequisite and share one recognizer + +Implementation should first land `compaction-event-detection` (or implement it in chronological dependency order before this feature). The detector's rule table remains the only compaction signature authority. + +Continuity may need a pure request **preview** so the first post-compaction turn can be protected before B-leg Open. That preview should be factored from the same matcher/fingerprint logic but must not mutate detector state or emit lifecycle events. Committed start/completion still follows #312 semantics. + +## Decision D2 — Add a separate compaction preservation interceptor + +Do not mutate `compaction.Observer`. + +Add a distinct additive FeatureBundle contribution (name indicative): + +```go +type Preserver interface { + ID() string + BeforeRequest(ctx context.Context, call *lipapi.Call, meta RequestMeta, preview RequestPreview, svc Services) error + RequestOpened(ctx context.Context, call lipapi.Call, meta OpenMeta, derived []Event, svc Services) error + BeforeResponseRelease(ctx context.Context, ev *lipapi.Event, meta ResponseMeta, derived []Event, svc Services) error +} +``` + +Exact method names may differ, but the semantic stages are deliberate: + +- **BeforeRequest:** pre-open only; check pending reinjection, protect completion-only first turn, never emit detector truth. +- **RequestOpened:** successful-open only; commit sanitized source and start one background semantic job for a real strict compaction transaction. +- **BeforeResponseRelease:** final selected event; bounded join and verified plaintext augmentation/fallback marker before metadata observers/client release. + +The slice is merged/frozen separately from `CompactionObservers`. + +## Decision D3 — Add a narrow process-owned background auxiliary collector + +Add an additive auxiliary capability rather than expanding `Client` into a task framework. Conceptually: + +```go +type JobID string + +type BackgroundClient interface { + SubmitCollect(ctx context.Context, req Request, opts SubmitOptions) (JobID, error) + Await(ctx context.Context, id JobID) (lipapi.Collected, error) + Forget(id JobID) +} +``` + +`SubmitCollect` must synchronously: + +1. validate/copy the canonical child request; +2. resolve the current Executor runner; +3. retain `genpin.KindAsync` while the request still has spawn authority; +4. clone only required principal/scope/correlation context; +5. enqueue into a bounded process scheduler. + +Workers execute with a scheduler-rooted context and configured timeout. A job registry supports coalescing and a bounded await barrier. Raw results have a short TTL and are forgotten after parse/validation. ProcessServices owns shutdown/cancel/join. + +This capability remains narrowly about background auxiliary model collection. It is not a durable queue, cron service, arbitrary function executor, or external workflow engine. + +## Decision D4 — Use a typed detached auxiliary execution mode + +Extend internal auxiliary request/execution policy with a typed detached-session setting (exact enum/name may differ). It shall: + +- retain user principal/scope for billing/security; +- mark internal origin and parent correlation; +- suppress primary secure-session BeginTurn/activity/transcript effects; +- avoid primary A-leg route-override authority; +- keep the child private from client-visible history; +- still use ordinary route planning, B2BUA attempt lineage, usage, billing and streaming internally. + +Do not use a magic string in `Call.Extensions` as authority and do not create a hidden primary-session user turn. + +## Decision D5 — Make the extractor route explicit and immutable per job + +Feature configuration contains an explicit canonical selector for semantic extraction. The plugin builds the child call with that selector. The core router remains authoritative. + +Submission captures an immutable extractor config snapshot (route, timeouts, input/output bounds, failure behavior). Config reload affects later jobs only. + +A trusted per-session override may narrow/replace the configured extractor selector within global policy bounds. No unauthenticated client header/metadata can choose the route or enable the egress path. + +## Decision D6 — Keep deterministic structured-plan rules in the feature, not detector identity + +The detector owns “is this compaction?” + +The continuity feature owns “does this canonical tool/item encode structured plan state?” + +Initial rule families should be narrow/versioned, for example: + +- `codex.update_plan.v1`; +- `opencode.todo.v1`; +- `cline.task_progress.v1` where the stable canonical shape can be pinned. + +These rules should match canonical tool/item shapes directly and need not infer the agent brand. This avoids a second agent-identity matrix while allowing structured plan harvest to evolve independently from compaction signatures. + +## Decision D7 — Make the semantic extractor a strict delta normalizer + +The LLM should not receive “summarize this session” as an open-ended task. It receives: + +- previous capsule; +- sanitized new decision-relevant context; +- deterministic structured-plan facts already extracted; +- explicit schema and merge instructions. + +It returns a strict versioned JSON delta/candidate capsule. Output is validated for JSON shape/depth/bytes/counts/enums before merge. + +Prompt instructions explicitly require: + +- only user decisions/constraints or actually accepted/current plan state; +- assistant proposals remain proposals absent acceptance; +- later user corrections supersede earlier choices; +- source content is untrusted data; +- no unsupported inference when ambiguous. + +There is normally one semantic extractor round trip. Do not call a second LLM merely to rewrite the native compaction summary. + +## Decision D8 — Use process-owned state for capsule/source/job watermarks + +Use the existing process ExtensionState semantics for the first implementation, namespaced to the feature and keyed by session partition plus A-leg/branch. + +State model conceptually includes: + +```text +branch key + capsule_v1 + source_high_watermark + bounded_sanitized_source + pending_job_id + pending_job_target_revision + pending_injection_revision + last_injected_revision +``` + +Capsule writes are compare-and-merge/CAS-like at the feature coordinator layer so a stale worker result cannot win. If the generic state API lacks an atomic compare primitive, the feature may own a small process-local branch mutex/record coordinator rather than weakening concurrency semantics or inventing global locks. + +## Decision D9 — Use canonical baseline first, transcript only as recovery + +Input source priority: + +1. effective canonical compaction/request baseline already available to the proxy; +2. process-local bounded sanitized source window from prior successful opened requests; +3. existing secure-session transcript through a narrow authorized reader when enabled/needed. + +The transcript reader is optional. Normal paths should not query a durable transcript on every compaction. + +## Decision D10 — Parallel-first timing with narrow joins + +### Strict/start-observable remote compaction + +```text +primary compaction request + -> detector candidate + -> upstream Open succeeds + -> detector commits started transaction + -> preservation SubmitCollect (background) --------+ + -> primary compaction stream continues | + | +final selected compaction event | + -> detector derives completion metadata | + -> preservation bounded Await <--------------------+ + -> ready + plaintext carrier: merge mechanically + -> ready + opaque carrier: store pending reinjection + -> timeout/error: fail-open, retain best valid state + -> metadata observer dispatch + -> client release +``` + +### Completion-only/local compaction first seen on next request + +```text +next request before backend Open + -> pure detector preview says likely installed compaction + -> preservation checks capsule/pending job + -> if needed: background SubmitCollect from previous sanitized source + -> bounded Await barrier + -> inject ready capsule or fail-open + -> normal Open + -> detector commits completion only after successful Open +``` + +The extractor is still off-session/background in both cases. The difference is whether useful parallel time existed before the first turn that needs the result. + +## Decision D11 — Reinjection is the universal safe fallback + +When native compaction content is opaque, store the capsule and inject a bounded proxy-owned continuity block into the first eligible post-compaction request. + +Injection must respect canonical authority: + +- message-authoritative call -> proxy-owned instruction/developer/system message representation allowed by current canonical contract; +- item-authoritative call -> canonical message item representation, not simultaneous legacy `Instructions`/`Messages` that would violate `Call.Validate`. + +Use one helper that preserves call validity rather than protocol-specific encoders. + +A revision watermark prevents duplicate injection. + +## Decision D12 — User billing is normal billing, not special charging code + +The detached child retains the original `scope.PrincipalID`, so ordinary billing resolves the same account. The child gets a separate BillingCallID and per-B-leg records. + +Add only a content-free workload/origin projection such as: + +```text +workload = auxiliary +aux_role = compaction_continuity_extractor +parent_session_id_hash / parent_trace correlation as allowed +``` + +This lets reports distinguish continuity overhead while the money ledger/journal remains unchanged. + +If the child is denied before provider submission, preservation fails open by default. If provider work was submitted, usage remains billable even if the result is discarded later. + +## Decision D13 — Global config plus trusted per-session narrowing/override + +Conceptual feature config: + +```yaml +# exact nesting follows standard feature-plugin opaque config conventions +compaction_continuity: + enabled: false + preserve: + plan: true + user_decisions: true + constraints: true + rationale: true + rejected_alternatives: true + extractor: + route: "openai-responses:small-model" + timeout: 8s + max_input_tokens: 12000 + max_output_tokens: 2000 + max_concurrency: 2 + queue_capacity: 16 + result_ttl: 30s + barrier_timeout: 2s + max_capsule_tokens: 2500 + source_ttl: 2h + failure_mode: fail_open +``` + +Values above are examples, not normative defaults. Implementation tests should pin chosen defaults/validation. + +Per-session overrides come only from trusted proxy-owned policy/session state and cannot exceed global egress/resource maxima unless explicitly authorized. + +## Decision D14 — Do not claim restart durability in v1 + +The first implementation guarantees continuity across compaction and generation reload **within the process**. It does not create a new durable capsule database. + +If an authorized secure-session transcript exists after restart, the capsule can be reconstructed on demand. Otherwise the feature resumes without prior process-only capsule state and fails open. This is truthful and materially simpler than introducing a new durable state subsystem for a UX enhancement. + +## Rejected Alternatives + +### Run extractor inline on the main request goroutine + +Rejected. It violates the off-session/background requirement and adds full extractor latency directly before native compaction. + +### Fire-and-forget goroutine calling current `Aux.Collect` + +Rejected. It can lose generation spawn rights, inherit canceled request lifetime, leak on shutdown, and has no bounded admission/result ownership. + +### Direct provider/HTTP client for the extractor + +Rejected. It would bypass routing, principal scope, usage authority, billing, B2BUA attempts, retries and current provider abstractions. + +### Put extractor call into the primary session as another assistant/user turn + +Rejected. It pollutes conversation state, can influence agent behavior, changes transcript/turn semantics and makes auxiliary billing indistinguishable from primary inference. + +### Append text to every `CompactionItem` + +Rejected. Native compaction may be encrypted/opaque provider state and does not expose a universal safe text carrier. + +### Second LLM call to rewrite/improve native summary + +Rejected by default. The validated continuity capsule is already the semantic result; deterministic merge/reinjection is cheaper and more reliable. + +### Re-send the entire raw session on every compaction + +Rejected. It creates O(session age) cost, unnecessary privacy egress and prompt-injection surface. Use delta + prior capsule + bounded recovery sources. + +### Build a second durable transcript/capsule/job database now + +Rejected. Existing secure transcript can be used when authorized; process state is sufficient for the primary compaction UX objective. A general durable job/state system is disproportionate. + +### Use the #312 history heuristic as semantic truth about accepted plans + +Rejected. Compaction detection and user-decision semantics are different problems. Heuristics may decide when to call the semantic extractor, not manufacture user decisions. + +## Main Risks and Mitigations + +| Risk | Mitigation | +|---|---| +| semantic extractor hallucinates acceptance | strict schema/provenance, deterministic merge, explicit-user precedence, ambiguity omission | +| extra user cost | deterministic-first/eligibility gate, independently cheap route, visible auxiliary cost classification | +| worker leaks across reload/shutdown | ProcessServices ownership, submit-time KindAsync pin, bounded scheduler, race/goleak tests | +| extractor changes primary session state | typed detached-session execution mode and session/transcript regression tests | +| stale job overwrites correction | branch revision/high-watermark compare-and-merge | +| opaque compaction corrupted | exact byte-preservation invariant; reinjection fallback | +| first post-compaction turn races job | pre-open pure preview + bounded Await barrier | +| privacy egress of tool/file dumps | canonical sanitizer, redaction, excluded external/tool data by default | +| duplicate billable jobs on retry | transaction/revision coalescing key | +| config reload changes running job unexpectedly | immutable submission snapshot + retained generation | +| restart loses process capsule | truthful v1 contract; authorized transcript reconstruction only when available | + +## Research Conclusion + +The feature has high UX value and fits Go-LIP's current architecture if implemented as **structured process continuity + a narrow background auxiliary capability**, not as a generic memory subsystem or compactor rewrite. The highest-risk brownfield work is not the LLM prompt; it is lifecycle/session/accounting correctness around a detached asynchronous child call. The design and tasks should therefore put RED tests around those boundaries before implementing semantic extraction details. diff --git a/.kiro/specs/compaction-continuity-preservation/spec.json b/.kiro/specs/compaction-continuity-preservation/spec.json new file mode 100644 index 00000000..b8068c2d --- /dev/null +++ b/.kiro/specs/compaction-continuity-preservation/spec.json @@ -0,0 +1,23 @@ +{ + "feature_name": "compaction-continuity-preservation", + "created_at": "2026-08-17T14:48:00+02:00", + "updated_at": "2026-08-17T14:48:00+02:00", + "language": "en", + "phase": "tasks-generated", + "approvals": { + "requirements": { + "generated": true, + "approved": false + }, + "design": { + "generated": true, + "approved": false + }, + "tasks": { + "generated": true, + "approved": false + } + }, + "ready_for_implementation": false, + "project_description": "Preserve the latest accepted plan, explicit user decisions, constraints, rationale, meaningful rejections, and remaining work across lossy coding-agent compaction. Reuse the compaction-recognition authority specified by compaction-event-detection without weakening its metadata-only observer contract. Maintain a bounded revisioned continuity capsule, harvest structured plan state deterministically, and use an optional independently routed background auxiliary model only for unstructured semantic extraction. Auxiliary extraction is off the primary agent session, has process-owned bounded worker lifetime and narrow synchronization barriers, and is normal billable model usage attributed by default to the originating authenticated user/account. Preserve existing routing, B2BUA, secure-session, billing, usage, streaming, and provider-opaque compaction semantics; add no second transcript database, provider client path, hidden system billing account, generic task runtime, or mutation of encrypted/opaque compaction payloads." +} diff --git a/.kiro/specs/compaction-continuity-preservation/tasks.md b/.kiro/specs/compaction-continuity-preservation/tasks.md new file mode 100644 index 00000000..fe15c012 --- /dev/null +++ b/.kiro/specs/compaction-continuity-preservation/tasks.md @@ -0,0 +1,420 @@ +# Implementation Plan + +## Execution Rules + +- Follow TDD strictly: characterization/RED tests and contract gates precede production implementation in each dependency layer. +- `compaction-event-detection` runtime is a hard prerequisite. Do not duplicate its signature matrix if it is not yet implemented; land that spec first. +- Keep every task independently reviewable with **no more than five concrete actions**. +- Preserve existing routing, B2BUA, secure-session, generation pinning, output commitment, usage authority, BillingCallID accounting, and provider-opaque compaction semantics. +- Auxiliary semantic extraction is a separate off-session background model call, independently routed and billed to the originating user/account by default; no new billable continuity child may be submitted before the triggering primary request successfully opens. +- Capture the parent continuity BranchKey before detached child A-leg creation; child A-leg identity is execution/billing lineage only and never continuity-state authority. +- Do not add provider-specific continuity branches, a direct provider client, second transcript store, feature-owned money ledger, generic task/workflow engine, or second LLM summary-rewrite pass. +- No real model credentials are required for correctness tests; use deterministic local/fake backends and fixture JSON. + +## Phase 1 — Freeze Contracts and Failure Boundaries With RED Tests + +### Task 1.1 — Freeze prerequisite detector preview and final-release semantics + +- Verify the #312 runtime detector exists; if absent, stop this spec's implementation and complete `compaction-event-detection` first rather than adding fallback signatures. +- Add RED tests for pure `PreviewRequest` and `PreviewResponse` sharing the same rule/fingerprint authority as committed detection without mutating detector state or emitting lifecycle events. +- Prove a strict or completion-only preview on a request that never opens cannot create a billable semantic extraction submission; completion-only may create only a bounded non-billable preview intent. +- Prove committed `ResponseReleased` receives the exact event after permitted preservation finalization, not a pre-final copy. +- Add near-miss and completion-only preview fixtures, including stable preview boundary/fingerprint identity when no committed transaction exists. + +_Requirements: 1.1–1.13, 4.10, 7.1, 7.12, 11.3, 11.10_ + +_Validation: focused `internal/core/compactiondetect` RED tests; no duplicated rule catalog._ + +### Task 1.2 — Freeze the separate preservation extension contract + +- Add RED SDK tests for a content-bearing `compaction.Preserver`-style contract that is distinct from metadata-only `compaction.Observer`. +- Add RED FeatureBundle/merge/runtime-snapshot tests for ordered additive preservers, defensive copies, nil validation and frozen generation semantics. +- Prove ordinary compaction observer event payloads still contain no canonical request/response content and cannot mutate traffic. +- Freeze preservation callback ordering: BeforeRequest -> successful-open callback -> response-preview finalization -> committed detector observation -> metadata observer dispatch. +- Add RED callback error/panic tests proving BeforeRequest/BeforeResponseRelease mutations roll back to the exact prior canonical object and all preserver failures remain feature-local fail-open rather than primary provider/retry errors. + +_Requirements: 1.2–1.5, 1.9–1.11, 7.16, 10.4–10.5, 10.13, 11.12–11.13_ + +_Validation: `go test ./pkg/lipsdk/... ./internal/featurebundle/... ./internal/core/extensions/...` is RED for the new contract._ + +### Task 1.3 — Freeze background auxiliary scheduling and generation ownership + +- Add deterministic RED scheduler tests for bounded queue/concurrency, committed-transaction coalescing, result retention, Await/Forget, and saturation without goroutine fallback. +- Prove `SubmitCollect` rejects/never uses an empty transaction-derived continuity coalescing key, retains `genpin.KindAsync` synchronously while the parent spawn right is live, and a worker never retains a replacement generation later. +- Add enqueue-failure/cancel/timeout/shutdown paths proving every retained pin is released exactly once. +- Add a delayed-start test where the parent request context is canceled before the worker runs but the worker uses its own bounded context safely. +- Add `goleak` and race tests for queue saturation, shutdown, concurrent Await/Forget and late job completion. + +_Requirements: 4.4–4.15, 8.9–8.10, 11.2, 11.11_ + +_Validation: focused `internal/core/auxreq` tests are RED and deterministic._ + +### Task 1.4 — Freeze detached child A-leg, routing, and session isolation + +- Add RED tests proving a detached child creates a private auxiliary A-leg and never reuses the parent's A-leg as execution authority. +- Prove parent SessionID/A-leg/trace survive only as lineage for child execution while the separately captured parent BranchKey remains continuity-state authority and primary secure-session BeginTurn/transcript/activity/turn count remain unchanged. +- Prove the child uses its explicit extractor selector and a primary A-leg runtime route override cannot rewrite it. +- Prove detached mode is trusted auxiliary metadata unavailable from frontend/wire canonical fields. +- Cover private child B-leg failover/terminal lineage without client-visible session headers/history. + +_Requirements: 4.16, 5.1–5.15, 8.1–8.3, 9.11, 11.4–11.5, 11.16_ + +_Validation: runtime/auxreq/secure-session/B2BUA focused tests are RED._ + +### Task 1.5 — Freeze capsule semantics, billing attribution, and repeated-compaction outcomes + +- Add RED table tests for capsule branch binding/digest, revision, decision conflict-key/supersession precedence, rejection, plan-step progress, stale merge rejection and deterministic pruning. +- Add RED structured carrier fixtures for Codex update-plan, OpenCode todo and supported Cline task-progress shapes plus near misses. +- Add RED billing tests requiring originating-account attribution, separate BillingCallID/B-legs, positive AttemptSeq, valid usage/cost/Source/Authority/DedupeKey evidence, retry/failover accounting and auxiliary workload classification. +- Prove primary protocol-visible usage excludes extractor tokens while account/operator totals include the independent child usage and an `AttemptSeq <= 0` independent leg is rejected rather than silently counted. +- Add an end-to-end RED scenario with at least three compactions, user decision correction, completed/pending plan steps, same-revision distinct-boundary reinjection, dedupe and bounded capsule growth. + +_Requirements: 2.1–2.17, 3.1–3.7, 6.1–6.14, 7.11, 8.4–8.12, 11.6–11.8, 11.17_ + +_Validation: feature/billing/integration target tests are RED before production feature code._ + +## Phase 2 — Implement Minimal Shared Infrastructure + +### Task 2.1 — Implement pure detector request/response previews + +- Refactor prerequisite detector match/fingerprint code so preview and committed paths call one shared internal recognition authority. +- Implement `PreviewRequest` without state mutation, exposing stable completion-only boundary/fingerprint identity when no transaction exists, and preserve existing successful-Open commit behavior in `RequestOpened`. +- Implement `PreviewResponse` without completion mutation and keep `ResponseReleased` as the committed final-event boundary. +- Make Task 1.1 positives/near misses/state-snapshot assertions green without changing observer payloads. +- Keep unsupported protocol controls and provider DTOs outside the detector. + +_Requirements: 1.1, 1.5–1.13, 4.10_ + +_Design: D1; D13–D14_ + +_Validation: focused detector suites green; existing #312 tests remain unchanged/green._ + +### Task 2.2 — Implement and compose the preservation SDK surface + +- Add the separate preservation types/services/interfaces beside the existing compaction observer contract. +- Add `CompactionPreservers` to FeatureBundle, one merge surface and frozen request-snapshot accessor with existing validation conventions. +- Add core extension dispatch helpers that isolate preserver panic/error and record content-free failure outcomes without propagating them into primary traffic. +- Make request/event preservation mutation transactional with a bounded clone/undo helper so callback failure, panic, or canonical validation failure restores the pre-preservation object; keep ordinary response hooks before finalization. +- Make Task 1.2 SDK/merge/order/rollback tests green without introducing a generic transaction or mutating StageID framework. + +_Requirements: 1.2–1.4, 1.10–1.11, 7.16, 10.3–10.5, 10.13_ + +_Design: D2_ + +_Validation: SDK/featurebundle/extensions tests green._ + +### Task 2.3 — Implement the process-owned BackgroundAux collector + +- Add additive BackgroundClient/JobID/options APIs while keeping synchronous `auxiliary.Client` source-compatible. +- Implement the bounded `internal/core/auxreq` scheduler with committed keyed coalescing, worker pool, result registry and process-root context; do not admit pre-open preview intents as jobs. +- Implement submit-time runner capture/KindAsync retention and exact cleanup on enqueue failure, terminal completion, cancellation and shutdown. +- Implement useful bounded pending-result retention plus Await/Forget without arbitrary callbacks or durable task semantics. +- Register scheduler ownership/Close through ProcessServices and make Task 1.3 green under race/goleak. + +_Requirements: 4.4–4.15, 8.9–8.10, 10.11–10.12_ + +_Design: D3; D22_ + +_Validation: focused auxreq/runtimebundle tests green; scheduler has no network dependency._ + +### Task 2.4 — Implement detached auxiliary execution with a private child A-leg + +- Add trusted internal detached-session policy on auxiliary execution without adding a frontend/wire `lipapi.Call` control. +- Factor Executor preparation so detached calls preserve principal/scope but skip primary secure-session BeginTurn/activity/transcript/resume effects. +- Create/touch a private child A-leg through existing B2BUA lifecycle, keep parent A-leg/session values as lineage only for child execution, and preserve the separately captured parent branch binding untouched. +- Ensure detached route planning does not read the parent's route override and ordinary child request authority/B-legs still execute. +- Make Task 1.4 session/B2BUA/routing tests green without copying the Executor. + +_Requirements: 4.16, 5.5–5.15, 9.11, 11.4–11.5, 11.13, 11.16_ + +_Design: D4–D5_ + +_Validation: focused runtime/auxreq/B2BUA/secure-session tests green._ + +### Task 2.5 — Implement the process branch coordinator + +- Add a narrow ProcessServices-owned coordinator keyed by the authoritative **parent** SessionID + primary A-leg/branch or principal-isolated primary A-leg fallback; capture this key before detached child A-leg creation. +- Serialize branch revision/high-watermark/pending-job, non-billable preview-intent -> committed-transaction binding, and compound injection-watermark updates using process ExtensionState as opaque backing where practical. +- Require pending JobID/branch-binding consistency for Await/merge/revision/reinjection and never derive continuity keys from the private child A-leg. +- Enforce max entries/TTL/lazy cleanup and never call model/provider/plugin code while coordinator locks are held. +- Add reload/concurrency tests with distinct parent/child A-legs proving old/new generations/workers cannot overwrite or move capsule state to the auxiliary branch. + +_Requirements: 4.10, 4.16, 7.11–7.12, 8.1–8.15, 10.12, 11.11–11.12, 11.16–11.17_ + +_Design: D7; D14–D15; D22–D23_ + +_Validation: coordinator unit/race tests green and no generic transactional state framework appears._ + +## Phase 3 — Implement the Continuity Feature Semantics + +### Task 3.1 — Implement feature configuration and prerequisite validation + +- Register the official `compaction-continuity` feature and decode bounded preserve/extractor/worker/barrier/capsule/source/result/failure settings. +- Require explicit extractor route or explicit inherit when semantic extraction is enabled; feature remains disabled by default and shipped examples stay `enabled: false`. +- Validate finite positive maxima and consistency of pending-result/source/branch retention. +- Fail enabled generation/startup composition clearly when detector preview/commit, BranchCoordinator or BackgroundAux services are absent. +- Add config reload tests proving in-flight jobs retain immutable submission-time config while new jobs use new generation config. + +_Requirements: 9.1–9.2, 10.1–10.3, 10.9–10.12_ + +_Design: Dependency Gate; D17–D19_ + +_Validation: config/runtimebundle feature tests green._ + +### Task 3.2 — Implement Continuity Capsule v1 and deterministic carrier rules + +- Implement the versioned capsule envelope with parent `branch_binding`, canonical `content_digest`, revision/high-watermark, fact/plan types, stable IDs, decision `conflict_key`/`supersedes`, source authority/status enums and strict validation. +- Implement canonical digest encoding/verification plus deterministic conflict precedence, stale-revision rejection, at-most-one-active-decision-per-conflict-key, dedupe and whole-fact bounded pruning/re-digest. +- Add versioned canonical carrier rules for the researched Codex/OpenCode/Cline plan shapes without agent-brand inference. +- Normalize supported carrier updates into capsule plan state without an LLM call while preserving parent branch binding. +- Make Task 1.5 capsule/carrier tests green including branch mismatch/digest mismatch/unknown supersedes/malformed/near-miss fixtures. + +_Requirements: 2.1–2.17, 3.1–3.5, 8.1, 8.4, 8.11–8.12_ + +_Design: D7–D9_ + +_Validation: feature pure-unit tests green._ + +### Task 3.3 — Implement sanitized incremental source preparation and eligibility + +- Walk canonical calls into a bounded source envelope prioritizing user decisions, relevant assistant planning and recognized plan carriers. +- Drop/truncate ordinary tool results, logs, file/code dumps, media/binaries, unnecessary reasoning and unrelated external content. +- Mark any narrowly retained external/tool material as untrusted data and apply existing redaction/secret treatment before egress. +- Implement local semantic-eligibility heuristics that only decide whether to pay for extraction and never establish accepted intent themselves. +- Commit sanitized source/high-watermark only after successfully opened primary requests and make zero-call/no-candidate tests green. + +_Requirements: 3.6–3.16, 9.3–9.10_ + +_Design: D10; D12_ + +_Validation: sanitizer/eligibility privacy fixtures green with no external calls._ + +### Task 3.4 — Implement the semantic extractor child call and strict result parser + +- Build one detached no-tools auxiliary child using the configured independent selector, prior locally validated capsule semantic payload, deterministic plan facts and sanitized delta. +- Suppress the continuity plugin and stamp private role/origin/parent lineage without primary session authority or raw BranchKey exposure. +- Define the fixed extraction prompt/schema requiring explicit-user precedence, accepted/current plan semantics, normalized decision conflict keys, validated supersedes references and ambiguity omission. +- Parse exactly one bounded JSON result with schema/base-revision/enums/depth/count/string/source-ref/conflict-key/supersedes validation; discard malformed/authority-escalating/cross-branch output. +- Add fake-backend tests proving no second LLM summary-rewrite call is generated. + +_Requirements: 2.6–2.7, 2.17, 3.8–3.13, 5.1–5.9, 9.4–9.6, 10.6, 11.15_ + +_Design: D5; D8; D11–D12_ + +_Validation: deterministic child-call/parser tests green._ + +### Task 3.5 — Integrate validated deltas with parent branch state and late results + +- On Await success, verify JobID parent branch binding, capsule branch binding/digest and base revision, then merge the extractor delta through BranchCoordinator into a new bounded/re-digested capsule revision on that parent branch. +- Reject/forget stale, digest-invalid, wrong-branch or conflict-invalid results without changing active decisions or reinjection watermarks. +- Keep PendingJobID/result useful across a timed-out response barrier until bounded next-turn consumption or coherent expiry. +- Immediately Forget raw collected output after validation/merge and store only normalized capsule/source/job metadata. +- Add concurrent explicit-user-correction versus late-worker tests with different parent/child A-legs proving newer intent wins and state never migrates to the child branch. + +_Requirements: 2.12–2.17, 4.13–4.16, 8.1, 8.4–8.5, 10.12, 11.16_ + +_Design: D3; D7–D8_ + +_Validation: branch/feature concurrency tests green under `-race`._ + +## Phase 4 — Integrate Preservation at Compaction Boundaries + +### Task 4.1 — Schedule extraction after a real strict compaction Open + +- At successful primary request Open, obtain committed detector events and invoke preservation with the effective canonical baseline/correlation and captured parent BranchKey. +- Commit the sanitized source and deterministic plan updates before deciding whether semantic extraction is necessary. +- Derive `H(parent_branch_binding | committed transaction | target source revision)` and submit at most one coalesced semantic job; never submit from an unopened strict preview or with an empty transaction ID. +- Let the primary compaction stream proceed immediately after submission without waiting for extractor completion. +- Cover retry/failover start dedupe, parent-vs-child branch identity, and failed-Open zero-billing behavior. + +_Requirements: 1.5–1.8, 3.13, 4.9–4.10, 4.16, 7.1–7.2, 11.3, 11.16_ + +_Design: D3; D13_ + +_Validation: runtime integration tests green with deterministic delayed extractor backend._ + +### Task 4.2 — Implement response-preview finalization and bounded completion barrier + +- Route each final selected event through pure detector `PreviewResponse` before committed `ResponseReleased`. +- Let preservation resolve/await the matching parent-branch job only for the configured bounded barrier and merge a valid ready result. +- Keep ordinary hooks/gates/finalizers before this stage and committed detector observation/metadata observers/client delivery after it. +- On timeout/error/panic/invalid preservation mutation, restore the pre-preservation event where needed, mark appropriate pending state and continue fail-open without blocking indefinitely. +- Prove committed detector and client both see the exact post-preservation or correctly restored final event. + +_Requirements: 1.10–1.11, 7.2–7.5, 7.16, 10.4–10.5, 11.10_ + +_Design: D1–D2; D13; D19_ + +_Validation: final-stream release/ordering/fail-open tests green across live/gated/recovery paths._ + +### Task 4.3 — Protect completion-only/local first post-compaction turns without pre-open billing + +- Before B-leg Open, use pure request preview to recognize installed/completion-only local compaction, capture the parent branch, and derive/store one non-billable preview-intent key from branch + detector preview boundary/fingerprint + target source revision without committing detector state or submitting BackgroundAux. +- Load/validate prior capsule/source, apply deterministic plan updates, and await only a matching job that was already submitted by an earlier successfully opened request; inject the best already-ready/deterministic capsule before Open when required. +- After successful primary Open, commit detector completion, bind the preview intent to the committed transaction/source revision, then submit one coalesced semantic job if eligibility still requires it; the new job improves later continuity rather than retroactively justifying pre-open provider work. +- Continue fail-open on preview/await/injection failure; if primary Open fails, retain required pending injection but discard/expire the non-billable intent and prove zero new child BillingCallID/B-leg/provider usage was created. +- Cover retries, distinct parent/child A-legs, reset/new-A-leg/near-miss cases so unrelated rewrites do not trigger extraction/injection or cross-branch state. + +_Requirements: 1.8–1.9, 4.10, 4.16, 7.1, 7.3–7.4, 7.8, 7.12–7.14, 8.1, 8.5–8.7, 11.3, 11.16_ + +_Design: D14_ + +_Validation: local-compaction history/integration/billing tests green, including failed-Open zero-child-work._ + +### Task 4.4 — Implement authority-aware boundary-scoped first-turn reinjection + +- Serialize a locally branch/digest-validated capsule deterministically into one versioned/delimited bounded continuity block and inject through a canonical helper that preserves message/item authority and `Call.Validate` invariants. +- Track `PendingInjection{BoundaryKey,CapsuleRevision}` plus `LastReleasedInjection{BranchBinding,BoundaryKey,CapsuleRevision}`; use a call-local marker only to prevent duplicate insertion inside one primary retry/failover lifecycle. +- On callback error/panic, validation failure, or primary Open failure, restore the pre-injection call and keep pending injection unchanged; bind a completion-only preview boundary to its committed transaction after successful Open. +- Advance `LastReleasedInjection` and clear matching pending state only after successful final client release so a failed/aborted turn retries, while a later distinct compaction boundary may reinject the same capsule revision. +- Add legacy-message/item-authority tests plus same-revision-two-boundaries, validation-failure, failed-Open, aborted-release-then-retry, retry/failover and reload idempotency tests. + +_Requirements: 7.7–7.11, 7.16, 8.1–8.5, 10.5, 11.13, 11.17_ + +_Design: D2; D15_ + +_Validation: lipapi/runtime feature tests green for both canonical authorities and all reinjection commit/failure points._ + +### Task 4.5 — Implement verified plaintext augmentation and opaque exact-preservation fallback + +- Define the minimal verified plaintext continuation-carrier capability/matcher; default unknown/native compaction paths to no response mutation. +- Mechanically merge the ready capsule projection only on an allowed plaintext carrier; never use another LLM to rewrite it. +- Mark boundary/revision-scoped pending first-turn reinjection whenever carrier is opaque/unsupported or extractor result is not ready. +- Add exact byte comparisons for `CompactionItem.EncryptedContent`, `Opaque`, signatures and unknown extension blobs with feature enabled/disabled. +- Prove reinjection alone preserves continuity when result augmentation is unavailable and a later boundary can reuse the same capsule revision safely. + +_Requirements: 7.5–7.7, 7.11, 7.15, 11.9, 11.15, 11.17_ + +_Design: D15–D16_ + +_Validation: opaque/native protocol fixtures green with byte identity._ + +## Phase 5 — Complete Billing, Policy, Privacy, and Observability + +### Task 5.1 — Project auxiliary workload identity through normal billing/metering + +- Reuse existing auxiliary lineage as the source of a bounded workload class/role in usage, metering, billing and report correlation. +- Keep pricing/rating behavior unchanged unless existing operator policy explicitly differentiates the selected route/model. +- Prove every auxiliary/failover B-leg has its own BillingCallID association, positive authoritative AttemptSeq, usage/cost presence evidence, Source, Authority and DedupeKey required by the existing independent terminal-accounting boundary while account identity remains the originating principal. +- Cover child failover, `AttemptSeq <= 0` rejection, pre-submit credit rejection and submitted-but-discarded result accounting without losing provider COGS/usage obligations. +- Make Task 1.5 billing tests green without adding a feature-owned financial store or alternate settlement path. + +_Requirements: 6.1–6.14, 10.8, 11.6–11.7_ + +_Design: D6_ + +_Validation: focused billing/metering/report tests green._ + +### Task 5.2 — Certify primary protocol-usage and secure-session isolation + +- Prove primary frontend usage events/totals contain only primary call usage even when the extractor runs concurrently. +- Prove account/operator totals include both primary and auxiliary records and can distinguish continuity workload. +- Prove detached child causes no primary session transcript entry, TurnID increment, last-activity mutation, resume effect or client session header. +- Prove private child A-leg/attempt lineage remains available for operator/accounting diagnostics while continuity state remains keyed by the captured parent BranchKey. +- Cover cancellation/timeout/failover paths for both parent and child without cross-settlement or cross-branch merge. + +_Requirements: 4.1–4.3, 4.16, 5.10–5.15, 6.4–6.9, 11.4, 11.6, 11.16_ + +_Validation: frontend/session/B2BUA/billing integration tests green._ + +### Task 5.3 — Implement trusted per-session policy and egress controls + +- Resolve effective continuity policy as operator hard maxima > trusted session values > global feature defaults. +- Allow only explicitly approved per-session enable/category/route/tighter-limit overrides; reject/ignore unauthenticated client attempts as specified. +- Apply existing redaction/secret treatment before semantic child egress and preserve tenant/workspace authorization on any optional transcript read. +- Keep transcript-disabled sessions from acquiring a hidden durable transcript, isolate bounded source by parent branch key, and keep raw BranchKey identifiers out of remote extractor input unless independently required/authorized. +- Add adversarial prompt-injection/secret/tool-output fixtures proving source text cannot override extractor instructions or leak excluded payloads. + +_Requirements: 2.16, 9.1–9.13, 10.1–10.2_ + +_Design: D8; D17; D20_ + +_Validation: feature/security/session-policy tests green._ + +### Task 5.4 — Implement failure handling and content-free observability + +- Add metrics for previews/events, preview intents, carrier hits, eligibility skips, job queue/outcomes, token usage, capsule digest/size/revisions, barriers, augmentation/reinjection/watermark outcomes, accounting rejection and preserver callback failures. +- Log only bounded IDs/hashes/status/counts and never extractor prompt/output/capsule text or raw BranchKey identifiers. +- Implement explicit fail-open handling for preserver callback error/panic/rollback, queue saturation, generation retain failure, child admission denial, provider failure, invalid schema/digest/conflict, stale result and barrier timeout. +- Ensure disable/reload stops new jobs but preserves bounded completion/accounting of already-submitted work and coherently expires non-billable preview intents. +- Add tests proving no failure path spins/retries indefinitely, chooses an unconfigured model, changes primary retry authority, or blocks shutdown indefinitely. + +_Requirements: 7.16, 9.6–9.7, 10.3–10.13_ + +_Design: D19–D21_ + +_Validation: metrics/log/failure-path tests green._ + +### Task 5.5 — Document operator behavior, billing cost, and durability limits + +- Document disabled-by-default feature enablement, independent extractor selector/model, worker/barrier/capsule bounds and fail-open semantics in standard configuration/operator docs. +- State explicitly that extractor inference is additional billable user-attributed usage, that no fresh child work starts before successful primary Open, and show how auxiliary cost is distinguished from primary inference. +- Document remote history egress/privacy implications and that the extractor is off-session/no-tools. +- Document v1 process/generation durability versus process-restart limitations and optional authorized transcript reconstruction behavior. +- Document #312 prerequisite and troubleshooting for missing detector/background services, callback fail-open, or rejected extractor billing/admission. + +_Requirements: 6.12–6.13, 7.1, 8.13–8.15, 9.1–9.2, 10.1–10.5_ + +_Validation: docs/config examples pass repository docs/config validation._ + +## Phase 6 — Certify Repeated Compaction, Concurrency, ROI, and Simplicity + +### Task 6.1 — Run the full repeated-compaction semantic matrix + +- Exercise at least three successive compactions with accepted plan, product decisions, constraints, rationale, rejections and open questions. +- Change a decision after the first compaction and prove the older value is deterministically superseded by conflict key/validated supersedes and remains non-active through later compactions. +- Advance plan steps through pending -> in-progress -> completed and prove pending/current work survives while completed history can be pruned and the capsule digest revalidates after each revision. +- Exercise deterministic-only, semantic-extractor and mixed paths and prove no duplicate model call when deterministic state is sufficient. +- Verify capsule/prompt size remains bounded and two opaque compaction boundaries can reinject the same capsule revision once per boundary without monotonically duplicated facts/injection blocks. + +_Requirements: 2.6–2.17, 3.5–3.7, 7.11, 8.10–8.12, 11.8, 11.17_ + +_Validation: deterministic multi-compaction integration suite green._ + +### Task 6.2 — Certify concurrency, generation reload, parent-branch ownership, and stale-result safety + +- Race concurrent primary turns, late extractor completion and explicit user correction on one parent branch; newer explicit intent must win. +- Reload to a new immutable generation while an extractor job is queued/running and prove BranchCoordinator/job ownership survives with the original captured parent branch binding despite a distinct child A-leg. +- Change/disable extractor config across reload and prove old jobs retain captured route/budgets while new jobs use new policy or do not submit. +- Exercise branch reset/new A-leg/fork-no-parent and prove no capsule/job/preview-intent/injection leakage between branches. +- Verify pending result/job/preview-intent expiry coherently clears BranchState without stale injection or child-branch migration. + +_Requirements: 4.13–4.16, 8.1–8.10, 10.9–10.12, 11.11, 11.16_ + +_Validation: focused `-race` reload/concurrency suite green._ + +### Task 6.3 — Certify worker/resource shutdown and leak freedom + +- Run queue saturation, worker timeout, client cancellation, process shutdown and late completion under `-race` and goleak. +- Assert every successful background submission retains/releases exactly one `KindAsync` pin, non-billable preview intents retain none, and no job starts after scheduler close linearizes. +- Assert no provider/model call or plugin callback runs while scheduler/BranchCoordinator internal locks are held. +- Assert bounded queue/result/branch/preview-intent state under sustained synthetic compaction load. +- Run existing generation pin/ProcessServices shutdown tests to ensure no ownership regression. + +_Requirements: 4.4–4.15, 8.9, 11.2, 11.11–11.13_ + +_Validation: race/goleak/process lifecycle tests green._ + +### Task 6.4 — Run architecture and security scope gates + +- Add/execute architecture tests forbidding provider/frontend DTO imports, direct provider clients and provider-specific continuity branches in core/feature packages. +- Prove detached mode cannot be set through frontend/wire fields and continuity source/capsule/raw result/raw BranchKey is absent from ordinary logs/money records and unnecessary extractor egress. +- Prove no second transcript database, feature-owned money ledger, generic arbitrary-task scheduler/service locator or durable job framework was added. +- Prove prerequisite detector retains one signature authority, preview intents use its boundary identity, metadata observers remain content-free/non-mutating, and preserver error handling does not become generic retry authority. +- Prove opaque/encrypted compaction bytes, existing no-retry-after-output/request authority semantics, and billing independent-leg validation remain unchanged. + +_Requirements: 1.1–1.4, 2.16, 7.6, 7.16, 9.3–9.13, 11.9, 11.12–11.15_ + +_Validation: `go test ./internal/archtest/...` plus security/contract suites green._ + +### Task 6.5 — Run final repository gates and simplification review + +- Run focused packages, `make quality-checks`, `make test-unit`, required race/goleak suites and deterministic config/docs checks without external model credentials. +- Review implementation diff for redundant abstractions, duplicate rule/state ownership, unbounded work, hidden provider/session/billing bypasses, child-vs-parent branch confusion and unnecessary public API expansion. +- Confirm normal requests with feature disabled/no candidate state incur only negligible bounded checks and no auxiliary model calls or preview-intent leaks. +- Record implementation evidence separating UX/cost trade-off: preserved continuity and extra billed auxiliary usage/latency only after successful Open plus bounded barriers for already-submitted work. +- If the feature requires a general workflow engine, second money path, pre-open billable child, child-keyed continuity state, or unsafe opaque mutation to pass tests, re-scope rather than weaken the frozen requirements. + +_Requirements: 10.3–10.13, 11.1–11.17_ + +_Validation: full repository release-quality gates green; final architecture remains within the spec boundaries._