Skip to content

WIP: refactor(agent) runtime around event-sourced protocol boundaries - #47

Draft
Fodesu wants to merge 287 commits into
felinics:feat/sdk-step-primitivesfrom
Fodesu:feat/agent-runtime
Draft

Fodesu wants to merge 287 commits into
felinics:feat/sdk-step-primitivesfrom
Fodesu:feat/agent-runtime

Conversation

@Fodesu

@Fodesu Fodesu commented Aug 28, 2026

Copy link
Copy Markdown
Member

Summary

see spec

@Fodesu
Fodesu force-pushed the feat/agent-runtime branch 2 times, most recently from 6dc5728 to ffe9493 Compare August 29, 2026 06:37
@Fodesu Fodesu changed the title refactor(agent) runtime around event-sourced protocol boundaries WIP: refactor(agent) runtime around event-sourced protocol boundaries Aug 31, 2026
Fodesu and others added 27 commits September 9, 2026 14:40
WaitForExecutionRecovery is no longer an Effect. Model Executing
and ToolStep with no Pending calls both yield Idle. Loop sets
ExecutionRecovery from NeedsRecovery on the snapshot. Application
reads WaitingCalls and ExecutingCalls from MachineState.
ProtocolFor(Header.SchemaVersion) returns the implementation for
digest, decode, Decide, Evolve, and BuildEnvelope. Loop and
EvaluateCommit call those methods without a version argument.
v1 replay uses digestRequestV1 directly instead of re-dispatching
through DigestRequest(schemaVersion, ...).
Protocol is a struct whose fields are the digest, decode, Decide,
and Evolve functions for one schema version. ProtocolFor fills them
once from the header. There is no protocolV1 type; v2 adds a
ProtocolFor branch and the functions that actually differ.
SubmitModelResult snapshots ToolExecution and MaxParallel onto
ToolStepOpened. Resume honors the frozen ToolStep.Scheduling instead
of the live Loop ExecutionPolicy. Empty mode stays empty so default
parallel does not enter the v1 golden state digest. ModelCatalog must
resolve a ModelRef to equivalent execution for the life of a Run.
ModelCatalog.Resolve failure or a nil invoker recovers the ModelStep
and returns from Loop. The model was never invoked, so the Run stays
active instead of ending as provider_failure.
CancelRun still ends as stopped/cancelled, but Executing tool calls
and an Executing ModelStep are recorded on RunStoppedEnd and copied
onto RunResult as UncertainCalls and UncertainModel.
RejectToolCall on ApprovalRequired stays permission_denied.
Rejecting an ExternalResponse wait records response_rejected.
The spec no longer keeps ToolStepClosed as a compatibility fact.
Update type listings for RunResult, ToolScheduling, and Protocol.
Document Protocol.BuildEnvelope, FoldRun per-record schema, Next's
terminal error, Evolve closing ToolStep, and NeedsRecovery as the
ExecutionRecovery signal including Model Executing.
List ModelStep and the full Protocol method set. Qualify that Loop
stops starting DirectExecution calls when the outer context is
cancelled. Record that Cancel without an Evolve close leaves
LastToolStep unchanged. Extend Loop conformance with catalog
resolve, frozen scheduling, and uncertain cancel projection.
Runtime evaluates commits; Store persists header, state, log, and
leases. MemoryStore is the in-process Store. NewRuntime(store) is the
single Runtime implementation. Expired leases set recoveryValid so
grantless Recover works; zero deadline never expires.
Keep a single Runtime constructor over MemoryStore. Rebuild is a Store
operation so diagnostic refold does not depend on a Memory-only type.
Unknown now records ToolCallFailed for that executing call and leaves
the Run active. Occupancy without a step is Open, so AcceptInput and
Prepare no longer key off a nil Current. Feature coverage moves into
the runtest driver.
EventType is twilight/module/name. Chatlog owns content, Turn owns
round and Run linkage, and the reference assembly records Binding,
Planner, and the shared user-text payload.
Keep one package. Loop construction and Run stay in loop.go; start
caches, model execution, tool settlement, and EventSink live beside it.
Keep empty ToolExecution unset until freeze. Resume executing tools
without grantless Known failures. Export RecoverExpired, bind grantless
model recovery to the lease claim, and FoldRun through Rebuild.
…dec and lease renewal

Store contract
- Replace View/Update(fn)/ListIDs with LoadHead, LoadLog(from), LoadRecord,
  LookupTransition, Commit(fn RunTx) (*Append), RenewLease, ExpiredLeases,
  ReplaceSnapshot. No method can delete or rewrite a transition.
- Store.Commit is the Run's critical section: Runtime evaluates against the
  RunTx head and the Store persists the returned Append in the same
  transaction (RUN-CMT-2). Load and Commit no longer touch the log.
- Drop the StartGrants table; the grant lives only on its lease.

Snapshot codec
- Protocol.EncodeMachineState/DecodeMachineState define a persisted wire for
  MachineState including Current. SQLite stores snapshot + log; Record and
  Rebuild verify the snapshot through FoldRun. Golden digests unchanged.

Lease renewal (RUN-CMT-8)
- Runtime.RenewLease extends a live lease; Loop workers heartbeat at
  ExecutionPolicy.LeaseRenewInterval and stop when renewal is rejected.
  LeaseTTL bounds recovery delay instead of tool duration.
- RecoverExpired loads only Runs returned by Store.ExpiredLeases.

Protocol surface
- Remove package-level Decide/Evolve/Digest*/BuildEnvelope/EncodeCommand,
  BuildRunHeader and currentSchemaVersion; version binds once at the Run
  boundary via ProtocolFor or RuntimeSnapshot.Protocol().

Specs
- agent-run.md: append-only Store section, RUN-CMT-8, updated RUN-WIR-3,
  RUN-MCH-3, RUN-CMT-2, RUN-CMT-7, RUN-LOP-1, RUN-CMP-2.
- Session, Artifact, Session Extension downgraded to draft until a Memory
  vertical slice passes conformance.
- Refactor doc: lease/recovery belongs to run.Runtime; sdk.Request freeze
  evaluation recorded in 4.4; sdk/request.go digest comment corrected.

Conformance
- New RunLeaseRenewalConformance; snapshot codec round-trip and malformed
  wire tests; SQLite reopen test; Loop long-tool heartbeat test. MemoryStore
  and SQLite pass Runtime, recovery and renewal suites; go test -race clean.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- evolve.go: validateFactTransition (cyclomatic 96) becomes guardFactV1
  dispatching to one guard per fact, plus small apply* folds. Guards share
  requireOpen / requireModelStep / requireCall / requireWaitingFor;
  ToolCallFailed reuses ValidateToolCallState for class/outcome agreement.
- state.go: ValidateMachineState (cyclomatic 48) split into
  validatePendingInputs / validateLastToolStep / validateCurrent /
  validateCurrentToolStep. Add String() on ModelStepStatus and
  ToolCallStatus, ToolCallStatus.Terminal(); move current() markers next to
  their types.
- Remove MachineState.LastClosedStep: it duplicated LastToolStep.RefValue.ID
  and needed a consistency check. PlanningHint.SourceStep now reads from
  LastToolStep. Snapshot wire drops lastClosedStep; v1 header and event
  stream golden digests re-frozen (pre-release).
- decide.go: waitingCall returns only error (unparam). Remove unused
  jsonMarshal.
- agent-run.md: MachineState block and closing rule updated.

gocyclo findings on agent/run: 2 -> 0. go test -race ./agent/... clean.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A godoc example that is also the first external caller of agent/run: a
host composes run + loop + sqlitestore, creates a Run, accepts input and
drives the Loop until a tool call is Executing. The Store is closed to
simulate the process dying with a live lease and no settlement.

A second Runtime reopens the same database after the lease TTL:
RecoverExpired settles the abandoned call as Unknown, the Run stays Active
on the same RunID, the Loop re-plans from the committed tool outcome and
completes. Record verifies all 9 transitions against the stored snapshot.

The example exercises PlanningHint.LastToolStep, NeedsRecovery,
ExecutingCalls, RuntimeOptions.Now, LeaseRenewInterval and the split
model/tool catalog interfaces from outside the package.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every command identity of one execution attempt now derives from its claim:
DeriveStartCommandID, DeriveSettlementCommandID, DeriveModelRecoveryCommandID
and DeriveToolRecoveryCommandID (RUN-WIR-3). EvaluateCommit enforces the
start derivation, so a hand-minted start id is rejected before it can mint
ownership. Tool recovery no longer writes the grant into the log.

Loop keeps a single ClaimStore instead of the starts/settlements caches;
memoryClaims is the default, ExecutionPolicy.Claims injects a durable one.
A replacement Loop sharing the store replays the derived start, recovers
the live grant and settles without waiting for lease expiry
(TestLoopReplacementFinishesInheritedClaim). Known failure of a Pending
call derives from the call alone.

Test helpers treat literal start ids as attempt labels; spec identity
table, RUN-LOP-3 and RUN-CMP-2 updated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The vertical slice of agent-turn.md: one Turn owns one primary Run.
Start appends started + input_delivered, creates the Run, accepts inputs
under their derived CommandIDs and drives. Resume rebuilds the Turn from
the log and the Runtime (no coordinator state), materializes every
transition above the coverage watermark through the v1 FactMapper
(ModelStepCompleted -> assistant, ToolCallCompleted/Answered/Failed ->
tool_result success/error/unknown), drives, then settles completed/failed
from RunEnded. Stop cancels under a Turn-derived CommandID.

Log is a Seq-addressed MemoryLog standing in for the Session kernel; the
Session, Extension and Artifact specs stay drafts. LoopDriver is the
reference RunDriver.

Tests: completion with materialization and idempotent Start; approval wait
and Resume; Stop settling as failed/stopped; Resume by a fresh coordinator
after a crash mid tool call and lease expiry.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ProtocolV1()

- EvaluateCommit: an envelope digest mismatch is a construction fault and
  returns a hard error instead of ErrCommandConflict, so callers no longer
  reload and retry it.
- Evolve: a duplicate InputAccepted is guarded as a corrupt log rather than
  silently deduplicated; Decide already rejects it and exact replay never
  reaches Evolve.
- decideSubmitModelResult split into checkToolCallBindings /
  checkBindingAgainstResult / openToolStep.
- RunEnded wire is now a tagged union ({"completed":{}} | {"stopped":{..}} |
  {"failed":{..}}) mirroring the Go sealed union; the flat status/reason
  shape and legacyEnd are gone. Golden digests unchanged (RunEnded is not in
  the state snapshot preimage).
- ModelCatalog.ResolveModel and ToolCatalog.ResolveTool replace the two
  same-named Resolve methods so one type can implement both.
- ProtocolV1 is a function; the package-level var could be reassigned.
- agent-run.md and refactor doc 4.3 updated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…iderCallID

The Run's CallID is now DeriveCallID(source, index): an authority-owned
identity that enters facts, lease keys, derived CommandIDs and chatlog, and
cannot collide across ModelSteps or depend on provider behaviour. The
model's tool_call_id rides along as ToolCallBinding.ProviderCallID /
ToolCallState.ProviderCallID, used only when a Planner echoes the call and
its result back to the model. Decide enforces both: CallID must be the
derived value, ProviderCallID must match the result at that index. Empty or
repeated provider ids (vLLM/llama.cpp style call_0, prompt-parsed tools)
no longer reject the result.

- loop.bindToolCalls derives ids and no longer treats duplicate provider ids
  as malformed; invalid UTF-8 input remains the malformed path.
- turn.MapTransition takes the record prefix so tool_result events carry
  the ProviderCallID from ToolStepOpened; payloads gain providerCallId.
- Planners in example, runtest and live test echo ProviderCallID.
- Test fixtures build bindings by (step, index, providerID); runtest resolves
  provider ids to derived CallIDs. New regression test covers provider id
  reuse and a forged non-derived CallID.
- v1 event-stream golden re-frozen (pre-release). Spec identity table,
  RUN-MCH-2 and TRN-MAP-2 updated.
- live_test: full payload logging, TWILIGHT_LIVE_RECORD_OUT dumps the
  RunRecord, 6 minute deadline.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The live run showed the second model request had no user message: the
example planner read only PlanningHint.Inputs, which the first prepare had
consumed. PlanningHint is boundary facts by design; the conversation must
come from the session log.

- ContextFold projects a session log into ordered Entries (delivered
  input, assistant, tool_result) across every Turn (CHT-CTX-1/2, minimal:
  no summary/checkpoint yet).
- ContextPlanner is the reference RequestPlanner (REF-PLN): system prompt,
  the fold as user/assistant/tool messages, then the tool step that closed
  inside this Loop.Run and is not yet materialized, taken from
  hint.LastModelResult/LastToolStep. sdk.Message is produced here and never
  stored. Tool results echo ProviderCallID and the model-facing tool name.
- ToolResultPayload gains Name; the mapper indexes provider id and name per
  derived CallID from ModelStepCompleted across the record prefix.
- Tests: the second request now carries system/user/assistant/tool; a second
  Turn on the same session sees the first Turn's whole conversation. The
  live test uses ContextPlanner; request 6 of the live record now includes
  the user message.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Replace the two-log design (per-Run RunHeader + TransitionRecord store,
Turn materialization into the Session) with one Session stream:

- agent-run.md: Run facts are twilight/run/ events; CommitID = CommandID;
  Runtime commits inside the Session critical section; MachineState is the
  twilight/run/machine projection. Facts keep only digests: request bodies
  go to a content-addressed FrozenValueStore, model/tool output goes to the
  chatlog companion events in the same commit (RUN-WIR-4). Prepare hard CAS
  keys on RunPosition, not the Session head. Companion/Attach on Commit.
- agent-turn.md: Turn:Run 1:N with attempt_failed / Retry / Settle;
  companion mapping replaces FactMapper, MaterializeAll, ResultReference,
  coverage and outbox; Stop settles via Attach.
- agent-session.md: add CommitIn critical-section port alongside CAS Commit.
- extension/chatlog/reference-assembly: run as first-party module,
  ProviderCallID and SourceDigest on chatlog parts, single-store composition.
- agent-runtime-refactor.md: record the decision, the byte analysis behind
  it, and the code migration list.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Apply the review findings on the single-Session-ES design:

- One write path: SemanticAppender gains AppendSemanticIn; run.Runtime
  writes through it, so companion/Attach events get codec, binding
  admission and claims in the same transaction (EXT-SCP-1, EXT-APP-3).
- Session Store gains a control-plane KV (SES-API-3) readable/writable
  inside SessionTx; lease, grant and artifact claims live there and
  commit atomically with the event group. RecoverExpired adds a
  no-lease fallback keyed on start time + TTL.
- Kernel ProtocolVersion covers envelope/commit only; payloads carry a
  top-level `v`, Registry keeps every codec version (SES-VER, EXT-REG-2);
  Run keeps its own created.SchemaVersion.
- v1 scope: Fork/ancestry/import, Application modules/Catalog, two-phase
  journal, artifact Prepared state/reconciler/import move to appendices.
- Snapshot is a droppable cache with SnapshotPolicy; MachineProjection
  drops terminal runs; turn surface records AttemptView.End.
- Run keeps only an opaque OwnerID (turn fills TurnID); Companion returns
  ModuleEvent; Attach carries the Stop settlement.
- Attempt-content policy moves to the reference Planner (REF-PLN-6).
- Fingerprint excludes RecordedAtUnixMilli; Replay/Tail gain EventType
  prefix filter; CoverageDigest removed in favor of Through.Digest.
- agent-runtime-refactor.md §7 records the review, the persistent
  structure/consistency table and the implementation order.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ncy pass

Kernel / framework layering
- Session control-plane KV gains ControlCompareAndPut, per-entry deadline
  and ControlExpired; the kernel carries no lease concept (SES-API-3).
- extension.Lease is the shared occupancy facility: Acquire/Release inside
  SemanticTx, Renew via conditional put, Expired via deadline scan, Token
  derived from the acquiring CommitID (EXT-LSE). run is its first consumer;
  the projection fallback scan and "renew via session lock" are dropped.
- ModuleDescriptor.Requires declares event-consumption dependencies; the
  Registry validates registration, acyclicity, projection Consumes and
  payload-version compatibility at build (EXT-REG-4, EXT-SCP-4).
- BuildRegistry takes descriptors from composition; extractors are declared
  per EventDefinition (BindingExtractor), so extension imports no module.
- ProjectionReader is the only out-of-section projection read path;
  Coordinator no longer holds session.Store.
- EventID for all first-party events = Digest(EventType, CommitID, index),
  assigned by the Appender (EXT-APP-5); per-module rules removed.

Mid-turn input
- Turn gains Deliver: one Run commit per input (AcceptInput + attached
  chatlog/input_delivered), never interrupting in-flight calls (TRN-DLV).
- AcceptInput is accepted in any non-terminal state; a model result with no
  tool calls but pending inputs returns to Open instead of ending; new
  WithdrawPreparedStep discards a frozen-but-unsent request when input
  arrives (RUN-MCH, RUN-LOP-8). turn surface tracks run/input_accepted;
  Retry replays all delivered inputs.
- Reference assembly adds SessionDriver (Send / OnTurnSettled) mapping the
  inbox model's next-step / next-turn onto Deliver / Start (REF-DRV).

Cleanup
- Status lines without dates or revision history; stale terms removed
  (resolved ancestry, RegistryID/Parts, ToolIndeterminate, InitialInputs).
- Redundancies removed: StartRequest.Inputs only; BindingPublic.Tools as
  PublicTool; Lease Attrs dropped for run (target parsed from Key).
- Gaps closed: Create idempotency, CausationID/CorrelationID semantics,
  ProjectionKey = ProjectionID, Record compares only active runs.
- agent-runtime-refactor.md §7.6–7.8 record the decisions.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Reshape agent/run per agent-run.md so facts carry execution state and
content digests only (RUN-WIR-4):
- ModelStepPrepared keeps RequestDigest; the request body lives in a
  FrozenValueStore (Runtime.FrozenRequest, MemoryFrozenValues).
- ModelStepCompleted/ToolCallCompleted/ToolCallAnswered carry digests;
  MachineState drops LastModelResult, RunResult drops Model,
  ToolSpec drops Definition and gains Name.
- RunCreated fact and BuildCreateGroup; MachineState gains Owner/Attempt.
- AcceptInput legal in any non-terminal state; WithdrawPreparedStep and
  Next -> WithdrawPrepared; no-call result with pending inputs reopens.
- PlanningHint carries boundary facts only (Owner, RunID, SourceStep).
Tests and runtest/runtimetest drivers follow; golden fixtures re-frozen.
Old agent/turn is behind the legacy_turn build tag until rewritten.
… carries the three

A Dispatch error is one of three things (RUN-EXE-3): a definite rejection
of the Assignment (plain error, nothing started), the Worker's own
refusal before the barrier because its record store was unavailable
(effect.ErrDispatchRetryable, nothing started, the same Assignment may be
offered again), or a lost answer (ErrDispatchUnknown). The HTTP server
maps them to 400/409, 503 and never 500; the client maps 4xx to a plain
error, 503 to ErrDispatchRetryable and other 5xx or transport failures to
ErrDispatchUnknown, so a deterministic rejection no longer leaves the
target Executing. The Loop re-offers a retryable refusal a bounded number
of times inside the Advance before treating it as a Known dispatch failure.
…the tool

Replay stays a tool-level capability. Whether a specific failure is worth a
second execution is decided per failure, the way the model call already
does it: the effect layer classifies the error into a FailureCode and reads
the disposition off the code (FailureCode.Retry), and ModelFailed carries
it. Tools follow the same shape: ToolFailure.Class names what went wrong
(not_found, invalid_input, timeout, unavailable, rate_limited, conflict,
internal) and ToolExecutionFailed{Failure, Retry} declares the disposition
for that failure alone. The tool-level RetryPolicy, ExecutionPolicy and
ExecutableTool.Retry() are removed; the wire carries the disposition on
WireError and ToolOutcomeEnvelope; the Worker retries any Known failure
that says RetryAllowed within its budget (RUN-EXE-11).

.pi is untracked and ignored.
…RetryAllowed asserts safety

ModelFailed loses its Retry field: the disposition is read off the
FailureCode (ModelFailed.Retry) wherever it is needed, so neither the wire
nor a literal can carry a disposition that disagrees with the code, and
the malformed-request failure that bypassed the constructor is no longer
inconsistent. WireError drops retry; ToolOutcomeEnvelope keeps it, because
a tool's disposition is its own declaration.

RUN-EXE-11 and the type comments now state what RetryAllowed asserts: this
Known failure suffices to confirm the attempt produced no external effect
that cannot safely be repeated. A failure that cannot confirm that (a
payment call that timed out) is a ToolExecutionUnknown, which enters replay
and reconciliation; the three cases Known+Never, Known+Allowed and Unknown
are spelled out.

Lint is clean under golangci-lint v2.11.3: test error types renamed
(errname), registerEvent extracted from BuildRegistryWithExtensions
(gocyclo), attempt ordinal compared in int64 (G115) and converted with
Record(p) (S1016), test helpers return after Fatalf (SA5011),
onOwnershipLost returns only its error (unparam), the unused
committedLocked removed, the attempt-shadowing parameters renamed
(importShadow), and the recovery lifetime's cancel annotated (G118).
… segment

SegmentHeader.Ext and Commit.Ext are the kernel's own extension objects
(SES-WIR-5): canonical JSON objects sealed into their digests byte for
byte, omitted when absent so existing digests and goldens are unchanged.
Later kernels add optional fields there instead of a new ProtocolVersion.

Derived identities no longer embed the kernel version (SES-VER-3):
ClaimID and ClaimOwner are keyed by the segment that holds the commit,
not by SessionID or ProtocolVersion; the writer, chatlog and fork claim
preimages carry their own derivation constants.
Each segment carries a CommitIndex: its own commits by CommitID with Seq,
digest and per-stream event counts, plus the head it covers. Adapters
extend it with every Append and cut it with every Truncate; Open uses it
after two O(1) checks against the head and rebuilds it from the commits
only when they fail. Committed, StreamHead and inherited membership are
answered from indexes alone (Backend.Locate replaces Contains).

The filestore persists it as index.jsonl next to log.jsonl with each
commit's byte range: written after the commit, so a crash leaves it at
most one short, which load repairs from the log tail; LookupCommit and
ReadSegment read only the bytes the index points at.
…ion to a later one (SES-ADV-1)

A Session upgrades by advancing onto a new, empty tip segment sealed under
the later ProtocolVersion, anchored at the current head (or carrying the
tip's own edge when the tip is empty). No segment or commit is rewritten:
an Ancestry may hold segments of different versions, each verified under
its own profile, and forks cross versions in both directions, so the
parent/child same-version check is gone.

The Ledger carries a profile table (WithProfile; ProfileVariant seals the
v1 rules under another number for fixtures) and adapters verify each
header through it. OpenWriter refuses a tip newer than its registry and
advances a tip older than it once ownership is held (EXT-WRT-1).
…iter reads from its earliest cache entry

Each segment records the head through which its commits were verified
(Backend.VerifiedMark/PutVerifiedMark): Open recomputes the digest chain
only past the mark, after confirming the mark through the CommitIndex and
resealing the marked commit itself, and advances it at Open and Close.
Commits behind a trusted mark are the explicit ValidateLedger's job.

OpenWriter judges each projection's cache entry by reading the one commit
it points at and reads the log from the earliest entry any projection
resumes from, so a clean Close reopens without reading commits.
…them, forks hold none (SES-GC-3)

A commit's retention claim is owned by the segment that holds it, so it
lives exactly as long as the segment: Delete only drops the root, and
Collect releases the claims of every segment it removes and of every
commit a truncation drops (CollectReport.Dropped names them). A fork
therefore needs no claim of its own: as a root reaching the prefix
segments it keeps them, and Fork no longer decodes the parent's prefix
or fails closed on unknown event types.
… (SES-OWN-1, EXT-WRT-11)

A Handle now holds Lease{Epoch, Owner, Until}: Acquire takes it for
OpenOptions.LeaseDuration, Renew extends it for the current lease only,
and an Open supersedes an expired lease without Takeover. Epoch fencing
still carries all safety; the lease only decides when a supersession is
allowed without an operator, mirroring the Execution Store's records.
A zero duration never expires. The clock is injected (OpenOptions.Clock).

OpenWriter starts a heartbeat at a third of the duration; a fenced Renew
marks the Writer lost like a fenced Append. Conformance ages a lease
through the clock on both adapters.
…RUN-LOP-10)

The Loop gains Settings.BeforePrepare, called while a Run is Open before
the plan reads the context; the Driver forwards it to a Planner with the
drive's Writer, and the Application implements the Planner with each
Session's automatic compaction policy. The checkpoint guard becomes
turn.RequireQuiescentRun: no active Turn, or a Run that is Open or in a
ToolStep with no Executing call. The retain closure keeps any assistant
whose call has no result yet, so a pair settled after the checkpoint
stays closed; RetainLast pulls such assistants into the suffix.
…ker to the Bus (RUN-EXE-12)

Backends publish model deltas and tool progress as ProgressFrames into the
Worker's ProgressHub, a bounded in-memory window per key stamped with a
generation and sequence; a re-dispatch emits a reset frame that voids the
earlier generation, a terminal record an end frame. The Worker serves them
as effect.ProgressPort (relayed through PortBackend and over HTTP as
server-sent events on /progress). Loop.Run subscribes for each dispatched
key and forwards frames to its EventSink; the Driver's sink publishes them
on the observe.Bus as transient Progress events next to the committed
facts. Outcome semantics are unchanged. The local backend now streams.
…er's Responder answers (SPN-1, DRV-4)

The spawn tool's ResponsePolicy is ExternalResponse: a call waits instead
of starting an effect, and no execution record is involved. The Driver
gains Responders by ToolRef: a drive that ends waiting on such a call
asks the Responder, commits its payload as SubmitToolResponse (or an
error as RejectToolCall) and drives on; a Session opening with such waits
asks again under its recovery lifetime. spawn.Responder replaces the
Worker backend and route: it creates or continues the child Session from
its durable state, drives it to settlement and returns the reply, so a
takeover continues the same child without adopting a record.
…es only persistent authorities

agent/authority becomes agent/owner and Authority becomes Owner: the
role that holds a Session's lease, runs its Loops and can crash and be
taken over. Observer (reads by SessionID), Worker (executes Assignments)
and Node (the hosting process) complete the role vocabulary, recorded as a
term rule in the design README. "authority" now means only what survives
a process: the ledger, the Execution Store and artifact's store instance.
Clause ids AUTH-* become OWN-* (AUTH-OWN-* is OWN-HDL-*).
…ed, never forgotten

Attach answers missing for a key without a record only when the absence
proves the execution absent: the store is durable, or every Backend
declares Colocated (RUN-EXE-3); otherwise orphaned, and Dispose writes a
terminal Unknown record for the key. A takeover whose backend cannot
confirm the execution holds the lease and asks again instead of restarting;
the Port adapter reports an unreadable Ref as an error. The Reconciler
refuses to dispose without an executor to ask (ErrNoExecutionPort) unless
the caller sets Abandon; RecoverInterrupted requires a reconciler.
Terminal records are collected, not evicted (RUN-EXE-13): the Loop
acknowledges each settlement through effect.Acknowledger, Worker.Collect
strips acknowledged (or CollectAfter-old) records to key, digest, state
and ExecutionRef, GetOutcome answers ErrOutcomeCollected, and a Dispatch of
the key starts nothing. HTTP carries /acknowledge, /collect and 410.
…gs and claims

The memory implementations are gone: session.MemoryStore, the execution
MemoryStore and FileStore, artifact.MemoryContentStore, MemoryBindingStore,
MemoryLedger and runmod.FrozenValuesInMemory. owner.New and app.Build
require every store and no longer fall back; the durability bundle
(Durable(), Artifacts.Ephemeral, ErrEphemeral*) has nothing left to guard
and is removed (OWN-PRT-3). agent/store/sqlite is the new durable store of
the small-row ports: one SQLite file (WAL, immediate transactions) carries
executor/store.Store, artifact.BindingStore and artifact.RetentionLedger,
so leases fence across processes. With no record store that can lose a
record, a key without a record is missing again and Colocated goes away;
collected records are kept, never evicted (RUN-EXE-13). EncodeOutcome
refuses a model result that does not freeze (malformed_result) instead of
letting JSON rewrite its invalid UTF-8 on the record or the wire.
Tests run over t.TempDir() stores via filestoretest, sqlitetest, runmodtest.
…and metadata

The text-generation client layer is gone: GenerateText, GenerateTextResult,
StreamText, GenerateParams, GenerateResult, StepResult, StreamResult, the
With* options and the loop, callbacks and approval flow they carried. The
seam is the whole API: Model.Generate/Stream take a Request and return a
ModelResult or ModelStream; ExecuteTools and BuildStepMessages are the
primitives a caller composes a loop from.
Every open field is closed. ToolArguments{JSON, Text} is what the model
supplied: a JSON document, or the verbatim text of a call that was not one,
which providers now report instead of dropping, and which ExecuteTools
answers with an error result instead of running. ToolOutput{Text, JSON} is
what a tool returned. ProviderMetadata is namespace -> key -> string.
Tool.Parameters and ToolDefinition.Parameters are *jsonschema.Schema;
RawPart.RawValue is json.RawMessage. Response instants are UTC.
README, getting started, streaming, tools, providers, embeddings, the API
reference and the skill files show Model.Generate and Model.Stream with a
Request, ExecuteTools and BuildStepMessages for the caller's own loop, and
ToolArguments, ToolOutput and ProviderMetadata where the guides used to show
any. The sections that documented the deleted loop, its options and callbacks
are removed.
…ontract

The SDK now closes its opaque fields: ToolArguments (JSON or verbatim
Text), ToolOutput (Text or JSON), ProviderMetadata (string tokens by
namespace) and a *jsonschema.Schema for tool parameters. The agent's
persisted mirror follows the same shape.

- model.ToolArguments / model.ToolOutput mirror the SDK types; Input and
  Result of message parts and tool calls carry them. Canonical() is the
  binding form (RUN-MCH-2): document, empty object, or quoted text.
- model.ProviderMetadata is map[string]map[string]string; no JSON freeze.
- sdkconv: FreezeToolArguments / FreezeToolOutput replace FreezeToolCallInput
  and the any-based metadata freeze; ToolDefinition decodes the schema.
- model.CanonicalToolArguments / RawToolArguments removed.
- Legacy sdk client layer, request_adapter and the SA1019 lint rules go.
- executor test closes its Worker before the SQLite TempDir is removed.
@Fodesu
Fodesu changed the base branch from main to feat/sdk-step-primitives September 21, 2026 19:03
ModelResult.Response was a pointer so that a reply without metadata could
omit the object. The presence test that came with it was applied in some
providers and paths and not others, and time.Unix(0, 0) passed it, so a
reply without created got a 1970-01-01 timestamp and Copilot's two paths
disagreed on the same reply.

- ResponseMetadata is always present; IsZero drives omitzero, so an empty
  object is still omitted from JSON. No presence tests remain.
- TimestampFromUnix maps a missing Unix seconds field to the zero time and
  renders the rest in UTC; hardenResult normalizes as its comment said.
- providertest gains a paths-agree case: response metadata, text metadata,
  raw finish reason, sources and reasoning tokens must match between
  Generate and Stream. Completions' stream fixture now repeats created and
  model the way the real wire does.
…d-trip test

sdk.ModelResult.Response is a value now, so the persisted mirror follows:
model.ResponseMetadata is always present, IsZero drives omitzero, and the
freeze/thaw pair no longer handles nil. The JSON is unchanged for both an
absent and a present value, so no golden drifted.

Also adds TestToolDefinitionRoundTripKeepsCanonicalBytes: a schema frozen,
thawed into *jsonschema.Schema and frozen again keeps its canonical bytes,
including Extra keywords and boolean schemas, so a replayed Request cannot
carry a definition whose digest differs from the frozen ToolSpec.
README, getting-started, streaming, tools, providers, embeddings, the API
reference and the skill files describe what the SDK now is: Model.Generate
and Model.Stream as the only chat entry points, ExecuteTools and
BuildStepMessages as the loop primitives, ToolArguments / ToolOutput /
ProviderMetadata as the closed field types, and the Client as the carrier
of the other modalities. Every mention of the deleted multi-step loop,
MaxSteps and the With* options is gone.
@Fodesu
Fodesu force-pushed the feat/sdk-step-primitives branch from 80cdea2 to 53c2e79 Compare September 21, 2026 20:10
# Conflicts:
#	.golangci.yml
#	docs/api-reference.md
#	docs/getting-started.md
#	docs/providers.md
#	provider/github/copilot/copilot.go
#	provider/openai/codex/codex.go
#	provider/openai/completions/completions.go
#	provider/openai/completions/completions_test.go
#	provider/openai/completions/thinking_response_test.go
#	provider/openai/responses/reasoning_roundtrip_test.go
#	provider/openai/responses/responses.go
#	provider/providertest/providertest.go
#	sdk/client.go
#	sdk/malformed_tool_args_test.go
#	sdk/model_call.go
#	sdk/request.go
#	skill/reference.md
@Fodesu
Fodesu force-pushed the feat/sdk-step-primitives branch from 53c2e79 to 44a22e5 Compare September 21, 2026 20:20
# Conflicts:
#	docs/api-reference.md
#	skill/reference.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant