# Harness

The harness (`crates/tinyagents-harness/`) is the model/tool orchestration
layer: the agent loop, provider-neutral model calls, typed tools, middleware,
structured output, streaming, usage/cost accounting, and observability. It
does not depend on the graph runtime — you can run a harness agent on its own,
without building a graph.

Message, model-request/response, provider, and embedding types are not owned
by this crate. They live in the vendored `tinyinference` crate
(`vendor/tinyinference/crates/tinyinference/`) and are used directly as
`tinyinference::message::Message`, `tinyinference::model::ModelRequest`, and so
on — `tinyagents-harness` is not a facade over them, it depends on them.

Cargo features (`crates/tinyagents-harness/Cargo.toml`, none on by default):
`sqlite` (bundled SQLite via `rusqlite`), `tools` (pulls in `chrono-tz` for the
built-in tool set), `multimodal` (`base64` + `flate2` for image/data-URI
content), `tracing`.

## Module map

| Module (`crates/tinyagents-harness/src/…`) | Role |
| --- | --- |
| `agent_loop` | The model -> tool -> model loop, entry points on `AgentHarness` |
| `runtime` | `AgentHarness<State, Ctx>` — registers models, tools, middleware, policy |
| `context` | `RunConfig`, `RunContext`, depth/limit tracking |
| `ids` | Typed `RunId`/`ThreadId`/`CallId`/… identifiers |
| `prompt` | Prompt templates and cache-segmented `PromptBuilder` |
| `tool` | Typed tool trait, schemas, registry |
| `tool_calling` | Provider tool-call dialect parsing |
| `tools` (feature `tools`) | Built-in tools |
| `middleware` | before/after hooks around agent, model, tool calls |
| `structured` | Typed/JSON-schema response extraction |
| `stream` | Harness-level streaming (`StreamSink`, `StreamMode`) |
| `cost` | Pricing and cost roll-ups (`CostTotals`) |
| `limits` / `retry` | Caps, timeouts, backoff, fallback, rate limiting |
| `cache` | Local response cache and prompt-cache layout |
| `memory` | Short-term thread memory / chat history |
| `store` | Pluggable persistence backends |
| `events` | Typed event stream and run status |
| `observability` | Durable journals, status stores, sinks, latency metrics, Langfuse export |
| `subagent` | Wrapping an agent as a tool for another agent |
| `steering` | Typed runtime control of a running agent |
| `summarization` | Context-window-aware transcript compaction |
| `cancel` | Cooperative `CancellationToken` |
| `workspace` | Filesystem sandboxing for tools |
| `artifacts` | Tool-produced artifact tracking |
| `handoff` | Cross-run handoff records |
| `host` | Host-provided runtime hooks |
| `no_progress` | Detection of stalled/repeating loops |
| `run_queue` | `RunQueue`/`QueueLane` scheduling primitives |
| `token_estimation` | Cheap token-count estimates |
| `model_registry` | `ModelRegistry`, name-based model selection |
| `testkit` | Fakes, recorders, trajectory assertions |
| `multimodal` (feature `multimodal`) | Image/data-URI content helpers |
| `config` | Shared config types |
| `error` | `TinyAgentsError`, `Result` |

## The agent loop (`agent_loop`)

The loop is implemented as async methods on `AgentHarness<State, Ctx>`
(`crates/tinyagents-harness/src/agent_loop/entry.rs`):

- `invoke(state, ctx_data, config, input)` — full control over run identity
  and context data.
- `invoke_default(state, input)` — convenience wrapper that builds a default
  `RunConfig`.
- `invoke_with_status` / `invoke_in_context` / `invoke_in_context_with_status`
  — variants that run inside an existing `RunContext`, which is how a
  sub-agent call inherits the parent's identity.
- `invoke_streaming`, `invoke_streaming_default`,
  `invoke_streaming_in_context[_with_status]` — the same loop, emitting
  `ModelStreamItem`s.
- `invoke_collecting_partial` / `invoke_in_context_collecting_partial` —
  streaming variants that also collect the final `AgentRun`.

The lifecycle (model -> tool -> model):

```text
input messages
  -> build RunContext, emit RunStarted
  -> before_agent middleware
  -> loop:
       enforce model-call cap + wall-clock deadline (fail-closed)
       build ModelRequest (messages + tool schemas + response format)
       before_model middleware; emit ModelStarted
       resolve + invoke model (with retry + fallback)
       after_model middleware; emit ModelCompleted; fold Usage into AgentRun
       append assistant message
       if tool calls -> enforce tool cap, before_tool, run tools,
                        after_tool, append tool results, continue
       else -> extract structured output (if configured) and break
  -> after_agent middleware; emit RunCompleted
```

On error the loop emits `RunFailed`, runs `on_error` middleware, and returns
the error. Retry backoff is computed via `RetryPolicy::backoff_for_attempt`
without the loop itself sleeping, which keeps tests deterministic.

`AgentRun` (from `middleware::AgentRun`) holds the final messages, folded
`Usage`, and any extracted structured response.

## Provider-neutral model calls

Model invocation goes through `tinyinference::model::ChatModel<State>`, with
`invoke` (unary) and `stream` (incremental). A call is a `ModelRequest`,
answered by a `ModelResponse` carrying the assistant message, `Usage`, finish
reason, and a `ResolvedModel` recording which provider/model actually served
the call. `tinyagents_harness::model_registry::ModelRegistry` resolves a named
model registration to a concrete `ChatModel` handle; `ModelSelection` and
`ResolvedModelBinding` record what was chosen and why.

`tinyinference::providers` holds the adapters: `MockModel` for offline tests,
and `OpenAiModel`, which speaks the OpenAI Chat Completions wire format and
also serves every OpenAI-compatible endpoint through preset constructors
(`deepseek`, `anthropic`, `groq`, `xai`, `openrouter`, `together`, `mistral`,
`ollama`, plus `compatible(base_url, model)` for anything else). See
[Providers](Providers).

## Typed tools (`tool`)

Tools implement `tinyagents_harness::tool::Tool<State>` with `name`,
`description`, `schema() -> ToolSchema`, and an async
`call(state, ctx, call) -> Result<ToolResult>`. `ToolCall` (from
`tinyinference::tool`) carries an id, name, and JSON arguments; `ToolResult` is
built with `ToolResult::text(...)` / `ToolResult::error(...)`. `ToolRegistry`
keys tools by name and is consulted by the loop when the model requests one.

An entire agent can be exposed as a tool — see [Sub-agents](#sub-agents).

## Middleware (`middleware`)

`Middleware<State, Ctx>` exposes hooks the loop calls into:
`before_agent`/`after_agent`, `before_model`/`on_model_delta`/`after_model`,
`before_tool`/`on_tool_delta`/`after_tool`, and `on_error`. They run through a
`MiddlewareStack` (`before_*` in registration order, `after_*` in reverse).
Built-ins under `middleware::library` include logging, message trimming,
context compression, prompt-cache guarding, and usage accounting middleware.

## Structured output (`structured`)

`StructuredExtractor` extracts a typed value from the final `ModelResponse`
according to a `StructuredStrategy`. `response_format_for_strategy(...)` maps
a strategy onto a `ResponseFormat`. The parsed value lands in
`StructuredOutput`, exposing `as_value()` and `parse::<T>()`. Set
`RunPolicy::default_response_format` to attach a schema to every model
request in a run; the loop runs extraction automatically before completing.
See the `openai_structured` example.

## Streaming (`stream`)

At the provider edge, `ChatModel::stream` yields a `ModelStream` of
`ModelStreamItem`s. At the harness edge, `stream::StreamSink` exposes
`StreamChunk`s with selectable `StreamMode`s. The `invoke_streaming*`
agent-loop methods surface model deltas as the loop runs.

## Usage and cost (`cost`)

`Usage` (from `tinyinference::usage`) records input/output tokens, cached-token
bookkeeping, and reasoning tokens when a provider reports them. The loop folds
each call's `Usage` into the `AgentRun`. `cost::estimate_cost(pricing, usage)`
turns a `tinyagents_registry::ModelPricing` and `Usage` into `CostTotals`.
Because a sub-agent call executes through the parent's `RunContext`, child
usage and cost roll up into the parent's totals.

## Limits, retry, fallback (`limits`, `retry`)

`RunLimits` (`with_max_model_calls`, `with_max_tool_calls`,
`with_max_wall_clock_ms`, `with_max_retries_per_call`, `with_max_concurrency`,
`with_max_depth`) is enforced fail-closed. Reaching a cap returns
`TinyAgentsError::LimitExceeded`; the deadline returns
`TinyAgentsError::Timeout`. `max_depth` bounds how deep nested sub-agents may
recurse — exceeding it fails with `TinyAgentsError::SubAgentDepth`.

`retry` provides `RetryPolicy` (attempts, backoff, multiplier, jitter), an
`is_retryable(err)` classifier, `FallbackPolicy` (an ordered model fallback
chain), and a token-bucket `RateLimiter`.

Bound per-tool timeouts by installing `ToolTimeoutSettings` on the harness:

```rust
use tinyagents_harness::runtime::AgentHarness;
use tinyagents_harness::tool::ToolTimeoutSettings;

let mut harness: AgentHarness<()> = AgentHarness::new();
harness.with_tool_timeout_settings(ToolTimeoutSettings::new(
    120_000, // inherited default
    1_000,   // minimum explicit budget
    3_600_000, // maximum explicit budget
    5_000,   // scheduling grace for explicit budgets
));
```

Tools return `ToolTimeout::Inherit` by default; a tool may opt out with
`Unbounded` or request a clamped, explicit `Millis` budget. A per-tool
deadline produces a recoverable tool-error message and the agent loop
continues so the model can retry or choose another tool; the run's
wall-clock deadline stays the outer hard abort.

## Cache (`cache`)

Two distinct ideas:

- **Local response cache** — `ResponseCache` (with `InMemoryResponseCache`),
  keyed by `cache_key(request)`. Attach it with
  `AgentHarness::with_response_cache`; the loop checks it before each
  provider call and stores successful responses.
- **Provider prompt/KV-cache layout** — `PromptCacheLayout` makes the stable
  prompt prefix explicit so middleware editing model-visible prompt segments
  can be detected. Keep `thread_id` stable across parent agents, sub-agents,
  and nested harness calls so a provider's prompt cache sees one logical
  conversation.

## Memory (`memory`)

`memory` owns conversation continuity: the `ChatHistory` trait
(`InMemoryChatHistory`, `StoreChatHistory<S>`) and `ShortTermMemory<H>`, loaded
before a loop and saved after. Embedding-based retrieval
(`tinyinference::embeddings`: `EmbeddingModel`, `VectorStore`, `Retriever`,
`cosine_similarity`) lives in `tinyinference`, not in this crate.

## Sub-agents

A `SubAgent<State, Ctx>` (`subagent::types::SubAgent`) wraps an
`Arc<AgentHarness>` plus a stable `name`, `description`, and optional
`system_prompt`:

- `SubAgent::invoke` / `invoke_with_events` run a fresh child loop.
- `SubAgent::invoke_in_parent(state, ctx_data, parent, input)` threads the
  live parent `RunContext`, so the child runs at `parent.depth() + 1` and its
  events and usage surface on the parent's stream.
- `SubAgentTool` adapts a sub-agent into a `Tool`, so a parent model calls an
  entire agent the way it calls any other tool. The child input is read from
  the `SUBAGENT_INPUT_FIELD` (`"input"`) argument. Depth is fixed at
  construction (`with_parent_depth`) because `Tool::call` has no live parent
  context.
- If a sub-agent invoked through `SubAgentTool` hits a child run limit, the
  tool returns an error `ToolResult` telling the parent it hit its limit,
  rather than mistaking a partial answer for a complete one.
- `SubAgentSession` keeps the same sub-agent alive across turns, accumulating
  a transcript for reuse (for example human-in-the-loop follow-up).

The graph equivalent is `tinyagents_graph::subagent_node` (`SubAgentNode`),
where a graph node embeds another compiled graph instead of a plain harness
agent. See the `orchestrator_subagents` example.

## Steering (`steering`)

Steering is typed runtime control of an already-running agent, distinct from
sub-agent session reuse (which acts between runs). A `SteeringHandle` sends a
`SteeringCommand` — `Pause`, `PauseWith { reason }`, `Resume`, `Cancel`,
`InjectMessage(Message)`, `Redirect { instruction }`, or `SetMetadata` — and
the loop drains pending commands at a safe checkpoint (before each model
call). A `SteeringPolicy` allowlist checks each command's
`SteeringCommandKind` (`SteeringPolicy::new` permits nothing by default;
`allow_all` is for tests); a disallowed command is rejected with
`TinyAgentsError::Steering`. Applying a batch yields a `SteeringOutcome`
(`Continue` / `Pause` / `Cancel`).

## Summarization (`summarization`)

`estimate_tokens(text)` and `TokenEstimate` provide a cheap budget;
`trim_messages(messages, strategy)` applies a `TrimStrategy`; and
`SummarizationPolicy` decides when to compact, optionally derived from a
model's own profile (`from_profile`, `with_context_window`).
`should_summarize(messages)` / `plan(messages)` split the transcript, the
`Summarizer` trait (with `ConcatSummarizer`) produces the summary, and
`SummaryRecord` / `CompressionProvenance` record what was compressed.

## Events and observability (`events`, `observability`)

`events::AgentEvent` enumerates lifecycle boundaries — run, model, tool,
middleware, sub-agent, cache hit/miss, retry/rate-limit/fallback, usage/cost,
limit-reached, memory, compression. Events flow through an `EventSink` to
listeners and are journaled in an `EventJournal`; a run's overall state is
`HarnessRunStatus`. Because a child run shares the parent's `EventSink`, one
stream shows the whole call tree with correct depth annotations.

`observability` is the durable side: `AgentObservation` wraps each event with
correlation metadata (`event_id`, `run_id`, `parent_run_id`/`root_run_id`,
stream offset, timestamp). `HarnessEventJournal` (`InMemoryEventJournal`,
`StoreEventJournal<A>`) durably records observations; `HarnessStatusStore`
(`InMemoryStatusStore`) snapshots run status. Sinks compose:
`FanOutSink` (multiplex), `RedactingSink` (scrub sensitive fields),
`JournalSink` (write through to a journal), `JsonlSink` (append JSONL).
`AgentLatencyMetrics::from_record(...)` summarizes a run's timing.

Export observations to Langfuse:

```rust
use tinyagents_harness::{LangfuseClient, LangfuseTraceConfig};

let client = LangfuseClient::proxy("https://api.tinyhumans.ai", backend_jwt)?;
client
    .send_observations(
        LangfuseTraceConfig {
            user_id: Some("user_123".to_string()),
            session_id: Some("thread_abc".to_string()),
            ..Default::default()
        },
        &observations,
    )
    .await?;
```

`LangfuseClient::proxy` sends through a backend `/telemetry/langfuse/ingestion`
endpoint with bearer auth; `LangfuseClient::direct(langfuse_url, public_key,
secret_key)` talks to Langfuse directly. `tinyagents_graph::GraphLangfuseExporter`
reuses the same transport for graph runs (see [Graph Runtime](Graph-Runtime)).

## Prompt templates (`prompt`)

`PromptTemplate` renders `{{var}}` substitutions with a `TemplateRole`, and
`MessagesTemplate` renders an ordered multi-message conversation.
`PromptBuilder` assembles a request out of named, ordered segments —
`push_system`, `push_tools_segment`, `push_instructions`, `push_history`,
`push_volatile` — then `build(tail)` emits a `ModelRequest`. Stable segments
before volatile ones let `fingerprint()` identify the cacheable prefix, used
by the prompt-cache layout in `cache`.

## Testkit (`testkit`)

`ScriptedModel` / `StreamingMock` / `SlowModel` fake providers, `FakeTool`,
`DeterministicClock`, `DeterministicIds`, `EventRecorder`, and a `Trajectory`
assertion helper (`from_events(...).assert_tool_called(...)`,
`assert_model_called_times(n)`, `assert_order(&[...])`, `assert_completed()`)
make loop behavior deterministically testable.

## Unknown-tool recovery

When a model calls an unregistered tool, `RunPolicy::unknown_tool:
UnknownToolPolicy` decides what happens: `Fail` (default) aborts with
`TinyAgentsError::ToolNotFound`; `ReturnToolError` injects a tool-error result
and continues so the model can retry; `Rewrite { tool_name }` retargets the
call to a fixed compatibility tool once, falling back to `ReturnToolError` if
that target is also missing. Every recovery consumes a tool-call slot and
emits `AgentEvent::UnknownToolCall`.

## Workspace isolation (`workspace`)

A `WorkspaceDescriptor` (root, trusted roots, `policy_id`, `SandboxMode`)
tells a tool which paths it may touch; `RunContext::with_workspace(descriptor)`
threads it into every tool call context. `WorkspaceIsolation` providers
prepare and tear down per-agent environments (`SharedRootWorkspace` is the
built-in). `WorkspaceDescriptor::enforce(path, &events)` is a fail-closed
lexical gate: an out-of-root path emits `WorkspaceViolation` and returns
`TinyAgentsError::Validation`.

## Tool policy enforcement

Each tool advertises a serializable `ToolPolicy` (`side_effects`, `runtime`,
`access`, `classified`) via `Tool::policy()`. `ToolPolicyMiddleware` enforces
it both at exposure (`before_model` hides the tool) and at execution
(`before_tool` rejects with `Validation`).

## Tool exposure

`ContextualToolSelectionMiddleware` filters model-visible tools per call
using a predicate over `ToolSelectionContext { run_id, depth, tags,
requested_model }`, emitting `AgentEvent::ToolsFiltered` whenever it withholds
tools. `ContextualToolSelectionMiddleware::inheriting(parent_allow,
parent_deny, child_allow, child_deny)` composes a child policy against an
inherited parent one so a sub-agent can only narrow (deny is additive, allow
is intersective).

## Middleware control

A middleware can steer the loop out-of-band with
`RunContext::request_control(MiddlewareControl)`: `StopWithFinal(text)` ends
the run with a final answer, `Interrupt { node, message }` pauses at the next
safe checkpoint. The highest-precedence pending request wins
(`Interrupt` > `StopWithFinal`), so a pause is never downgraded to a stop.

## Budget middleware

`BudgetMiddleware` enforces `BudgetLimits` across a run, or a whole sub-agent
tree when a shared `BudgetTracker` is handed to every sub-agent. Beyond
token/cost limits it adds `max_cached_input_tokens` and a preflight
reservation: `before_model` estimates the upcoming call's input tokens and
blocks before dispatch if it would breach budget; `after_model` reconciles
the estimate against provider-reported usage.

## See also

- [Graph Runtime](Graph-Runtime) — durable typed state graphs and subgraphs.
- [Providers](Providers) — configuring hosted providers.
- [Examples](Examples) — annotated, runnable catalog.