From a92dd8dd81002757b48213a92c7cbc704931b59d Mon Sep 17 00:00:00 2001 From: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> Date: Fri, 28 Aug 2026 14:35:18 -0700 Subject: [PATCH] docs(hooks): document ephemeral semantics on stateful providers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The ephemeral flag on HookResult/context_injection is enforced client-side only -- it controls whether Amplifier's local context module persists the injection, not what a stateful provider does with the message once it is sent. Hook authors currently have no documentation of the gap. Adds a concise 'Ephemeral Semantics on Stateful Providers' section to docs/HOOKS_API.md covering: - user/assistant-role injections ride the delta once, then persist server-side for the rest of the chain (no retraction API); repeated injections accumulate, bounded in practice by prompt caching + compaction chain resets - system-role injections ARE retractable on these providers (per-request instructions typically replace the chain root's) - never omit the system/instructions field on a chained request -- omission can clear it entirely rather than falling back - content already echoed into a visible assistant reply is permanent conversation history regardless of later retraction - stateless full-rebuild providers are unaffected; there, ephemeral additionally serves as a cache-breakpoint placement hint Also adds a one-sentence pointer from the ephemeral field's docstring (python/amplifier_core/models.py) to the new doc section. Docs and docstrings only -- zero behavior changes. Findings live-probed against a stateful, previous_response_id- chaining Responses-API provider on 2026-08-28 (probe script + results available on request). 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- docs/HOOKS_API.md | 37 +++++++++++++++++++++++++++++++++ python/amplifier_core/models.py | 5 ++++- 2 files changed, 41 insertions(+), 1 deletion(-) diff --git a/docs/HOOKS_API.md b/docs/HOOKS_API.md index f0afe833..c68dae99 100644 --- a/docs/HOOKS_API.md +++ b/docs/HOOKS_API.md @@ -137,6 +137,7 @@ When multiple handlers return different actions for the same event, they are res - Description: If `True`, injection is temporary (only for current LLM call, not stored in conversation history). Use for transient state that updates frequently (todo reminders, live status). Orchestrator appends ephemeral injection to messages without storing in context. - Use Cases: Todo state, live metrics, temporary warnings - Not Recommended For: Persistent feedback, linter errors that need to stay visible +- **Provider caveat**: this contract is enforced client-side only — see "Ephemeral Semantics on Stateful Providers" below for what it actually means once the message reaches a provider with server-held chain state. #### Approval Gate Fields @@ -180,6 +181,42 @@ When multiple handlers return different actions for the same event, they are res --- +### Ephemeral Semantics on Stateful Providers + +`ephemeral` is a **client-side-only** contract: it controls whether Amplifier's +local context module retains the injection, not what happens once the +message reaches the provider. On providers that chain conversation state +server-side (sending only a delta of new items per request, rather than +re-sending the full transcript each turn), the practical semantics differ +from "not stored": + +- A `user`/`assistant`-role ephemeral injection rides the delta **once** and + then persists in the provider's server-held state for the rest of the + chain — there is no retraction API. Re-injecting the same content on later + turns **accumulates** additional copies rather than replacing the earlier + one; in practice this is bounded, since accumulated history rides on the + provider's own prompt caching once past its auto-cache token floor, and a + context-compaction event (which resets the chain) flushes it. +- A `system`-role injection behaves differently and generally **is** + retractable: providers in this category typically re-derive a per-request + system/instructions field from the current turn rather than the chain + root, so withdrawing it on the next request causes it to disappear from + what the model sees. This is the mechanism mode enter/exit relies on. +- Never omit the system/instructions field on a chained request once one has + been sent — omitting it can clear the provider's system prompt entirely, + rather than falling back to an earlier value. +- Content the model has already echoed into a visible assistant reply is + permanent conversation history on these providers, regardless of any later + retraction of the field that originally prompted it. +- Providers that rebuild the full request from scratch each turn (no + server-held state) are unaffected: a local drop is a remote drop, and here + `ephemeral` additionally serves only as a cache-breakpoint placement hint. + +Live-probed against a stateful, `previous_response_id`-chaining Responses-API +provider on 2026-08-28. + +--- + ## Hook Registration Register hooks to handle specific events. diff --git a/python/amplifier_core/models.py b/python/amplifier_core/models.py index 218d6499..2bf33300 100644 --- a/python/amplifier_core/models.py +++ b/python/amplifier_core/models.py @@ -241,7 +241,10 @@ class HookResult(BaseModel): description=( "If True, injection is temporary (only for current LLM call, not stored in history). " "Use for transient state like todo reminders that update frequently. " - "Orchestrator must append ephemeral injection to messages without storing in context." + "Orchestrator must append ephemeral injection to messages without storing in context. " + "This is enforced client-side only: providers that chain conversation state " + "server-side may still retain the injected content across turns after it is sent. " + "See docs/HOOKS_API.md's 'Ephemeral Semantics on Stateful Providers' section." ), )