Skip to content

docs(hooks): document ephemeral semantics on stateful providers - #105

Merged
Brian Krabach (bkrabach) merged 1 commit into
mainfrom
docs/ephemeral-stateful-provider-semantics
Aug 28, 2026
Merged

docs(hooks): document ephemeral semantics on stateful providers#105
Brian Krabach (bkrabach) merged 1 commit into
mainfrom
docs/ephemeral-stateful-provider-semantics

Conversation

@bkrabach

Copy link
Copy Markdown
Collaborator

Summary

The ephemeral flag on HookResult/context injection (docs/HOOKS_API.md,
python/amplifier_core/models.py) is enforced client-side only — it
controls whether Amplifier's local context module persists the injection,
not what a given provider does with the message once it's sent. On
providers that chain conversation state server-side (rather than
re-sending the full transcript every turn), the real semantics diverge
from what "not stored in conversation history" implies, and hook authors
currently have no documentation of the gap.

This is a docs/docstrings-only PR — zero behavior changes.

What's added

A new "Ephemeral Semantics on Stateful Providers" section in
docs/HOOKS_API.md (worded provider-agnostically), covering:

  • user/assistant-role injections ride the delta once, then persist
    in the provider's server-held state for the rest of the chain — no
    retraction API. Repeated injections accumulate; bounded in practice
    because accumulated history rides the provider's own prompt caching once
    past its auto-cache token floor, and a context-compaction event (which
    resets the chain) flushes it.
  • system-role injections are different and generally are
    retractable — providers in this category typically re-derive the
    per-request system/instructions field from the current turn rather than
    the chain root, so withdrawing it makes it disappear. This is the
    mechanism mode enter/exit relies on.
  • Never omit the system/instructions field on a chained request once one
    has been sent — omission can clear the provider's system prompt
    entirely rather than falling back to an earlier value.
  • Content the model has already echoed into a visible assistant reply is
    permanent conversation history on these providers, regardless of later
    retraction of the field that prompted it.
  • Stateless full-rebuild providers are unaffected — a local drop is a
    remote drop; there, ephemeral additionally serves only as a
    cache-breakpoint placement hint.

Also adds a one-sentence pointer from the ephemeral field's docstring in
python/amplifier_core/models.py to the new doc section, and a
one-line "Provider caveat" cross-reference from the existing field bullet
in docs/HOOKS_API.md.

Why / evidence

Findings are live-probed, not theoretical. A standalone probe script (no
Amplifier imports) was run against a stateful, previous_response_id-
chaining Responses-API-style provider on 2026-08-28:

  • Instructions-field semantics: chained request's own instructions
    replaced the chain root's (U1_verdict: REPLACE) — confirms
    system-role retractability.
  • A user-role delta message (NONCE_BRAVO) injected on turn 1 was still
    visible to the model on turn 2 with no re-send (U2_verdict: CONFIRMED)
    — confirms delta persistence with no retraction.
  • Content already echoed into a visible assistant reply (SECRET_CHARLIE)
    remained visible on turn 3 even after the field that introduced it was
    no longer sent (R3_lingering_verdict: LINGERS) — confirms echoed
    content becomes permanent history.
  • Omitting instructions entirely on a chained request cleared the
    model's view of the previously-set secret rather than falling back to
    the chain root's value (U5_verdict: CLEARED_ON_OMIT).
  • A ~2KB filler block re-sent as a new tail user message on 4 consecutive
    chained turns: input token count grew monotonically, the model's own
    count of visible marker copies matched the request count exactly
    (case4_verdict: COUNT_CONFIRMS_ACCUMULATION, model replied "4" for 4
    injections), and cached-token accounting showed the accumulated history
    rides free once past the provider's auto-cache floor (turn 3 wrote a
    1329-token cache entry; turn 4 read all 1329 back as cached tokens).

Probe script and raw results available on request (kept outside this repo
per house convention — not committed here to avoid depending on this
specific probe/task history from a public doc).

Testing

Docs and one docstring string only. python -m py_compile on the
modified module succeeds; python_check reports only pre-existing
warnings unrelated to this change (verified against the pre-change file).
No code paths, schemas, or defaults changed.


🤖 Generated with Amplifier

The ephemeral flag on HookResult/context_injection is enforced
client-side only -- it controls whether Amplifier's local context
module persists the injection, not what a stateful provider does
with the message once it is sent. Hook authors currently have no
documentation of the gap.

Adds a concise 'Ephemeral Semantics on Stateful Providers' section
to docs/HOOKS_API.md covering:
- user/assistant-role injections ride the delta once, then persist
  server-side for the rest of the chain (no retraction API);
  repeated injections accumulate, bounded in practice by prompt
  caching + compaction chain resets
- system-role injections ARE retractable on these providers
  (per-request instructions typically replace the chain root's)
- never omit the system/instructions field on a chained request --
  omission can clear it entirely rather than falling back
- content already echoed into a visible assistant reply is
  permanent conversation history regardless of later retraction
- stateless full-rebuild providers are unaffected; there, ephemeral
  additionally serves as a cache-breakpoint placement hint

Also adds a one-sentence pointer from the ephemeral field's
docstring (python/amplifier_core/models.py) to the new doc section.

Docs and docstrings only -- zero behavior changes.

Findings live-probed against a stateful, previous_response_id-
chaining Responses-API provider on 2026-08-28 (probe script +
results available on request).

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@bkrabach

Copy link
Copy Markdown
Collaborator Author

Admin merge notice: this PR is self-authored (docs/docstrings only, zero
behavior changes) and is being merged via `--admin` at the repo owner's
direction, since branch protection requires a review this account cannot
self-approve. All CI checks are green (Rust Kernel Tests, Node.js Binding
Tests, Python 3.11/3.12/3.13, license/cla) — merging only after that,
never over a failing check.

@bkrabach
Brian Krabach (bkrabach) merged commit bc6e3c8 into main Aug 28, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants