Shared identity function for agent invocations (prompt normalization + content hashing)

[lambdabaa]

Summary

Multiple planned subsystems need to answer the same question — "is this agent invocation the same as that one?" — and each would currently have to invent its own answer:

Consumer	What it needs the answer for
Nix-style node-level caching / early cutoff (the Ledger proposal)	Skip re-running a node whose inputs are byte-identical to a previous run
Record-and-replay integration testing	Look up a recorded cassette for a deterministic replay
Artifact provenance / manifest DAG	Attribute each output artifact to the invocation that produced it

Today there is no canonical identity for an agent invocation, and the naive approach — hashing the raw request — does not work, because prompts are not stable. This issue proposes extracting a single, well-tested identity function as shared infrastructure before any of the consumers are built.

Why naive hashing fails

An AgentRunRequest (factory/models.py:594) carries prompt, task, role, model, cwd, and session fields. The prompt embeds:

• absolute paths (tmp dirs differ per machine and per test run),
• timestamps and run IDs,
• content from earlier nodes in the cycle (prior artifacts, evolving strategy state).

So hash(prompt) is machine-specific and time-specific: a cache keyed on it never hits, and a replay system keyed on it never matches. The correctness of any caching or replay system lives entirely in the normalization rules applied before hashing — and those rules have a sharp edge in both directions:

• Too aggressive (strips semantic content): a cache that silently treats a changed prompt as unchanged — it hides prompt regressions and replays stale outputs.
• Too timid (keeps volatility): a cache that never hits, or a test suite whose cassettes must be re-recorded on every innocuous change.

The correctness trap: attempt number must be part of the identity

The subtle requirement, discovered while reviewing the Ledger proposal's identity cache: a node that gets rejected and retried (e.g. via a RELOOP verdict, factory/workflow/primitives.py:60) runs with byte-identical inputs on the retry. If the identity excludes the attempt number:

1. Node fails → output rejected → RELOOP.
2. Retry has identical inputs → identity-cache hit → the rejected output is replayed.
3. Gate rejects it again → RELOOP → infinite loop that consumes no budget and never converges.

The retry loop count (or attempt number) must be a component of the identity, and the cache must treat "same inputs, different attempt" as a miss. This applies equally to replay-based tests: a retry within a workflow must not replay the recorded first-attempt output.

Proposal

A standalone module (e.g. factory/identity.py) exposing:

def normalize_prompt(prompt: str, *, project_path: Path | None = None) -> str:
    """Canonicalize a prompt: absolute paths → placeholders, strip run IDs /
    timestamps / session identifiers. Documented, deterministic, and
    deliberately conservative — only true volatility is removed."""

def invocation_identity(
    request: AgentRunRequest,
    *,
    attempt: int = 0,
    input_artifact_hashes: list[str] | None = None,
) -> str:
    """Stable digest over (role, model, normalized prompt, normalized task,
    attempt, sorted input artifact hashes)."""

Properties the implementation must hold (and be property-tested for):

1. Path-independence: the same logical request yields the same digest on any machine, any tmp dir, any checkout path.
2. Semantic sensitivity: changing the meaning of a prompt or task (new instruction, changed constraint) changes the digest.
3. Attempt sensitivity: incrementing attempt changes the digest.
4. Model sensitivity: changing model changes the digest.
5. Order-insensitivity for unordered components (input artifact hashes are sorted).
6. Determinism: no clocks, no randomness, no environment-dependent input.

Normalization rules should live in one table (what is stripped, what is templated, what is kept) so that reviewers can reason about rule changes the same way they reason about prompt-template changes — a normalization change is a cache-invalidation event for every consumer and should be visible as such.

What is explicitly out of scope

• The storage layer for cached artifacts (CAS, SQLite, Dolt, or plain files) — the identity function must be backend-agnostic so the storage decision can be made independently.
• Capturing outputs and workspace side effects (the "cassette" format) — a separate concern; the identity function only defines the key.
• The caching/replay machinery itself.

Acceptance criteria

• factory/identity.py (or equivalent) with the two functions above, consumed by nothing yet — it is infrastructure for the consumers listed above.
• A property-test suite covering the six properties, including the RELOOP/attempt regression as an explicit test case.
• A documented normalization table.
• The module has no dependency on any specific storage backend or consumer.