Skip to content

[Priority 1] Add provenance logging and lineage tracking #60

@marknutter

Description

@marknutter

Problem

We can't answer:

  • 'How was this memory created?' (trace back to original session)
  • 'What recall sessions used this entry?' (forward lineage)
  • 'Replay this recall session with exact context state'

No systematic logging of transformations (archive, summarize, recall, validate).

Proposal (from AIGNE paper analysis)

Add provenance logging for every transformation:

  • archive — session → memory entries
  • summarize — full transcript → summary entry
  • recall — query → retrieved entries → subagent findings → answer
  • validate — answer → confidence score → human override

Schema

CREATE TABLE provenance_log (
    event_id TEXT PRIMARY KEY,
    timestamp REAL,
    event_type TEXT,  -- archive, summarize, recall, validate
    source_ids TEXT,  -- JSON array of input entry IDs
    output_ids TEXT,  -- JSON array of resulting entry IDs
    agent TEXT,       -- Which component performed this
    metadata TEXT     -- JSON with confidence, reasoning, etc.
);

CLI

  • rlm provenance <entry_id> — show full lineage
  • rlm provenance --replay <session_id> — reconstruct exact context

Impact

  • Debugging (why did this recall fail?)
  • Reproducibility (exact replay)
  • Accountability (audit trail)
  • Trust (transparent transformations)

Effort

1-2 days

Related

  • Provenance/traceability from 'Everything is Context' paper
  • Context Evaluator (needs provenance for validation)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions