Skip to content

IMPLEMENTATION_SPEC.md

Latest commit

 

History

History
545 lines (454 loc) · 27.4 KB

IMPLEMENTATION_SPEC.md

File metadata and controls

545 lines (454 loc) · 27.4 KB

IMPLEMENTATION_SPEC — ECHOFORM (Ghost Memory)

Built from dossier: C:/Users/ekada/OneDrive/Desktop/Mind_bending_Scientist/inventions/2026-05-18-echoform-ghost-memory.md Dossier SHA-256: e350745e21d61e8e238bd0c3f75b1ea360402d8fa346e85c803be05c070e1c03 Crystallized at: 2026-05-18T00:00:00Z Crystallizer: mind-build@v2 Phase-0.5 / Chief Engineer Reconciled at: 2026-05-18 (Phase 3 — Architecture Council) Repo slug: ghost-memory Python internal package: echoform Public SDK import: import ghost_memory as gm (façade over echoform.sdk) License: Apache-2.0

0. Mission

ECHOFORM is the only LLM memory substrate that:

  1. Stores unbounded episodic history as a single side-channel FHRR hypervector (zero context tokens).
  2. Injects memory via a residual-stream bias produced by a 4-param/head adapter (no weight update).
  3. Ships a mathematically-derivable signed forgetting certificate that turns the FHRR capacity ceiling into a compliance primitive (Dossier §7.5 — Move 1: Inversion + Ghost Archive composition).

The spec below is the single source of truth for Phase 4 assembly, reconciled against 30 Phase-2 artifacts. Every disagreement raised in Phase 2 is decided here, not left open.

1. Module Graph (15 subsystems)

# Subsystem ID Purpose Stability Owner pair
1 echoform.core.fhrr FHRR binding / unbinding / bundling / capacity_curve at D=8192 (complex64; FFT path; optional Triton fused kernel) internal R3 + E2
2 echoform.core.compression Mamba-2 130M coprocessor with semantic rate-distortion loss; v0.1 ships IdentityCompressor fallback internal R2 + E13
3 echoform.core.adapter Per-head residual-stream injection (4 params × n_heads × n_layers); v0.1 = Llama-3.1-70B layer-16 only internal R4 + E3
4 echoform.core.consolidation CLS-inspired offline replay scheduler with salience-weighted re-binding (recency-decayed in v0.1; HiCL DG/CA3/CA1 in v0.2) internal R9 + E4
5 echoform.storage.episodic Hot tier — recent raw episodes (Postgres 16 + pgcrypto + asyncpg 0.31) internal R14 + E5
6 echoform.storage.semantic FHRR superposition state — single tensor, versioned snapshots, optimistic version locking internal R14 + E6
7 echoform.storage.archive Ghost Archive — cold WORM fallback (S3/MinIO; LocalArchive fallback in v0.1) internal R15 + E7
8 echoform.api.public FastAPI HTTP/REST surface: remember / bias / forget / export / certificate / health public R8 + E8
9 echoform.api.proxy Managed inference proxy wrapping OpenAI/Anthropic/local LLMs (501 in v0.1, live in v0.2) experimental R16 + E9
10 ghost_memory (SDK) Public Python SDK façade re-exporting echoform.sdk.{Sync,Async}GhostMemory public R27 + E10
11 echoform.sdk.cli typer 0.15.1 CLI: `echoform serve remember bias
12 echoform.obs.telemetry structlog 25.5.0 + OpenTelemetry 1.41.1 + prometheus-client 0.21 platform R10 + E12
13 echoform.sec.certificate Ed25519 JWS forgetting certificate mint; TEE-attested quote (stub v0.1, live v1.0) platform R6 + E13-crypto
14 echoform.ops.k8s Kubernetes manifests + Helm + TEE-attested Sovereign deployment platform R1 + E14
15 echoform.safety.guardrails Presidio PII redactor + MINJA poison detector (v0.2) + red-team eval suite platform R20 + E15

Dependency DAG: see _artifacts/phase3/architecture-adr.md §2. No cycles. echoform.types is the leaf-most root.

2. Public API Surface

2.1 Canonical types — echoform/types.py (single source of truth)

# echoform/types.py — every module imports from here; no duplicates allowed.
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime
from typing import Literal, NewType

TenantId      = NewType("TenantId", str)
AgentId       = NewType("AgentId", str)
TraceId       = NewType("TraceId", str)
EpisodeId     = NewType("EpisodeId", str)        # ULID 26-char Crockford b32
CertificateId = NewType("CertificateId", str)    # ULID
ReceiptId     = NewType("ReceiptId", str)        # ULID
ScanId        = NewType("ScanId", str)           # ULID
HypervectorBytes = NewType("HypervectorBytes", bytes)

D_DEFAULT:    int   = 8192
BETA_DEFAULT: float = 0.03
EPISODE_MAX_TOKENS:        int = 2048
EPISODE_MAX_CONTENT_BYTES: int = 65_536   # 64 KiB → HTTP 413

@dataclass(frozen=True, slots=True)
class Episode:
    episode_id: EpisodeId
    tenant_id:  TenantId
    agent_id:   AgentId
    content:    str
    salience:   float
    timestamp:  datetime
    trace_id:   TraceId | None = None
    subject_pseudonym: str | None = None

@dataclass(frozen=True, slots=True)
class MemoryWrite:
    episode_id: EpisodeId
    agent_id:   AgentId
    content:    str
    salience:   float = 0.5
    trace_id:   TraceId | None = None

@dataclass(frozen=True, slots=True)
class WriteAck:
    episode_id:     EpisodeId
    accepted:       bool
    capacity_pct:   float
    certificate_id: CertificateId

@dataclass(frozen=True, slots=True)
class BiasVector:
    layer:          int
    vector_b64:     str
    model_family:   str
    issued_at:      datetime
    certificate_id: CertificateId

@dataclass(frozen=True, slots=True)
class ForgettingCertificate:
    certificate_id:        CertificateId
    tenant_id:             TenantId
    D:                     int
    beta:                  float
    episode_count:         int
    archive_count:         int
    n_star_certified:      int
    issued_at:             datetime
    valid_until:           datetime
    forgetting_curve:      list[dict]
    adapter_sha256:        str
    coprocessor_sha256:    str
    proxy_calibration_eps: float
    enclave_quote_b64:     str | None
    signature_jws:         str

@dataclass(frozen=True, slots=True)
class ErasureReceipt:
    receipt_id:           ReceiptId
    tenant_id:            TenantId
    subject_pseudonym:    str
    erased_episode_count: int
    pre_hash:             str
    post_hash:            str
    post_erasure_cosine:  float
    issued_at:            datetime
    signature_jws:        str

@dataclass(frozen=True, slots=True)
class FilterDecision:
    allow:        bool
    threat_class: Literal["minja","indirect_injection","jailbreak","pii_leak"] | None
    confidence:   float
    filter_id:    str

@dataclass(frozen=True, slots=True)
class DriftMetric:
    cumulative_drift:        float
    replay_count:            int
    saturation_pre:          float
    saturation_post:         float
    certificate_still_valid: bool

2.2 HTTP endpoints

All endpoints accept Authorization: Bearer <tenant-jwt> (HS256 self-hosted; EdDSA Sovereign). All mutating routes accept Idempotency-Key: <uuid4> (24 h replay-cache in Redis). All responses include x-trace-id (W3C).

Method Path Stab. Pydantic schemas (req → res) Errors p99 budget
POST /v1/remember stable MemoryWriteRequest → MemoryWriteResponse 400/401/413/422/429/503/507 5 ms
POST /v1/bias stable BiasRequest → BiasResponse 400/401/404/409/503 20 ms
GET /v1/certificate stable query → ForgettingCertificateResponse 401/404 50 ms
DELETE /v1/forget stable ForgetRequest → ForgetResponse (JWS) 401/403/404 200 ms
GET /v1/export stable query → NDJSON stream EpisodeRecord 401/404 2 s
POST /v1/proxy/messages experimental OpenAI/Anthropic-compat + gm_agent_id 4xx/5xx pass-through upstream +30 ms
GET /healthz//readyz//metrics stable — 503 only on readyz ≤ 10 ms
POST /v1/replay/trigger internal ReplayTriggerRequest → ReplayTriggerResponse 401/403 200 ms
GET /v1/audit/{event_id} internal path → AuditEvent 401/403/404 50 ms

2.3 Error taxonomy

E_INVALID_INPUT          400   request schema mismatch
E_UNAUTHENTICATED        401   missing or invalid bearer token
E_FORGET_NOT_ALLOWED     403   legal hold prevents erasure
E_NOT_FOUND              404   tenant/agent/episode unknown
E_CERTIFICATE_STALE      409   adapter or coprocessor SHA changed
E_PAYLOAD_TOO_LARGE      413   episode.content > 64 KiB
E_VALIDATION_FAILED      422   pydantic schema failure (incl. missing Idempotency-Key)
E_QUOTA_EXCEEDED         429   per-tenant rate limit (slowapi token bucket)
E_DEPENDENCY_DOWN        503   pg/redis/s3 unavailable
E_CAPACITY_EXHAUSTED     507   FHRR capacity ≥ 0.95; auto-spill to Ghost Archive

2.4 SDK surface (ghost_memory)

import ghost_memory as gm                       # facade over echoform.sdk
mem = gm.GhostMemory(
    tenant_id="acme-prod",
    model_family="llama-3.1-70b",
    endpoint="https://ghost.example.com",
)
mem.remember("user asked about Q3 forecast")    # POST /v1/remember
bias = mem.bias(query="what did the user ask earlier?")  # POST /v1/bias
cert = mem.certificate()                        # GET  /v1/certificate
mem.forget(subject_pseudonym="usr_91abf")       # DELETE /v1/forget
# AsyncGhostMemory mirrors the same surface.

2.5 CLI surface (echoform)

echoform serve         # uvicorn-based dev server
echoform remember --episode @file.json
echoform bias --agent a1 --model llama-3.1-70b --layer 16
echoform certificate --tenant acme-prod --verify
echoform forget --episode-glob "user:91abf:*" --dry-run
echoform replay --salience-threshold 0.7
echoform version

3. Data Model (Postgres 16)

Schema gm. Migrations by Alembic 1.13.3. pgcrypto enabled for column-level encryption on Direct/Sensitive PII.

-- 0001_init.sql (Alembic-managed)
CREATE EXTENSION IF NOT EXISTS pgcrypto;
CREATE EXTENSION IF NOT EXISTS pg_trgm;
CREATE SCHEMA gm;

CREATE TABLE gm.episodes (
    episode_id        TEXT PRIMARY KEY,
    tenant_id         TEXT NOT NULL,
    agent_id          TEXT NOT NULL,
    content_enc       BYTEA NOT NULL,         -- pgp_sym_encrypt(content, key)
    salience          REAL NOT NULL DEFAULT 0.5 CHECK (salience BETWEEN 0 AND 1),
    written_at        TIMESTAMPTZ NOT NULL DEFAULT now(),
    consolidated_at   TIMESTAMPTZ,
    trace_id          TEXT,
    subject_pseudonym TEXT,
    archive_uri       TEXT
);
CREATE INDEX idx_ep_tenant_time ON gm.episodes (tenant_id, written_at DESC);
CREATE INDEX idx_ep_subject     ON gm.episodes (subject_pseudonym);
CREATE INDEX idx_ep_unconsol    ON gm.episodes (tenant_id) WHERE consolidated_at IS NULL;

CREATE TABLE gm.superpositions (
    tenant_id      TEXT NOT NULL,
    agent_id       TEXT NOT NULL,
    version        BIGINT NOT NULL,
    hypervector    BYTEA NOT NULL,            -- complex64 D=8192 → 65,536 B
    episode_count  INTEGER NOT NULL,
    saturation_pct REAL NOT NULL,
    updated_at     TIMESTAMPTZ NOT NULL DEFAULT now(),
    PRIMARY KEY (tenant_id, agent_id, version)
);

CREATE TABLE gm.certificates (
    certificate_id        TEXT PRIMARY KEY,
    tenant_id             TEXT NOT NULL,
    d_param               INTEGER NOT NULL,
    beta_param            REAL NOT NULL,
    episode_count         INTEGER NOT NULL,
    archive_count         INTEGER NOT NULL,
    n_star_certified      INTEGER NOT NULL,
    issued_at             TIMESTAMPTZ NOT NULL DEFAULT now(),
    valid_until           TIMESTAMPTZ NOT NULL,
    forgetting_curve      JSONB NOT NULL,
    adapter_sha256        TEXT NOT NULL,
    coprocessor_sha256    TEXT NOT NULL,
    proxy_calibration_eps REAL NOT NULL,
    enclave_quote_b64     TEXT,                -- NULL outside Sovereign
    signature_jws         TEXT NOT NULL
);
CREATE INDEX idx_cert_tenant_time ON gm.certificates (tenant_id, issued_at DESC);

CREATE TABLE gm.audit_log (
    event_id         BIGSERIAL PRIMARY KEY,
    occurred_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
    tenant_id        TEXT NOT NULL,
    actor            TEXT NOT NULL,
    event_kind       TEXT NOT NULL,        -- remember|bias|forget|export|cert|replay
    fhrr_digest      TEXT,
    episode_set_hash TEXT,
    trace_id         TEXT,
    payload_redacted JSONB
);
CREATE INDEX idx_audit_tenant_time ON gm.audit_log (tenant_id, occurred_at DESC);

CREATE TABLE gm.erasure_receipts (
    receipt_id           TEXT PRIMARY KEY,
    tenant_id            TEXT NOT NULL,
    subject_pseudonym    TEXT NOT NULL,
    erased_episode_count INTEGER NOT NULL,
    pre_hash             TEXT NOT NULL,
    post_hash            TEXT NOT NULL,
    post_erasure_cosine  REAL NOT NULL,        -- AC-10 audit field
    issued_at            TIMESTAMPTZ NOT NULL DEFAULT now(),
    signature_jws        TEXT NOT NULL
);

CREATE TABLE gm.idempotency_keys (             -- 24 h dedup
    tenant_id   TEXT NOT NULL,
    key         TEXT NOT NULL,
    method      TEXT NOT NULL,
    path        TEXT NOT NULL,
    response    JSONB NOT NULL,
    expires_at  TIMESTAMPTZ NOT NULL,
    PRIMARY KEY (tenant_id, key)
);

3.1 Data classification (Rail 13)

Entity PII class Retention Encrypted at rest Backup Erasable
gm.episodes Direct (content) + Pseudonymous (ids) Hot 90d → Archive yes (pgcrypto AES-256) continuous WAL yes (Art. 17)
gm.superpositions None (opaque tensor) indefinite (versioned) yes (transparent FS) continuous indirect (re-bind)
gm.certificates Pseudonymous 7 years yes daily no (legal hold)
gm.audit_log Pseudonymous + redacted 7 years (Art. 12 floor: 6 mo) yes daily no (regulatory)
gm.erasure_receipts Pseudonymous 10 years yes daily no (legal hold)
gm.idempotency_keys None 24 h n/a none n/a
Archive (S3) Direct (encrypted blobs) indefinite (configurable per tenant) yes (SSE-KMS) cross-region yes (object DELETE)

4. Performance Budgets

Reference node: c7gd.4xlarge (16 vCPU, 32 GiB RAM, 1× A10G GPU).

Endpoint p50 p99 Throughput Dominant cost Optimization
/v1/remember 1.5 ms 5 ms 50 k QPS/node pgcrypto write batch insert; WAL flush async
/v1/bias 8 ms 20 ms 10 k QPS/node FHRR cdot probe + adapter MLP Triton fused complex GEMV + TTLCache on superposition
/v1/forget 50 ms 200 ms 100 QPS/node FHRR re-bind + Archive purge salience-weighted re-bind only over affected slice
/v1/certificate 5 ms 50 ms 5 k QPS/node JSONB read cachetools LRU on superposition.version
/v1/export 200 ms 2 s 50 QPS/node NDJSON stream server-side cursor + gzip
/v1/proxy/messages upstream +5 ms upstream +30 ms upstream-bound bias compute

Critical invariants (Phase 5 verifies):

  • AC-12: bias is O(1) in episode count — p50 variance ≤ 15 % across episode_count ∈ {10, 1k, 100k}.
  • AC-9: bias p99 ≤ 20 ms at 100 QPS sustained for 60 s (Locust).
  • AC-4: FHRR round-trip cosine ≥ 0.95 for n ≤ n*(D).

5. Security Boundaries (STRIDE)

[ Client SDK ] ─TLS 1.3─► [ Envoy / api-gw ] ─mTLS─► [ gm-api (stateless) ]
                                                          │
                                                          ├─ asyncpg ─► [ Postgres 16 ]
                                                          ├─ boto3   ─► [ S3 / MinIO ]
                                                          └─ redis   ─► [ Redis 7 ]
                                                                            ▲
                                                       [ gm-worker (consolidation) ┘ ]
Threat Vector Mitigation
Spoofing Forged tenant JWT EdDSA-signed JWTs (Sovereign); per-tenant keys in HSM/AWS-KMS; pyjwt 2.9.0
Tampering Mutated FHRR state superposition row version BIGINT optimistic locking + SHA-256 chain
Repudiation "We never remembered that" gm.audit_log per event w/ W3C trace_id; certificate signs episode_set_hash
Info disclosure Cross-tenant via shared coprocessor weights Per-tenant adapter fine-tune (Pro/Sovereign); DP-SGD on base; certificate attests training-data isolation
DoS Bias path saturation slowapi 0.1.9 per-tenant token bucket; GPU pool quota; circuit-breaker on /v1/bias
EoP MINJA-style memory poisoning safety.guardrails filter; adapter inputs L2-clipped; replay scheduler refuses bind on low-trust episodes (AC-11)

TEE attestation (Sovereign): AMD SEV-SNP / Intel TDX / AWS Nitro. Quote embedded in every ForgettingCertificate.enclave_quote_b64. Verified via echoform certificate --verify --enclave-policy <policy.json>.

Secret handling: 12-Factor; pydantic-settings 2.5.2 validates at startup; gitleaks in CI; structlog redact_pii=True filter mandatory.

6. Runtime Topology

client SDK ─TLS─► Envoy ─mTLS─► gm-api (FastAPI 0.136.1 + uvicorn 0.32.1, 3+ replicas)
                                  │       │
                                  │       └─► Redis 7 (replay queue, cert cache, idempotency, rate-limit)
                                  ▼
                              Postgres 16 (primary + 2 RR, pgcrypto, pgBackRest)
                                  │
                                  ▼ (gm-worker reads streams)
                              gm-worker (APScheduler + Redis Streams; single replica)
                                  │
                                  ▼
                              MinIO / S3 (Ghost Archive, COMPLIANCE object-lock, cross-region)

7. The Value Path (smoke target)

A client calls POST /v1/remember 100 times with diverse episodes, then calls POST /v1/bias with a query semantically similar to episode 42; the returned bias vector produces ≥ 0.15 higher cosine similarity against the episode-42 FHRR key than against a randomly-selected episode key; the round-trip completes in < 30 s end-to-end; the response includes a valid certificate_id signed by the deployment key.

Implemented at tests/smoke/test_value_path.py; run via python tasks.py smoke. Phase 5 runs this verbatim.

8. Acceptance Criteria

Each AC is binary, verifiable by exit code or grep.

  • AC-1: python tasks.py smoke exits 0 in ≤ 30 s on a clean clone after docker compose up -d.
  • AC-2: python tasks.py test reports 100 % pass and ≥ 80 % line coverage on echoform/core/.
  • AC-3: docker compose up → /healthz 200 within 60 s; /readyz reports all deps healthy.
  • AC-4: FHRR bind/unbind round-trip cosine ≥ 0.95 for first 1,000 bundled items at D=8192 (tests/property/test_fhrr_roundtrip.py).
  • AC-5: Every emitted ForgettingCertificate validates against schemas/forgetting_certificate.schema.json (Draft 2020-12) and its signature_jws verifies under the published deployment ed25519 key.
  • AC-6: pip-licenses --fail-on='GPL;LGPL;AGPL;SSPL;BUSL;Commons Clause' exits 0 across all locked deps.
  • AC-7: gitleaks detect --no-git --redact reports zero findings.
  • AC-8: syft packages dir:. -o cyclonedx-json > sbom.json && cyclonedx validate --input-file sbom.json succeeds.
  • AC-9: /v1/bias p99 ≤ 20 ms at 100 QPS sustained 60 s in tests/perf/test_bias_p99.py (Locust + pytest-benchmark; gates ±10 %).
  • AC-10: GDPR Art. 17 erasure: after DELETE /v1/forget, replay of subject's episodes through core.fhrr.unbind yields cosine ≤ 0.05 against any episode key across both live superposition AND spilled Archive objects; erasure JWS verifies (tests/integration/test_gdpr_erasure.py).
  • AC-11: Memory-poisoning regression: MINJA-style indirect injection (arxiv 2601.05504) detected with ≥ 90 % recall on included eval set (tests/security/test_memory_poisoning.py).
  • AC-12: Bias path is O(1) in episode count: p50 variance across episode_count ∈ {10, 1k, 100k} ≤ 15 % (tests/perf/test_bias_o1.py).
  • AC-13: BUILT_FROM.md exists, contains dossier SHA-256, and is parseable by scripts/check_built_from.py.
  • AC-14: docs/security-baseline.md covers all 6 STRIDE categories.
  • AC-15: CI matrix passes on {ubuntu-latest, macos-latest, windows-latest} × {python-3.11, python-3.12} for library tests; service tests gated to ubuntu-latest × python-3.11.
  • AC-16 (NEW): Cross-tier erasure: post_erasure_cosine field in every ErasureReceipt is computed against the post-spill superposition, not the pre-spill (covered by AC-10 but also tracked separately).
  • AC-17 (NEW): Import contract: lint-imports.cfg (import-linter) reports zero violations against the DAG in _artifacts/phase3/architecture-adr.md §2.

9. Dossier §7.5 Mitigation — Capacity Ceiling → Compliance Primitive

Meta-con: FHRR superposition capacity ceiling of ~D / (β · ln D) reliably-decodable bindings makes "unlimited memory" technically false.

Implementation:

  1. Inversion → echoform.sec.certificate. Each deployment emits a ForgettingCertificate with closed-form forgetting_curve derived from (D, β, n) via echoform.core.fhrr.capacity.forgetting_curve(). Signed ed25519 + JWS detached; TEE-attested under Sovereign.

  2. Composition → echoform.storage.archive. Episodes flagged at capacity_pct ≥ 0.95 (or below per-tenant salience threshold) spill to Ghost Archive cold WORM (S3 x-amz-object-lock-mode: COMPLIANCE). Archive objects appear in /v1/export, count in archive_count, and are subject to the same erasure receipt.

  3. Cross-module invariants (Phase 5 verifies):

    • certificate.episode_count + certificate.archive_count == sum(superposition.episode_count) + archive_count
    • forgetting_curve[0].p_recall ≥ 0.95 when episode_count ≤ n_star_certified ≤ floor(D / (β · ln D))
    • Any episode reachable via /v1/export is either (a) in active superposition with cos ≥ 0.95 recall or (b) in Archive with hash-chain back to the certificate.
    • After forget(): post_erasure_cosine ≤ 0.05 against erased key, AND any archived copy of the subject is DELETEd from S3 with a tombstone signed by the same certificate key.

Phase 4.5 audits this section hardest.

10. Stack Decisions (reconciled — see _artifacts/phase3/architecture-adr.md §1 for full table)

Layer Choice Pinned
Python CPython 3.11 (CI also 3.12 for library)
Web FastAPI ==0.136.1
ASGI uvicorn[standard] ==0.32.1
Validation pydantic ==2.9.2
Settings pydantic-settings ==2.5.2
HTTP client httpx ==0.27.2 (both service AND SDK)
Retries tenacity ==9.0.0
ML core torch ==2.6.0+cu124
SSM mamba-ssm ==2.3.2.post1
Causal conv causal-conv1d ==1.4.0
HF transformers ==4.46.3
Accelerate accelerate ==1.0.1
HDC ref torch-hd ==5.8.4
Numerics numpy ==2.1.3
Scientific scipy ==1.14.1
Triton triton ==3.2.0
Postgres asyncpg ==0.31.0
SQL/Migr sqlalchemy / alembic ==2.0.36 / ==1.13.3
S3 boto3 ==1.35.45
Redis redis ==5.0.8
Caches cachetools ==5.5.0 (was fabricated 7.1.3; corrected)
Crypto cryptography ==43.0.3
JWS pyjwt[crypto] ==2.9.0 (replaces python-jose)
ULID python-ulid ==3.0.0
OTel SDK opentelemetry-sdk + exporter-otlp ==1.41.1
OTel instr opentelemetry-instrumentation-* ==0.62b1
Metrics prometheus-client ==0.21.0
Logging structlog ==25.5.0 (spec said "25.x — confirm"; confirmed)
PII presidio-{analyzer,anonymizer} ==2.2.355
spaCy spacy ==3.7.5 (+ en_core_web_lg==3.7.1)
Rate limit slowapi ==0.1.9
Corr-id asgi-correlation-id ==4.3.4
CLI typer ==0.15.1 (was 0.13 in spec; corrected)
CLI render rich ==14.0.0
Test pytest / pytest-asyncio ==8.3.3 / ==0.24.0
Property hypothesis ==6.114.0
HTTP fakes pytest-httpx ==0.32.0
Containers testcontainers[postgres,redis,minio] ==4.8.2
Bench pytest-benchmark ==4.0.0
Load locust ==2.31.5
Cov coverage[toml] ==7.6.4
Lint ruff ==0.7.1
Typecheck mypy ==1.13.0
Licenses pip-licenses ==5.0.0
SBOM syft (CLI) 1.16.0
Safetensors safetensors ==0.4.5 (was fabricated 0.7.0; corrected)
Base (api) python:3.11-slim digest-pinned
Base (worker) nvidia/cuda:12.4.1-runtime-ubuntu22.04 digest-pinned
K8s kubectl ≥ 1.28, kustomize ≥ 5.3, helm 3.16, cosign 2.4 as cited

Forbidden: distroless base for v0.1 (mamba-ssm/glibc constraint); python-jose (CVE-prone, unmaintained); cachetools 7.x (does not exist); Rust hot path for v0.1 (revisit Y2).

11. Open Questions — RESOLVED in Phase 3

All Phase-2 disagreement noted in the original spec §10 is now decided. Tracked remaining items (small, deferred to Phase 4 / v0.2):

  • Q1 (adapter retrain cadence): DECIDED — auto-retrain on every minor model checkpoint behind a feature flag; 72 h SLA for major releases. api.proxy returns 409 E_CERTIFICATE_STALE on SHA mismatch.
  • Q2 (replay priority): DECIDED — recency-decayed salience in v0.1; HiCL DG/CA3/CA1 in v0.2.
  • Q3 (poisoning FP ceiling): DEFERRED to Phase 4 — design-partner survey gates the threshold; AC-11 default 90 % recall.
  • Q4 (cert payload size): DECIDED — JSON in v0.1; CBOR/COSE in v1.0.
  • Q5 (proxy mode): DECIDED — logit-bias fallback ships in v0.2; switch when Anthropic Activations API GAs; documented_gap=true field in cert until then.
  • Mamba-2 state size N: DEFERRED to Phase 4 — default 64; E13 benchmarks 16/64/128 at first training run.
  • Rate term R(z): DECIDED — 32-component GMM in v0.1, VIB learned prior in v0.2.
  • Contrastive negative source: DECIDED — public ShareGPT shard v0.1; synthetic paraphrase v0.2.

12. References

(Identical to original — see dossier and ADR for full citations.)

  1. Plate (1995) — HRR.
  2. Kanerva (2009) — HDC.
  3. Gu & Dao (2023) — Mamba (arxiv 2312.00752).
  4. Clarkson-Ubaru-Yang (2023) — VSA Capacity (arxiv 2301.10352).
  5. Frady-Sommer (2021) — bundling capacity.
  6. Turner et al. (2023) — ActAdd (arxiv 2308.10248).
  7. Representation Engineering survey (2025, arxiv 2502.17601).
  8. HiCL (2025, arxiv 2508.16651).
  9. MINJA (arxiv 2601.05504).
  10. EU AI Act Article 12.
  11. GDPR Article 17.
  12. mem0.ai State of AI Agent Memory 2026.
  13. PyTorch 2.6 release (pytorch.org/blog/pytorch2-6/).
  14. mamba-ssm 2.3.2.post1 (pypi.org/project/mamba-ssm/).
  15. FastAPI 0.136.1, opentelemetry-sdk 1.41.1, structlog 25.5.0, cachetools 5.5.0, pyjwt 2.9.0 — all version-verified on PyPI as of 2026-05-18.

13. Filling-rule honesty check

  • No "TBD" or "to be decided" anywhere — every Phase-2 conflict resolved in §1 or §10.
  • Every library version exists on PyPI as of 2026-05-18; two prior fabrications (cachetools 7.1.3, safetensors 0.7.0) corrected.
  • Package root reconciled: internal echoform/, public façade ghost_memory/.
  • The Value Path (§7) is one sentence and end-to-end testable in tests/smoke/.
  • §9 (Dossier §7.5 mitigation) names the two implementing modules (sec.certificate, storage.archive) and the cross-tier erasure invariant Phase 4.5 must verify.
  • §10 stack table is reconciled against all 30 Phase-2 artifacts; ADR §1 records every change.

The dossier wins for intent; this spec wins for interface; the ADR (_artifacts/phase3/architecture-adr.md) wins for any disagreement between the two.

— end IMPLEMENTATION_SPEC.md —