Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
[![Papers](https://img.shields.io/badge/papers-989-brightgreen.svg)](papers/index.md)
[![PDFs](https://img.shields.io/badge/local_PDFs-534-orange.svg)](papers/pdfs/)
[![Memory products](https://img.shields.io/badge/memory%20products-38-purple.svg)](products/)
[![Benchmarks](https://img.shields.io/badge/benchmarks-21-blueviolet.svg)](benchmarks/)
[![Benchmarks](https://img.shields.io/badge/benchmarks-25-blueviolet.svg)](benchmarks/)
[![Surveys](https://img.shields.io/badge/meta_surveys-6-yellow.svg)](docs/meta-surveys.md)
[![Updated](https://img.shields.io/badge/updated-2026--07-lightgrey.svg)](docs/signals.md)

Expand Down Expand Up @@ -58,10 +58,10 @@ separate buckets.
| Paper index | 989 scraped papers + manual radar additions through 2026-07 | [`papers/index.md`](papers/index.md) | Searchable entry point for agent-memory papers; the 989 count is the 2026-05 scrape baseline. |
| Paper stubs | 988 stubs | [`papers/stubs/`](papers/stubs/) | Track papers that are covered but not yet fully read. |
| Local PDFs | 534 files | [`papers/pdfs/`](papers/pdfs/) | Re-read sources and audit paper notes. |
| Full / seed paper notes | 7 full + 15 seed | [`papers/`](papers/) | Use human-read notes for architectural decisions. |
| Full / seed paper notes | 7 full + 18 seed | [`papers/`](papers/) | Use human-read notes for architectural decisions. |
| Memory product notes | 38 notes | [`products/`](products/) | Compare memory layers, memory SDKs, managed memory, and memory-enabled agents. |
| Product page archives | 37 snapshots | [`products/archives/`](products/archives/) | Audit product claims after source pages change. |
| Benchmark catalog | 21 catalog rows | [`benchmarks/index.md`](benchmarks/index.md) | First-class benchmark records plus stub-backed candidate rows and a usage-claim ledger. |
| Benchmark catalog | 25 catalog rows | [`benchmarks/index.md`](benchmarks/index.md) | First-class benchmark records plus stub-backed candidate rows and a usage-claim ledger. |
| Claims ledger | Structured YAML ledger | [`benchmarks/claims/claims.yaml`](benchmarks/claims/claims.yaml) | Separate vendor claims, paper evaluations, critiques, and reproductions. |
| Cost-savings lane | Seed landscape | [`docs/cost-savings-landscape.md`](docs/cost-savings-landscape.md) | Find agent-memory papers, methods, code paths, and products that reduce token, latency, or runtime cost. |
| Survey and taxonomy | 1 living survey + 6 meta-survey records | [`docs/agent-memory-survey.md`](docs/agent-memory-survey.md) · [`docs/meta-surveys.md`](docs/meta-surveys.md) | Build a field-level view before choosing an implementation. |
Expand Down Expand Up @@ -180,11 +180,11 @@ flowchart LR

| Path | Purpose |
|---|---|
| [`papers/`](papers/) | 7 full paper notes, 15 seed notes, and the master [`index.md`](papers/index.md). |
| [`papers/`](papers/) | 7 full paper notes, 18 seed notes, and the master [`index.md`](papers/index.md). |
| [`papers/stubs/`](papers/stubs/) | 988 generated stubs for papers not yet fully read. |
| [`papers/pdfs/`](papers/pdfs/) | 534 archived PDFs, about 1.8 GB. See the archival policy below. |
| [`papers/_scrape/`](papers/_scrape/) | Reproducibility artifacts: scrape script and dedup JSON. |
| [`benchmarks/`](benchmarks/) | 21 benchmark catalog rows, including protocol notes, stub-backed candidate rows, and the note template. |
| [`benchmarks/`](benchmarks/) | 25 benchmark catalog rows, including protocol notes, stub-backed candidate rows, and the note template. |
| [`benchmarks/claims/`](benchmarks/claims/) | Usage-event ledger for benchmark mentions, vendor claims, critiques, and reproductions. |
| [`benchmarks/archives/`](benchmarks/archives/) | Optional source-page snapshots for benchmark pages, repositories, or dataset cards. |
| [`docs/benchmarks-landscape.md`](docs/benchmarks-landscape.md) | Benchmark landscape by capability, usage type, and evidence independence. |
Expand Down
10 changes: 5 additions & 5 deletions README_cn.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
[![Papers](https://img.shields.io/badge/papers-989-brightgreen.svg)](papers/index.md)
[![PDFs](https://img.shields.io/badge/local_PDFs-534-orange.svg)](papers/pdfs/)
[![记忆产品](https://img.shields.io/badge/memory%20products-38-purple.svg)](products/)
[![Benchmarks](https://img.shields.io/badge/benchmarks-21-blueviolet.svg)](benchmarks/)
[![Benchmarks](https://img.shields.io/badge/benchmarks-25-blueviolet.svg)](benchmarks/)
[![Surveys](https://img.shields.io/badge/meta_surveys-6-yellow.svg)](docs/meta-surveys.md)
[![Updated](https://img.shields.io/badge/updated-2026--07-lightgrey.svg)](docs/signals.md)

Expand Down Expand Up @@ -56,10 +56,10 @@
| 论文索引 | 989 篇抓取论文 + 截至 2026-07 的手工 radar 新增 | [`papers/index.md`](papers/index.md) | 搜索 agent-memory 论文和发现线索;989 是 2026-05 抓取基线。 |
| 论文 stub | 988 个 stub | [`papers/stubs/`](papers/stubs/) | 跟踪已覆盖但尚未 full 阅读的论文。 |
| 本地 PDF | 534 个文件 | [`papers/pdfs/`](papers/pdfs/) | 复读来源和审计论文笔记。 |
| full / seed 论文笔记 | 7 个 full + 15 个 seed | [`papers/`](papers/) | 为架构决策引用人工阅读笔记。 |
| full / seed 论文笔记 | 7 个 full + 18 个 seed | [`papers/`](papers/) | 为架构决策引用人工阅读笔记。 |
| 记忆产品笔记 | 38 个笔记 | [`products/`](products/) | 对比 memory layer、memory SDK、managed memory 和带记忆的 agent 产品。 |
| 产品页面快照 | 37 个快照 | [`products/archives/`](products/archives/) | 在源页面变化后审计产品 claims。 |
| Benchmark 目录 | 21 个 catalog 行 | [`benchmarks/index.md`](benchmarks/index.md) | 理解 memory benchmark、stub-backed 候选行及其 claims 来源。 |
| Benchmark 目录 | 25 个 catalog 行 | [`benchmarks/index.md`](benchmarks/index.md) | 理解 memory benchmark、stub-backed 候选行及其 claims 来源。 |
| Claims ledger | 结构化 YAML ledger | [`benchmarks/claims/claims.yaml`](benchmarks/claims/claims.yaml) | 区分厂商自报、论文评测、方法批评和独立复现。 |
| 成本节省专题 | seed landscape | [`docs/cost-savings-landscape.md`](docs/cost-savings-landscape.md) | 查找能减少 token、延迟或运行成本的 agent-memory 论文、方法、代码和产品。 |
| 综述与 taxonomy | 1 份活综述 + 6 条 meta-survey 记录 | [`docs/agent-memory-survey.md`](docs/agent-memory-survey.md) · [`docs/meta-surveys.md`](docs/meta-surveys.md) | 在选型或设计前建立领域视角。 |
Expand Down Expand Up @@ -175,11 +175,11 @@ flowchart LR

| 路径 | 用途 |
|---|---|
| [`papers/`](papers/) | 7 个 full 论文笔记、15 个 seed 笔记和主索引 [`index.md`](papers/index.md)。 |
| [`papers/`](papers/) | 7 个 full 论文笔记、18 个 seed 笔记和主索引 [`index.md`](papers/index.md)。 |
| [`papers/stubs/`](papers/stubs/) | 988 个尚未 full 阅读论文的生成 stub。 |
| [`papers/pdfs/`](papers/pdfs/) | 534 个本地 PDF,约 1.8 GB。详见下方存档策略。 |
| [`papers/_scrape/`](papers/_scrape/) | 可复现产物:抓取脚本和 dedup JSON。 |
| [`benchmarks/`](benchmarks/) | 21 个 benchmark catalog 行,包含协议笔记、stub-backed 候选行和笔记模板。 |
| [`benchmarks/`](benchmarks/) | 25 个 benchmark catalog 行,包含协议笔记、stub-backed 候选行和笔记模板。 |
| [`benchmarks/claims/`](benchmarks/claims/) | benchmark 提及、厂商自报、方法批评和复现的 usage-event ledger。 |
| [`benchmarks/archives/`](benchmarks/archives/) | benchmark 页面、repo 或 dataset card 的可选审计快照。 |
| [`docs/benchmarks-landscape.md`](docs/benchmarks-landscape.md) | 按能力、使用方式和证据独立性整理的 benchmark 全景。 |
Expand Down
78 changes: 78 additions & 0 deletions benchmarks/bad-memory-prompt-injection.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
---
title: Bad Memory — prompt-injection risk from persistent agent memory
benchmark_id: bad-memory-prompt-injection
name: Bad Memory
aliases:
- Bad Memory
status: seed
origin_type: paper_origin
origin_source: https://arxiv.org/abs/2607.14611
first_public_date: 2026-07
domain: memory_security
modality: text
task_grain: persistent_memory_prompt_injection
capability_axes:
- memory_file_integrity
- persistent_prompt_injection
- multi_session_attack
- memory_update_defense
data_nature: sandboxed_synthetic_workspace
metrics:
- attack_success_rate
- payload_persistence
judge_type: check
code_available: check
data_available: check
license: check
known_limitations:
- seed note; sandbox setup, systems, model versions, and released package need full read
canonical_sources:
- https://arxiv.org/abs/2607.14611
confidence: medium
memory_modules:
- evaluator-benchmark
- policy-privacy
- ingest-adapter
last_revised: 2026-07-20
---

# Bad Memory

## What It Measures

Bad Memory evaluates prompt-injection risks introduced by persistent memory
files, behavioral preferences, and knowledge bases in agentic systems. It is
directly relevant to coding-agent memory because the paper studies planted
payloads that can influence future sessions through durable memory state.

## Protocol

The arXiv abstract describes a sandboxed synthetic workspace and evaluations
over Claude Code and OpenAI Codex-style systems. It distinguishes difficulty in
getting an agent to overwrite its own memory from the risk of payloads already
present in persistent memory files. Full workspace construction, adversarial
goals, judge prompts, released package, and model-version details need a deeper
read.

## Baselines and Reported Results

No normalized scores are logged here. Reported attack success and payload
persistence remain paper-origin claims until the setup is mapped and compared.

## Comparability Notes

Compare with MemPoison and memory-poisoning trajectory-forensics work on threat
model, not raw attack success. Bad Memory is especially useful for file-backed
agent memory and coding-agent memory systems.

## Related Papers

- Paper:https://arxiv.org/abs/2607.14611

## Impact Use

- `policy-privacy`:candidate protocol for protecting persistent memory updates
and file-backed memory state.
- `ingest-adapter`:tests whether untrusted content can cross into durable
memory.
- Ready for ImpactReport:no, upgrade after full protocol read.
80 changes: 80 additions & 0 deletions benchmarks/claims/claims.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,26 @@ events:
- benchmarks/memsyco-bench.md
- https://arxiv.org/abs/2607.01071
extract_confidence: medium
- event_id: pm-bench-origin-2026
benchmark_id: pm-bench
source_kind: paper
source_id: benchmarks/pm-bench.md
source_date: 2026-07
actor: PM-Bench authors
actor_type: paper_authors
usage_type: originates_benchmark
claim_direction: methodological_only
compared_systems: []
reported_metrics: []
experimental_setup: "Prospective-memory benchmark over a simulated seven-day week; local note is seed quality."
reproduction_status: not_reproduced
evidence_level: primary_pdf
independence_class: survey_only
comparability_notes: "Origin protocol only; author-reported F1 scores, agent configurations, released data, and code need full normalization before use."
evidence_refs:
- benchmarks/pm-bench.md
- https://arxiv.org/abs/2607.12385
extract_confidence: medium
- event_id: memleak-origin-2026
benchmark_id: memleak
source_kind: paper
Expand Down Expand Up @@ -195,6 +215,66 @@ events:
- benchmarks/memdelta.md
- https://arxiv.org/abs/2606.29914
extract_confidence: medium
- event_id: mempoison-origin-2026
benchmark_id: mempoison
source_kind: paper
source_id: benchmarks/mempoison.md
source_date: 2026-07
actor: MemPoison authors
actor_type: paper_authors
usage_type: originates_benchmark
claim_direction: methodological_only
compared_systems: []
reported_metrics: []
experimental_setup: "Persistent memory-poisoning benchmark and analysis over hand-validated cases; local note is seed quality."
reproduction_status: not_reproduced
evidence_level: primary_pdf
independence_class: survey_only
comparability_notes: "Origin protocol only; reported attack success and defense rates need full setup normalization before use."
evidence_refs:
- benchmarks/mempoison.md
- https://arxiv.org/abs/2607.14651
extract_confidence: medium
- event_id: memops-origin-2026
benchmark_id: memops
source_kind: paper
source_id: benchmarks/memops.md
source_date: 2026-07
actor: MemOps authors
actor_type: paper_authors
usage_type: originates_benchmark
claim_direction: methodological_only
compared_systems: []
reported_metrics: []
experimental_setup: "Memory-operation benchmark over remember, forget, update, reflect, and no-op choices; local note is seed quality."
reproduction_status: not_reproduced
evidence_level: primary_pdf
independence_class: survey_only
comparability_notes: "Origin protocol only; operation traces, scoring rules, and reported model behavior need full normalization before use."
evidence_refs:
- benchmarks/memops.md
- https://arxiv.org/abs/2607.12893
extract_confidence: medium
- event_id: bad-memory-origin-2026
benchmark_id: bad-memory
source_kind: paper
source_id: benchmarks/bad-memory-prompt-injection.md
source_date: 2026-07
actor: Bad Memory authors
actor_type: paper_authors
usage_type: originates_benchmark
claim_direction: methodological_only
compared_systems: []
reported_metrics: []
experimental_setup: "Persistent prompt-injection attacks against memory files, preferences, knowledge bases, and defenses; local note is seed quality."
reproduction_status: not_reproduced
evidence_level: primary_pdf
independence_class: survey_only
comparability_notes: "Origin protocol only; attack success rates, defense setups, and agent configurations need full normalization before use."
evidence_refs:
- benchmarks/bad-memory-prompt-injection.md
- https://arxiv.org/abs/2607.14611
extract_confidence: medium
- event_id: fact-memory-vs-long-context-baseline-2026
benchmark_id: fact-memory-vs-long-context
source_kind: paper
Expand Down
4 changes: 4 additions & 0 deletions benchmarks/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ official repository, dataset card, or independent reproduction.
| LoCoMo | seed | paper-origin | very long-term conversational memory | retrieval / temporal / causal | [`locomo.md`](locomo.md) | yes |
| ConvoMem | full | paper-origin | conversational memory scaling | multi-evidence / preference / abstention | [`convomem.md`](convomem.md) | yes |
| MemoryAgentBench | seed | paper-origin | incremental multi-turn agent memory | retrieval / learning / long-range / forgetting | [`memoryagentbench.md`](memoryagentbench.md) | yes |
| PM-Bench | seed | paper-origin | prospective memory | delayed intention / cue monitoring / ongoing activity | [`pm-bench.md`](pm-bench.md) | yes |
| MemOps | seed | paper-origin | memory operation selection | remember / forget / update / reflect / no-op | [`memops.md`](memops.md) | yes |
| GateMem | seed | paper-origin | multi-principal shared memory governance | utility / access control / active forgetting | [`gatemem.md`](gatemem.md) | backlog |
| StructMemEval | seed | paper-origin | structured memory organization | structure selection / state tracking / task-specific organization | [`structmemeval.md`](structmemeval.md) | yes |
| BEAM | candidate | paper-origin | million-token memory scale | long-scale recall / temporal degradation | [`beam.md`](beam.md) | yes |
Expand All @@ -49,6 +51,8 @@ official repository, dataset card, or independent reproduction.
| MemSyco-Bench | seed | paper-origin | memory-induced sycophancy | scope / conflict resolution / update / valid personalization | [`memsyco-bench.md`](memsyco-bench.md) | yes |
| MemLeak | seed | paper-origin | multimodal deletion leakage | deletion compliance / provenance / residual image leakage | [`memleak.md`](memleak.md) | yes |
| MemDelta | seed | paper-origin | memory-evaluation baseline control | component delta / model-family sensitivity / write-path cost | [`memdelta.md`](memdelta.md) | yes |
| Bad Memory | seed | paper-origin | persistent memory prompt injection | memory files / preference stores / KB poisoning / mitigation | [`bad-memory-prompt-injection.md`](bad-memory-prompt-injection.md) | yes |
| MemPoison | seed | paper-origin | persistent memory poisoning | compositional corruption / dormant trigger / defense blind spots | [`mempoison.md`](mempoison.md) | yes |

## Evidence Ledgers

Expand Down
81 changes: 81 additions & 0 deletions benchmarks/memops.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
---
title: MemOps — lifecycle memory operations benchmark
benchmark_id: memops
name: MemOps
aliases:
- MemOps
status: seed
origin_type: paper_origin
origin_source: https://arxiv.org/abs/2607.12893
first_public_date: 2026-07
domain: memory_lifecycle_operations
modality: text
task_grain: operation_level_conversational_memory
capability_axes:
- remembering
- forgetting
- updating
- reflecting
- evidence_binding
- state_transition
data_nature: generated_long_horizon_conversations
metrics:
- operation_trace_accuracy
- probe_accuracy
judge_type: check
code_available: check
data_available: check
license: check
known_limitations:
- seed note; protocol, generation pipeline, resources, and reported results need full read
canonical_sources:
- https://arxiv.org/abs/2607.12893
confidence: medium
memory_modules:
- evaluator-benchmark
- memorydiff-generator
- retriever-reranker
last_revised: 2026-07-20
---

# MemOps

## What It Measures

MemOps evaluates long-term conversational memory as a lifecycle of explicit
operations rather than final-answer recall alone. It focuses on remembering,
forgetting, updating, reflecting, operation composition, and whether each memory
event is tied to the right trigger, target, scope, state transition, and
supporting evidence.

## Protocol

The arXiv abstract describes a controllable generation pipeline that embeds
memory operations into long task-oriented conversations and produces gold
operation traces plus six categories of operation-level probes. Full probe
categories, dataset construction, released resources, and scoring details need a
deeper read.

## Baselines and Reported Results

No normalized scores are logged here. The abstract reports comparisons across
long-context, retrieval-based, parametric, and managed-memory systems, but those
remain author-reported results until the setup is normalized.

## Comparability Notes

Compare MemOps with A-TMA and MemDelta on failure decomposition, not leaderboard
rank. It is most useful when the design question is whether a memory system can
explain the operation that created, updated, forgot, or reflected on a memory.

## Related Papers

- Paper:https://arxiv.org/abs/2607.12893

## Impact Use

- `evaluator-benchmark`:candidate protocol for operation-level memory
diagnosis.
- `memorydiff-generator`:maps naturally to explicit memory event traces and
state transitions.
- Ready for ImpactReport:no, upgrade after full protocol read.
Loading