diff --git a/.claude/agents/codebase-analyst.md b/.claude/agents/codebase-analyst.md index 7cd2d72a..24f14241 100644 --- a/.claude/agents/codebase-analyst.md +++ b/.claude/agents/codebase-analyst.md @@ -9,9 +9,9 @@ maxTurns: 15 skills: [prp-advisor] --- -You are a code analysis specialist. You analyze implementation details, trace data flow, and explain technical workings with precise file:line references. +## CRITICAL: Your only job is to analyze existing code and document how it works -## CRITICAL: Document What Exists, Nothing More +> This agent follows the prompt structure in `plugins/prp-core/references/agent-prompt-style.md`. - **DO NOT** suggest improvements, propose enhancements, or critique implementations - **DO NOT** perform root cause analysis or comment on quality/performance/security @@ -19,7 +19,6 @@ You are a code analysis specialist. You analyze implementation details, trace da - **ONLY** describe what exists, how it works, and how components interact - **ALWAYS** Call `prp-advisor` skill BEFORE assuming work is done or building on an assumption. If the task requires orientation first (finding files, fetching a source, seeing what's there), do that, then call `prp-advisor` agent. Orientation is not substantive work. Writing, editing, and declaring an answer are. - You are a documentarian, not a critic or consultant. ## Core Responsibilities diff --git a/.claude/agents/codebase-explorer.md b/.claude/agents/codebase-explorer.md index 721331e0..b9e6ea45 100644 --- a/.claude/agents/codebase-explorer.md +++ b/.claude/agents/codebase-explorer.md @@ -8,9 +8,9 @@ maxTurns: 15 skills: [prp-advisor] --- -You are a codebase explorer. You find WHERE code lives and show HOW it's implemented with concrete examples, precise file:line references, and actual code patterns. +## CRITICAL: Your only job is to map where code lives and show how it is implemented -## CRITICAL: Document What Exists, Nothing More +> This agent follows the prompt structure in `plugins/prp-core/references/agent-prompt-style.md`. - **DO NOT** suggest improvements, critique implementations, or identify problems - **DO NOT** recommend refactoring, reorganization, or evaluate pattern quality diff --git a/.claude/agents/plan-critic.md b/.claude/agents/plan-critic.md index 2fd97d5c..8152cbe6 100644 --- a/.claude/agents/plan-critic.md +++ b/.claude/agents/plan-critic.md @@ -9,7 +9,9 @@ maxTurns: 10 You are a PRP artifact critic. You review consolidated findings and draft content for implementation plans and product requirement documents. Your goal is to catch gaps, blind spots, and weak assumptions BEFORE the artifact is written — when course corrections are still cheap. -## CRITICAL: Actionable Feedback Only +## CRITICAL: Your only job is to validate PRP artifacts for completeness, coherence, and blind spots + +> This agent follows the prompt structure in `plugins/prp-core/references/agent-prompt-style.md`. - **DO** identify specific gaps with concrete suggestions to fix them - **DO** challenge untested assumptions with evidence from the codebase diff --git a/.claude/agents/prp-advisor.md b/.claude/agents/prp-advisor.md index 8f7c1b3c..cd93cf4f 100644 --- a/.claude/agents/prp-advisor.md +++ b/.claude/agents/prp-advisor.md @@ -8,7 +8,9 @@ color: purple maxTurns: 10 --- -You are a senior technical advisor. Your only job is to review the current task state and give precise, actionable guidance. You receive the full conversation history — every tool call, every result. +## CRITICAL: Your only job is to review task state and give precise, actionable guidance + +> This agent follows the prompt structure in `plugins/prp-core/references/agent-prompt-style.md`. ## Constraints diff --git a/.claude/agents/web-researcher.md b/.claude/agents/web-researcher.md index 054d6253..60aa5d33 100644 --- a/.claude/agents/web-researcher.md +++ b/.claude/agents/web-researcher.md @@ -7,7 +7,19 @@ tools: [WebSearch, WebFetch, Bash] maxTurns: 10 --- -You are a web research specialist. You find accurate, relevant information from web sources and synthesize it into actionable knowledge with proper citations. +## CRITICAL: Your only job is to find and synthesize accurate information from web sources + +> This agent follows the prompt structure in `plugins/prp-core/references/agent-prompt-style.md`. + +- DO NOT speculate when you can search +- DO NOT present findings without source citations +- DO NOT claim expertise beyond what sources support + +## Core Responsibilities + +1. Find accurate, up-to-date information from authoritative web sources +2. Synthesize findings into actionable knowledge with proper citations +3. Flag conflicting information, outdated content, and coverage gaps ## Process diff --git a/.claude/references/agent-prompt-style.md b/.claude/references/agent-prompt-style.md new file mode 100644 index 00000000..c83a10d3 --- /dev/null +++ b/.claude/references/agent-prompt-style.md @@ -0,0 +1,60 @@ +# Agent Prompt Style + +PRP agents follow a structured prompt style that keeps each agent's job explicit, its boundaries clear, and its output predictable. The pattern is: one declared job, explicit DO NOT boundaries, a short strategy, and a fixed output shape. + +## Required structure + +Every agent prompt must include these sections in this order: + +### 1. Critical role declaration + +State the agent's single job in the opening line. Use this form: + +``` +## CRITICAL: Your only job is to [specific role] +``` + +Follow with explicit `DO NOT` boundaries for the two or three most common scope violations. + +### 2. Core Responsibilities + +A numbered list of the concrete actions the agent takes. Keep it to five or fewer items. Each item should be specific enough that another agent could audit whether it was followed. + +### 3. Strategy + +A numbered stepwise approach the agent should follow. This is not a restatement of responsibilities — it is the execution order and any key decision rules. + +### 4. Output Format + +Define the required output shape. For agents with a repeatable task, the output format should be explicit: section names, order, and any required fields. + +## Hard boundaries + +- One job per agent. An agent that reviews code must not also modify files. An agent that explores must not also commit. +- Minimal tool surface. Grant only the tools the agent genuinely needs. Advisory agents typically need read-only access (grep, glob, view, bash). No file modification tools. +- `maxTurns` must be declared. Advisory agents: `maxTurns: 10`. Exploration agents: `maxTurns: 15`. + +## PRP-specific frontmatter + +Retain all PRP-specific frontmatter fields (`name`, `description`, `model`, `color`, `tools`, `maxTurns`, `skills`) unless a change is explicitly justified by the plugin's actual runtime needs. Do not silently drop fields when adapting prompt structure. + +Model selection: +- `haiku` — structured, fast tasks (explorer, comment-analyzer, docs-impact) +- `sonnet` — reasoning-heavy tasks (analyst, reviewer, type-design, advisor) + +## Adaptation guidance + +When normalizing an existing agent against this style: + +1. Add the critical role declaration if it is missing. +2. Add hard `DO NOT` boundaries if implicit. +3. Consolidate scattered responsibilities into numbered Core Responsibilities. +4. Add a Strategy section if the agent's approach is not stated. +5. Make the output format explicit if it is currently open-ended. +6. Do not remove PRP-specific frontmatter or behavioral capabilities without reviewing the plugin's runtime contract. + +## Related references + +- `harness-taxonomy.md` — advisory vs. utility component classification +- `execution-policy.md` — when an agent receives a subagent delegation +- `context-budget-policy.md` — brief size limits for agent output diff --git a/.claude/references/artifact-lifecycle.md b/.claude/references/artifact-lifecycle.md new file mode 100644 index 00000000..718b18b6 --- /dev/null +++ b/.claude/references/artifact-lifecycle.md @@ -0,0 +1,42 @@ +# Artifact Lifecycle + +PRP artifacts are first-class outputs. Every artifact must declare where it lives, when it is archived, and who is responsible for it. + +## Artifact locations + +All PRP runtime artifacts live under `.claude/PRPs/` in the target project: + +| Type | Path | Created by | +|------|------|------------| +| PRDs | `.claude/PRPs/prds/` | `prp-prd` | +| Plans | `.claude/PRPs/plans/` | `prp-plan` | +| Completed plans | `.claude/PRPs/plans/completed/` | `prp-implement` (archives after execution) | +| Reports | `.claude/PRPs/reports/` | `prp-implement` | +| Issues | `.claude/PRPs/issues/` | `prp-issue-investigate` | +| Completed issues | `.claude/PRPs/issues/completed/` | `prp-issue-fix` | +| Reviews | `.claude/PRPs/reviews/` | `prp-review` | +| Ralph archives | `.claude/PRPs/ralph-archives/` | `prp-ralph` (on completion) | + +## Lifecycle rules + +1. **Every artifact must declare its location before writing.** Skills should state the output path in the first step of their workflow, not after the artifact is written. + +2. **Archive completed artifacts consistently.** A plan is not complete until it is moved to `plans/completed/`. An issue investigation is not resolved until the artifact is in `issues/completed/`. Archival is a workflow step, not optional cleanup. + +3. **Durable artifacts carry the workflow forward; transcripts do not.** Only the compact artifact (PRD, plan, report, issue plan) should be passed between phases. Raw conversation history, exploration transcripts, and verbose tool outputs are temporary and must not be treated as the handoff artifact. + +4. **Keep repo-only eval outputs outside the shipped plugin payload.** Prompt-evaluation fixtures, test case files, and CI scripts live in the repository's `tests/` and `scripts/` directories. They do not live in `plugins/prp-core/` and must not be referenced from shipped skills. + +## Artifact naming + +- PRDs: `.prd.md` +- Plans: `.plan.md` +- Reports: `-report.md` +- Issues: `issue--.md` +- Reviews: `pr--review.md` + +## Related references + +- `harness-taxonomy.md` — which skills own which artifact class +- `context-budget-policy.md` — brief size limits that govern handoff artifacts +- `execution-policy.md` — delegation modes that determine which agent writes the artifact diff --git a/.claude/references/context-budget-policy.md b/.claude/references/context-budget-policy.md new file mode 100644 index 00000000..46c33278 --- /dev/null +++ b/.claude/references/context-budget-policy.md @@ -0,0 +1,39 @@ +# Context Budget Policy + +The PRP harness treats context as a budget. Every token must earn its place. Noise in the context window degrades reasoning quality measurably — 3,000 filler tokens drop simple math accuracy from 0.92 to 0.68 (Levy et al., ACL 2024). + +## Core rules + +1. **Keep orchestration in the smart zone by default.** Run the primary workflow in a context window that stays under 40% capacity. Above that threshold, compaction or a subagent handoff is required before the next phase starts. + +2. **Compact noisy outputs into bounded briefs before handoff.** Raw exploration transcripts, verbose tool outputs, and multi-file search dumps must be summarized before they pass to the next skill or phase. Brief size limits: + - Discovery brief: ≤ 50 lines + - Execution brief: ≤ 30 lines + - Validation brief: ≤ 20 lines + +3. **Prefer references over repeated inline policy text.** Cross-cutting policy belongs in shared reference files, not duplicated in each skill. Skills should link to the relevant reference instead of restating the same guidance. + +4. **Treat raw agent transcripts as temporary context.** Subagent work products belong in compact artifacts (plan files, reports, brief sections), not in the main conversation thread. Discard exploration transcripts once the artifact is written. + +5. **Progressive disclosure for instructions.** Load skill context, agents, and path-scoped rules on demand. Do not front-load all workflow instructions into every session. Always-loaded instructions must stay under 200 lines total. + +## Compaction checkpoints + +Each orchestration-heavy skill must define at least one compaction checkpoint — a point where noisy intermediate outputs are summarized before the next step proceeds. Typical checkpoints: + +- After initial codebase exploration → summarize into a discovery brief +- After running all review agents → summarize findings into a combined report +- After investigation → summarize into an issue plan before implementation + +## When to start a fresh context + +Start a fresh context or subagent when: +- The current context is over 40% capacity and more noisy work remains +- The next task is genuinely independent of the current exploration +- The task is a large-corpus search, multi-file read, or iterative test loop + +## Related references + +- `harness-taxonomy.md` — component classes and their context profiles +- `execution-policy.md` — when to delegate to a subagent vs. stay inline +- `artifact-lifecycle.md` — where durable outputs live diff --git a/.claude/references/execution-policy.md b/.claude/references/execution-policy.md new file mode 100644 index 00000000..259e7333 --- /dev/null +++ b/.claude/references/execution-policy.md @@ -0,0 +1,69 @@ +# Execution Policy + +The PRP harness defines four delegation modes. Choose the right mode based on task size, isolation, and determinism. + +## Delegation modes + +### 1. Stay inline + +Use for short, deterministic, single-scope work that does not generate noisy intermediate output. + +Criteria: +- Task completes in a handful of tool calls +- Output is compact and directly consumed by the next step +- No heavy exploration or multi-file corpus search needed + +Examples: reading a single file, writing a short commit message, running a schema check. + +### 2. Use an isolated subagent + +Use for noisy exploration, large-corpus research, or analysis that would pollute the main context. + +Criteria: +- Task requires reading many files, searching broadly, or running iterative queries +- Output will be summarized into a compact brief before handoff +- The work is independent enough that the main context does not need to observe every step + +Examples: codebase exploration, external documentation research, review analysis. + +Return only the compact result (discovery brief, research summary, review report) to the main context. Discard the exploration transcript. + +### 3. Parallelize + +Use only when scopes are genuinely independent — no shared files, no shared state, no ordering dependency. + +Criteria: +- Two or more subagents can run without reading each other's outputs +- Merging their results does not require re-running either agent +- Parallelism does not create conflicting writes + +If in doubt, keep execution sequential. + +### 4. Use a documented repo-local evaluation harness + +Use for deterministic batch verification that would otherwise repeat the same manual checks in the main context. + +Criteria: +- A documented evaluation script already exists for this check +- The check is repeatable, schema-bound, or file-comparison-based +- The result is pass/fail, not a reasoning judgment + +Do not assume evaluation harness scripts ship in the plugin payload. They are repo-only tools. Do not name repo-only script paths inside shipped skills. + +## Parallelism guard + +Before parallelizing, answer: "Do these scopes share any file, state variable, or sequencing constraint?" If yes, keep them sequential. + +## Research-team policy + +When `prp-research-team` is used, the orchestrating agent must: +1. Decompose the question into genuinely independent sub-questions +2. Assign each sub-question to a specialist with a bounded scope +3. Collect compact summaries from each specialist +4. Synthesize into the required six-section research plan + +## Related references + +- `context-budget-policy.md` — brief size limits and compaction rules +- `harness-taxonomy.md` — which component class owns each decision +- `artifact-lifecycle.md` — where outputs land after delegation diff --git a/.claude/references/harness-taxonomy.md b/.claude/references/harness-taxonomy.md new file mode 100644 index 00000000..0e575c7f --- /dev/null +++ b/.claude/references/harness-taxonomy.md @@ -0,0 +1,48 @@ +# Harness Taxonomy + +The PRP harness groups all shipped components into three classes. Every skill and agent belongs to exactly one class. + +## Component Classes + +### Sequential artifact skills + +Create durable PRP artifacts that later phases consume. Each skill owns one phase of the research → plan → implement → validate lifecycle and hands forward a compact artifact, not raw exploration output. + +- `prp-prd` — PRD from a feature request +- `prp-plan` — implementation plan from a PRD or brief +- `prp-implement` — execution from a plan +- `prp-issue-investigate` — investigation artifact from a GitHub issue +- `prp-issue-fix` — fix and PR from an investigation artifact +- `prp-research-team` — research plan from a question +- `prp-codebase-question` — scoped research answer + +### Advisory components + +Challenge, review, and refine decisions. Advisory components do not create primary workflow artifacts and do not modify files directly. + +- `prp-advisor` — pre-work and mid-work critique gate +- `prp-review` — PR review artifact +- `prp-review-agents` — multi-aspect specialized review +- `prp-prd` — also acts as an advisory scope-clarifier before planning +- `plan-critic` agent — targeted plan critique +- `prp-advisor` agent — advisor subagent for the skill +- `codebase-analyst` agent — deep implementation analysis +- `codebase-explorer` agent — file and structure exploration +- `web-researcher` agent — external documentation research + +### Utility components + +Execute one narrow, repeatable job as the final step of a flow. Utilities should not contain research or planning logic. + +- `prp-commit` — atomic commit from staged changes +- `prp-pr` — pull request from a branch +- `prp-verification-before-completion` — final validation gate before claiming done +- `prp-ralph` / `prp-ralph-cancel` / `prp-ralph-loop` — autonomous loop control +- `prp-debug` — root-cause analysis helper + +## Design intent + +- Each class has a different context profile. Artifact skills need bounded briefs and compaction checkpoints. Advisory components need read access and a fixed output shape. Utility components should keep their context footprint small. +- A skill that crosses two classes (for example, an artifact skill that also reviews) should be refactored or the review step should be delegated to an advisory agent. +- See `context-budget-policy.md` for compaction rules and brief size limits. +- See `execution-policy.md` for when to stay inline vs. delegate to a subagent. diff --git a/.claude/rules/agent-conventions.md b/.claude/rules/agent-conventions.md index 6bb9d61d..b38d9b72 100644 --- a/.claude/rules/agent-conventions.md +++ b/.claude/rules/agent-conventions.md @@ -3,10 +3,14 @@ paths: - "plugins/prp-core/agents/*.md" - ".claude/agents/*.md" --- + # Agent File Conventions +> **Authoritative prompt structure**: `plugins/prp-core/references/agent-prompt-style.md` +> **Component class definitions**: `plugins/prp-core/references/harness-taxonomy.md` + - YAML frontmatter requires: `name`, `description`, `model`, `color` - Model selection: `haiku` for structured/fast tasks (explorer, comment-analyzer, docs-impact), `sonnet` for reasoning-heavy tasks (analyst, reviewer, type-design) - Include `tools` and `maxTurns` in frontmatter: advisory agents get `maxTurns: 10`, exploration agents get `maxTurns: 15` diff --git a/.claude/rules/artifact-paths.md b/.claude/rules/artifact-paths.md index 02d855ad..87d4e918 100644 --- a/.claude/rules/artifact-paths.md +++ b/.claude/rules/artifact-paths.md @@ -3,10 +3,13 @@ paths: - "plugins/prp-core/skills/*/SKILL.md" - ".claude/skills/*/SKILL.md" --- + # PRP Artifact Path Contracts +> **Authoritative policy**: `plugins/prp-core/references/artifact-lifecycle.md` — this rule is a scoped quick-reference for skill authors. + All PRP artifacts go under `.claude/PRPs/` with this structure: | Type | Path | Created by | diff --git a/.claude/rules/harness-engineering.md b/.claude/rules/harness-engineering.md new file mode 100644 index 00000000..fb3723da --- /dev/null +++ b/.claude/rules/harness-engineering.md @@ -0,0 +1,41 @@ +--- +paths: + - "plugins/prp-core/**/*.md" + - ".claude/**/*.md" + - "scripts/**/*.py" + - "tests/**/*.json" +--- +# Harness Engineering Conventions + +Authoritative sources: `plugins/prp-core/references/context-budget-policy.md`, `plugins/prp-core/references/harness-taxonomy.md`, `wiki/knowledge/harness-engineering.md`. + +## Smart Zone (Priority 1 — CRITICAL) + +- Context < 40% capacity: normal execution. +- Context 40–70%: compact aggressively. Write current state to an artifact; start next phase with a fresh window. +- Context > 70%: delegate remaining work to a subagent; pass only the compacted artifact, not raw context. +- 3,000 filler tokens drop math accuracy from 0.92 → 0.68 (Levy et al., ACL 2024). Every token must earn its place. + +## Harness Rails (Priority 2 — HIGH) + +- Add a test, lint check, or hook constraint before adding a corrective prompt instruction. +- A behavioral rule repeated more than twice signals a harness gap, not a wording gap. Fix the harness. +- Verify completion deterministically: `uv run python scripts/prp_core_prompt_eval.py --all` (6 cases). + +## Subagent Isolation (Priority 3 — HIGH) + +- Research, analysis, and exploration run in forked subagent contexts. +- Return only condensed output (~1,000 tokens) to the parent agent. +- Never carry raw tool output or exploration transcripts across phase boundaries. + +## Artifact Discipline (Priority 4 — MEDIUM) + +- Each phase (research → plan → implement → review) must produce a durable artifact before the next phase begins. +- Component taxonomy: artifact / advisory / utility — see `plugins/prp-core/references/harness-taxonomy.md`. +- Brief size limits: discovery ≤ 50 lines, execution ≤ 30 lines, validation ≤ 20 lines. + +## Progressive Disclosure (Priority 5 — MEDIUM) + +- Root `CLAUDE.md` → subdirectory `CLAUDE.md` → `.claude/rules/` (path-scoped) → skills (on-demand). +- Never duplicate policy inline in skills or agents — link to `plugins/prp-core/references/` instead. +- Skills link to reference files; reference files are the single source of truth. diff --git a/.claude/rules/hook-conventions.md b/.claude/rules/hook-conventions.md index dd76d074..b936f8e7 100644 --- a/.claude/rules/hook-conventions.md +++ b/.claude/rules/hook-conventions.md @@ -3,6 +3,7 @@ paths: - "plugins/prp-core/hooks/*.sh" - ".claude/hooks/*.sh" --- + # Hook Script Conventions diff --git a/.claude/rules/prp-workflow.md b/.claude/rules/prp-workflow.md index ed0ba776..929f6efe 100644 --- a/.claude/rules/prp-workflow.md +++ b/.claude/rules/prp-workflow.md @@ -6,13 +6,9 @@ paths: # PRP Workflow Execution -## Before Creating PRP Artifacts - -- Run the `prp-advisor` skill before writing any `*.prd.md` or `*.plan.md` file and before committing to any interpretation or assumption. - ## After `*.prd.md` Is Created -- Create a GitHub issue using the skill's `references/issue-template.md`. +- Create a GitHub issue using the `prp-prd` skill's `references/issue-template.md`. - Write the created issue URL into the `` field in the PRD file. ## After `*.prd.md` Is Updated @@ -21,7 +17,7 @@ paths: ## After `*.plan.md` Is Created -- Create a GitHub issue using the skill's `references/issue-template.md`. +- Create a GitHub issue using the `prp-plan` skill's `references/issue-template.md`. - Write the parent PRD issue URL into `` in the plan file. - Write the created plan issue URL into `` in the plan file. diff --git a/.claude/rules/reference-conventions.md b/.claude/rules/reference-conventions.md new file mode 100644 index 00000000..6699b61e --- /dev/null +++ b/.claude/rules/reference-conventions.md @@ -0,0 +1,23 @@ +--- +paths: + - "plugins/prp-core/references/*.md" + - ".claude/references/*.md" +--- +# Reference File Conventions + +Reference files in `references/` are the authoritative source of truth for cross-cutting policy. Skills and agents link to these files; they never duplicate policy inline. + +## Required Structure + +- `# [Title]` — one-line title +- Brief purpose sentence (what this file governs) +- Main content with clear headings +- `## Evidence` or inline citation where the policy is research-backed + +## Authoring Rules + +- Line budget: ≤ 100 lines per file. +- No external product names or attributions — describe patterns self-containedly. +- Link to other references with bare filenames: `context-budget-policy.md`, not absolute paths. +- Mirror parity: every change to `plugins/prp-core/references/` must be mirrored to `.claude/references/` byte-for-byte. +- Apply the deletion test: "Would removing this cause an agent to make mistakes?" If not, cut it. diff --git a/.claude/rules/skill-conventions.md b/.claude/rules/skill-conventions.md index 219b7b22..fae80886 100644 --- a/.claude/rules/skill-conventions.md +++ b/.claude/rules/skill-conventions.md @@ -3,6 +3,7 @@ paths: - "plugins/prp-core/skills/*/SKILL.md" - ".claude/skills/*/SKILL.md" --- + # Skill File Conventions diff --git a/.claude/skills/prp-advisor/SKILL.md b/.claude/skills/prp-advisor/SKILL.md index 5cd9836d..9bb53733 100644 --- a/.claude/skills/prp-advisor/SKILL.md +++ b/.claude/skills/prp-advisor/SKILL.md @@ -9,6 +9,8 @@ user-invocable: false ## Step 1: Detect Advisor Path +> **Policy**: This is an advisory component per `plugins/prp-core/references/harness-taxonomy.md`. It reviews and challenges decisions but does not create primary workflow artifacts. + Check whether the native `advisor` tool is available in your current tool set. - **`advisor` tool available** → proceed to [Native Advisor](#native-advisor) diff --git a/.claude/skills/prp-codebase-question/SKILL.md b/.claude/skills/prp-codebase-question/SKILL.md index 9613632c..a904ae84 100644 --- a/.claude/skills/prp-codebase-question/SKILL.md +++ b/.claude/skills/prp-codebase-question/SKILL.md @@ -11,6 +11,16 @@ Answer codebase questions by spawning parallel specialized agents, synthesizing **CRITICAL**: DO NOT suggest improvements, critique implementations, or propose changes. Only describe what exists, where it lives, and how it works. +## Context Contract + +Before starting, read `plugins/prp-core/references/context-budget-policy.md` and `plugins/prp-core/references/execution-policy.md`. + +- Compact noisy findings into a **bounded brief** before each handoff (discovery ≤ 50 lines, execution ≤ 30 lines, validation ≤ 20 lines). +- Parallelize only independent scopes; keep sequential when scopes share files or state. +- Isolate noisy exploration in subagents; return only the compact artifact to the main context. +- For deterministic batch verification, use the repo-local evaluation harness. +- All PRP artifacts go under `.claude/PRPs/` — see `artifact-lifecycle.md`. + ## Phase 1: PARSE 1. If user mentions specific files, read them FULLY first diff --git a/.claude/skills/prp-commit/SKILL.md b/.claude/skills/prp-commit/SKILL.md index 8d2f9df6..68379a0a 100644 --- a/.claude/skills/prp-commit/SKILL.md +++ b/.claude/skills/prp-commit/SKILL.md @@ -7,6 +7,8 @@ description: "Atomic commits by logical scope following Conventional Commits 1.0 ## Strict Rules +> **Policy**: This is a utility component per `plugins/prp-core/references/harness-taxonomy.md`. It does one narrow job: create atomic commits. It must not contain research or planning logic. + - NEVER use `git add .` or `git add -A` — stage files by explicit path only - NEVER add `Co-Authored-By` or any AI attribution footer - One commit per logical scope — never mix unrelated changes diff --git a/.claude/skills/prp-implement/SKILL.md b/.claude/skills/prp-implement/SKILL.md index 73181e5e..54e0e9da 100644 --- a/.claude/skills/prp-implement/SKILL.md +++ b/.claude/skills/prp-implement/SKILL.md @@ -11,6 +11,18 @@ Execute the plan end-to-end with rigorous self-validation. You are autonomous. --- +## Context Contract + +Before starting, read `plugins/prp-core/references/context-budget-policy.md` and `plugins/prp-core/references/execution-policy.md`. + +- Compact noisy findings into a **bounded brief** before each handoff (discovery ≤ 50 lines, execution ≤ 30 lines, validation ≤ 20 lines). +- Parallelize only independent scopes; keep sequential when scopes share files or state. +- Isolate noisy exploration in subagents; return only the compact artifact to the main context. +- For deterministic batch verification, use the repo-local evaluation harness. +- All PRP artifacts go under `.claude/PRPs/` — see `artifact-lifecycle.md`. + +--- + ## Phase 0: DETECT — Project Environment ### 0.1 Identify Package Manager diff --git a/.claude/skills/prp-issue-fix/SKILL.md b/.claude/skills/prp-issue-fix/SKILL.md index f12c0346..a73e32e5 100644 --- a/.claude/skills/prp-issue-fix/SKILL.md +++ b/.claude/skills/prp-issue-fix/SKILL.md @@ -21,6 +21,18 @@ Execute the implementation plan from the `prp-issue-investigate` skill: --- +## Context Contract + +Before starting, read `plugins/prp-core/references/context-budget-policy.md` and `plugins/prp-core/references/execution-policy.md`. + +- Compact noisy findings into a **bounded brief** before each handoff (discovery ≤ 50 lines, execution ≤ 30 lines, validation ≤ 20 lines). +- Parallelize only independent scopes; keep sequential when scopes share files or state. +- Isolate noisy exploration in subagents; return only the compact artifact to the main context. +- For deterministic batch verification, use the repo-local evaluation harness. +- All PRP artifacts go under `.claude/PRPs/` — see `artifact-lifecycle.md`. + +--- + ## Phase 0: DETECT — Base Branch Load the base branch detection protocol: diff --git a/.claude/skills/prp-issue-investigate/SKILL.md b/.claude/skills/prp-issue-investigate/SKILL.md index 46ea875e..447eaec9 100644 --- a/.claude/skills/prp-issue-investigate/SKILL.md +++ b/.claude/skills/prp-issue-investigate/SKILL.md @@ -17,6 +17,18 @@ Investigate the issue/problem and produce a comprehensive implementation plan th --- +## Context Contract + +Before starting, read `plugins/prp-core/references/context-budget-policy.md` and `plugins/prp-core/references/execution-policy.md`. + +- Compact noisy findings into a **bounded brief** before each handoff (discovery ≤ 50 lines, execution ≤ 30 lines, validation ≤ 20 lines). +- Parallelize only independent scopes; keep sequential when scopes share files or state. +- Isolate noisy exploration in subagents; return only the compact artifact to the main context. +- For deterministic batch verification, use the repo-local evaluation harness. +- All PRP artifacts go under `.claude/PRPs/` — see `artifact-lifecycle.md`. + +--- + ## Phase 1: PARSE — Understand Input ### 1.1 Determine Input Type diff --git a/.claude/skills/prp-plan/SKILL.md b/.claude/skills/prp-plan/SKILL.md index b188b3c2..06616c53 100644 --- a/.claude/skills/prp-plan/SKILL.md +++ b/.claude/skills/prp-plan/SKILL.md @@ -11,6 +11,18 @@ description: "Create a comprehensive feature implementation plan with deep codeb --- +## Context Contract + +Before starting, read `plugins/prp-core/references/context-budget-policy.md` and `plugins/prp-core/references/execution-policy.md`. + +- Compact noisy findings into a **bounded brief** before each handoff (discovery ≤ 50 lines, execution ≤ 30 lines, validation ≤ 20 lines). +- Parallelize only independent scopes; keep sequential when scopes share files or state. +- Isolate noisy exploration in subagents; return only the compact artifact to the main context. +- For deterministic batch verification, use the repo-local evaluation harness. +- All PRP artifacts go under `.claude/PRPs/` — see `artifact-lifecycle.md`. + +--- + ## Phase 0: DETECT — Input Type Resolution | Input Pattern | Type | Action | diff --git a/.claude/skills/prp-prd/SKILL.md b/.claude/skills/prp-prd/SKILL.md index a793e411..d2a3b7b9 100644 --- a/.claude/skills/prp-prd/SKILL.md +++ b/.claude/skills/prp-prd/SKILL.md @@ -17,6 +17,14 @@ INITIATE → FOUNDATION → GROUNDING → DEEP DIVE → FEASIBILITY → DECISION Each phase builds on previous answers. Grounding phases validate assumptions via agents. +## Context Contract + +Before starting, read `plugins/prp-core/references/context-budget-policy.md` and `plugins/prp-core/references/execution-policy.md`. + +- Compact noisy findings into a **bounded brief** before handing off to the plan phase. +- Parallelize only independent research scopes; keep sequential when scopes share state. +- All PRP artifacts go under `.claude/PRPs/prds/` — see `artifact-lifecycle.md`. + ## Phase 1: INITIATE **No input**: Ask "What do you want to build?" diff --git a/.claude/skills/prp-research-team/SKILL.md b/.claude/skills/prp-research-team/SKILL.md index a8175ddb..581fd3b8 100644 --- a/.claude/skills/prp-research-team/SKILL.md +++ b/.claude/skills/prp-research-team/SKILL.md @@ -13,6 +13,18 @@ Design a dynamic team of research agents and a structured research plan. Targets --- +## Context Contract + +Before starting, read `plugins/prp-core/references/context-budget-policy.md` and `plugins/prp-core/references/execution-policy.md`. + +- Compact noisy findings into a **bounded brief** before each handoff (discovery ≤ 50 lines, execution ≤ 30 lines, validation ≤ 20 lines). +- Parallelize only independent scopes; keep sequential when scopes share files or state. +- Isolate noisy exploration in subagents; return only the compact artifact to the main context. +- For deterministic batch verification, use the repo-local evaluation harness. +- All PRP artifacts go under `.claude/PRPs/` — see `artifact-lifecycle.md`. + +--- + ## Variables | Variable | Source | Default | diff --git a/.claude/skills/prp-review-agents/SKILL.md b/.claude/skills/prp-review-agents/SKILL.md index 4468d1f5..cb148153 100644 --- a/.claude/skills/prp-review-agents/SKILL.md +++ b/.claude/skills/prp-review-agents/SKILL.md @@ -7,6 +7,16 @@ description: "Comprehensive PR review using specialized agents. Runs code qualit Run a multi-agent review on a pull request, with each agent focusing on a specific aspect of code quality. +## Context Contract + +Before starting, read `plugins/prp-core/references/context-budget-policy.md` and `plugins/prp-core/references/execution-policy.md`. + +- Compact noisy findings into a **bounded brief** before each handoff (discovery ≤ 50 lines, execution ≤ 30 lines, validation ≤ 20 lines). +- Parallelize only independent scopes; keep sequential when scopes share files or state. +- Isolate noisy exploration in subagents; return only the compact artifact to the main context. +- For deterministic batch verification, use the repo-local evaluation harness. +- All PRP artifacts go under `.claude/PRPs/` — see `artifact-lifecycle.md`. + ## Pre-Review Setup 1. **Identify the PR** diff --git a/.claude/skills/prp-review/SKILL.md b/.claude/skills/prp-review/SKILL.md index eaa687d1..a7463dd4 100644 --- a/.claude/skills/prp-review/SKILL.md +++ b/.claude/skills/prp-review/SKILL.md @@ -11,6 +11,18 @@ Perform a thorough, senior-engineer-level code review. --- +## Context Contract + +Before starting, read `plugins/prp-core/references/context-budget-policy.md` and `plugins/prp-core/references/execution-policy.md`. + +- Compact noisy findings into a **bounded brief** before each handoff (discovery ≤ 50 lines, execution ≤ 30 lines, validation ≤ 20 lines). +- Parallelize only independent scopes; keep sequential when scopes share files or state. +- Isolate noisy exploration in subagents; return only the compact artifact to the main context. +- For deterministic batch verification, use the repo-local evaluation harness. +- All PRP artifacts go under `.claude/PRPs/` — see `artifact-lifecycle.md`. + +--- + ## Phase 1: FETCH — Get PR Context ### 1.1 Parse Input diff --git a/.claude/skills/prp-verification-before-completion/SKILL.md b/.claude/skills/prp-verification-before-completion/SKILL.md index f8ee1f2e..6e766c59 100644 --- a/.claude/skills/prp-verification-before-completion/SKILL.md +++ b/.claude/skills/prp-verification-before-completion/SKILL.md @@ -1,12 +1,14 @@ --- -name: verification-before-completion -description: Use when about to claim a plan or work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always +name: prp-verification-before-completion +description: Use when about to claim a plan or work is complete, fixed, or passing, before completion of any PRP task, committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always --- # Verification Before Completion ## Overview +> **Policy**: This is a utility component per `plugins/prp-core/references/harness-taxonomy.md`. It does one narrow job: verify that work is complete before any completion claim. + Claiming work is complete without verification is dishonesty, not efficiency. **Core principle:** Evidence before claims, always. diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 00000000..fbb7f706 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,4 @@ +.claude/** linguist-generated +docs/** linguist-generated +graphify-out/** linguist-generated +wiki/** linguist-generated \ No newline at end of file diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index f2526a91..4b0bffde 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -1,5 +1,8 @@ # Copilot Instructions +- **graphify** (`~/.claude/skills/graphify/SKILL.md`) - any input to knowledge graph. Use the `graphify` MCP tools or Trigger: `/graphify` +When the user types `/graphify`, invoke the Skill tool with `skill: "graphify"` before doing anything else. + ## Build, test, and lint - Use `uv` for repo-level Python tasks. @@ -21,4 +24,4 @@ - Keep manifest and install naming aligned across docs and config: the marketplace publishes `prp-core` from `./plugins/prp-core`, and install examples should use `prp-core@prp-engineering-marketplace`. - Keep `CLAUDE.md` minimal. Project-wide essentials belong there; file-pattern-specific rules belong in `.claude/rules/*.md`. - Keep `.claude/rules/` narrowly scoped and one-topic-per-file. For example, SKILL.md conventions should target `**/skills/*/SKILL.md`, and hook-script conventions should target shell hooks rather than README or JSON files. -- When editing `.github/instructions/*.instructions.md`, include YAML frontmatter with `applyTo`, keep each file under 4,000 characters, and write concrete review checks rather than generic quality guidance. +- When editing `.github/instructions/*.instructions.md`, include YAML frontmatter with `applyTo`, keep each file under 4,000 characters, and write concrete review checks rather than generic quality guidance. \ No newline at end of file diff --git a/.github/instructions/harness-engineering.instructions.md b/.github/instructions/harness-engineering.instructions.md new file mode 100644 index 00000000..14b40dc3 --- /dev/null +++ b/.github/instructions/harness-engineering.instructions.md @@ -0,0 +1,31 @@ +--- +applyTo: "**" +--- +# Harness Engineering Review Guidelines + +## Smart Zone + +- Flag any CLAUDE.md or rules file change that increases always-loaded context without a compensating removal. +- Flag additions of directory listings, skills inventories, or version numbers to always-loaded files. +- Flag inline policy duplication that should reference `plugins/prp-core/references/` instead. + +## Harness Rails + +- Flag any new behavioral instruction that should instead be a hook, test, or lint constraint. +- Verify that shipped skills link to `plugins/prp-core/references/` for policy rather than duplicating inline. +- Flag changes to deterministic contracts (hook sentinels, eval cases, schema validators) without corresponding test updates. + +## Subagent Isolation + +- Flag skills that return raw tool output to the parent context instead of condensed summaries. +- Verify that advisory agents (review, analysis) include a `## CRITICAL: Your only job is to [role]` header. + +## Plugin Boundary + +- Flag any content that crosses the `plugins/prp-core/` ↔ root `.claude/` boundary without explicit justification. +- Verify root-only extras (`prp-core-runner`, `update-review-instructions`, `gpui-researcher`, `rubber-duck`) are not documented or shipped as plugin payload. + +## Artifact Discipline + +- Flag plan or implementation work that skips producing a durable artifact (PRP, plan file, research note) before the next phase begins. +- Verify the eval harness (`tests/prp-core/prompt-cases.json`) is updated when new skills or agents are added. diff --git a/.github/workflows/prp-core-prompt-evals.yml b/.github/workflows/prp-core-prompt-evals.yml new file mode 100644 index 00000000..7911d789 --- /dev/null +++ b/.github/workflows/prp-core-prompt-evals.yml @@ -0,0 +1,36 @@ +name: prp-core-prompt-evals + +on: + pull_request: + paths: + - "plugins/prp-core/**" + - ".claude/**" + - "scripts/prp_core_prompt_eval.py" + - "tests/prp-core/**" + +jobs: + prompt-evals: + name: Prompt contract checks + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.11" + + - name: Install uv + run: pip install uv + + - name: List cases + run: uv run python scripts/prp_core_prompt_eval.py --list + + - name: Run mirror-parity check + run: uv run python scripts/prp_core_prompt_eval.py --case mirror-parity + + - name: Run hook schema check + run: uv run python scripts/prp_core_prompt_eval.py --case research-team-hook-schema + + - name: Run all contract checks + run: uv run python scripts/prp_core_prompt_eval.py --all diff --git a/.gitignore b/.gitignore index 3c8effcc..9acb5935 100644 --- a/.gitignore +++ b/.gitignore @@ -11,4 +11,15 @@ wheels/ .env -.ruff_cache \ No newline at end of file +.ruff_cache + +graphify-out/cache/ +graphify-out/chunks/ +graphify-out/chunks_deep/ +graphify-out/memory/ +graphify-out/pdf_extracts/ +graphify-out/.graphify_chunk_*.txt +humanlayer/ +autoresearch-ref/ + +docs/ \ No newline at end of file diff --git a/.graphifyignore b/.graphifyignore new file mode 100644 index 00000000..cd50bdda --- /dev/null +++ b/.graphifyignore @@ -0,0 +1,4 @@ +autoresearch-ref/ +humanlayer/ +docs/superpowers/ +superpowers/ diff --git a/.mcp.json b/.mcp.json new file mode 100644 index 00000000..bbcfa270 --- /dev/null +++ b/.mcp.json @@ -0,0 +1,13 @@ +{ + "mcpServers": { + "graphify": { + "type": "stdio", + "command": ".venv/bin/python3", + "args": [ + "-m", + "graphify.serve", + "graphify-out/graph.json" + ] + } + } +} \ No newline at end of file diff --git a/CLAUDE.md b/CLAUDE.md index b6cca449..f2fa5fd7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,30 +1,27 @@ # PRP Framework — AI-powered workflow automation plugin for Claude Code. -## Skills +- **graphify**: `/graphify` → invoke `skill: "graphify"` before anything else. -Skills are the primary interface. Invoke by name (e.g., "run the prp-plan skill"). +## Priorities (highest first) -| Workflow | Skills | -|----------|--------| -| Full pipeline | `prp-core-runner` (orchestrates: plan → implement → commit → PR) | -| Plan & Build | `prp-plan`, `prp-implement`, `prp-commit`, `prp-pr` | -| Advisor | `prp-advisor` | -| Validation | `prp-verification-before-completion` | -| Research | `prp-codebase-question`, `prp-debug`, `prp-prd`, `prp-research-team` | -| Issues | `prp-issue-investigate`, `prp-issue-fix` | -| Review | `prp-review`, `prp-review-agents` | -| Autonomous | `prp-ralph` (start loop), `prp-ralph-cancel` (stop loop) | +1. **Smart Zone** [CRITICAL] — Keep context window < 40% capacity. Compact before reaching limit; delegate heavy work to subagents with fresh windows. +2. **Harness-First** [HIGH] — Deterministic rails (tests, hooks, linters) over corrective prompting. Prove completion via verification, not self-reported confidence. +3. **Subagent Isolation** [HIGH] — Research and analysis in isolated contexts; return only condensed summaries (~1,000 tokens) to the parent agent. +4. **Artifact-First** [MEDIUM] — PRPs, plans, and review artifacts are first-class outputs. Each phase writes a durable artifact before the next phase starts. +5. **Progressive Disclosure** [MEDIUM] — Load context in tiers. Authoritative policy: `plugins/prp-core/references/`. ## Context - `plugins/prp-core/` is the packaged plugin distribution. -- Root `.claude/` mirrors plugin artifacts for local development and also contains project-only extras such as `prp-core-runner`, `update-review-instructions`, and `gpui-researcher`. +- Root `.claude/` mirrors plugin artifacts for local development; root-only extras: `prp-core-runner`, `update-review-instructions`, `gpui-researcher`, `rubber-duck`. - Use `plugins/prp-core/CLAUDE.md` when changing the shipped plugin payload. ## PRP Methodology **PRP = PRD + curated codebase intelligence + agent/runbook** — enables one-pass implementation. -When creating PRPs, include: goal, business value, user-visible behavior, all needed context (docs, examples, gotchas), implementation blueprint with task list, and executable validation commands. +Run the `prp-advisor` skill before writing any `*.prd.md` or `*.plan.md` file or committing to any interpretation. -When executing PRPs: load and understand all context → create plan with todos → implement following blueprint → validate at each step → fix failures before proceeding. +When creating PRPs: include goal, business value, user-visible behavior, all needed context, implementation blueprint with task list, and executable validation commands. + +When executing PRPs: load context → create plan with todos → implement → validate at each step → fix failures before proceeding. diff --git a/README.md b/README.md index bdaae601..e234de69 100644 --- a/README.md +++ b/README.md @@ -410,15 +410,24 @@ Artifacts are plain markdown files. They accumulate over time, forming a project ├── .claude/ # Repo-local mirror + project-only extras │ ├── agents/ # 14 agents (12 plugin + gpui-researcher + rubber-duck) │ ├── hooks/ # Manual Ralph hook setup +│ ├── references/ # Mirror of plugin references/ (policy docs) │ ├── rules/ # Path-scoped execution rules (e.g. prp-workflow) │ └── skills/ # 19 skills (17 plugin + prp-core-runner, update-review-instructions) +├── docs/ +│ ├── prp-core-harness-architecture.md # Component taxonomy, budget policy, agent structure +│ └── prp-core-prompt-evals.md # Eval harness usage and test case guide ├── plugins/ │ └── prp-core/ # Installable plugin package (v3.1.2) │ ├── .claude-plugin/plugin.json │ ├── agents/ # 12 agents │ ├── hooks/ # 2 Stop hooks +│ ├── references/ # Shared policy documents (context-budget, execution-policy, etc.) │ ├── rules/ # 1 path-scoped rule (prp-workflow) │ └── skills/ # 17 skills +├── scripts/ +│ └── prp_core_prompt_eval.py # Repo-only deterministic contract checker +├── tests/ +│ └── prp-core/ # Prompt eval cases and fixtures ├── README.md └── README-for-DUMMIES.md ``` diff --git a/graphify-out/.graphify_chunks.json b/graphify-out/.graphify_chunks.json new file mode 100644 index 00000000..887c7571 --- /dev/null +++ b/graphify-out/.graphify_chunks.json @@ -0,0 +1,92 @@ +[ + [ + "docs/README.md", + "docs/harness-engineering.md", + "docs/agent-protocols/a2a-protocol-huggingface-space.md", + "docs/agent-protocols/advancing-agentic-ai-communication-protocols.md", + "docs/agent-protocols/ai-agent-protocols-2026-guide.md", + "docs/agent-protocols/anthropic-engineer-2026-forecast-full-connectivity-mcp.md", + "docs/agent-protocols/architectural-paradigms-advanced-agentic-systems.md", + "docs/agent-protocols/fluid-human-agent-collaboration-pmc.md", + "docs/agentic-engineering/agentic-software-modernization-markus-harrer.md", + "docs/agentic-engineering/building-agent-harness-martin-richards.md", + "docs/agentic-engineering/claude-cookbook-anthropic.md", + "docs/agentic-engineering/research-plan-implement-review-tyler-burleigh.md", + "docs/agentic-engineering/research-plan-implement-rpi.md", + "docs/claude-code/claude-prompting-best-practices.md", + "docs/claude-code/hooks/automate-workflow-with-hooks.md", + "docs/claude-code/hooks/claude-hook-reference-doc.md", + "docs/claude-code/memory/how-claude-remembers-a-project.md", + "docs/claude-code/plugins/claude-create-plugin-doc.md", + "docs/claude-code/skills/extend-claude-with-skills.md", + "docs/claude-code/skills/research-claude-code-skills-format.md" + ], + [ + "docs/claude-code/subagents/claude-orchestrate-of-claude-code-sessions.md", + "docs/claude-code/subagents/creating-custom-subagents.md", + "docs/context-engineering/advanced-context-engineering-coding-agents-dev.md", + "docs/context-engineering/agents-md-is-a-liability-paddo.md", + "docs/context-engineering/context-engineering-commercial-agents-jeremy-daly.md", + "docs/context-engineering/context-engineering-most-important-skill-dev.md", + "docs/context-engineering/context-stops-being-scarce-paddo.md", + "docs/context-engineering/pi-context-zone-github.md", + "docs/context-engineering/progressive-disclosure-ai-agents.md", + "docs/context-engineering/shedding-dead-context-ryan-spletzer.md", + "docs/general-llm/Call_Me_A_Jerk_Persuading_AI_to_Comply_with_Objectionable_Requests.pdf", + "docs/general-llm/Evaluating-AGENTS-paper.md", + "docs/general-llm/Evaluating-AGENTS-paper.pdf", + "docs/general-llm/a-guide-to-agents.md", + "docs/general-llm/persuasion-principles.md", + "docs/general-llm/prompt-engineering-guide.md", + "docs/general-llm/research-agent-workflows-and-patterns.md", + "docs/general-llm/research-context-engineering-comprehensive.md", + "docs/general-llm/research-context-rot-and-management.md", + "docs/general-llm/research-multilingual-performance.md", + "docs/general-llm/research-whitespace-and-formatting.md" + ], + [ + "docs/general-llm/subagents/research-subagent-best-practices.md", + "docs/long-context-research/README.md", + "docs/long-context-research/lost-in-the-middle-acl.md", + "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "docs/long-context-research/lost-in-the-middle-arxiv.md", + "docs/long-context-research/lost-in-the-middle-researchgate.md", + "docs/mcp/a2a-protocol-huggingface.md", + "docs/mcp/about-mcp-github-docs.md", + "docs/mcp/anthropic-mcp-github-topics.md", + "docs/mcp/long-live-mcp-aqfer.md", + "docs/mcp/long-live-mcp-aws.md", + "docs/mcp/mcp-http2-http3-reddit.md", + "docs/mcp/mcp-specification.md", + "docs/mcp/mcp-typescript-sdk-github.md", + "docs/mcp/mcp-vs-a2a-dzone.md", + "docs/mcp/modelcontextprotocol-servers-github.md", + "docs/mcp/skills-over-mcp-meeting-notes-2248.md", + "docs/mcp/skills-over-mcp-office-hours-2460.md", + "docs/mcp/skills-vs-mcp-speakeasy.md", + "docs/shared/skill-authoring-best-practices.md" + ], + [ + "docs/shared/skills-standard/README.md", + "docs/shared/skills-standard/agentskills-best-practices.md", + "docs/shared/skills-standard/agentskills-evaluating-skills.md", + "docs/shared/skills-standard/agentskills-optimizing-descriptions.md", + "docs/shared/skills-standard/agentskills-specification.md", + "docs/shared/skills-standard/agentskills-using-scripts.md", + "docs/shared/skills-standard/agentskills-what-are-skills.md", + "docs/spec-driven-development/README.md", + "docs/spec-driven-development/spec-driven-development-arxiv.md", + "docs/spec-driven-development/spec-driven-development-main.md", + "docs/spec-driven-development/spec-driven-development-variant.md", + "docs/structured-outputs/README.md", + "docs/structured-outputs/anthropic-implement-tool-use.md", + "docs/structured-outputs/anthropic-increase-consistency.md", + "docs/structured-outputs/anthropic-strict-tool-use.md", + "docs/structured-outputs/anthropic-structured-outputs.md", + "docs/tool-calling/README.md", + "docs/tool-calling/cameronking4-programmatic-tool-calling-github.md", + "docs/tool-calling/mcp-programmatic-tool-calling-opensandbox-dev.md", + "docs/tool-calling/programmatic-tool-calling-claude-api.md", + "docs/tool-calling/tool-search-redefining-agent-tool-calling-epsilla.md" + ] +] \ No newline at end of file diff --git a/graphify-out/.graphify_python b/graphify-out/.graphify_python new file mode 100644 index 00000000..43c7abab --- /dev/null +++ b/graphify-out/.graphify_python @@ -0,0 +1 @@ +/home/rodrigo/.local/share/pipx/venvs/graphifyy/bin/python \ No newline at end of file diff --git a/graphify-out/GRAPH_REPORT.md b/graphify-out/GRAPH_REPORT.md new file mode 100644 index 00000000..54081c1f --- /dev/null +++ b/graphify-out/GRAPH_REPORT.md @@ -0,0 +1,194 @@ +# Graph Report - /home/rodrigo/Workspace/prp-agentic-engineering (2026-04-22) + +## Corpus Check +- 1 files · ~359,895 words +- Verdict: corpus is large enough that graph structure adds value. + +## Summary +- 339 nodes · 455 edges · 21 communities detected +- Extraction: 83% EXTRACTED · 17% INFERRED · 1% AMBIGUOUS · INFERRED: 76 edges (avg confidence: 0.77) +- Token cost: 0 input · 0 output + +## Community Hubs (Navigation) +- [[_COMMUNITY_Community 0|Community 0]] +- [[_COMMUNITY_Community 1|Community 1]] +- [[_COMMUNITY_Community 2|Community 2]] +- [[_COMMUNITY_Community 3|Community 3]] +- [[_COMMUNITY_Community 4|Community 4]] +- [[_COMMUNITY_Community 5|Community 5]] +- [[_COMMUNITY_Community 6|Community 6]] +- [[_COMMUNITY_Community 7|Community 7]] +- [[_COMMUNITY_Community 8|Community 8]] +- [[_COMMUNITY_Community 9|Community 9]] +- [[_COMMUNITY_Community 10|Community 10]] +- [[_COMMUNITY_Community 11|Community 11]] +- [[_COMMUNITY_Community 12|Community 12]] +- [[_COMMUNITY_Community 13|Community 13]] +- [[_COMMUNITY_Community 14|Community 14]] +- [[_COMMUNITY_Community 15|Community 15]] +- [[_COMMUNITY_Community 16|Community 16]] +- [[_COMMUNITY_Community 17|Community 17]] +- [[_COMMUNITY_Community 18|Community 18]] +- [[_COMMUNITY_Community 19|Community 19]] +- [[_COMMUNITY_Community 20|Community 20]] + +## God Nodes (most connected - your core abstractions) +1. `Claude Code subagent definition best practices` - 12 edges +2. `Spec-Driven Development` - 10 edges +3. `Strict Tool Use` - 9 edges +4. `Context Engineering` - 9 edges +5. `Structured Outputs` - 8 edges +6. `Create plugins` - 8 edges +7. `Skill authoring best practices` - 8 edges +8. `Model Context Protocol (MCP)` - 7 edges +9. `Agent Skills Standard — Official Documentation` - 7 edges +10. `Lost in the Middle (ACL/TACL)` - 7 edges + +## Surprising Connections (you probably didn't know these) +- `Context Engineering` --conceptually_related_to--> `Parahuman AI Behavior` [INFERRED] + docs/context-engineering/context-engineering-most-important-skill-dev.md → docs/general-llm/Call_Me_A_Jerk_Persuading_AI_to_Comply_with_Objectionable_Requests.pdf +- `Context Engineering` --conceptually_related_to--> `Effective Context Engineering for AI Agents` [INFERRED] + docs/general-llm/research-context-engineering-comprehensive.md → docs/general-llm/research-context-rot-and-management.md +- `Spec-Driven Development` --semantically_similar_to--> `Structured Outputs` [INFERRED] [semantically similar] + docs/spec-driven-development/spec-driven-development-main.md → docs/structured-outputs/anthropic-structured-outputs.md +- `Strict Tool Use` --references--> `Anthropic Claude tool_search` [EXTRACTED] + docs/structured-outputs/anthropic-strict-tool-use.md → docs/tool-calling/README.md +- `Anthropic Claude Cookbook` --references--> `Model Context Protocol (MCP)` [INFERRED] + docs/agentic-engineering/claude-cookbook-anthropic.md → docs/mcp/mcp-vs-a2a-dzone.md + +## Hyperedges (group relationships) +- **Anthropic Reliability Stack** — anthropic_structured_outputs_structured_outputs, anthropic_structured_outputs_json_outputs, anthropic_strict_tool_use_strict_tool_use, anthropic_structured_outputs_constrained_decoding [EXTRACTED 1.00] +- **HumanLayer Runtime Stack** — humanlayer_repository_analysis_humanlayer_wui, humanlayer_repository_analysis_hlyr, humanlayer_repository_analysis_hld, humanlayer_repository_analysis_claudecode_go, humanlayer_repository_analysis_approval_loop [EXTRACTED 1.00] +- **RPI Workflow Implementations** — advanced-context-engineering_rpi_workflow, research-plan-implement_humanlayer_rpi, rpir-tyler-burleigh_rpir_workflow, building-agent-harness_atelier [EXTRACTED 0.95] +- **Context Degradation Phenomena** — advanced-context-engineering_dumb_zone, progressive-disclosure_context_rot, shedding-dead-context_dead_context, context-stops-being-scarce_compaction_problem [INFERRED 0.85] +- **MCP Ecosystem Components** — mcp-vs-a2a_mcp, mcp-typescript-sdk_typescript_sdk, mcp-servers_reference_servers, mcp-transport_mrtr_stateless, mcp-transport_server_cards, anthropic-mcp-topics_mcp_ecosystem [EXTRACTED 0.90] +- **Long-Context Mitigation Experiments** — lost_in_the_middle_and_in_between_multi_hop_qa, lost_in_the_middle_and_in_between_chain_of_thought_prompting, lost_in_the_middle_and_in_between_knowledge_graph_triple_extraction, lost_in_the_middle_and_in_between_document_summarization [EXTRACTED 1.00] +- **Context Engineering Scoped File Set** — research_context_engineering_comprehensive_doc, research_context_rot_and_management_doc, research_whitespace_and_formatting_doc, research_multilingual_performance_doc, research_agent_workflows_and_patterns_doc [EXTRACTED 1.00] +- **Agent Skills Standard Document Set** — agentskills_readme_doc, agentskills_what_are_skills_doc, agentskills_specification_doc, agentskills_best_practices_doc, agentskills_using_scripts_doc, agentskills_optimizing_descriptions_doc, agentskills_evaluating_skills_doc [EXTRACTED 1.00] +- **Spec-Driven Development Tool Landscape** — spec_driven_development_readme_doc, spec_driven_development_variant_kiro, spec_driven_development_variant_spec_kit, spec_driven_development_variant_tessl [EXTRACTED 1.00] +- **Skills over MCP Discovery Stack** — skills_over_mcp_meeting_notes_2248_prompts_as_skills, skills_over_mcp_office_hours_2460_well_known_skill_index, skills_over_mcp_office_hours_2460_lazy_loading_pattern, mcp_specification_prompts_primitive, skills_vs_mcp_speakeasy_server_prompt_bridge [INFERRED 0.86] +- **Programmatic Tool Calling Stack** — long_live_mcp_aqfer_progressive_tool_discovery, tool_search_redefining_agent_tool_calling_epsilla_tool_search, programmatic_tool_calling_claude_api_tool_search, mcp_programmatic_tool_calling_opensandbox_code_mode, cameronking4_programmatic_tool_calling_programmatic_tool_calling [INFERRED 0.88] +- **Claude Extensibility Stack** — claude_hook_reference_doc_hook_lifecycle, extend_claude_with_skills_context_fork, creating_custom_subagents_custom_subagents, claude_orchestrate_of_claude_code_sessions_agent_teams, research_claude_code_skills_format_marketplace_format [INFERRED 0.82] +- **Project memory stack** — how_claude_remembers_a_project_claude_md_files, how_claude_remembers_a_project_auto_memory, how_claude_remembers_a_project_memory_command [EXTRACTED 0.95] +- **Plugin package layout** — claude_create_plugin_doc_plugin_manifest, claude_create_plugin_doc_plugin_skills_directory, claude_create_plugin_doc_plugin_root_structure, claude_create_plugin_doc_plugin_settings_json [EXTRACTED 0.96] +- **Subagent guardrail bundle** — research_subagent_best_practices_tool_restrictions, research_subagent_best_practices_pretooluse_hooks, research_subagent_best_practices_worktree_isolation [INFERRED 0.84] + +## Communities + +### Community 0 - "Community 0" +Cohesion: 0.08 +Nodes (40): Copilot MCP Surfaces, GitHub MCP Server, Push Protection, GitHub MCP Toolsets, code_execution Meta-Tool, MCP Bridge, Programmatic Tool Calling, Progressive Tool Discovery (+32 more) + +### Community 1 - "Community 1" +Cohesion: 0.08 +Nodes (38): Plugin skills directory, Auto memory, claudeMdExcludes, CLAUDE.md files, .claude/rules, Concise specific instructions, Context noise reduction, How Claude remembers your project (+30 more) + +### Community 2 - "Community 2" +Cohesion: 0.07 +Nodes (34): Dumb Zone, Intentional Compaction, Research-Plan-Implement (RPI) Workflow, Critic Agent Pattern, Atelier, Harness Concept, Mitchell Hashimoto, Parahuman AI Behavior (+26 more) + +### Community 3 - "Community 3" +Cohesion: 0.08 +Nodes (32): A2A Protocol, Agent Card, A2aprotocol — a Hugging Face Space by a2aprotocol, Agent Network Protocol (ANP), Advancing Agentic AI through Communication Protocols, Best practices for skill creators, Gotchas Pattern, Validation Loop (+24 more) + +### Community 4 - "Community 4" +Cohesion: 0.11 +Nodes (24): Why Strict Tool Use Matters for Agents, Strict Tool PHI Schema Exclusion, Strict Tool Schema Complexity Limits, Strict Tool Use, tool_choice any, Validated Tool Inputs, Constrained Decoding, Grammar Compilation Cache (+16 more) + +### Community 5 - "Community 5" +Cohesion: 0.12 +Nodes (20): Chain-of-Thought Prompting, Why Re-ranking Becomes Impractical, Document Summarization, Why Context Reduction Remains Fragile, In-Between Effect, Knowledge Graph Triple Extraction, Levy et al. 2024, Liu et al. 2024 (+12 more) + +### Community 6 - "Community 6" +Cohesion: 0.15 +Nodes (18): AGENTS.md, A Complete Guide To AGENTS.md, Instruction Budget, Path-Scoped Rules, Stale Documentation Poisons Context, What are skills?, Progressive Skill Discovery, Skill Directory Structure (+10 more) + +### Community 7 - "Community 7" +Cohesion: 0.2 +Nodes (18): Deterministic Hook Automation, Hook Type Selection, Hook Decision Control, Hook Lifecycle, Subagent and Team Hooks, Agent Teams, Subagents vs Agent Teams, Custom Subagents (+10 more) + +### Community 8 - "Community 8" +Cohesion: 0.14 +Nodes (14): Entropy in Legacy Codebases, 500 Instruction Ceiling, Attention Dilution, Attention Sinks, IFScale Benchmark, Prompt Repetition Hack, Context Selection, Context Structuring (+6 more) + +### Community 9 - "Community 9" +Cohesion: 0.23 +Nodes (14): Agentic Context Engineering (ACE), Architectural Paradigms of Advanced Agentic Systems, The Prompt Report, Context Engineering, LLM Context Engineering: Comprehensive Research Synthesis, Smallest Set of High-Signal Tokens, LLM Multilingual Performance & Language Overhead, Tokenization Unfairness Between Languages (+6 more) + +### Community 10 - "Community 10" +Cohesion: 0.18 +Nodes (14): Input Examples, Strict Tool Choice, Tool Definition Quality, Prompt Chaining, Prompt Consistency Techniques, Retrieval Grounding, Structured Outputs for Guarantees, Long Context Prompting (+6 more) + +### Community 11 - "Community 11" +Cohesion: 0.25 +Nodes (11): Approval Loop, claudecode-go, CodeLayer, Why hld Owns Orchestration, hld Daemon, hlyr CLI, HumanLayer Local Runtime, humanlayer-wui (+3 more) + +### Community 12 - "Community 12" +Cohesion: 0.25 +Nodes (11): Long Context Paper Collection, Knowledge Graph Extraction Fragility, Multi-Hop Follow-up Paper, Long-Context Evaluation Protocols, Lost in the Middle (ACL/TACL), Positional Bias, U-Shaped Performance Curve, Lost in the Middle (arXiv Preprint) (+3 more) + +### Community 13 - "Community 13" +Cohesion: 0.27 +Nodes (11): Conflict prevention, Create plugins, --plugin-dir flag, Plugin manifest, Standalone-to-plugin migration, Plugin root structure, Plugin settings.json, /reload-plugins command (+3 more) + +### Community 14 - "Community 14" +Cohesion: 0.29 +Nodes (9): evaluate_mirror_parity(), evaluate_must_contain(), load_cases(), main(), Deterministic prompt-contract evaluator for prp-core. Checks shipped skills, ag, Check that each target file contains all required strings., Check that each (shipped, mirror) pair is identical., Run one case. Returns (pass_count, fail_count). (+1 more) + +### Community 15 - "Community 15" +Cohesion: 0.22 +Nodes (10): Agent Cards, MCP Ecosystem Projects, Anthropic Claude Cookbook, AWS MCP Adoption, MCP Reference Servers, MRTR Stateless Transport, MCP Server Cards, MCP TypeScript SDK (+2 more) + +### Community 16 - "Community 16" +Cohesion: 0.38 +Nodes (10): Spec-Driven Development: From Code to Contract in the Age of AI Coding Assistants, Spec-Anchored, Spec-as-Source, Spec-First, Spec-Driven Development — Research Overview, Spec-Driven Development, Spec-Driven Development: Exploring Three SDD Tools, Kiro (+2 more) + +### Community 17 - "Community 17" +Cohesion: 0.5 +Nodes (4): Call Me A Jerk, Commitment Principle, Persuasion Principles for Agent Skill Engineering, Parahuman Effect + +### Community 18 - "Community 18" +Cohesion: 1.0 +Nodes (3): Towards Fluid Human-Agent Collaboration, Dynamic Mentalizing, Fluid Collaboration + +### Community 19 - "Community 19" +Cohesion: 1.0 +Nodes (2): CIMD, XAA + +### Community 20 - "Community 20" +Cohesion: 1.0 +Nodes (1): Multi Round-Trip Requests + +## Ambiguous Edges - Review These +- `Spec-Anchored` → `Spec Kit` [AMBIGUOUS] + docs/spec-driven-development/spec-driven-development-variant.md · relation: conceptually_related_to +- `Resource URIs` → `GitHub MCP Toolsets` [AMBIGUOUS] + docs/mcp/skills-over-mcp-office-hours-2460.md · relation: conceptually_related_to +- `Evaluation-driven development` → `Confidence-based filtering` [AMBIGUOUS] + docs/shared/skill-authoring-best-practices.md · relation: conceptually_related_to + +## Knowledge Gaps +- **85 isolated node(s):** `Deterministic prompt-contract evaluator for prp-core. Checks shipped skills, ag`, `Check that each target file contains all required strings.`, `Check that each (shipped, mirror) pair is identical.`, `Run one case. Returns (pass_count, fail_count).`, `AWS Kiro` (+80 more) + These have ≤1 connection - possible missing edges or undocumented components. +- **Thin community `Community 19`** (2 nodes): `CIMD`, `XAA` + Too small to be a meaningful cluster - may be noise or needs more connections extracted. +- **Thin community `Community 20`** (1 nodes): `Multi Round-Trip Requests` + Too small to be a meaningful cluster - may be noise or needs more connections extracted. + +## Suggested Questions +_Questions this graph is uniquely positioned to answer:_ + +- **What is the exact relationship between `Spec-Anchored` and `Spec Kit`?** + _Edge tagged AMBIGUOUS (relation: conceptually_related_to) - confidence is low._ +- **What is the exact relationship between `Resource URIs` and `GitHub MCP Toolsets`?** + _Edge tagged AMBIGUOUS (relation: conceptually_related_to) - confidence is low._ +- **What is the exact relationship between `Evaluation-driven development` and `Confidence-based filtering`?** + _Edge tagged AMBIGUOUS (relation: conceptually_related_to) - confidence is low._ +- **Why does `Practical Agent Workflows and Patterns` connect `Community 3` to `Community 9`, `Community 6`?** + _High betweenness centrality (0.024) - this node is a cross-community bridge._ +- **Why does `LLM Context Engineering: Comprehensive Research Synthesis` connect `Community 9` to `Community 3`, `Community 6`?** + _High betweenness centrality (0.014) - this node is a cross-community bridge._ +- **Why does `Context Engineering` connect `Community 2` to `Community 8`?** + _High betweenness centrality (0.013) - this node is a cross-community bridge._ +- **What connects `Deterministic prompt-contract evaluator for prp-core. Checks shipped skills, ag`, `Check that each target file contains all required strings.`, `Check that each (shipped, mirror) pair is identical.` to the rest of the system?** + _85 weakly-connected nodes found - possible documentation gaps or missing edges._ \ No newline at end of file diff --git a/graphify-out/cost.json b/graphify-out/cost.json new file mode 100644 index 00000000..407c03ed --- /dev/null +++ b/graphify-out/cost.json @@ -0,0 +1,42 @@ +{ + "runs": [ + { + "date": "2026-04-22T01:06:04.010061+00:00", + "input_tokens": 0, + "output_tokens": 0, + "files": 81 + }, + { + "date": "2026-04-22T01:32:45.374045+00:00", + "input_tokens": 0, + "output_tokens": 0, + "files": 82 + }, + { + "date": "2026-04-22T01:59:39.080240+00:00", + "input_tokens": 0, + "output_tokens": 0, + "files": 83 + }, + { + "date": "2026-04-22T03:15:29.267189+00:00", + "input_tokens": 0, + "output_tokens": 0, + "files": 238 + }, + { + "date": "2026-04-22T12:42:26.282483+00:00", + "input_tokens": 0, + "output_tokens": 0, + "files": 88 + }, + { + "date": "2026-04-22T13:05:47.306063+00:00", + "input_tokens": 0, + "output_tokens": 0, + "files": 85 + } + ], + "total_input_tokens": 0, + "total_output_tokens": 0 +} \ No newline at end of file diff --git a/graphify-out/graph.html b/graphify-out/graph.html new file mode 100644 index 00000000..43017ba7 --- /dev/null +++ b/graphify-out/graph.html @@ -0,0 +1,257 @@ + + + + +graphify - /home/rodrigo/Workspace/prp-agentic-engineering/graphify-out/graph.html + + + + +
+ + + + + \ No newline at end of file diff --git a/graphify-out/graph.json b/graphify-out/graph.json new file mode 100644 index 00000000..d54fd54a --- /dev/null +++ b/graphify-out/graph.json @@ -0,0 +1,10277 @@ +{ + "directed": false, + "multigraph": false, + "graph": { + "hyperedges": [ + { + "id": "anthropic_reliability_stack", + "label": "Anthropic Reliability Stack", + "nodes": [ + "anthropic_structured_outputs_structured_outputs", + "anthropic_structured_outputs_json_outputs", + "anthropic_strict_tool_use_strict_tool_use", + "anthropic_structured_outputs_constrained_decoding" + ], + "relation": "form", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-structured-outputs.md" + }, + { + "id": "humanlayer_runtime_stack", + "label": "HumanLayer Runtime Stack", + "nodes": [ + "humanlayer_repository_analysis_humanlayer_wui", + "humanlayer_repository_analysis_hlyr", + "humanlayer_repository_analysis_hld", + "humanlayer_repository_analysis_claudecode_go", + "humanlayer_repository_analysis_approval_loop" + ], + "relation": "participate_in", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md" + }, + { + "id": "rpi_workflow_implementations", + "label": "RPI Workflow Implementations", + "nodes": [ + "advanced-context-engineering_rpi_workflow", + "research-plan-implement_humanlayer_rpi", + "rpir-tyler-burleigh_rpir_workflow", + "building-agent-harness_atelier" + ], + "relation": "implement", + "confidence": "EXTRACTED", + "confidence_score": 0.95, + "source_file": "docs/agentic-engineering/research-plan-implement-rpi.md" + }, + { + "id": "context_degradation_phenomena", + "label": "Context Degradation Phenomena", + "nodes": [ + "advanced-context-engineering_dumb_zone", + "progressive-disclosure_context_rot", + "shedding-dead-context_dead_context", + "context-stops-being-scarce_compaction_problem" + ], + "relation": "form", + "confidence": "INFERRED", + "confidence_score": 0.85, + "source_file": "docs/context-engineering/advanced-context-engineering-coding-agents-dev.md" + }, + { + "id": "mcp_ecosystem_components", + "label": "MCP Ecosystem Components", + "nodes": [ + "mcp-vs-a2a_mcp", + "mcp-typescript-sdk_typescript_sdk", + "mcp-servers_reference_servers", + "mcp-transport_mrtr_stateless", + "mcp-transport_server_cards", + "anthropic-mcp-topics_mcp_ecosystem" + ], + "relation": "participate_in", + "confidence": "EXTRACTED", + "confidence_score": 0.9, + "source_file": "docs/mcp/mcp-vs-a2a-dzone.md" + }, + { + "id": "long_context_mitigation_experiments", + "label": "Long-Context Mitigation Experiments", + "nodes": [ + "lost_in_the_middle_and_in_between_multi_hop_qa", + "lost_in_the_middle_and_in_between_chain_of_thought_prompting", + "lost_in_the_middle_and_in_between_knowledge_graph_triple_extraction", + "lost_in_the_middle_and_in_between_document_summarization" + ], + "relation": "participate_in", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md" + }, + { + "id": "context_engineering_scoped_bundle", + "label": "Context Engineering Scoped File Set", + "nodes": [ + "research_context_engineering_comprehensive_doc", + "research_context_rot_and_management_doc", + "research_whitespace_and_formatting_doc", + "research_multilingual_performance_doc", + "research_agent_workflows_and_patterns_doc" + ], + "relation": "form", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-context-engineering-comprehensive.md" + }, + { + "id": "agent_skills_standard_docset", + "label": "Agent Skills Standard Document Set", + "nodes": [ + "agentskills_readme_doc", + "agentskills_what_are_skills_doc", + "agentskills_specification_doc", + "agentskills_best_practices_doc", + "agentskills_using_scripts_doc", + "agentskills_optimizing_descriptions_doc", + "agentskills_evaluating_skills_doc" + ], + "relation": "form", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skills-standard/README.md" + }, + { + "id": "spec_driven_development_tool_landscape", + "label": "Spec-Driven Development Tool Landscape", + "nodes": [ + "spec_driven_development_readme_doc", + "spec_driven_development_variant_kiro", + "spec_driven_development_variant_spec_kit", + "spec_driven_development_variant_tessl" + ], + "relation": "participate_in", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/README.md" + }, + { + "id": "skills_over_mcp_discovery_stack", + "label": "Skills over MCP Discovery Stack", + "nodes": [ + "skills_over_mcp_meeting_notes_2248_prompts_as_skills", + "skills_over_mcp_office_hours_2460_well_known_skill_index", + "skills_over_mcp_office_hours_2460_lazy_loading_pattern", + "mcp_specification_prompts_primitive", + "skills_vs_mcp_speakeasy_server_prompt_bridge" + ], + "relation": "form", + "confidence": "INFERRED", + "confidence_score": 0.86, + "source_file": "docs/mcp/skills-over-mcp-office-hours-2460.md" + }, + { + "id": "programmatic_tool_calling_stack", + "label": "Programmatic Tool Calling Stack", + "nodes": [ + "long_live_mcp_aqfer_progressive_tool_discovery", + "tool_search_redefining_agent_tool_calling_epsilla_tool_search", + "programmatic_tool_calling_claude_api_tool_search", + "mcp_programmatic_tool_calling_opensandbox_code_mode", + "cameronking4_programmatic_tool_calling_programmatic_tool_calling" + ], + "relation": "form", + "confidence": "INFERRED", + "confidence_score": 0.88, + "source_file": "docs/tool-calling/programmatic-tool-calling-claude-api.md" + }, + { + "id": "claude_extensibility_stack", + "label": "Claude Extensibility Stack", + "nodes": [ + "claude_hook_reference_doc_hook_lifecycle", + "extend_claude_with_skills_context_fork", + "creating_custom_subagents_custom_subagents", + "claude_orchestrate_of_claude_code_sessions_agent_teams", + "research_claude_code_skills_format_marketplace_format" + ], + "relation": "participate_in", + "confidence": "INFERRED", + "confidence_score": 0.82, + "source_file": "docs/README.md" + }, + { + "id": "project_memory_stack", + "label": "Project memory stack", + "nodes": [ + "how_claude_remembers_a_project_claude_md_files", + "how_claude_remembers_a_project_auto_memory", + "how_claude_remembers_a_project_memory_command" + ], + "relation": "form", + "confidence": "EXTRACTED", + "confidence_score": 0.95, + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md" + }, + { + "id": "plugin_package_layout", + "label": "Plugin package layout", + "nodes": [ + "claude_create_plugin_doc_plugin_manifest", + "claude_create_plugin_doc_plugin_skills_directory", + "claude_create_plugin_doc_plugin_root_structure", + "claude_create_plugin_doc_plugin_settings_json" + ], + "relation": "form", + "confidence": "EXTRACTED", + "confidence_score": 0.96, + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md" + }, + { + "id": "subagent_guardrail_bundle", + "label": "Subagent guardrail bundle", + "nodes": [ + "research_subagent_best_practices_tool_restrictions", + "research_subagent_best_practices_pretooluse_hooks", + "research_subagent_best_practices_worktree_isolation" + ], + "relation": "form", + "confidence": "INFERRED", + "confidence_score": 0.84, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md" + } + ] + }, + "nodes": [ + { + "label": "prp_core_prompt_eval.py", + "file_type": "code", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L1", + "id": "home_rodrigo_workspace_prp_agentic_engineering_scripts_prp_core_prompt_eval_py", + "community": 14, + "norm_label": "prp_core_prompt_eval.py" + }, + { + "label": "load_cases()", + "file_type": "code", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L34", + "id": "prp_core_prompt_eval_load_cases", + "community": 14, + "norm_label": "load_cases()" + }, + { + "label": "evaluate_must_contain()", + "file_type": "code", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L41", + "id": "prp_core_prompt_eval_evaluate_must_contain", + "community": 14, + "norm_label": "evaluate_must_contain()" + }, + { + "label": "evaluate_mirror_parity()", + "file_type": "code", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L60", + "id": "prp_core_prompt_eval_evaluate_mirror_parity", + "community": 14, + "norm_label": "evaluate_mirror_parity()" + }, + { + "label": "run_case()", + "file_type": "code", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L76", + "id": "prp_core_prompt_eval_run_case", + "community": 14, + "norm_label": "run_case()" + }, + { + "label": "main()", + "file_type": "code", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L97", + "id": "prp_core_prompt_eval_main", + "community": 14, + "norm_label": "main()" + }, + { + "label": "Deterministic prompt-contract evaluator for prp-core. Checks shipped skills, ag", + "file_type": "rationale", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L1", + "community": 14, + "norm_label": "deterministic prompt-contract evaluator for prp-core. checks shipped skills, ag", + "id": "prp_core_prompt_eval_rationale_1" + }, + { + "label": "Check that each target file contains all required strings.", + "file_type": "rationale", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L42", + "community": 14, + "norm_label": "check that each target file contains all required strings.", + "id": "prp_core_prompt_eval_rationale_42" + }, + { + "label": "Check that each (shipped, mirror) pair is identical.", + "file_type": "rationale", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L61", + "community": 14, + "norm_label": "check that each (shipped, mirror) pair is identical.", + "id": "prp_core_prompt_eval_rationale_61" + }, + { + "label": "Run one case. Returns (pass_count, fail_count).", + "file_type": "rationale", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L77", + "community": 14, + "norm_label": "run one case. returns (pass_count, fail_count).", + "id": "prp_core_prompt_eval_rationale_77" + }, + { + "label": "Spec-Driven Development", + "file_type": "document", + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "source_url": "https://dev.to/bobbyblaine/spec-driven-development-write-the-spec-not-the-code-2p5o", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "spec-driven development", + "id": "spec_driven_development_main_spec_driven_development" + }, + { + "label": "Vibe Coding", + "file_type": "document", + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "source_url": "https://dev.to/bobbyblaine/spec-driven-development-write-the-spec-not-the-code-2p5o", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "vibe coding", + "id": "spec_driven_development_main_vibe_coding" + }, + { + "label": "GitHub Spec Kit", + "file_type": "document", + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "source_url": "https://dev.to/bobbyblaine/spec-driven-development-write-the-spec-not-the-code-2p5o", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "github spec kit", + "id": "spec_driven_development_main_github_spec_kit" + }, + { + "label": "AWS Kiro", + "file_type": "document", + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "source_url": "https://dev.to/bobbyblaine/spec-driven-development-write-the-spec-not-the-code-2p5o", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "aws kiro", + "id": "spec_driven_development_main_aws_kiro" + }, + { + "label": "Tessl Framework", + "file_type": "document", + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "source_url": "https://dev.to/bobbyblaine/spec-driven-development-write-the-spec-not-the-code-2p5o", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "tessl framework", + "id": "spec_driven_development_main_tessl_framework" + }, + { + "label": "Spec-First Adoption", + "file_type": "document", + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "source_url": "https://dev.to/bobbyblaine/spec-driven-development-write-the-spec-not-the-code-2p5o", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "spec-first adoption", + "id": "spec_driven_development_main_spec_first" + }, + { + "label": "Spec-Anchored Adoption", + "file_type": "document", + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "source_url": "https://dev.to/bobbyblaine/spec-driven-development-write-the-spec-not-the-code-2p5o", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "spec-anchored adoption", + "id": "spec_driven_development_main_spec_anchored" + }, + { + "label": "Spec-as-Source Adoption", + "file_type": "document", + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "source_url": "https://dev.to/bobbyblaine/spec-driven-development-write-the-spec-not-the-code-2p5o", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "spec-as-source adoption", + "id": "spec_driven_development_main_spec_as_source" + }, + { + "label": "Why Specs Reduce AI Guesswork", + "file_type": "document", + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "source_url": "https://dev.to/bobbyblaine/spec-driven-development-write-the-spec-not-the-code-2p5o", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "why specs reduce ai guesswork", + "id": "spec_driven_development_main_ambiguity_reduction_rationale" + }, + { + "label": "Why SDD Breaks Down for Some Work", + "file_type": "document", + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "source_url": "https://dev.to/bobbyblaine/spec-driven-development-write-the-spec-not-the-code-2p5o", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "why sdd breaks down for some work", + "id": "spec_driven_development_main_review_overhead_rationale" + }, + { + "label": "Structured Outputs", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-structured-outputs.md", + "source_location": null, + "source_url": "https://docs.anthropic.com/en/docs/build-with-claude/structured-outputs", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "structured outputs", + "id": "anthropic_structured_outputs_structured_outputs" + }, + { + "label": "JSON Outputs", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-structured-outputs.md", + "source_location": null, + "source_url": "https://docs.anthropic.com/en/docs/build-with-claude/structured-outputs", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "json outputs", + "id": "anthropic_structured_outputs_json_outputs" + }, + { + "label": "Constrained Decoding", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-structured-outputs.md", + "source_location": null, + "source_url": "https://docs.anthropic.com/en/docs/build-with-claude/structured-outputs", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "constrained decoding", + "id": "anthropic_structured_outputs_constrained_decoding" + }, + { + "label": "Schema Complexity Limits", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-structured-outputs.md", + "source_location": null, + "source_url": "https://docs.anthropic.com/en/docs/build-with-claude/structured-outputs", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "schema complexity limits", + "id": "anthropic_structured_outputs_schema_complexity_limits" + }, + { + "label": "Grammar Compilation Cache", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-structured-outputs.md", + "source_location": null, + "source_url": "https://docs.anthropic.com/en/docs/build-with-claude/structured-outputs", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "grammar compilation cache", + "id": "anthropic_structured_outputs_grammar_compilation_cache" + }, + { + "label": "HIPAA and ZDR Constraints", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-structured-outputs.md", + "source_location": null, + "source_url": "https://docs.anthropic.com/en/docs/build-with-claude/structured-outputs", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "hipaa and zdr constraints", + "id": "anthropic_structured_outputs_hipaa_zdr_constraints" + }, + { + "label": "Why Structured Outputs Improve Reliability", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-structured-outputs.md", + "source_location": null, + "source_url": "https://docs.anthropic.com/en/docs/build-with-claude/structured-outputs", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "why structured outputs improve reliability", + "id": "anthropic_structured_outputs_reliability_rationale" + }, + { + "label": "Why PHI Must Stay Out of Schemas", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-structured-outputs.md", + "source_location": null, + "source_url": "https://docs.anthropic.com/en/docs/build-with-claude/structured-outputs", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "why phi must stay out of schemas", + "id": "anthropic_structured_outputs_phi_schema_exclusion_rationale" + }, + { + "label": "Strict Tool Use", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-strict-tool-use.md", + "source_location": null, + "source_url": "https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/strict-tool-use", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "strict tool use", + "id": "anthropic_strict_tool_use_strict_tool_use" + }, + { + "label": "tool_choice any", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-strict-tool-use.md", + "source_location": null, + "source_url": "https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/strict-tool-use", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "tool_choice any", + "id": "anthropic_strict_tool_use_tool_choice_any" + }, + { + "label": "Validated Tool Inputs", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-strict-tool-use.md", + "source_location": null, + "source_url": "https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/strict-tool-use", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "validated tool inputs", + "id": "anthropic_strict_tool_use_validated_tool_inputs" + }, + { + "label": "Why Strict Tool Use Matters for Agents", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-strict-tool-use.md", + "source_location": null, + "source_url": "https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/strict-tool-use", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "why strict tool use matters for agents", + "id": "anthropic_strict_tool_use_agent_reliability_rationale" + }, + { + "label": "Strict Tool Schema Complexity Limits", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-strict-tool-use.md", + "source_location": null, + "source_url": "https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/strict-tool-use", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "strict tool schema complexity limits", + "id": "anthropic_strict_tool_use_schema_complexity_limits" + }, + { + "label": "Strict Tool PHI Schema Exclusion", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-strict-tool-use.md", + "source_location": null, + "source_url": "https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/strict-tool-use", + "captured_at": null, + "author": null, + "contributor": null, + "community": 4, + "norm_label": "strict tool phi schema exclusion", + "id": "anthropic_strict_tool_use_phi_schema_exclusion" + }, + { + "label": "CodeLayer", + "file_type": "document", + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 11, + "norm_label": "codelayer", + "id": "humanlayer_repository_analysis_codelayer" + }, + { + "label": "HumanLayer Local Runtime", + "file_type": "document", + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 11, + "norm_label": "humanlayer local runtime", + "id": "humanlayer_repository_analysis_humanlayer_local_runtime" + }, + { + "label": "hlyr CLI", + "file_type": "document", + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 11, + "norm_label": "hlyr cli", + "id": "humanlayer_repository_analysis_hlyr" + }, + { + "label": "hld Daemon", + "file_type": "document", + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 11, + "norm_label": "hld daemon", + "id": "humanlayer_repository_analysis_hld" + }, + { + "label": "humanlayer-wui", + "file_type": "document", + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 11, + "norm_label": "humanlayer-wui", + "id": "humanlayer_repository_analysis_humanlayer_wui" + }, + { + "label": "claudecode-go", + "file_type": "document", + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 11, + "norm_label": "claudecode-go", + "id": "humanlayer_repository_analysis_claudecode_go" + }, + { + "label": "SQLite Persistence", + "file_type": "document", + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 11, + "norm_label": "sqlite persistence", + "id": "humanlayer_repository_analysis_sqlite_persistence" + }, + { + "label": "Approval Loop", + "file_type": "document", + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 11, + "norm_label": "approval loop", + "id": "humanlayer_repository_analysis_approval_loop" + }, + { + "label": "Hybrid Monorepo", + "file_type": "document", + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 11, + "norm_label": "hybrid monorepo", + "id": "humanlayer_repository_analysis_hybrid_monorepo" + }, + { + "label": "Two Persistence Worlds", + "file_type": "document", + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 11, + "norm_label": "two persistence worlds", + "id": "humanlayer_repository_analysis_two_persistence_worlds" + }, + { + "label": "Why hld Owns Orchestration", + "file_type": "document", + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 11, + "norm_label": "why hld owns orchestration", + "id": "humanlayer_repository_analysis_daemon_orchestration_rationale" + }, + { + "label": "Harness Concept", + "file_type": "document", + "source_file": "docs/agentic-engineering/building-agent-harness-martin-richards.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "harness concept", + "id": "building-agent-harness_harness_concept" + }, + { + "label": "Atelier", + "file_type": "document", + "source_file": "docs/agentic-engineering/building-agent-harness-martin-richards.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "atelier", + "id": "building-agent-harness_atelier" + }, + { + "label": "Mitchell Hashimoto", + "file_type": "document", + "source_file": "docs/agentic-engineering/building-agent-harness-martin-richards.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "mitchell hashimoto", + "id": "building-agent-harness_mitchell_hashimoto" + }, + { + "label": "HumanLayer RPI Framework", + "file_type": "document", + "source_file": "docs/agentic-engineering/research-plan-implement-rpi.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "humanlayer rpi framework", + "id": "research-plan-implement_humanlayer_rpi" + }, + { + "label": "Leverage Model (Bad Research \u2192 1000s Bad Lines)", + "file_type": "document", + "source_file": "docs/agentic-engineering/research-plan-implement-rpi.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "leverage model (bad research \u2192 1000s bad lines)", + "id": "research-plan-implement_leverage_model" + }, + { + "label": "Why Phased Workflows Prevent Error Cascade", + "file_type": "document", + "source_file": "docs/agentic-engineering/research-plan-implement-rpi.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "why phased workflows prevent error cascade", + "id": "rpi-rationale_why_phases_matter" + }, + { + "label": "Research-Plan-Implement-Review (RPIR) Workflow", + "file_type": "document", + "source_file": "docs/agentic-engineering/research-plan-implement-review-tyler-burleigh.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "research-plan-implement-review (rpir) workflow", + "id": "rpir-tyler-burleigh_rpir_workflow" + }, + { + "label": "Anthropic Claude Cookbook", + "file_type": "document", + "source_file": "docs/agentic-engineering/claude-cookbook-anthropic.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 15, + "norm_label": "anthropic claude cookbook", + "id": "claude-cookbook_anthropic_cookbook" + }, + { + "label": "Entropy in Legacy Codebases", + "file_type": "document", + "source_file": "docs/agentic-engineering/agentic-software-modernization-markus-harrer.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 8, + "norm_label": "entropy in legacy codebases", + "id": "agentic-modernization_entropy_problem" + }, + { + "label": "Critic Agent Pattern", + "file_type": "document", + "source_file": "docs/agentic-engineering/agentic-software-modernization-markus-harrer.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "critic agent pattern", + "id": "agentic-modernization_critic_agent" + }, + { + "label": "Context Engineering", + "file_type": "document", + "source_file": "docs/context-engineering/context-engineering-most-important-skill-dev.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "context engineering", + "id": "context-engineering-most-important-skill-dev_context_engineering" + }, + { + "label": "Five Pillars of Context Engineering", + "file_type": "document", + "source_file": "docs/context-engineering/context-engineering-most-important-skill-dev.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 8, + "norm_label": "five pillars of context engineering", + "id": "context-engineering-most-important-skill-dev_five_pillars" + }, + { + "label": "Context Selection", + "file_type": "document", + "source_file": "docs/context-engineering/context-engineering-most-important-skill-dev.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 8, + "norm_label": "context selection", + "id": "context-engineering-most-important-skill-dev_context_selection" + }, + { + "label": "Signal-to-Noise Principle", + "file_type": "document", + "source_file": "docs/context-engineering/context-engineering-most-important-skill-dev.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 8, + "norm_label": "signal-to-noise principle", + "id": "context-engineering-most-important-skill-dev_signal_to_noise" + }, + { + "label": "Context Structuring", + "file_type": "document", + "source_file": "docs/context-engineering/context-engineering-most-important-skill-dev.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 8, + "norm_label": "context structuring", + "id": "context-engineering-most-important-skill-dev_context_structuring" + }, + { + "label": "Lost in the Middle Problem", + "file_type": "document", + "source_file": "docs/context-engineering/context-engineering-most-important-skill-dev.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 8, + "norm_label": "lost in the middle problem", + "id": "context-engineering-most-important-skill-dev_lost_in_middle" + }, + { + "label": "Why Context Engineering > Prompt Engineering", + "file_type": "document", + "source_file": "docs/context-engineering/context-engineering-most-important-skill-dev.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "why context engineering > prompt engineering", + "id": "context-eng-rationale_why_context_engineering_matters" + }, + { + "label": "Dumb Zone", + "file_type": "document", + "source_file": "docs/context-engineering/advanced-context-engineering-coding-agents-dev.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "dumb zone", + "id": "advanced-context-engineering_dumb_zone" + }, + { + "label": "Intentional Compaction", + "file_type": "document", + "source_file": "docs/context-engineering/advanced-context-engineering-coding-agents-dev.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "intentional compaction", + "id": "advanced-context-engineering_intentional_compaction" + }, + { + "label": "Research-Plan-Implement (RPI) Workflow", + "file_type": "document", + "source_file": "docs/context-engineering/advanced-context-engineering-coding-agents-dev.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "research-plan-implement (rpi) workflow", + "id": "advanced-context-engineering_rpi_workflow" + }, + { + "label": "1M Context Window", + "file_type": "document", + "source_file": "docs/context-engineering/context-stops-being-scarce-paddo.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "1m context window", + "id": "context-stops-being-scarce_1m_context_window" + }, + { + "label": "Compaction Problem", + "file_type": "document", + "source_file": "docs/context-engineering/context-stops-being-scarce-paddo.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "compaction problem", + "id": "context-stops-being-scarce_compaction_problem" + }, + { + "label": "Flat Context Pricing", + "file_type": "document", + "source_file": "docs/context-engineering/context-stops-being-scarce-paddo.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "flat context pricing", + "id": "context-stops-being-scarce_flat_pricing" + }, + { + "label": "Progressive Disclosure", + "file_type": "document", + "source_file": "docs/context-engineering/progressive-disclosure-ai-agents.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "progressive disclosure", + "id": "progressive-disclosure_progressive_disclosure" + }, + { + "label": "Context Rot", + "file_type": "document", + "source_file": "docs/context-engineering/progressive-disclosure-ai-agents.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "context rot", + "id": "progressive-disclosure_context_rot" + }, + { + "label": "Inverted U Failure Pattern", + "file_type": "document", + "source_file": "docs/context-engineering/progressive-disclosure-ai-agents.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "inverted u failure pattern", + "id": "progressive-disclosure_inverted_u_pattern" + }, + { + "label": "Index-First Loading Pattern", + "file_type": "document", + "source_file": "docs/context-engineering/progressive-disclosure-ai-agents.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "index-first loading pattern", + "id": "progressive-disclosure_index_first_loading" + }, + { + "label": "Scout Pattern", + "file_type": "document", + "source_file": "docs/context-engineering/progressive-disclosure-ai-agents.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "scout pattern", + "id": "progressive-disclosure_scout_pattern" + }, + { + "label": "Dead Context", + "file_type": "document", + "source_file": "docs/context-engineering/shedding-dead-context-ryan-spletzer.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "dead context", + "id": "shedding-dead-context_dead_context" + }, + { + "label": "Context Window as RAM Analogy", + "file_type": "document", + "source_file": "docs/context-engineering/shedding-dead-context-ryan-spletzer.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "context window as ram analogy", + "id": "shedding-dead-context_ram_analogy" + }, + { + "label": "Memory Taxonomy (Scoped & Typed)", + "file_type": "document", + "source_file": "docs/context-engineering/context-engineering-commercial-agents-jeremy-daly.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "memory taxonomy (scoped & typed)", + "id": "commercial-agents_memory_taxonomy" + }, + { + "label": "Truth vs. Acceleration Layers", + "file_type": "document", + "source_file": "docs/context-engineering/context-engineering-commercial-agents-jeremy-daly.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "truth vs. acceleration layers", + "id": "commercial-agents_truth_vs_acceleration" + }, + { + "label": "Context Engine Loop", + "file_type": "document", + "source_file": "docs/context-engineering/context-engineering-commercial-agents-jeremy-daly.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "context engine loop", + "id": "commercial-agents_context_engine_loop" + }, + { + "label": "500 Instruction Ceiling", + "file_type": "document", + "source_file": "docs/context-engineering/agents-md-is-a-liability-paddo.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 8, + "norm_label": "500 instruction ceiling", + "id": "agents-md-liability_500_instruction_ceiling" + }, + { + "label": "IFScale Benchmark", + "file_type": "document", + "source_file": "docs/context-engineering/agents-md-is-a-liability-paddo.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 8, + "norm_label": "ifscale benchmark", + "id": "agents-md-liability_ifscale_benchmark" + }, + { + "label": "Attention Dilution", + "file_type": "document", + "source_file": "docs/context-engineering/agents-md-is-a-liability-paddo.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 8, + "norm_label": "attention dilution", + "id": "agents-md-liability_attention_dilution" + }, + { + "label": "Attention Sinks", + "file_type": "document", + "source_file": "docs/context-engineering/agents-md-is-a-liability-paddo.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 8, + "norm_label": "attention sinks", + "id": "agents-md-liability_attention_sinks" + }, + { + "label": "Prompt Repetition Hack", + "file_type": "document", + "source_file": "docs/context-engineering/agents-md-is-a-liability-paddo.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 8, + "norm_label": "prompt repetition hack", + "id": "agents-md-liability_prompt_repetition" + }, + { + "label": "Pi Coding Agent Extension", + "file_type": "document", + "source_file": "docs/context-engineering/pi-context-zone-github.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "pi coding agent extension", + "id": "pi-context-zone_pi_coding_agent" + }, + { + "label": "Smart/Warm/Dumb Zone Model", + "file_type": "document", + "source_file": "docs/context-engineering/pi-context-zone-github.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "smart/warm/dumb zone model", + "id": "pi-context-zone_smart_warm_dumb_zones" + }, + { + "label": "Model Context Protocol (MCP)", + "file_type": "document", + "source_file": "docs/mcp/mcp-vs-a2a-dzone.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 15, + "norm_label": "model context protocol (mcp)", + "id": "mcp-vs-a2a_mcp" + }, + { + "label": "Agent-to-Agent Protocol (A2A)", + "file_type": "document", + "source_file": "docs/mcp/mcp-vs-a2a-dzone.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 15, + "norm_label": "agent-to-agent protocol (a2a)", + "id": "mcp-vs-a2a_a2a" + }, + { + "label": "MRTR Stateless Transport", + "file_type": "document", + "source_file": "docs/mcp/mcp-http2-http3-reddit.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 15, + "norm_label": "mrtr stateless transport", + "id": "mcp-transport_mrtr_stateless" + }, + { + "label": "MCP Server Cards", + "file_type": "document", + "source_file": "docs/mcp/mcp-http2-http3-reddit.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 15, + "norm_label": "mcp server cards", + "id": "mcp-transport_server_cards" + }, + { + "label": "MCP TypeScript SDK", + "file_type": "document", + "source_file": "docs/mcp/mcp-typescript-sdk-github.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 15, + "norm_label": "mcp typescript sdk", + "id": "mcp-typescript-sdk_typescript_sdk" + }, + { + "label": "AWS MCP Adoption", + "file_type": "document", + "source_file": "docs/mcp/long-live-mcp-aws.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 15, + "norm_label": "aws mcp adoption", + "id": "long-live-mcp_aws_adoption" + }, + { + "label": "MCP Reference Servers", + "file_type": "document", + "source_file": "docs/mcp/modelcontextprotocol-servers-github.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 15, + "norm_label": "mcp reference servers", + "id": "mcp-servers_reference_servers" + }, + { + "label": "Agent Cards", + "file_type": "document", + "source_file": "docs/mcp/a2a-protocol-huggingface.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 15, + "norm_label": "agent cards", + "id": "a2a-protocol_agent_cards" + }, + { + "label": "MCP Ecosystem Projects", + "file_type": "document", + "source_file": "docs/mcp/anthropic-mcp-github-topics.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 15, + "norm_label": "mcp ecosystem projects", + "id": "anthropic-mcp-topics_mcp_ecosystem" + }, + { + "label": "Skill Issue Blog Post", + "file_type": "document", + "source_file": "docs/harness-engineering/skill-issue-harness-engineering-for-coding-agents.md", + "source_location": null, + "source_url": "https://www.humanlayer.dev/blog/skill-issue-harness-engineering-for-coding-agents", + "captured_at": "2026-04-22T03:06:43.748119+00:00", + "author": null, + "contributor": "unknown", + "community": 2, + "norm_label": "skill issue blog post", + "id": "skill-issue_skill_issue_blog" + }, + { + "label": "Harness Engineering", + "file_type": "document", + "source_file": "docs/harness-engineering/skill-issue-harness-engineering-for-coding-agents.md", + "source_location": null, + "source_url": "https://www.humanlayer.dev/blog/skill-issue-harness-engineering-for-coding-agents", + "captured_at": "2026-04-22T03:06:43.748119+00:00", + "author": null, + "contributor": "unknown", + "community": 2, + "norm_label": "harness engineering", + "id": "skill-issue_harness_engineering" + }, + { + "label": "Dex Horthy", + "file_type": "document", + "source_file": "docs/harness-engineering/skill-issue-harness-engineering-for-coding-agents.md", + "source_location": null, + "source_url": "https://www.humanlayer.dev/blog/skill-issue-harness-engineering-for-coding-agents", + "captured_at": "2026-04-22T03:06:43.748119+00:00", + "author": null, + "contributor": "unknown", + "community": 2, + "norm_label": "dex horthy", + "id": "skill-issue_dex_horthy" + }, + { + "label": "Harness as Operating System", + "file_type": "document", + "source_file": "docs/harness-engineering/harnessengineering-building-the-operating-system-for-autonomous-agents.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "harness as operating system", + "id": "harness-os_harness_as_os" + }, + { + "label": "Five Pillars (Harness)", + "file_type": "document", + "source_file": "docs/harness-engineering/harness-engineering.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 8, + "norm_label": "five pillars (harness)", + "id": "harness-engineering-definitive_five_pillars" + }, + { + "label": "Programmatic Tool Calling", + "file_type": "document", + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 5, + "norm_label": "programmatic tool calling", + "id": "readme_programmatic_tool_calling" + }, + { + "label": "Tool Search", + "file_type": "document", + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 5, + "norm_label": "tool search", + "id": "readme_tool_search" + }, + { + "label": "Context Bloat Problem", + "file_type": "document", + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 5, + "norm_label": "context bloat problem", + "id": "readme_context_bloat_problem" + }, + { + "label": "Code Execution Sandbox", + "file_type": "document", + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 5, + "norm_label": "code execution sandbox", + "id": "readme_code_execution_sandbox" + }, + { + "label": "Anthropic Claude tool_search", + "file_type": "document", + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 5, + "norm_label": "anthropic claude tool_search", + "id": "readme_anthropic_claude_tool_search" + }, + { + "label": "Vercel AI SDK withProgrammaticCalling", + "file_type": "document", + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 5, + "norm_label": "vercel ai sdk withprogrammaticcalling", + "id": "readme_vercel_ai_sdk_withprogrammaticcalling" + }, + { + "label": "Spring AI ToolSearchToolCallAdvisor", + "file_type": "document", + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 5, + "norm_label": "spring ai toolsearchtoolcalladvisor", + "id": "readme_spring_ai_toolsearchtoolcalladvisor" + }, + { + "label": ".NET MCP Three-Tool Pattern", + "file_type": "document", + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 5, + "norm_label": ".net mcp three-tool pattern", + "id": "readme_dotnet_mcp_three_tool_pattern" + }, + { + "label": "Why Tool Search Helps at Enterprise Scale", + "file_type": "document", + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 5, + "norm_label": "why tool search helps at enterprise scale", + "id": "readme_context_bloat_rationale" + }, + { + "label": "Lost in the Middle, and In-Between", + "file_type": "paper", + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "source_url": "https://arxiv.org/abs/2412.10079", + "captured_at": null, + "author": "George Arthur Baker; Ankush Raut; Sagi Shaier; Lawrence E Hunter; Katharina von der Wense", + "contributor": null, + "community": 5, + "norm_label": "lost in the middle, and in-between", + "id": "lost_in_the_middle_and_in_between_paper" + }, + { + "label": "Lost in the Middle", + "file_type": "paper", + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "source_url": "https://arxiv.org/abs/2412.10079", + "captured_at": null, + "author": "George Arthur Baker; Ankush Raut; Sagi Shaier; Lawrence E Hunter; Katharina von der Wense", + "contributor": null, + "community": 5, + "norm_label": "lost in the middle", + "id": "lost_in_the_middle_and_in_between_lost_in_the_middle" + }, + { + "label": "In-Between Effect", + "file_type": "paper", + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "source_url": "https://arxiv.org/abs/2412.10079", + "captured_at": null, + "author": "George Arthur Baker; Ankush Raut; Sagi Shaier; Lawrence E Hunter; Katharina von der Wense", + "contributor": null, + "community": 5, + "norm_label": "in-between effect", + "id": "lost_in_the_middle_and_in_between_in_between_effect" + }, + { + "label": "Multi-Hop Question Answering", + "file_type": "paper", + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "source_url": "https://arxiv.org/abs/2412.10079", + "captured_at": null, + "author": "George Arthur Baker; Ankush Raut; Sagi Shaier; Lawrence E Hunter; Katharina von der Wense", + "contributor": null, + "community": 5, + "norm_label": "multi-hop question answering", + "id": "lost_in_the_middle_and_in_between_multi_hop_qa" + }, + { + "label": "Chain-of-Thought Prompting", + "file_type": "paper", + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "source_url": "https://arxiv.org/abs/2412.10079", + "captured_at": null, + "author": "George Arthur Baker; Ankush Raut; Sagi Shaier; Lawrence E Hunter; Katharina von der Wense", + "contributor": null, + "community": 5, + "norm_label": "chain-of-thought prompting", + "id": "lost_in_the_middle_and_in_between_chain_of_thought_prompting" + }, + { + "label": "Knowledge Graph Triple Extraction", + "file_type": "paper", + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "source_url": "https://arxiv.org/abs/2412.10079", + "captured_at": null, + "author": "George Arthur Baker; Ankush Raut; Sagi Shaier; Lawrence E Hunter; Katharina von der Wense", + "contributor": null, + "community": 5, + "norm_label": "knowledge graph triple extraction", + "id": "lost_in_the_middle_and_in_between_knowledge_graph_triple_extraction" + }, + { + "label": "Document Summarization", + "file_type": "paper", + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "source_url": "https://arxiv.org/abs/2412.10079", + "captured_at": null, + "author": "George Arthur Baker; Ankush Raut; Sagi Shaier; Lawrence E Hunter; Katharina von der Wense", + "contributor": null, + "community": 5, + "norm_label": "document summarization", + "id": "lost_in_the_middle_and_in_between_document_summarization" + }, + { + "label": "Why Re-ranking Becomes Impractical", + "file_type": "paper", + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "source_url": "https://arxiv.org/abs/2412.10079", + "captured_at": null, + "author": "George Arthur Baker; Ankush Raut; Sagi Shaier; Lawrence E Hunter; Katharina von der Wense", + "contributor": null, + "community": 5, + "norm_label": "why re-ranking becomes impractical", + "id": "lost_in_the_middle_and_in_between_combinatorial_scaling_rationale" + }, + { + "label": "Why Context Reduction Remains Fragile", + "file_type": "paper", + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "source_url": "https://arxiv.org/abs/2412.10079", + "captured_at": null, + "author": "George Arthur Baker; Ankush Raut; Sagi Shaier; Lawrence E Hunter; Katharina von der Wense", + "contributor": null, + "community": 5, + "norm_label": "why context reduction remains fragile", + "id": "lost_in_the_middle_and_in_between_fragile_reasoning_chains_rationale" + }, + { + "label": "Liu et al. 2024", + "file_type": "paper", + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "source_url": "https://arxiv.org/abs/2412.10079", + "captured_at": null, + "author": "George Arthur Baker; Ankush Raut; Sagi Shaier; Lawrence E Hunter; Katharina von der Wense", + "contributor": null, + "community": 5, + "norm_label": "liu et al. 2024", + "id": "lost_in_the_middle_and_in_between_liu_etal_2024" + }, + { + "label": "Levy et al. 2024", + "file_type": "paper", + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "source_url": "https://arxiv.org/abs/2412.10079", + "captured_at": null, + "author": "George Arthur Baker; Ankush Raut; Sagi Shaier; Lawrence E Hunter; Katharina von der Wense", + "contributor": null, + "community": 5, + "norm_label": "levy et al. 2024", + "id": "lost_in_the_middle_and_in_between_levy_etal_2024" + }, + { + "label": "Seven Principles of Persuasion", + "file_type": "paper", + "source_file": "docs/general-llm/Call_Me_A_Jerk_Persuading_AI_to_Comply_with_Objectionable_Requests.pdf", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "seven principles of persuasion", + "id": "call-me-jerk_persuasion_principles" + }, + { + "label": "Parahuman AI Behavior", + "file_type": "paper", + "source_file": "docs/general-llm/Call_Me_A_Jerk_Persuading_AI_to_Comply_with_Objectionable_Requests.pdf", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 2, + "norm_label": "parahuman ai behavior", + "id": "call-me-jerk_parahuman_behavior" + }, + { + "label": "AGENTBENCH", + "file_type": "paper", + "source_file": "docs/general-llm/Evaluating-AGENTS-paper.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 8, + "norm_label": "agentbench", + "id": "evaluating-agents-paper_agentbench" + }, + { + "label": "ETH Zurich AGENTS.md Study", + "file_type": "paper", + "source_file": "docs/general-llm/Evaluating-AGENTS-paper.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 8, + "norm_label": "eth zurich agents.md study", + "id": "evaluating-agents-paper_eth_study" + }, + { + "label": "Practical Agent Workflows and Patterns", + "file_type": "document", + "source_file": "docs/general-llm/research-agent-workflows-and-patterns.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "practical agent workflows and patterns", + "id": "research_agent_workflows_and_patterns_doc" + }, + { + "label": "LLM Context Engineering: Comprehensive Research Synthesis", + "file_type": "document", + "source_file": "docs/general-llm/research-context-engineering-comprehensive.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 9, + "norm_label": "llm context engineering: comprehensive research synthesis", + "id": "research_context_engineering_comprehensive_doc" + }, + { + "label": "Prompt Engineering: Complete Technical Guide to Techniques from 2022 to 2026", + "file_type": "document", + "source_file": "docs/general-llm/prompt-engineering-guide.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "prompt engineering: complete technical guide to techniques from 2022 to 2026", + "id": "prompt_engineering_guide_doc" + }, + { + "label": "A Complete Guide To AGENTS.md", + "file_type": "document", + "source_file": "docs/general-llm/a-guide-to-agents.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 6, + "norm_label": "a complete guide to agents.md", + "id": "a_guide_to_agents_doc" + }, + { + "label": "Persuasion Principles for Agent Skill Engineering", + "file_type": "document", + "source_file": "docs/general-llm/persuasion-principles.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 17, + "norm_label": "persuasion principles for agent skill engineering", + "id": "persuasion_principles_doc" + }, + { + "label": "LLM Whitespace & Formatting in Context Windows", + "file_type": "document", + "source_file": "docs/general-llm/research-whitespace-and-formatting.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 9, + "norm_label": "llm whitespace & formatting in context windows", + "id": "research_whitespace_and_formatting_doc" + }, + { + "label": "LLM Context Rot and Context Window Management", + "file_type": "document", + "source_file": "docs/general-llm/research-context-rot-and-management.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 6, + "norm_label": "llm context rot and context window management", + "id": "research_context_rot_and_management_doc" + }, + { + "label": "LLM Multilingual Performance & Language Overhead", + "file_type": "document", + "source_file": "docs/general-llm/research-multilingual-performance.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 9, + "norm_label": "llm multilingual performance & language overhead", + "id": "research_multilingual_performance_doc" + }, + { + "label": "Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?", + "file_type": "paper", + "source_file": "docs/general-llm/Evaluating-AGENTS-paper.pdf", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 6, + "norm_label": "evaluating agents.md: are repository-level context files helpful for coding agents?", + "id": "evaluating_agents_paper_doc" + }, + { + "label": "Specification", + "file_type": "document", + "source_file": "docs/shared/skills-standard/agentskills-specification.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "specification", + "id": "agentskills_specification_doc" + }, + { + "label": "Evaluating skill output quality", + "file_type": "document", + "source_file": "docs/shared/skills-standard/agentskills-evaluating-skills.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "evaluating skill output quality", + "id": "agentskills_evaluating_skills_doc" + }, + { + "label": "Best practices for skill creators", + "file_type": "document", + "source_file": "docs/shared/skills-standard/agentskills-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "best practices for skill creators", + "id": "agentskills_best_practices_doc" + }, + { + "label": "Agent Skills Standard \u2014 Official Documentation", + "file_type": "document", + "source_file": "docs/shared/skills-standard/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "agent skills standard \u2014 official documentation", + "id": "agentskills_readme_doc" + }, + { + "label": "Optimizing skill descriptions", + "file_type": "document", + "source_file": "docs/shared/skills-standard/agentskills-optimizing-descriptions.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "optimizing skill descriptions", + "id": "agentskills_optimizing_descriptions_doc" + }, + { + "label": "What are skills?", + "file_type": "document", + "source_file": "docs/shared/skills-standard/agentskills-what-are-skills.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 6, + "norm_label": "what are skills?", + "id": "agentskills_what_are_skills_doc" + }, + { + "label": "Using scripts in skills", + "file_type": "document", + "source_file": "docs/shared/skills-standard/agentskills-using-scripts.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "using scripts in skills", + "id": "agentskills_using_scripts_doc" + }, + { + "label": "A2aprotocol \u2014 a Hugging Face Space by a2aprotocol", + "file_type": "document", + "source_file": "docs/agent-protocols/a2a-protocol-huggingface-space.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "a2aprotocol \u2014 a hugging face space by a2aprotocol", + "id": "a2a_protocol_huggingface_space_doc" + }, + { + "label": "Anthropic Engineer Forecasts 2026 as AI Agent 'Full Connectivity' Year, Driven by MCP", + "file_type": "document", + "source_file": "docs/agent-protocols/anthropic-engineer-2026-forecast-full-connectivity-mcp.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "anthropic engineer forecasts 2026 as ai agent 'full connectivity' year, driven by mcp", + "id": "anthropic_engineer_2026_forecast_full_connectivity_mcp_doc" + }, + { + "label": "AI Agent Protocols 2026: The Complete Guide to Standardizing AI Communication", + "file_type": "document", + "source_file": "docs/agent-protocols/ai-agent-protocols-2026-guide.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "ai agent protocols 2026: the complete guide to standardizing ai communication", + "id": "ai_agent_protocols_2026_guide_doc" + }, + { + "label": "Architectural Paradigms of Advanced Agentic Systems", + "file_type": "document", + "source_file": "docs/agent-protocols/architectural-paradigms-advanced-agentic-systems.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 9, + "norm_label": "architectural paradigms of advanced agentic systems", + "id": "architectural_paradigms_advanced_agentic_systems_doc" + }, + { + "label": "Towards Fluid Human-Agent Collaboration", + "file_type": "document", + "source_file": "docs/agent-protocols/fluid-human-agent-collaboration-pmc.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 18, + "norm_label": "towards fluid human-agent collaboration", + "id": "fluid_human_agent_collaboration_pmc_doc" + }, + { + "label": "Advancing Agentic AI through Communication Protocols", + "file_type": "document", + "source_file": "docs/agent-protocols/advancing-agentic-ai-communication-protocols.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "advancing agentic ai through communication protocols", + "id": "advancing_agentic_ai_communication_protocols_doc" + }, + { + "label": "Spec-Driven Development \u2014 Research Overview", + "file_type": "document", + "source_file": "docs/spec-driven-development/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 16, + "norm_label": "spec-driven development \u2014 research overview", + "id": "spec_driven_development_readme_doc" + }, + { + "label": "Spec-Driven Development: Exploring Three SDD Tools", + "file_type": "document", + "source_file": "docs/spec-driven-development/spec-driven-development-variant.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 16, + "norm_label": "spec-driven development: exploring three sdd tools", + "id": "spec_driven_development_variant_doc" + }, + { + "label": "Spec-Driven Development: From Code to Contract in the Age of AI Coding Assistants", + "file_type": "document", + "source_file": "docs/spec-driven-development/spec-driven-development-arxiv.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 16, + "norm_label": "spec-driven development: from code to contract in the age of ai coding assistants", + "id": "spec_driven_development_arxiv_doc" + }, + { + "label": "Context Engineering", + "file_type": "document", + "source_file": "docs/general-llm/research-context-engineering-comprehensive.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 9, + "norm_label": "context engineering", + "id": "research_context_engineering_comprehensive_context_engineering" + }, + { + "label": "Smallest Set of High-Signal Tokens", + "file_type": "document", + "source_file": "docs/general-llm/research-context-engineering-comprehensive.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 9, + "norm_label": "smallest set of high-signal tokens", + "id": "research_context_engineering_comprehensive_high_signal_tokens" + }, + { + "label": "Context Rot", + "file_type": "document", + "source_file": "docs/general-llm/research-context-rot-and-management.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 6, + "norm_label": "context rot", + "id": "research_context_rot_and_management_context_rot" + }, + { + "label": "Attention Budget", + "file_type": "document", + "source_file": "docs/general-llm/research-context-rot-and-management.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 6, + "norm_label": "attention budget", + "id": "research_context_rot_and_management_attention_budget" + }, + { + "label": "Lost in the Middle", + "file_type": "document", + "source_file": "docs/general-llm/research-context-rot-and-management.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 6, + "norm_label": "lost in the middle", + "id": "research_context_rot_and_management_lost_in_the_middle" + }, + { + "label": "Progressive Disclosure", + "file_type": "document", + "source_file": "docs/general-llm/research-context-rot-and-management.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 6, + "norm_label": "progressive disclosure", + "id": "research_context_rot_and_management_progressive_disclosure" + }, + { + "label": "Effective Context Engineering for AI Agents", + "file_type": "document", + "source_file": "docs/general-llm/research-context-rot-and-management.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 6, + "norm_label": "effective context engineering for ai agents", + "id": "research_context_rot_and_management_effective_context_engineering" + }, + { + "label": "Structural Formatting", + "file_type": "document", + "source_file": "docs/general-llm/research-whitespace-and-formatting.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 9, + "norm_label": "structural formatting", + "id": "research_whitespace_and_formatting_structural_formatting" + }, + { + "label": "PathPiece", + "file_type": "document", + "source_file": "docs/general-llm/research-whitespace-and-formatting.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 9, + "norm_label": "pathpiece", + "id": "research_whitespace_and_formatting_pathpiece" + }, + { + "label": "Tokenization Tax", + "file_type": "document", + "source_file": "docs/general-llm/research-multilingual-performance.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 9, + "norm_label": "tokenization tax", + "id": "research_multilingual_performance_tokenization_tax" + }, + { + "label": "Self-Translate Strategy", + "file_type": "document", + "source_file": "docs/general-llm/research-multilingual-performance.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 9, + "norm_label": "self-translate strategy", + "id": "research_multilingual_performance_self_translate_strategy" + }, + { + "label": "Portuguese-Specific LLMs", + "file_type": "document", + "source_file": "docs/general-llm/research-multilingual-performance.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 9, + "norm_label": "portuguese-specific llms", + "id": "research_multilingual_performance_portuguese_models" + }, + { + "label": "Tokenization Unfairness Between Languages", + "file_type": "document", + "source_file": "docs/general-llm/research-multilingual-performance.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 9, + "norm_label": "tokenization unfairness between languages", + "id": "research_multilingual_performance_petrov_tokenization_unfairness" + }, + { + "label": "Explore \u2192 Plan \u2192 Code \u2192 Verify", + "file_type": "document", + "source_file": "docs/general-llm/research-agent-workflows-and-patterns.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "explore \u2192 plan \u2192 code \u2192 verify", + "id": "research_agent_workflows_and_patterns_explore_plan_code_verify" + }, + { + "label": "Spec \u2192 Plan \u2192 Execute", + "file_type": "document", + "source_file": "docs/general-llm/research-agent-workflows-and-patterns.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "spec \u2192 plan \u2192 execute", + "id": "research_agent_workflows_and_patterns_spec_plan_execute" + }, + { + "label": "Two-Agent Architecture", + "file_type": "document", + "source_file": "docs/general-llm/research-agent-workflows-and-patterns.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "two-agent architecture", + "id": "research_agent_workflows_and_patterns_two_agent_architecture" + }, + { + "label": "ReAct Pattern", + "file_type": "document", + "source_file": "docs/general-llm/prompt-engineering-guide.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "react pattern", + "id": "prompt_engineering_guide_react_pattern" + }, + { + "label": "Two-Stage Structured Output", + "file_type": "document", + "source_file": "docs/general-llm/prompt-engineering-guide.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "two-stage structured output", + "id": "prompt_engineering_guide_structured_output_two_stage" + }, + { + "label": "The Prompt Report", + "file_type": "document", + "source_file": "docs/general-llm/prompt-engineering-guide.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 9, + "norm_label": "the prompt report", + "id": "prompt_engineering_guide_prompt_report" + }, + { + "label": "AGENTS.md", + "file_type": "document", + "source_file": "docs/general-llm/a-guide-to-agents.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 6, + "norm_label": "agents.md", + "id": "a_guide_to_agents_agents_md" + }, + { + "label": "Instruction Budget", + "file_type": "document", + "source_file": "docs/general-llm/a-guide-to-agents.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 6, + "norm_label": "instruction budget", + "id": "a_guide_to_agents_instruction_budget" + }, + { + "label": "Stale Documentation Poisons Context", + "file_type": "document", + "source_file": "docs/general-llm/a-guide-to-agents.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 6, + "norm_label": "stale documentation poisons context", + "id": "a_guide_to_agents_stale_documentation_poisoning" + }, + { + "label": "Path-Scoped Rules", + "file_type": "document", + "source_file": "docs/general-llm/a-guide-to-agents.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 6, + "norm_label": "path-scoped rules", + "id": "a_guide_to_agents_path_scoped_rules" + }, + { + "label": "Parahuman Effect", + "file_type": "document", + "source_file": "docs/general-llm/persuasion-principles.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 17, + "norm_label": "parahuman effect", + "id": "persuasion_principles_parahuman_effect" + }, + { + "label": "Commitment Principle", + "file_type": "document", + "source_file": "docs/general-llm/persuasion-principles.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 17, + "norm_label": "commitment principle", + "id": "persuasion_principles_commitment_principle" + }, + { + "label": "Call Me A Jerk", + "file_type": "document", + "source_file": "docs/general-llm/persuasion-principles.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 17, + "norm_label": "call me a jerk", + "id": "persuasion_principles_call_me_a_jerk_paper" + }, + { + "label": "Agent Skills Standard", + "file_type": "document", + "source_file": "docs/shared/skills-standard/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "agent skills standard", + "id": "agentskills_readme_agent_skills_standard" + }, + { + "label": "Skill Manifest Metadata", + "file_type": "document", + "source_file": "docs/shared/skills-standard/agentskills-specification.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "skill manifest metadata", + "id": "agentskills_specification_skill_manifest" + }, + { + "label": "Skill Directory Structure", + "file_type": "document", + "source_file": "docs/shared/skills-standard/agentskills-what-are-skills.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 6, + "norm_label": "skill directory structure", + "id": "agentskills_what_are_skills_skill_directory" + }, + { + "label": "Progressive Skill Discovery", + "file_type": "document", + "source_file": "docs/shared/skills-standard/agentskills-what-are-skills.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 6, + "norm_label": "progressive skill discovery", + "id": "agentskills_what_are_skills_progressive_discovery" + }, + { + "label": "Skill Triggering via Description", + "file_type": "document", + "source_file": "docs/shared/skills-standard/agentskills-optimizing-descriptions.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "skill triggering via description", + "id": "agentskills_optimizing_descriptions_skill_triggering" + }, + { + "label": "Eval-Driven Iteration", + "file_type": "document", + "source_file": "docs/shared/skills-standard/agentskills-evaluating-skills.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "eval-driven iteration", + "id": "agentskills_evaluating_skills_eval_driven_iteration" + }, + { + "label": "Self-Contained Skill Scripts", + "file_type": "document", + "source_file": "docs/shared/skills-standard/agentskills-using-scripts.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "self-contained skill scripts", + "id": "agentskills_using_scripts_self_contained_scripts" + }, + { + "label": "Gotchas Pattern", + "file_type": "document", + "source_file": "docs/shared/skills-standard/agentskills-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "gotchas pattern", + "id": "agentskills_best_practices_gotchas_pattern" + }, + { + "label": "Validation Loop", + "file_type": "document", + "source_file": "docs/shared/skills-standard/agentskills-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "validation loop", + "id": "agentskills_best_practices_validation_loop" + }, + { + "label": "A2A Protocol", + "file_type": "document", + "source_file": "docs/agent-protocols/a2a-protocol-huggingface-space.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "a2a protocol", + "id": "a2a_protocol_huggingface_space_a2a_protocol" + }, + { + "label": "Agent Card", + "file_type": "document", + "source_file": "docs/agent-protocols/a2a-protocol-huggingface-space.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "agent card", + "id": "a2a_protocol_huggingface_space_agent_card" + }, + { + "label": "Full Connectivity Stack", + "file_type": "document", + "source_file": "docs/agent-protocols/anthropic-engineer-2026-forecast-full-connectivity-mcp.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "full connectivity stack", + "id": "anthropic_engineer_2026_forecast_full_connectivity_mcp_full_connectivity_stack" + }, + { + "label": "Programmatic Tool Composition", + "file_type": "document", + "source_file": "docs/agent-protocols/anthropic-engineer-2026-forecast-full-connectivity-mcp.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "programmatic tool composition", + "id": "anthropic_engineer_2026_forecast_full_connectivity_mcp_programmatic_tool_composition" + }, + { + "label": "MCP Protocol", + "file_type": "document", + "source_file": "docs/agent-protocols/ai-agent-protocols-2026-guide.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "mcp protocol", + "id": "ai_agent_protocols_2026_guide_mcp_protocol" + }, + { + "label": "Layered Multi-Agent Stack", + "file_type": "document", + "source_file": "docs/agent-protocols/ai-agent-protocols-2026-guide.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "layered multi-agent stack", + "id": "ai_agent_protocols_2026_guide_layered_stack" + }, + { + "label": "Agentic Context Engineering (ACE)", + "file_type": "document", + "source_file": "docs/agent-protocols/architectural-paradigms-advanced-agentic-systems.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 9, + "norm_label": "agentic context engineering (ace)", + "id": "architectural_paradigms_advanced_agentic_systems_ace_framework" + }, + { + "label": "Fluid Collaboration", + "file_type": "document", + "source_file": "docs/agent-protocols/fluid-human-agent-collaboration-pmc.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 18, + "norm_label": "fluid collaboration", + "id": "fluid_human_agent_collaboration_pmc_fluid_collaboration" + }, + { + "label": "Dynamic Mentalizing", + "file_type": "document", + "source_file": "docs/agent-protocols/fluid-human-agent-collaboration-pmc.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 18, + "norm_label": "dynamic mentalizing", + "id": "fluid_human_agent_collaboration_pmc_dynamic_mentalizing" + }, + { + "label": "Agent Network Protocol (ANP)", + "file_type": "document", + "source_file": "docs/agent-protocols/advancing-agentic-ai-communication-protocols.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "agent network protocol (anp)", + "id": "advancing_agentic_ai_communication_protocols_anp_protocol" + }, + { + "label": "Spec-Driven Development", + "file_type": "document", + "source_file": "docs/spec-driven-development/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 16, + "norm_label": "spec-driven development", + "id": "spec_driven_development_readme_spec_driven_development" + }, + { + "label": "Spec-First", + "file_type": "document", + "source_file": "docs/spec-driven-development/spec-driven-development-arxiv.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 16, + "norm_label": "spec-first", + "id": "spec_driven_development_arxiv_spec_first" + }, + { + "label": "Spec-Anchored", + "file_type": "document", + "source_file": "docs/spec-driven-development/spec-driven-development-arxiv.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 16, + "norm_label": "spec-anchored", + "id": "spec_driven_development_arxiv_spec_anchored" + }, + { + "label": "Spec-as-Source", + "file_type": "document", + "source_file": "docs/spec-driven-development/spec-driven-development-arxiv.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 16, + "norm_label": "spec-as-source", + "id": "spec_driven_development_arxiv_spec_as_source" + }, + { + "label": "Specify \u2192 Plan \u2192 Implement \u2192 Validate", + "file_type": "document", + "source_file": "docs/spec-driven-development/spec-driven-development-arxiv.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 3, + "norm_label": "specify \u2192 plan \u2192 implement \u2192 validate", + "id": "spec_driven_development_arxiv_sdd_workflow" + }, + { + "label": "Kiro", + "file_type": "document", + "source_file": "docs/spec-driven-development/spec-driven-development-variant.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 16, + "norm_label": "kiro", + "id": "spec_driven_development_variant_kiro" + }, + { + "label": "Spec Kit", + "file_type": "document", + "source_file": "docs/spec-driven-development/spec-driven-development-variant.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 16, + "norm_label": "spec kit", + "id": "spec_driven_development_variant_spec_kit" + }, + { + "label": "Tessl Framework", + "file_type": "document", + "source_file": "docs/spec-driven-development/spec-driven-development-variant.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 16, + "norm_label": "tessl framework", + "id": "spec_driven_development_variant_tessl" + }, + { + "label": "AGENTBENCH", + "file_type": "paper", + "source_file": "docs/general-llm/Evaluating-AGENTS-paper.pdf", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 6, + "norm_label": "agentbench", + "id": "evaluating_agents_paper_agentbench" + }, + { + "label": "Repository Context Files", + "file_type": "paper", + "source_file": "docs/general-llm/Evaluating-AGENTS-paper.pdf", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 6, + "norm_label": "repository context files", + "id": "evaluating_agents_paper_repository_context_files" + }, + { + "label": "Minimal Context Files", + "file_type": "paper", + "source_file": "docs/general-llm/Evaluating-AGENTS-paper.pdf", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 6, + "norm_label": "minimal context files", + "id": "evaluating_agents_paper_minimal_context_files" + }, + { + "label": "Prompts as Skills", + "file_type": "document", + "source_file": "docs/mcp/skills-over-mcp-meeting-notes-2248.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "prompts as skills", + "id": "skills_over_mcp_meeting_notes_2248_prompts_as_skills" + }, + { + "label": "Agents Ignore Skills", + "file_type": "document", + "source_file": "docs/mcp/skills-over-mcp-meeting-notes-2248.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "agents ignore skills", + "id": "skills_over_mcp_meeting_notes_2248_agents_ignore_skills" + }, + { + "label": "Skills-as-Instructors", + "file_type": "document", + "source_file": "docs/mcp/skills-over-mcp-meeting-notes-2248.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "skills-as-instructors", + "id": "skills_over_mcp_meeting_notes_2248_skills_as_instructors" + }, + { + "label": "Skills-as-Helpers", + "file_type": "document", + "source_file": "docs/mcp/skills-over-mcp-meeting-notes-2248.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "skills-as-helpers", + "id": "skills_over_mcp_meeting_notes_2248_skills_as_helpers" + }, + { + "label": "Vendor-Neutral Skill Schema", + "file_type": "document", + "source_file": "docs/mcp/skills-over-mcp-meeting-notes-2248.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "vendor-neutral skill schema", + "id": "skills_over_mcp_meeting_notes_2248_vendor_neutral_skill_schema" + }, + { + "label": "Discovery Metadata Rationale", + "file_type": "document", + "source_file": "docs/mcp/skills-over-mcp-meeting-notes-2248.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "discovery metadata rationale", + "id": "skills_over_mcp_meeting_notes_2248_discovery_metadata_rationale" + }, + { + "label": "Well-Known Skill Index", + "file_type": "document", + "source_file": "docs/mcp/skills-over-mcp-office-hours-2460.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "well-known skill index", + "id": "skills_over_mcp_office_hours_2460_well_known_skill_index" + }, + { + "label": "Lazy Loading Pattern", + "file_type": "document", + "source_file": "docs/mcp/skills-over-mcp-office-hours-2460.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "lazy loading pattern", + "id": "skills_over_mcp_office_hours_2460_lazy_loading_pattern" + }, + { + "label": "Script-Bearing Skills", + "file_type": "document", + "source_file": "docs/mcp/skills-over-mcp-office-hours-2460.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "script-bearing skills", + "id": "skills_over_mcp_office_hours_2460_script_bearing_skills" + }, + { + "label": "Resource URIs", + "file_type": "document", + "source_file": "docs/mcp/skills-over-mcp-office-hours-2460.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "resource uris", + "id": "skills_over_mcp_office_hours_2460_resource_uris" + }, + { + "label": "Reject Custom Skill URI Scheme", + "file_type": "document", + "source_file": "docs/mcp/skills-over-mcp-office-hours-2460.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "reject custom skill uri scheme", + "id": "skills_over_mcp_office_hours_2460_skill_scheme_rejection_rationale" + }, + { + "label": "JSON-RPC Host Client Server Roles", + "file_type": "document", + "source_file": "docs/mcp/mcp-specification.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "json-rpc host client server roles", + "id": "mcp_specification_json_rpc_roles" + }, + { + "label": "Prompts Primitive", + "file_type": "document", + "source_file": "docs/mcp/mcp-specification.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "prompts primitive", + "id": "mcp_specification_prompts_primitive" + }, + { + "label": "Resources Primitive", + "file_type": "document", + "source_file": "docs/mcp/mcp-specification.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "resources primitive", + "id": "mcp_specification_resources_primitive" + }, + { + "label": "Tools Primitive", + "file_type": "document", + "source_file": "docs/mcp/mcp-specification.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "tools primitive", + "id": "mcp_specification_tools_primitive" + }, + { + "label": "Security Principles", + "file_type": "document", + "source_file": "docs/mcp/mcp-specification.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "security principles", + "id": "mcp_specification_security_principles" + }, + { + "label": "Skills Layer", + "file_type": "document", + "source_file": "docs/mcp/skills-vs-mcp-speakeasy.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "skills layer", + "id": "skills_vs_mcp_speakeasy_skills_layer" + }, + { + "label": "MCP Layer", + "file_type": "document", + "source_file": "docs/mcp/skills-vs-mcp-speakeasy.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "mcp layer", + "id": "skills_vs_mcp_speakeasy_mcp_layer" + }, + { + "label": "Two-Layer Architecture", + "file_type": "document", + "source_file": "docs/mcp/skills-vs-mcp-speakeasy.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "two-layer architecture", + "id": "skills_vs_mcp_speakeasy_two_layer_architecture" + }, + { + "label": "server.prompt Bridge", + "file_type": "document", + "source_file": "docs/mcp/skills-vs-mcp-speakeasy.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "server.prompt bridge", + "id": "skills_vs_mcp_speakeasy_server_prompt_bridge" + }, + { + "label": "Copilot MCP Surfaces", + "file_type": "document", + "source_file": "docs/mcp/about-mcp-github-docs.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "copilot mcp surfaces", + "id": "about_mcp_github_docs_copilot_surfaces" + }, + { + "label": "GitHub MCP Server", + "file_type": "document", + "source_file": "docs/mcp/about-mcp-github-docs.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "github mcp server", + "id": "about_mcp_github_docs_github_mcp_server" + }, + { + "label": "GitHub MCP Toolsets", + "file_type": "document", + "source_file": "docs/mcp/about-mcp-github-docs.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "github mcp toolsets", + "id": "about_mcp_github_docs_toolsets" + }, + { + "label": "Push Protection", + "file_type": "document", + "source_file": "docs/mcp/about-mcp-github-docs.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "push protection", + "id": "about_mcp_github_docs_push_protection" + }, + { + "label": "Progressive Tool Discovery", + "file_type": "document", + "source_file": "docs/mcp/long-live-mcp-aqfer.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "progressive tool discovery", + "id": "long_live_mcp_aqfer_progressive_tool_discovery" + }, + { + "label": "Multi Round-Trip Requests", + "file_type": "document", + "source_file": "docs/mcp/long-live-mcp-aqfer.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 20, + "norm_label": "multi round-trip requests", + "id": "long_live_mcp_aqfer_mrtr" + }, + { + "label": "CIMD", + "file_type": "document", + "source_file": "docs/mcp/long-live-mcp-aqfer.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 19, + "norm_label": "cimd", + "id": "long_live_mcp_aqfer_cimd" + }, + { + "label": "XAA", + "file_type": "document", + "source_file": "docs/mcp/long-live-mcp-aqfer.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 19, + "norm_label": "xaa", + "id": "long_live_mcp_aqfer_xaa" + }, + { + "label": "Lost in the Middle (ACL/TACL)", + "file_type": "paper", + "source_file": "docs/long-context-research/lost-in-the-middle-acl.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 12, + "norm_label": "lost in the middle (acl/tacl)", + "id": "lost_in_the_middle_acl_paper" + }, + { + "label": "Positional Bias", + "file_type": "paper", + "source_file": "docs/long-context-research/lost-in-the-middle-acl.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 12, + "norm_label": "positional bias", + "id": "lost_in_the_middle_acl_positional_bias" + }, + { + "label": "U-Shaped Performance Curve", + "file_type": "paper", + "source_file": "docs/long-context-research/lost-in-the-middle-acl.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 12, + "norm_label": "u-shaped performance curve", + "id": "lost_in_the_middle_acl_u_shaped_curve" + }, + { + "label": "Long-Context Evaluation Protocols", + "file_type": "paper", + "source_file": "docs/long-context-research/lost-in-the-middle-acl.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 12, + "norm_label": "long-context evaluation protocols", + "id": "lost_in_the_middle_acl_eval_protocols" + }, + { + "label": "Lost in the Middle (arXiv Preprint)", + "file_type": "paper", + "source_file": "docs/long-context-research/lost-in-the-middle-arxiv.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 12, + "norm_label": "lost in the middle (arxiv preprint)", + "id": "lost_in_the_middle_arxiv_preprint" + }, + { + "label": "Preprint to TACL Publication Link", + "file_type": "paper", + "source_file": "docs/long-context-research/lost-in-the-middle-arxiv.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 12, + "norm_label": "preprint to tacl publication link", + "id": "lost_in_the_middle_arxiv_publication_link" + }, + { + "label": "Lost in the Middle (ResearchGate Reference)", + "file_type": "paper", + "source_file": "docs/long-context-research/lost-in-the-middle-researchgate.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 12, + "norm_label": "lost in the middle (researchgate reference)", + "id": "lost_in_the_middle_researchgate_reference" + }, + { + "label": "ResearchGate Access Note", + "file_type": "paper", + "source_file": "docs/long-context-research/lost-in-the-middle-researchgate.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 12, + "norm_label": "researchgate access note", + "id": "lost_in_the_middle_researchgate_access_note" + }, + { + "label": "Long Context Paper Collection", + "file_type": "document", + "source_file": "docs/long-context-research/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 12, + "norm_label": "long context paper collection", + "id": "long_context_research_readme_collection" + }, + { + "label": "Multi-Hop Follow-up Paper", + "file_type": "document", + "source_file": "docs/long-context-research/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 12, + "norm_label": "multi-hop follow-up paper", + "id": "long_context_research_readme_multihop_followup" + }, + { + "label": "RAG Edge Placement Guidance", + "file_type": "document", + "source_file": "docs/long-context-research/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 10, + "norm_label": "rag edge placement guidance", + "id": "long_context_research_readme_rag_edge_placement" + }, + { + "label": "Knowledge Graph Extraction Fragility", + "file_type": "document", + "source_file": "docs/long-context-research/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 12, + "norm_label": "knowledge graph extraction fragility", + "id": "long_context_research_readme_kg_extraction_fragility" + }, + { + "label": "Tool Search with OpenSandbox", + "file_type": "document", + "source_file": "docs/tool-calling/mcp-programmatic-tool-calling-opensandbox-dev.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "tool search with opensandbox", + "id": "mcp_programmatic_tool_calling_opensandbox_tool_search" + }, + { + "label": "Code Mode", + "file_type": "document", + "source_file": "docs/tool-calling/mcp-programmatic-tool-calling-opensandbox-dev.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "code mode", + "id": "mcp_programmatic_tool_calling_opensandbox_code_mode" + }, + { + "label": "Sandboxed Execution", + "file_type": "document", + "source_file": "docs/tool-calling/mcp-programmatic-tool-calling-opensandbox-dev.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "sandboxed execution", + "id": "mcp_programmatic_tool_calling_opensandbox_sandboxed_execution" + }, + { + "label": "search get_schema execute Flow", + "file_type": "document", + "source_file": "docs/tool-calling/mcp-programmatic-tool-calling-opensandbox-dev.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "search get_schema execute flow", + "id": "mcp_programmatic_tool_calling_opensandbox_search_getschema_execute" + }, + { + "label": "Sandbox Isolation Rationale", + "file_type": "document", + "source_file": "docs/tool-calling/mcp-programmatic-tool-calling-opensandbox-dev.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "sandbox isolation rationale", + "id": "mcp_programmatic_tool_calling_opensandbox_sandbox_rationale" + }, + { + "label": "Programmatic Tool Calling", + "file_type": "document", + "source_file": "docs/tool-calling/cameronking4-programmatic-tool-calling-github.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "programmatic tool calling", + "id": "cameronking4_programmatic_tool_calling_programmatic_tool_calling" + }, + { + "label": "code_execution Meta-Tool", + "file_type": "document", + "source_file": "docs/tool-calling/cameronking4-programmatic-tool-calling-github.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "code_execution meta-tool", + "id": "cameronking4_programmatic_tool_calling_code_execution_metatool" + }, + { + "label": "MCP Bridge", + "file_type": "document", + "source_file": "docs/tool-calling/cameronking4-programmatic-tool-calling-github.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "mcp bridge", + "id": "cameronking4_programmatic_tool_calling_mcp_bridge" + }, + { + "label": "Tool Search", + "file_type": "document", + "source_file": "docs/tool-calling/tool-search-redefining-agent-tool-calling-epsilla.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "tool search", + "id": "tool_search_redefining_agent_tool_calling_epsilla_tool_search" + }, + { + "label": "Just-in-Time Retrieval", + "file_type": "document", + "source_file": "docs/tool-calling/tool-search-redefining-agent-tool-calling-epsilla.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "just-in-time retrieval", + "id": "tool_search_redefining_agent_tool_calling_epsilla_jitr" + }, + { + "label": "Prompt Caching Stability", + "file_type": "document", + "source_file": "docs/tool-calling/tool-search-redefining-agent-tool-calling-epsilla.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "prompt caching stability", + "id": "tool_search_redefining_agent_tool_calling_epsilla_prompt_caching_stability" + }, + { + "label": "Anthropic Server Tools", + "file_type": "document", + "source_file": "docs/tool-calling/programmatic-tool-calling-claude-api.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "anthropic server tools", + "id": "programmatic_tool_calling_claude_api_server_tools" + }, + { + "label": "Anthropic Tool Search", + "file_type": "document", + "source_file": "docs/tool-calling/programmatic-tool-calling-claude-api.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "anthropic tool search", + "id": "programmatic_tool_calling_claude_api_tool_search" + }, + { + "label": "Anthropic code_execution Pattern", + "file_type": "document", + "source_file": "docs/tool-calling/programmatic-tool-calling-claude-api.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "anthropic code_execution pattern", + "id": "programmatic_tool_calling_claude_api_code_execution" + }, + { + "label": "MCP Connector", + "file_type": "document", + "source_file": "docs/tool-calling/programmatic-tool-calling-claude-api.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 0, + "norm_label": "mcp connector", + "id": "programmatic_tool_calling_claude_api_mcp_connector" + }, + { + "label": "Tool Definition Quality", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-implement-tool-use.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 10, + "norm_label": "tool definition quality", + "id": "anthropic_implement_tool_use_tool_definition_quality" + }, + { + "label": "Input Examples", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-implement-tool-use.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 10, + "norm_label": "input examples", + "id": "anthropic_implement_tool_use_input_examples" + }, + { + "label": "Strict Tool Choice", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-implement-tool-use.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 10, + "norm_label": "strict tool choice", + "id": "anthropic_implement_tool_use_strict_tool_choice" + }, + { + "label": "Structured Outputs", + "file_type": "document", + "source_file": "docs/structured-outputs/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 10, + "norm_label": "structured outputs", + "id": "structured_outputs_readme_structured_outputs" + }, + { + "label": "CFG Decoding", + "file_type": "document", + "source_file": "docs/structured-outputs/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 10, + "norm_label": "cfg decoding", + "id": "structured_outputs_readme_cfg_decoding" + }, + { + "label": "Strict Tool Use", + "file_type": "document", + "source_file": "docs/structured-outputs/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 10, + "norm_label": "strict tool use", + "id": "structured_outputs_readme_strict_tool_use" + }, + { + "label": "Prompt Consistency Techniques", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-increase-consistency.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 10, + "norm_label": "prompt consistency techniques", + "id": "anthropic_increase_consistency_prompt_techniques" + }, + { + "label": "Retrieval Grounding", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-increase-consistency.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 10, + "norm_label": "retrieval grounding", + "id": "anthropic_increase_consistency_retrieval_grounding" + }, + { + "label": "Prompt Chaining", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-increase-consistency.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 10, + "norm_label": "prompt chaining", + "id": "anthropic_increase_consistency_prompt_chaining" + }, + { + "label": "Structured Outputs for Guarantees", + "file_type": "document", + "source_file": "docs/structured-outputs/anthropic-increase-consistency.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 10, + "norm_label": "structured outputs for guarantees", + "id": "anthropic_increase_consistency_schema_guarantee_rationale" + }, + { + "label": "Hook Lifecycle", + "file_type": "document", + "source_file": "docs/claude-code/hooks/claude-hook-reference-doc.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 7, + "norm_label": "hook lifecycle", + "id": "claude_hook_reference_doc_hook_lifecycle" + }, + { + "label": "Hook Decision Control", + "file_type": "document", + "source_file": "docs/claude-code/hooks/claude-hook-reference-doc.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 7, + "norm_label": "hook decision control", + "id": "claude_hook_reference_doc_decision_control" + }, + { + "label": "Subagent and Team Hooks", + "file_type": "document", + "source_file": "docs/claude-code/hooks/claude-hook-reference-doc.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 7, + "norm_label": "subagent and team hooks", + "id": "claude_hook_reference_doc_subagent_team_hooks" + }, + { + "label": "Deterministic Hook Automation", + "file_type": "document", + "source_file": "docs/claude-code/hooks/automate-workflow-with-hooks.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 7, + "norm_label": "deterministic hook automation", + "id": "automate_workflow_with_hooks_deterministic_automation" + }, + { + "label": "Hook Type Selection", + "file_type": "document", + "source_file": "docs/claude-code/hooks/automate-workflow-with-hooks.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 7, + "norm_label": "hook type selection", + "id": "automate_workflow_with_hooks_hook_type_selection" + }, + { + "label": "Reference vs Task Skill Content", + "file_type": "document", + "source_file": "docs/claude-code/skills/extend-claude-with-skills.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 7, + "norm_label": "reference vs task skill content", + "id": "extend_claude_with_skills_reference_vs_task_content" + }, + { + "label": "Skill Frontmatter Controls", + "file_type": "document", + "source_file": "docs/claude-code/skills/extend-claude-with-skills.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 7, + "norm_label": "skill frontmatter controls", + "id": "extend_claude_with_skills_skill_frontmatter" + }, + { + "label": "context: fork", + "file_type": "document", + "source_file": "docs/claude-code/skills/extend-claude-with-skills.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 7, + "norm_label": "context: fork", + "id": "extend_claude_with_skills_context_fork" + }, + { + "label": "Three-Layer Extensibility System", + "file_type": "document", + "source_file": "docs/claude-code/skills/research-claude-code-skills-format.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 7, + "norm_label": "three-layer extensibility system", + "id": "research_claude_code_skills_format_three_layer_extensibility" + }, + { + "label": "Progressive Disclosure Skill Layout", + "file_type": "document", + "source_file": "docs/claude-code/skills/research-claude-code-skills-format.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 7, + "norm_label": "progressive disclosure skill layout", + "id": "research_claude_code_skills_format_progressive_disclosure" + }, + { + "label": "Marketplace Format", + "file_type": "document", + "source_file": "docs/claude-code/skills/research-claude-code-skills-format.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 7, + "norm_label": "marketplace format", + "id": "research_claude_code_skills_format_marketplace_format" + }, + { + "label": "Agent Teams", + "file_type": "document", + "source_file": "docs/claude-code/subagents/claude-orchestrate-of-claude-code-sessions.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 7, + "norm_label": "agent teams", + "id": "claude_orchestrate_of_claude_code_sessions_agent_teams" + }, + { + "label": "Subagents vs Agent Teams", + "file_type": "document", + "source_file": "docs/claude-code/subagents/claude-orchestrate-of-claude-code-sessions.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 7, + "norm_label": "subagents vs agent teams", + "id": "claude_orchestrate_of_claude_code_sessions_subagents_vs_teams" + }, + { + "label": "Custom Subagents", + "file_type": "document", + "source_file": "docs/claude-code/subagents/creating-custom-subagents.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 7, + "norm_label": "custom subagents", + "id": "creating_custom_subagents_custom_subagents" + }, + { + "label": "Subagent Frontmatter Controls", + "file_type": "document", + "source_file": "docs/claude-code/subagents/creating-custom-subagents.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 7, + "norm_label": "subagent frontmatter controls", + "id": "creating_custom_subagents_frontmatter_controls" + }, + { + "label": "Subagent Skills Preloading", + "file_type": "document", + "source_file": "docs/claude-code/subagents/creating-custom-subagents.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 7, + "norm_label": "subagent skills preloading", + "id": "creating_custom_subagents_skills_preloading" + }, + { + "label": "Documentation Scope Partitioning", + "file_type": "document", + "source_file": "docs/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 7, + "norm_label": "documentation scope partitioning", + "id": "docs_readme_scope_partitioning" + }, + { + "label": "Task-Based Reading Order", + "file_type": "document", + "source_file": "docs/README.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 7, + "norm_label": "task-based reading order", + "id": "docs_readme_task_reading_order" + }, + { + "label": "Long Context Prompting", + "file_type": "document", + "source_file": "docs/claude-code/claude-prompting-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 10, + "norm_label": "long context prompting", + "id": "claude_prompting_best_practices_long_context_prompting" + }, + { + "label": "Quote Grounding", + "file_type": "document", + "source_file": "docs/claude-code/claude-prompting-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 10, + "norm_label": "quote grounding", + "id": "claude_prompting_best_practices_quote_grounding" + }, + { + "label": "XML Document Structure", + "file_type": "document", + "source_file": "docs/claude-code/claude-prompting-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 10, + "norm_label": "xml document structure", + "id": "claude_prompting_best_practices_xml_document_structure" + }, + { + "label": "How Claude remembers your project", + "file_type": "document", + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "how claude remembers your project", + "id": "how_claude_remembers_a_project_doc" + }, + { + "label": "CLAUDE.md files", + "file_type": "document", + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "claude.md files", + "id": "how_claude_remembers_a_project_claude_md_files" + }, + { + "label": "Auto memory", + "file_type": "document", + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "auto memory", + "id": "how_claude_remembers_a_project_auto_memory" + }, + { + "label": ".claude/rules", + "file_type": "document", + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": ".claude/rules", + "id": "how_claude_remembers_a_project_claude_rules" + }, + { + "label": "/memory command", + "file_type": "document", + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "/memory command", + "id": "how_claude_remembers_a_project_memory_command" + }, + { + "label": "Managed CLAUDE.md", + "file_type": "document", + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "managed claude.md", + "id": "how_claude_remembers_a_project_managed_claude_md" + }, + { + "label": "claudeMdExcludes", + "file_type": "document", + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "claudemdexcludes", + "id": "how_claude_remembers_a_project_claude_md_excludes" + }, + { + "label": "MEMORY.md entrypoint", + "file_type": "document", + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "memory.md entrypoint", + "id": "how_claude_remembers_a_project_memory_md_entrypoint" + }, + { + "label": "Concise specific instructions", + "file_type": "document", + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "concise specific instructions", + "id": "how_claude_remembers_a_project_concise_specific_instructions" + }, + { + "label": "Context noise reduction", + "file_type": "document", + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "context noise reduction", + "id": "how_claude_remembers_a_project_context_noise_reduction" + }, + { + "label": "Create plugins", + "file_type": "document", + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 13, + "norm_label": "create plugins", + "id": "claude_create_plugin_doc_doc" + }, + { + "label": "Plugin manifest", + "file_type": "document", + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 13, + "norm_label": "plugin manifest", + "id": "claude_create_plugin_doc_plugin_manifest" + }, + { + "label": "Plugin skills directory", + "file_type": "document", + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "plugin skills directory", + "id": "claude_create_plugin_doc_plugin_skills_directory" + }, + { + "label": "Skill namespacing", + "file_type": "document", + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 13, + "norm_label": "skill namespacing", + "id": "claude_create_plugin_doc_skill_namespacing" + }, + { + "label": "Plugin root structure", + "file_type": "document", + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 13, + "norm_label": "plugin root structure", + "id": "claude_create_plugin_doc_plugin_root_structure" + }, + { + "label": "--plugin-dir flag", + "file_type": "document", + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 13, + "norm_label": "--plugin-dir flag", + "id": "claude_create_plugin_doc_plugin_dir_flag" + }, + { + "label": "/reload-plugins command", + "file_type": "document", + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 13, + "norm_label": "/reload-plugins command", + "id": "claude_create_plugin_doc_reload_plugins" + }, + { + "label": "Plugin settings.json", + "file_type": "document", + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 13, + "norm_label": "plugin settings.json", + "id": "claude_create_plugin_doc_plugin_settings_json" + }, + { + "label": "Standalone-to-plugin migration", + "file_type": "document", + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 13, + "norm_label": "standalone-to-plugin migration", + "id": "claude_create_plugin_doc_plugin_migration" + }, + { + "label": "Sharing and reuse", + "file_type": "document", + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 13, + "norm_label": "sharing and reuse", + "id": "claude_create_plugin_doc_sharing_and_reuse" + }, + { + "label": "Conflict prevention", + "file_type": "document", + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 13, + "norm_label": "conflict prevention", + "id": "claude_create_plugin_doc_conflict_prevention" + }, + { + "label": "Skill authoring best practices", + "file_type": "document", + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "skill authoring best practices", + "id": "skill_authoring_best_practices_doc" + }, + { + "label": "SKILL.md YAML frontmatter", + "file_type": "document", + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "skill.md yaml frontmatter", + "id": "skill_authoring_best_practices_yaml_frontmatter" + }, + { + "label": "Description field", + "file_type": "document", + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "description field", + "id": "skill_authoring_best_practices_description_field" + }, + { + "label": "Progressive disclosure", + "file_type": "document", + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "progressive disclosure", + "id": "skill_authoring_best_practices_progressive_disclosure" + }, + { + "label": "Workflow checklists", + "file_type": "document", + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "workflow checklists", + "id": "skill_authoring_best_practices_workflow_checklists" + }, + { + "label": "Feedback loops", + "file_type": "document", + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "feedback loops", + "id": "skill_authoring_best_practices_feedback_loops" + }, + { + "label": "Evaluation-driven development", + "file_type": "document", + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "evaluation-driven development", + "id": "skill_authoring_best_practices_evaluation_driven_development" + }, + { + "label": "Utility scripts", + "file_type": "document", + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "utility scripts", + "id": "skill_authoring_best_practices_utility_scripts" + }, + { + "label": "Skill runtime environment", + "file_type": "document", + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "skill runtime environment", + "id": "skill_authoring_best_practices_runtime_environment" + }, + { + "label": "Context-window efficiency", + "file_type": "document", + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "context-window efficiency", + "id": "skill_authoring_best_practices_context_window_efficiency" + }, + { + "label": "Machine verifiability", + "file_type": "document", + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "machine verifiability", + "id": "skill_authoring_best_practices_machine_verifiability" + }, + { + "label": "Claude Code subagent definition best practices", + "file_type": "document", + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "claude code subagent definition best practices", + "id": "research_subagent_best_practices_doc" + }, + { + "label": "Subagents", + "file_type": "document", + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "subagents", + "id": "research_subagent_best_practices_subagents" + }, + { + "label": "Explore agent", + "file_type": "document", + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "explore agent", + "id": "research_subagent_best_practices_explore_agent" + }, + { + "label": "context: fork", + "file_type": "document", + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "context: fork", + "id": "research_subagent_best_practices_context_fork" + }, + { + "label": "agent field", + "file_type": "document", + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "agent field", + "id": "research_subagent_best_practices_agent_field" + }, + { + "label": "Tool restrictions", + "file_type": "document", + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "tool restrictions", + "id": "research_subagent_best_practices_tool_restrictions" + }, + { + "label": "PreToolUse hooks", + "file_type": "document", + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "pretooluse hooks", + "id": "research_subagent_best_practices_pretooluse_hooks" + }, + { + "label": "Persistent memory for subagents", + "file_type": "document", + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "persistent memory for subagents", + "id": "research_subagent_best_practices_persistent_memory" + }, + { + "label": "Worktree isolation", + "file_type": "document", + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "worktree isolation", + "id": "research_subagent_best_practices_worktree_isolation" + }, + { + "label": "Confidence-based filtering", + "file_type": "document", + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "confidence-based filtering", + "id": "research_subagent_best_practices_confidence_filtering" + }, + { + "label": "Plugin-shipped agents", + "file_type": "document", + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 13, + "norm_label": "plugin-shipped agents", + "id": "research_subagent_best_practices_plugin_shipped_agents" + }, + { + "label": "Agent teams", + "file_type": "document", + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "agent teams", + "id": "research_subagent_best_practices_agent_teams" + }, + { + "label": "Actionable descriptions", + "file_type": "document", + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "actionable descriptions", + "id": "research_subagent_best_practices_actionable_descriptions" + }, + { + "label": "Context preservation", + "file_type": "document", + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "context preservation", + "id": "research_subagent_best_practices_context_preservation" + }, + { + "label": "Safety guardrails", + "file_type": "document", + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "safety guardrails", + "id": "research_subagent_best_practices_safety_guardrails" + }, + { + "label": "everything-claude-code repository", + "file_type": "document", + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "everything-claude-code repository", + "id": "research_subagent_best_practices_everything_claude_code" + }, + { + "label": "Anthropic Docs \u2014 Subagents", + "file_type": "document", + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "source_url": null, + "captured_at": null, + "author": null, + "contributor": null, + "community": 1, + "norm_label": "anthropic docs \u2014 subagents", + "id": "research_subagent_best_practices_anthropic_subagents_docs" + } + ], + "links": [ + { + "relation": "contains", + "confidence": "EXTRACTED", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L34", + "weight": 1.0, + "_src": "home_rodrigo_workspace_prp_agentic_engineering_scripts_prp_core_prompt_eval_py", + "_tgt": "prp_core_prompt_eval_load_cases", + "source": "home_rodrigo_workspace_prp_agentic_engineering_scripts_prp_core_prompt_eval_py", + "target": "prp_core_prompt_eval_load_cases", + "confidence_score": 1.0 + }, + { + "relation": "contains", + "confidence": "EXTRACTED", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L41", + "weight": 1.0, + "_src": "home_rodrigo_workspace_prp_agentic_engineering_scripts_prp_core_prompt_eval_py", + "_tgt": "prp_core_prompt_eval_evaluate_must_contain", + "source": "home_rodrigo_workspace_prp_agentic_engineering_scripts_prp_core_prompt_eval_py", + "target": "prp_core_prompt_eval_evaluate_must_contain", + "confidence_score": 1.0 + }, + { + "relation": "contains", + "confidence": "EXTRACTED", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L60", + "weight": 1.0, + "_src": "home_rodrigo_workspace_prp_agentic_engineering_scripts_prp_core_prompt_eval_py", + "_tgt": "prp_core_prompt_eval_evaluate_mirror_parity", + "source": "home_rodrigo_workspace_prp_agentic_engineering_scripts_prp_core_prompt_eval_py", + "target": "prp_core_prompt_eval_evaluate_mirror_parity", + "confidence_score": 1.0 + }, + { + "relation": "contains", + "confidence": "EXTRACTED", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L76", + "weight": 1.0, + "_src": "home_rodrigo_workspace_prp_agentic_engineering_scripts_prp_core_prompt_eval_py", + "_tgt": "prp_core_prompt_eval_run_case", + "source": "home_rodrigo_workspace_prp_agentic_engineering_scripts_prp_core_prompt_eval_py", + "target": "prp_core_prompt_eval_run_case", + "confidence_score": 1.0 + }, + { + "relation": "contains", + "confidence": "EXTRACTED", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L97", + "weight": 1.0, + "_src": "home_rodrigo_workspace_prp_agentic_engineering_scripts_prp_core_prompt_eval_py", + "_tgt": "prp_core_prompt_eval_main", + "source": "home_rodrigo_workspace_prp_agentic_engineering_scripts_prp_core_prompt_eval_py", + "target": "prp_core_prompt_eval_main", + "confidence_score": 1.0 + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L1", + "weight": 1.0, + "_src": "prp_core_prompt_eval_rationale_1", + "_tgt": "home_rodrigo_workspace_prp_agentic_engineering_scripts_prp_core_prompt_eval_py", + "source": "home_rodrigo_workspace_prp_agentic_engineering_scripts_prp_core_prompt_eval_py", + "target": "prp_core_prompt_eval_rationale_1", + "confidence_score": 1.0 + }, + { + "relation": "calls", + "confidence": "EXTRACTED", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L107", + "weight": 1.0, + "_src": "prp_core_prompt_eval_main", + "_tgt": "prp_core_prompt_eval_load_cases", + "source": "prp_core_prompt_eval_load_cases", + "target": "prp_core_prompt_eval_main", + "confidence_score": 1.0 + }, + { + "relation": "calls", + "confidence": "EXTRACTED", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L85", + "weight": 1.0, + "_src": "prp_core_prompt_eval_run_case", + "_tgt": "prp_core_prompt_eval_evaluate_must_contain", + "source": "prp_core_prompt_eval_evaluate_must_contain", + "target": "prp_core_prompt_eval_run_case", + "confidence_score": 1.0 + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L42", + "weight": 1.0, + "_src": "prp_core_prompt_eval_rationale_42", + "_tgt": "prp_core_prompt_eval_evaluate_must_contain", + "source": "prp_core_prompt_eval_evaluate_must_contain", + "target": "prp_core_prompt_eval_rationale_42", + "confidence_score": 1.0 + }, + { + "relation": "calls", + "confidence": "EXTRACTED", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L83", + "weight": 1.0, + "_src": "prp_core_prompt_eval_run_case", + "_tgt": "prp_core_prompt_eval_evaluate_mirror_parity", + "source": "prp_core_prompt_eval_evaluate_mirror_parity", + "target": "prp_core_prompt_eval_run_case", + "confidence_score": 1.0 + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L61", + "weight": 1.0, + "_src": "prp_core_prompt_eval_rationale_61", + "_tgt": "prp_core_prompt_eval_evaluate_mirror_parity", + "source": "prp_core_prompt_eval_evaluate_mirror_parity", + "target": "prp_core_prompt_eval_rationale_61", + "confidence_score": 1.0 + }, + { + "relation": "calls", + "confidence": "EXTRACTED", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L120", + "weight": 1.0, + "_src": "prp_core_prompt_eval_main", + "_tgt": "prp_core_prompt_eval_run_case", + "source": "prp_core_prompt_eval_run_case", + "target": "prp_core_prompt_eval_main", + "confidence_score": 1.0 + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "source_file": "/home/rodrigo/Workspace/prp-agentic-engineering/scripts/prp_core_prompt_eval.py", + "source_location": "L77", + "weight": 1.0, + "_src": "prp_core_prompt_eval_rationale_77", + "_tgt": "prp_core_prompt_eval_run_case", + "source": "prp_core_prompt_eval_run_case", + "target": "prp_core_prompt_eval_rationale_77", + "confidence_score": 1.0 + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_main_spec_driven_development", + "_tgt": "spec_driven_development_main_vibe_coding", + "source": "spec_driven_development_main_spec_driven_development", + "target": "spec_driven_development_main_vibe_coding" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_main_spec_driven_development", + "_tgt": "spec_driven_development_main_github_spec_kit", + "source": "spec_driven_development_main_spec_driven_development", + "target": "spec_driven_development_main_github_spec_kit" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_main_spec_driven_development", + "_tgt": "spec_driven_development_main_aws_kiro", + "source": "spec_driven_development_main_spec_driven_development", + "target": "spec_driven_development_main_aws_kiro" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_main_spec_driven_development", + "_tgt": "spec_driven_development_main_tessl_framework", + "source": "spec_driven_development_main_spec_driven_development", + "target": "spec_driven_development_main_tessl_framework" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_main_spec_driven_development", + "_tgt": "spec_driven_development_main_spec_first", + "source": "spec_driven_development_main_spec_driven_development", + "target": "spec_driven_development_main_spec_first" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_main_spec_driven_development", + "_tgt": "spec_driven_development_main_spec_anchored", + "source": "spec_driven_development_main_spec_driven_development", + "target": "spec_driven_development_main_spec_anchored" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_main_spec_driven_development", + "_tgt": "spec_driven_development_main_spec_as_source", + "source": "spec_driven_development_main_spec_driven_development", + "target": "spec_driven_development_main_spec_as_source" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_main_spec_driven_development", + "_tgt": "spec_driven_development_main_ambiguity_reduction_rationale", + "source": "spec_driven_development_main_spec_driven_development", + "target": "spec_driven_development_main_ambiguity_reduction_rationale" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_main_spec_driven_development", + "_tgt": "spec_driven_development_main_review_overhead_rationale", + "source": "spec_driven_development_main_spec_driven_development", + "target": "spec_driven_development_main_review_overhead_rationale" + }, + { + "relation": "semantically_similar_to", + "confidence": "INFERRED", + "confidence_score": 0.67, + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_main_spec_driven_development", + "_tgt": "anthropic_structured_outputs_structured_outputs", + "source": "spec_driven_development_main_spec_driven_development", + "target": "anthropic_structured_outputs_structured_outputs" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_main_vibe_coding", + "_tgt": "spec_driven_development_main_spec_first", + "source": "spec_driven_development_main_vibe_coding", + "target": "spec_driven_development_main_spec_first" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_main_github_spec_kit", + "_tgt": "spec_driven_development_main_spec_first", + "source": "spec_driven_development_main_github_spec_kit", + "target": "spec_driven_development_main_spec_first" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-main.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_main_tessl_framework", + "_tgt": "spec_driven_development_main_spec_as_source", + "source": "spec_driven_development_main_tessl_framework", + "target": "spec_driven_development_main_spec_as_source" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-structured-outputs.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_structured_outputs_structured_outputs", + "_tgt": "anthropic_structured_outputs_json_outputs", + "source": "anthropic_structured_outputs_structured_outputs", + "target": "anthropic_structured_outputs_json_outputs" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-strict-tool-use.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_structured_outputs_structured_outputs", + "_tgt": "anthropic_strict_tool_use_strict_tool_use", + "source": "anthropic_structured_outputs_structured_outputs", + "target": "anthropic_strict_tool_use_strict_tool_use" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-structured-outputs.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_structured_outputs_structured_outputs", + "_tgt": "anthropic_structured_outputs_constrained_decoding", + "source": "anthropic_structured_outputs_structured_outputs", + "target": "anthropic_structured_outputs_constrained_decoding" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-structured-outputs.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_structured_outputs_structured_outputs", + "_tgt": "anthropic_structured_outputs_schema_complexity_limits", + "source": "anthropic_structured_outputs_structured_outputs", + "target": "anthropic_structured_outputs_schema_complexity_limits" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-structured-outputs.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_structured_outputs_structured_outputs", + "_tgt": "anthropic_structured_outputs_grammar_compilation_cache", + "source": "anthropic_structured_outputs_structured_outputs", + "target": "anthropic_structured_outputs_grammar_compilation_cache" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-structured-outputs.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_structured_outputs_structured_outputs", + "_tgt": "anthropic_structured_outputs_hipaa_zdr_constraints", + "source": "anthropic_structured_outputs_structured_outputs", + "target": "anthropic_structured_outputs_hipaa_zdr_constraints" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-structured-outputs.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_structured_outputs_structured_outputs", + "_tgt": "anthropic_structured_outputs_reliability_rationale", + "source": "anthropic_structured_outputs_structured_outputs", + "target": "anthropic_structured_outputs_reliability_rationale" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-structured-outputs.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_structured_outputs_json_outputs", + "_tgt": "anthropic_structured_outputs_constrained_decoding", + "source": "anthropic_structured_outputs_json_outputs", + "target": "anthropic_structured_outputs_constrained_decoding" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-structured-outputs.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_structured_outputs_schema_complexity_limits", + "_tgt": "anthropic_structured_outputs_grammar_compilation_cache", + "source": "anthropic_structured_outputs_schema_complexity_limits", + "target": "anthropic_structured_outputs_grammar_compilation_cache" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-strict-tool-use.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_structured_outputs_schema_complexity_limits", + "_tgt": "anthropic_strict_tool_use_strict_tool_use", + "source": "anthropic_structured_outputs_schema_complexity_limits", + "target": "anthropic_strict_tool_use_strict_tool_use" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-structured-outputs.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_structured_outputs_hipaa_zdr_constraints", + "_tgt": "anthropic_structured_outputs_phi_schema_exclusion_rationale", + "source": "anthropic_structured_outputs_hipaa_zdr_constraints", + "target": "anthropic_structured_outputs_phi_schema_exclusion_rationale" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-strict-tool-use.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_structured_outputs_hipaa_zdr_constraints", + "_tgt": "anthropic_strict_tool_use_strict_tool_use", + "source": "anthropic_structured_outputs_hipaa_zdr_constraints", + "target": "anthropic_strict_tool_use_strict_tool_use" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-strict-tool-use.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_strict_tool_use_strict_tool_use", + "_tgt": "anthropic_strict_tool_use_validated_tool_inputs", + "source": "anthropic_strict_tool_use_strict_tool_use", + "target": "anthropic_strict_tool_use_validated_tool_inputs" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-strict-tool-use.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_strict_tool_use_strict_tool_use", + "_tgt": "anthropic_strict_tool_use_tool_choice_any", + "source": "anthropic_strict_tool_use_strict_tool_use", + "target": "anthropic_strict_tool_use_tool_choice_any" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-strict-tool-use.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_strict_tool_use_strict_tool_use", + "_tgt": "anthropic_strict_tool_use_agent_reliability_rationale", + "source": "anthropic_strict_tool_use_strict_tool_use", + "target": "anthropic_strict_tool_use_agent_reliability_rationale" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-strict-tool-use.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_strict_tool_use_strict_tool_use", + "_tgt": "anthropic_strict_tool_use_schema_complexity_limits", + "source": "anthropic_strict_tool_use_strict_tool_use", + "target": "anthropic_strict_tool_use_schema_complexity_limits" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-strict-tool-use.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_strict_tool_use_strict_tool_use", + "_tgt": "anthropic_strict_tool_use_phi_schema_exclusion", + "source": "anthropic_strict_tool_use_strict_tool_use", + "target": "anthropic_strict_tool_use_phi_schema_exclusion" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_strict_tool_use_strict_tool_use", + "_tgt": "readme_anthropic_claude_tool_search", + "source": "anthropic_strict_tool_use_strict_tool_use", + "target": "readme_anthropic_claude_tool_search" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "weight": 1.0, + "_src": "humanlayer_repository_analysis_codelayer", + "_tgt": "humanlayer_repository_analysis_humanlayer_wui", + "source": "humanlayer_repository_analysis_codelayer", + "target": "humanlayer_repository_analysis_humanlayer_wui" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "weight": 1.0, + "_src": "humanlayer_repository_analysis_codelayer", + "_tgt": "humanlayer_repository_analysis_humanlayer_local_runtime", + "source": "humanlayer_repository_analysis_codelayer", + "target": "humanlayer_repository_analysis_humanlayer_local_runtime" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "weight": 1.0, + "_src": "humanlayer_repository_analysis_codelayer", + "_tgt": "humanlayer_repository_analysis_hybrid_monorepo", + "source": "humanlayer_repository_analysis_codelayer", + "target": "humanlayer_repository_analysis_hybrid_monorepo" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "weight": 1.0, + "_src": "humanlayer_repository_analysis_humanlayer_local_runtime", + "_tgt": "humanlayer_repository_analysis_hlyr", + "source": "humanlayer_repository_analysis_humanlayer_local_runtime", + "target": "humanlayer_repository_analysis_hlyr" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "weight": 1.0, + "_src": "humanlayer_repository_analysis_humanlayer_local_runtime", + "_tgt": "humanlayer_repository_analysis_hld", + "source": "humanlayer_repository_analysis_humanlayer_local_runtime", + "target": "humanlayer_repository_analysis_hld" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "weight": 1.0, + "_src": "humanlayer_repository_analysis_humanlayer_local_runtime", + "_tgt": "humanlayer_repository_analysis_claudecode_go", + "source": "humanlayer_repository_analysis_humanlayer_local_runtime", + "target": "humanlayer_repository_analysis_claudecode_go" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "weight": 1.0, + "_src": "humanlayer_repository_analysis_humanlayer_local_runtime", + "_tgt": "humanlayer_repository_analysis_two_persistence_worlds", + "source": "humanlayer_repository_analysis_humanlayer_local_runtime", + "target": "humanlayer_repository_analysis_two_persistence_worlds" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "weight": 1.0, + "_src": "humanlayer_repository_analysis_hlyr", + "_tgt": "humanlayer_repository_analysis_approval_loop", + "source": "humanlayer_repository_analysis_hlyr", + "target": "humanlayer_repository_analysis_approval_loop" + }, + { + "relation": "calls", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "weight": 1.0, + "_src": "humanlayer_repository_analysis_hld", + "_tgt": "humanlayer_repository_analysis_claudecode_go", + "source": "humanlayer_repository_analysis_hld", + "target": "humanlayer_repository_analysis_claudecode_go" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "weight": 1.0, + "_src": "humanlayer_repository_analysis_hld", + "_tgt": "humanlayer_repository_analysis_sqlite_persistence", + "source": "humanlayer_repository_analysis_hld", + "target": "humanlayer_repository_analysis_sqlite_persistence" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "weight": 1.0, + "_src": "humanlayer_repository_analysis_hld", + "_tgt": "humanlayer_repository_analysis_approval_loop", + "source": "humanlayer_repository_analysis_hld", + "target": "humanlayer_repository_analysis_approval_loop" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "weight": 1.0, + "_src": "humanlayer_repository_analysis_hld", + "_tgt": "humanlayer_repository_analysis_humanlayer_wui", + "source": "humanlayer_repository_analysis_hld", + "target": "humanlayer_repository_analysis_humanlayer_wui" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "weight": 1.0, + "_src": "humanlayer_repository_analysis_hld", + "_tgt": "humanlayer_repository_analysis_daemon_orchestration_rationale", + "source": "humanlayer_repository_analysis_hld", + "target": "humanlayer_repository_analysis_daemon_orchestration_rationale" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md", + "source_location": null, + "weight": 1.0, + "_src": "humanlayer_repository_analysis_sqlite_persistence", + "_tgt": "humanlayer_repository_analysis_two_persistence_worlds", + "source": "humanlayer_repository_analysis_sqlite_persistence", + "target": "humanlayer_repository_analysis_two_persistence_worlds" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agentic-engineering/building-agent-harness-martin-richards.md", + "source_location": null, + "weight": 1.0, + "_src": "building-agent-harness_harness_concept", + "_tgt": "context-engineering-most-important-skill-dev_context_engineering", + "source": "building-agent-harness_harness_concept", + "target": "context-engineering-most-important-skill-dev_context_engineering" + }, + { + "relation": "cites", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agentic-engineering/building-agent-harness-martin-richards.md", + "source_location": null, + "weight": 1.0, + "_src": "building-agent-harness_harness_concept", + "_tgt": "building-agent-harness_mitchell_hashimoto", + "source": "building-agent-harness_harness_concept", + "target": "building-agent-harness_mitchell_hashimoto" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agentic-engineering/building-agent-harness-martin-richards.md", + "source_location": null, + "weight": 1.0, + "_src": "building-agent-harness_harness_concept", + "_tgt": "building-agent-harness_atelier", + "source": "building-agent-harness_harness_concept", + "target": "building-agent-harness_atelier" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/harness-engineering/skill-issue-harness-engineering-for-coding-agents.md", + "source_location": null, + "weight": 1.0, + "_src": "building-agent-harness_harness_concept", + "_tgt": "skill-issue_harness_engineering", + "source": "building-agent-harness_harness_concept", + "target": "skill-issue_harness_engineering" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/harness-engineering/harnessengineering-building-the-operating-system-for-autonomous-agents.md", + "source_location": null, + "weight": 1.0, + "_src": "building-agent-harness_harness_concept", + "_tgt": "harness-os_harness_as_os", + "source": "building-agent-harness_harness_concept", + "target": "harness-os_harness_as_os" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agentic-engineering/building-agent-harness-martin-richards.md", + "source_location": null, + "weight": 1.0, + "_src": "building-agent-harness_atelier", + "_tgt": "advanced-context-engineering_rpi_workflow", + "source": "building-agent-harness_atelier", + "target": "advanced-context-engineering_rpi_workflow" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agentic-engineering/research-plan-implement-rpi.md", + "source_location": null, + "weight": 1.0, + "_src": "research-plan-implement_humanlayer_rpi", + "_tgt": "advanced-context-engineering_rpi_workflow", + "source": "research-plan-implement_humanlayer_rpi", + "target": "advanced-context-engineering_rpi_workflow" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agentic-engineering/research-plan-implement-rpi.md", + "source_location": null, + "weight": 1.0, + "_src": "research-plan-implement_humanlayer_rpi", + "_tgt": "research-plan-implement_leverage_model", + "source": "research-plan-implement_humanlayer_rpi", + "target": "research-plan-implement_leverage_model" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agentic-engineering/research-plan-implement-rpi.md", + "source_location": null, + "weight": 1.0, + "_src": "rpi-rationale_why_phases_matter", + "_tgt": "advanced-context-engineering_rpi_workflow", + "source": "rpi-rationale_why_phases_matter", + "target": "advanced-context-engineering_rpi_workflow" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agentic-engineering/research-plan-implement-review-tyler-burleigh.md", + "source_location": null, + "weight": 1.0, + "_src": "rpir-tyler-burleigh_rpir_workflow", + "_tgt": "advanced-context-engineering_rpi_workflow", + "source": "rpir-tyler-burleigh_rpir_workflow", + "target": "advanced-context-engineering_rpi_workflow" + }, + { + "relation": "references", + "confidence": "INFERRED", + "confidence_score": 0.7, + "source_file": "docs/agentic-engineering/claude-cookbook-anthropic.md", + "source_location": null, + "weight": 1.0, + "_src": "claude-cookbook_anthropic_cookbook", + "_tgt": "mcp-vs-a2a_mcp", + "source": "claude-cookbook_anthropic_cookbook", + "target": "mcp-vs-a2a_mcp" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.8, + "source_file": "docs/agentic-engineering/agentic-software-modernization-markus-harrer.md", + "source_location": null, + "weight": 1.0, + "_src": "agentic-modernization_entropy_problem", + "_tgt": "context-engineering-most-important-skill-dev_signal_to_noise", + "source": "agentic-modernization_entropy_problem", + "target": "context-engineering-most-important-skill-dev_signal_to_noise" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agentic-engineering/agentic-software-modernization-markus-harrer.md", + "source_location": null, + "weight": 1.0, + "_src": "agentic-modernization_critic_agent", + "_tgt": "advanced-context-engineering_rpi_workflow", + "source": "agentic-modernization_critic_agent", + "target": "advanced-context-engineering_rpi_workflow" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/context-engineering-most-important-skill-dev.md", + "source_location": null, + "weight": 1.0, + "_src": "context-engineering-most-important-skill-dev_context_engineering", + "_tgt": "context-engineering-most-important-skill-dev_five_pillars", + "source": "context-engineering-most-important-skill-dev_context_engineering", + "target": "context-engineering-most-important-skill-dev_five_pillars" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/context-engineering-most-important-skill-dev.md", + "source_location": null, + "weight": 1.0, + "_src": "context-engineering-most-important-skill-dev_context_engineering", + "_tgt": "context-eng-rationale_why_context_engineering_matters", + "source": "context-engineering-most-important-skill-dev_context_engineering", + "target": "context-eng-rationale_why_context_engineering_matters" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/advanced-context-engineering-coding-agents-dev.md", + "source_location": null, + "weight": 1.0, + "_src": "context-engineering-most-important-skill-dev_context_engineering", + "_tgt": "advanced-context-engineering_dumb_zone", + "source": "context-engineering-most-important-skill-dev_context_engineering", + "target": "advanced-context-engineering_dumb_zone" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/progressive-disclosure-ai-agents.md", + "source_location": null, + "weight": 1.0, + "_src": "context-engineering-most-important-skill-dev_context_engineering", + "_tgt": "progressive-disclosure_progressive_disclosure", + "source": "context-engineering-most-important-skill-dev_context_engineering", + "target": "progressive-disclosure_progressive_disclosure" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/context-engineering-commercial-agents-jeremy-daly.md", + "source_location": null, + "weight": 1.0, + "_src": "context-engineering-most-important-skill-dev_context_engineering", + "_tgt": "commercial-agents_memory_taxonomy", + "source": "context-engineering-most-important-skill-dev_context_engineering", + "target": "commercial-agents_memory_taxonomy" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/context-engineering-commercial-agents-jeremy-daly.md", + "source_location": null, + "weight": 1.0, + "_src": "context-engineering-most-important-skill-dev_context_engineering", + "_tgt": "commercial-agents_context_engine_loop", + "source": "context-engineering-most-important-skill-dev_context_engineering", + "target": "commercial-agents_context_engine_loop" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/harness-engineering/skill-issue-harness-engineering-for-coding-agents.md", + "source_location": null, + "weight": 1.0, + "_src": "context-engineering-most-important-skill-dev_context_engineering", + "_tgt": "skill-issue_harness_engineering", + "source": "context-engineering-most-important-skill-dev_context_engineering", + "target": "skill-issue_harness_engineering" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.6, + "source_file": "docs/general-llm/Call_Me_A_Jerk_Persuading_AI_to_Comply_with_Objectionable_Requests.pdf", + "source_location": null, + "weight": 1.0, + "_src": "context-engineering-most-important-skill-dev_context_engineering", + "_tgt": "call-me-jerk_parahuman_behavior", + "source": "context-engineering-most-important-skill-dev_context_engineering", + "target": "call-me-jerk_parahuman_behavior" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/context-engineering-most-important-skill-dev.md", + "source_location": null, + "weight": 1.0, + "_src": "context-engineering-most-important-skill-dev_five_pillars", + "_tgt": "context-engineering-most-important-skill-dev_context_selection", + "source": "context-engineering-most-important-skill-dev_five_pillars", + "target": "context-engineering-most-important-skill-dev_context_selection" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/context-engineering-most-important-skill-dev.md", + "source_location": null, + "weight": 1.0, + "_src": "context-engineering-most-important-skill-dev_five_pillars", + "_tgt": "context-engineering-most-important-skill-dev_context_structuring", + "source": "context-engineering-most-important-skill-dev_five_pillars", + "target": "context-engineering-most-important-skill-dev_context_structuring" + }, + { + "relation": "semantically_similar_to", + "confidence": "INFERRED", + "confidence_score": 0.85, + "source_file": "docs/harness-engineering/harness-engineering.md", + "source_location": null, + "weight": 1.0, + "_src": "context-engineering-most-important-skill-dev_five_pillars", + "_tgt": "harness-engineering-definitive_five_pillars", + "source": "context-engineering-most-important-skill-dev_five_pillars", + "target": "harness-engineering-definitive_five_pillars" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/context-engineering-most-important-skill-dev.md", + "source_location": null, + "weight": 1.0, + "_src": "context-engineering-most-important-skill-dev_context_selection", + "_tgt": "context-engineering-most-important-skill-dev_signal_to_noise", + "source": "context-engineering-most-important-skill-dev_context_selection", + "target": "context-engineering-most-important-skill-dev_signal_to_noise" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/context-engineering-most-important-skill-dev.md", + "source_location": null, + "weight": 1.0, + "_src": "context-engineering-most-important-skill-dev_context_structuring", + "_tgt": "context-engineering-most-important-skill-dev_lost_in_middle", + "source": "context-engineering-most-important-skill-dev_context_structuring", + "target": "context-engineering-most-important-skill-dev_lost_in_middle" + }, + { + "relation": "semantically_similar_to", + "confidence": "INFERRED", + "confidence_score": 0.9, + "source_file": "docs/context-engineering/agents-md-is-a-liability-paddo.md", + "source_location": null, + "weight": 1.0, + "_src": "context-engineering-most-important-skill-dev_lost_in_middle", + "_tgt": "agents-md-liability_attention_dilution", + "source": "context-engineering-most-important-skill-dev_lost_in_middle", + "target": "agents-md-liability_attention_dilution" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/advanced-context-engineering-coding-agents-dev.md", + "source_location": null, + "weight": 1.0, + "_src": "advanced-context-engineering_dumb_zone", + "_tgt": "advanced-context-engineering_intentional_compaction", + "source": "advanced-context-engineering_dumb_zone", + "target": "advanced-context-engineering_intentional_compaction" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.9, + "source_file": "docs/context-engineering/advanced-context-engineering-coding-agents-dev.md", + "source_location": null, + "weight": 1.0, + "_src": "advanced-context-engineering_dumb_zone", + "_tgt": "context-stops-being-scarce_compaction_problem", + "source": "advanced-context-engineering_dumb_zone", + "target": "context-stops-being-scarce_compaction_problem" + }, + { + "relation": "semantically_similar_to", + "confidence": "INFERRED", + "confidence_score": 0.9, + "source_file": "docs/context-engineering/progressive-disclosure-ai-agents.md", + "source_location": null, + "weight": 1.0, + "_src": "advanced-context-engineering_dumb_zone", + "_tgt": "progressive-disclosure_context_rot", + "source": "advanced-context-engineering_dumb_zone", + "target": "progressive-disclosure_context_rot" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/pi-context-zone-github.md", + "source_location": null, + "weight": 1.0, + "_src": "advanced-context-engineering_dumb_zone", + "_tgt": "pi-context-zone_smart_warm_dumb_zones", + "source": "advanced-context-engineering_dumb_zone", + "target": "pi-context-zone_smart_warm_dumb_zones" + }, + { + "relation": "cites", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/harness-engineering/skill-issue-harness-engineering-for-coding-agents.md", + "source_location": null, + "weight": 1.0, + "_src": "advanced-context-engineering_dumb_zone", + "_tgt": "skill-issue_dex_horthy", + "source": "advanced-context-engineering_dumb_zone", + "target": "skill-issue_dex_horthy" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/advanced-context-engineering-coding-agents-dev.md", + "source_location": null, + "weight": 1.0, + "_src": "advanced-context-engineering_intentional_compaction", + "_tgt": "advanced-context-engineering_rpi_workflow", + "source": "advanced-context-engineering_intentional_compaction", + "target": "advanced-context-engineering_rpi_workflow" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/context-stops-being-scarce-paddo.md", + "source_location": null, + "weight": 1.0, + "_src": "advanced-context-engineering_intentional_compaction", + "_tgt": "context-stops-being-scarce_compaction_problem", + "source": "advanced-context-engineering_intentional_compaction", + "target": "context-stops-being-scarce_compaction_problem" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/context-stops-being-scarce-paddo.md", + "source_location": null, + "weight": 1.0, + "_src": "context-stops-being-scarce_1m_context_window", + "_tgt": "context-stops-being-scarce_compaction_problem", + "source": "context-stops-being-scarce_1m_context_window", + "target": "context-stops-being-scarce_compaction_problem" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/context-stops-being-scarce-paddo.md", + "source_location": null, + "weight": 1.0, + "_src": "context-stops-being-scarce_1m_context_window", + "_tgt": "context-stops-being-scarce_flat_pricing", + "source": "context-stops-being-scarce_1m_context_window", + "target": "context-stops-being-scarce_flat_pricing" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/progressive-disclosure-ai-agents.md", + "source_location": null, + "weight": 1.0, + "_src": "progressive-disclosure_progressive_disclosure", + "_tgt": "progressive-disclosure_context_rot", + "source": "progressive-disclosure_progressive_disclosure", + "target": "progressive-disclosure_context_rot" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/progressive-disclosure-ai-agents.md", + "source_location": null, + "weight": 1.0, + "_src": "progressive-disclosure_progressive_disclosure", + "_tgt": "progressive-disclosure_index_first_loading", + "source": "progressive-disclosure_progressive_disclosure", + "target": "progressive-disclosure_index_first_loading" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/progressive-disclosure-ai-agents.md", + "source_location": null, + "weight": 1.0, + "_src": "progressive-disclosure_progressive_disclosure", + "_tgt": "progressive-disclosure_scout_pattern", + "source": "progressive-disclosure_progressive_disclosure", + "target": "progressive-disclosure_scout_pattern" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/progressive-disclosure-ai-agents.md", + "source_location": null, + "weight": 1.0, + "_src": "progressive-disclosure_context_rot", + "_tgt": "progressive-disclosure_inverted_u_pattern", + "source": "progressive-disclosure_context_rot", + "target": "progressive-disclosure_inverted_u_pattern" + }, + { + "relation": "semantically_similar_to", + "confidence": "INFERRED", + "confidence_score": 0.85, + "source_file": "docs/context-engineering/shedding-dead-context-ryan-spletzer.md", + "source_location": null, + "weight": 1.0, + "_src": "progressive-disclosure_context_rot", + "_tgt": "shedding-dead-context_dead_context", + "source": "progressive-disclosure_context_rot", + "target": "shedding-dead-context_dead_context" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/shedding-dead-context-ryan-spletzer.md", + "source_location": null, + "weight": 1.0, + "_src": "shedding-dead-context_dead_context", + "_tgt": "shedding-dead-context_ram_analogy", + "source": "shedding-dead-context_dead_context", + "target": "shedding-dead-context_ram_analogy" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/context-engineering-commercial-agents-jeremy-daly.md", + "source_location": null, + "weight": 1.0, + "_src": "commercial-agents_memory_taxonomy", + "_tgt": "commercial-agents_truth_vs_acceleration", + "source": "commercial-agents_memory_taxonomy", + "target": "commercial-agents_truth_vs_acceleration" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/context-engineering-commercial-agents-jeremy-daly.md", + "source_location": null, + "weight": 1.0, + "_src": "commercial-agents_memory_taxonomy", + "_tgt": "commercial-agents_context_engine_loop", + "source": "commercial-agents_memory_taxonomy", + "target": "commercial-agents_context_engine_loop" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/agents-md-is-a-liability-paddo.md", + "source_location": null, + "weight": 1.0, + "_src": "agents-md-liability_500_instruction_ceiling", + "_tgt": "agents-md-liability_ifscale_benchmark", + "source": "agents-md-liability_500_instruction_ceiling", + "target": "agents-md-liability_ifscale_benchmark" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/agents-md-is-a-liability-paddo.md", + "source_location": null, + "weight": 1.0, + "_src": "agents-md-liability_500_instruction_ceiling", + "_tgt": "agents-md-liability_attention_dilution", + "source": "agents-md-liability_500_instruction_ceiling", + "target": "agents-md-liability_attention_dilution" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/agents-md-is-a-liability-paddo.md", + "source_location": null, + "weight": 1.0, + "_src": "agents-md-liability_500_instruction_ceiling", + "_tgt": "agents-md-liability_prompt_repetition", + "source": "agents-md-liability_500_instruction_ceiling", + "target": "agents-md-liability_prompt_repetition" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/Evaluating-AGENTS-paper.md", + "source_location": null, + "weight": 1.0, + "_src": "agents-md-liability_500_instruction_ceiling", + "_tgt": "evaluating-agents-paper_agentbench", + "source": "agents-md-liability_500_instruction_ceiling", + "target": "evaluating-agents-paper_agentbench" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/agents-md-is-a-liability-paddo.md", + "source_location": null, + "weight": 1.0, + "_src": "agents-md-liability_attention_dilution", + "_tgt": "agents-md-liability_attention_sinks", + "source": "agents-md-liability_attention_dilution", + "target": "agents-md-liability_attention_sinks" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/context-engineering/pi-context-zone-github.md", + "source_location": null, + "weight": 1.0, + "_src": "pi-context-zone_pi_coding_agent", + "_tgt": "pi-context-zone_smart_warm_dumb_zones", + "source": "pi-context-zone_pi_coding_agent", + "target": "pi-context-zone_smart_warm_dumb_zones" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/mcp-vs-a2a-dzone.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp-vs-a2a_mcp", + "_tgt": "mcp-vs-a2a_a2a", + "source": "mcp-vs-a2a_mcp", + "target": "mcp-vs-a2a_a2a" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/mcp-http2-http3-reddit.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp-vs-a2a_mcp", + "_tgt": "mcp-transport_mrtr_stateless", + "source": "mcp-vs-a2a_mcp", + "target": "mcp-transport_mrtr_stateless" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/mcp-typescript-sdk-github.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp-vs-a2a_mcp", + "_tgt": "mcp-typescript-sdk_typescript_sdk", + "source": "mcp-vs-a2a_mcp", + "target": "mcp-typescript-sdk_typescript_sdk" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/long-live-mcp-aws.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp-vs-a2a_mcp", + "_tgt": "long-live-mcp_aws_adoption", + "source": "mcp-vs-a2a_mcp", + "target": "long-live-mcp_aws_adoption" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/modelcontextprotocol-servers-github.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp-vs-a2a_mcp", + "_tgt": "mcp-servers_reference_servers", + "source": "mcp-vs-a2a_mcp", + "target": "mcp-servers_reference_servers" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/anthropic-mcp-github-topics.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp-vs-a2a_mcp", + "_tgt": "anthropic-mcp-topics_mcp_ecosystem", + "source": "mcp-vs-a2a_mcp", + "target": "anthropic-mcp-topics_mcp_ecosystem" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/a2a-protocol-huggingface.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp-vs-a2a_a2a", + "_tgt": "a2a-protocol_agent_cards", + "source": "mcp-vs-a2a_a2a", + "target": "a2a-protocol_agent_cards" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/mcp-http2-http3-reddit.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp-transport_mrtr_stateless", + "_tgt": "mcp-transport_server_cards", + "source": "mcp-transport_mrtr_stateless", + "target": "mcp-transport_server_cards" + }, + { + "relation": "semantically_similar_to", + "confidence": "INFERRED", + "confidence_score": 0.75, + "source_file": "docs/mcp/a2a-protocol-huggingface.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp-transport_server_cards", + "_tgt": "a2a-protocol_agent_cards", + "source": "mcp-transport_server_cards", + "target": "a2a-protocol_agent_cards" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/harness-engineering/skill-issue-harness-engineering-for-coding-agents.md", + "source_location": null, + "weight": 1.0, + "_src": "skill-issue_skill_issue_blog", + "_tgt": "skill-issue_harness_engineering", + "source": "skill-issue_skill_issue_blog", + "target": "skill-issue_harness_engineering" + }, + { + "relation": "cites", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/harness-engineering/skill-issue-harness-engineering-for-coding-agents.md", + "source_location": null, + "weight": 1.0, + "_src": "skill-issue_harness_engineering", + "_tgt": "skill-issue_dex_horthy", + "source": "skill-issue_harness_engineering", + "target": "skill-issue_dex_horthy" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "weight": 1.0, + "_src": "readme_programmatic_tool_calling", + "_tgt": "readme_code_execution_sandbox", + "source": "readme_programmatic_tool_calling", + "target": "readme_code_execution_sandbox" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "weight": 1.0, + "_src": "readme_programmatic_tool_calling", + "_tgt": "readme_tool_search", + "source": "readme_programmatic_tool_calling", + "target": "readme_tool_search" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "weight": 1.0, + "_src": "readme_programmatic_tool_calling", + "_tgt": "readme_context_bloat_problem", + "source": "readme_programmatic_tool_calling", + "target": "readme_context_bloat_problem" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "weight": 1.0, + "_src": "readme_programmatic_tool_calling", + "_tgt": "readme_vercel_ai_sdk_withprogrammaticcalling", + "source": "readme_programmatic_tool_calling", + "target": "readme_vercel_ai_sdk_withprogrammaticcalling" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "weight": 1.0, + "_src": "readme_programmatic_tool_calling", + "_tgt": "readme_dotnet_mcp_three_tool_pattern", + "source": "readme_programmatic_tool_calling", + "target": "readme_dotnet_mcp_three_tool_pattern" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "weight": 1.0, + "_src": "readme_tool_search", + "_tgt": "readme_context_bloat_problem", + "source": "readme_tool_search", + "target": "readme_context_bloat_problem" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "weight": 1.0, + "_src": "readme_tool_search", + "_tgt": "readme_context_bloat_rationale", + "source": "readme_tool_search", + "target": "readme_context_bloat_rationale" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "weight": 1.0, + "_src": "readme_tool_search", + "_tgt": "readme_anthropic_claude_tool_search", + "source": "readme_tool_search", + "target": "readme_anthropic_claude_tool_search" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "weight": 1.0, + "_src": "readme_tool_search", + "_tgt": "readme_spring_ai_toolsearchtoolcalladvisor", + "source": "readme_tool_search", + "target": "readme_spring_ai_toolsearchtoolcalladvisor" + }, + { + "relation": "semantically_similar_to", + "confidence": "INFERRED", + "confidence_score": 0.62, + "source_file": "docs/tool-calling/README.md", + "source_location": null, + "weight": 1.0, + "_src": "readme_tool_search", + "_tgt": "lost_in_the_middle_and_in_between_knowledge_graph_triple_extraction", + "source": "readme_tool_search", + "target": "lost_in_the_middle_and_in_between_knowledge_graph_triple_extraction" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_and_in_between_paper", + "_tgt": "lost_in_the_middle_and_in_between_lost_in_the_middle", + "source": "lost_in_the_middle_and_in_between_paper", + "target": "lost_in_the_middle_and_in_between_lost_in_the_middle" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_and_in_between_paper", + "_tgt": "lost_in_the_middle_and_in_between_in_between_effect", + "source": "lost_in_the_middle_and_in_between_paper", + "target": "lost_in_the_middle_and_in_between_in_between_effect" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_and_in_between_paper", + "_tgt": "lost_in_the_middle_and_in_between_multi_hop_qa", + "source": "lost_in_the_middle_and_in_between_paper", + "target": "lost_in_the_middle_and_in_between_multi_hop_qa" + }, + { + "relation": "cites", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_and_in_between_paper", + "_tgt": "lost_in_the_middle_and_in_between_liu_etal_2024", + "source": "lost_in_the_middle_and_in_between_paper", + "target": "lost_in_the_middle_and_in_between_liu_etal_2024" + }, + { + "relation": "cites", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_and_in_between_paper", + "_tgt": "lost_in_the_middle_and_in_between_levy_etal_2024", + "source": "lost_in_the_middle_and_in_between_paper", + "target": "lost_in_the_middle_and_in_between_levy_etal_2024" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_and_in_between_lost_in_the_middle", + "_tgt": "lost_in_the_middle_and_in_between_in_between_effect", + "source": "lost_in_the_middle_and_in_between_lost_in_the_middle", + "target": "lost_in_the_middle_and_in_between_in_between_effect" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_and_in_between_in_between_effect", + "_tgt": "lost_in_the_middle_and_in_between_multi_hop_qa", + "source": "lost_in_the_middle_and_in_between_in_between_effect", + "target": "lost_in_the_middle_and_in_between_multi_hop_qa" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_and_in_between_multi_hop_qa", + "_tgt": "lost_in_the_middle_and_in_between_chain_of_thought_prompting", + "source": "lost_in_the_middle_and_in_between_multi_hop_qa", + "target": "lost_in_the_middle_and_in_between_chain_of_thought_prompting" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_and_in_between_multi_hop_qa", + "_tgt": "lost_in_the_middle_and_in_between_knowledge_graph_triple_extraction", + "source": "lost_in_the_middle_and_in_between_multi_hop_qa", + "target": "lost_in_the_middle_and_in_between_knowledge_graph_triple_extraction" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_and_in_between_multi_hop_qa", + "_tgt": "lost_in_the_middle_and_in_between_document_summarization", + "source": "lost_in_the_middle_and_in_between_multi_hop_qa", + "target": "lost_in_the_middle_and_in_between_document_summarization" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_and_in_between_multi_hop_qa", + "_tgt": "lost_in_the_middle_and_in_between_combinatorial_scaling_rationale", + "source": "lost_in_the_middle_and_in_between_multi_hop_qa", + "target": "lost_in_the_middle_and_in_between_combinatorial_scaling_rationale" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_and_in_between_knowledge_graph_triple_extraction", + "_tgt": "lost_in_the_middle_and_in_between_fragile_reasoning_chains_rationale", + "source": "lost_in_the_middle_and_in_between_knowledge_graph_triple_extraction", + "target": "lost_in_the_middle_and_in_between_fragile_reasoning_chains_rationale" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_and_in_between_document_summarization", + "_tgt": "lost_in_the_middle_and_in_between_fragile_reasoning_chains_rationale", + "source": "lost_in_the_middle_and_in_between_document_summarization", + "target": "lost_in_the_middle_and_in_between_fragile_reasoning_chains_rationale" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/Call_Me_A_Jerk_Persuading_AI_to_Comply_with_Objectionable_Requests.pdf", + "source_location": null, + "weight": 1.0, + "_src": "call-me-jerk_persuasion_principles", + "_tgt": "call-me-jerk_parahuman_behavior", + "source": "call-me-jerk_persuasion_principles", + "target": "call-me-jerk_parahuman_behavior" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/Evaluating-AGENTS-paper.md", + "source_location": null, + "weight": 1.0, + "_src": "evaluating-agents-paper_agentbench", + "_tgt": "evaluating-agents-paper_eth_study", + "source": "evaluating-agents-paper_agentbench", + "target": "evaluating-agents-paper_eth_study" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-context-engineering-comprehensive.md", + "source_location": null, + "weight": 1.0, + "_src": "research_agent_workflows_and_patterns_doc", + "_tgt": "research_context_engineering_comprehensive_doc", + "source": "research_agent_workflows_and_patterns_doc", + "target": "research_context_engineering_comprehensive_doc" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-agent-workflows-and-patterns.md", + "source_location": null, + "weight": 1.0, + "_src": "research_agent_workflows_and_patterns_doc", + "_tgt": "research_agent_workflows_and_patterns_explore_plan_code_verify", + "source": "research_agent_workflows_and_patterns_doc", + "target": "research_agent_workflows_and_patterns_explore_plan_code_verify" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-agent-workflows-and-patterns.md", + "source_location": null, + "weight": 1.0, + "_src": "research_agent_workflows_and_patterns_doc", + "_tgt": "research_agent_workflows_and_patterns_spec_plan_execute", + "source": "research_agent_workflows_and_patterns_doc", + "target": "research_agent_workflows_and_patterns_spec_plan_execute" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-agent-workflows-and-patterns.md", + "source_location": null, + "weight": 1.0, + "_src": "research_agent_workflows_and_patterns_doc", + "_tgt": "research_agent_workflows_and_patterns_two_agent_architecture", + "source": "research_agent_workflows_and_patterns_doc", + "target": "research_agent_workflows_and_patterns_two_agent_architecture" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-agent-workflows-and-patterns.md", + "source_location": null, + "weight": 1.0, + "_src": "research_agent_workflows_and_patterns_doc", + "_tgt": "research_context_rot_and_management_progressive_disclosure", + "source": "research_agent_workflows_and_patterns_doc", + "target": "research_context_rot_and_management_progressive_disclosure" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-context-engineering-comprehensive.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_engineering_comprehensive_doc", + "_tgt": "research_context_engineering_comprehensive_context_engineering", + "source": "research_context_engineering_comprehensive_doc", + "target": "research_context_engineering_comprehensive_context_engineering" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-context-engineering-comprehensive.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_engineering_comprehensive_doc", + "_tgt": "research_context_engineering_comprehensive_high_signal_tokens", + "source": "research_context_engineering_comprehensive_doc", + "target": "research_context_engineering_comprehensive_high_signal_tokens" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-context-engineering-comprehensive.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_engineering_comprehensive_doc", + "_tgt": "research_context_rot_and_management_doc", + "source": "research_context_engineering_comprehensive_doc", + "target": "research_context_rot_and_management_doc" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-context-engineering-comprehensive.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_engineering_comprehensive_doc", + "_tgt": "research_whitespace_and_formatting_doc", + "source": "research_context_engineering_comprehensive_doc", + "target": "research_whitespace_and_formatting_doc" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-context-engineering-comprehensive.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_engineering_comprehensive_doc", + "_tgt": "research_multilingual_performance_doc", + "source": "research_context_engineering_comprehensive_doc", + "target": "research_multilingual_performance_doc" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/prompt-engineering-guide.md", + "source_location": null, + "weight": 1.0, + "_src": "prompt_engineering_guide_doc", + "_tgt": "prompt_engineering_guide_react_pattern", + "source": "prompt_engineering_guide_doc", + "target": "prompt_engineering_guide_react_pattern" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/prompt-engineering-guide.md", + "source_location": null, + "weight": 1.0, + "_src": "prompt_engineering_guide_doc", + "_tgt": "prompt_engineering_guide_structured_output_two_stage", + "source": "prompt_engineering_guide_doc", + "target": "prompt_engineering_guide_structured_output_two_stage" + }, + { + "relation": "cites", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/prompt-engineering-guide.md", + "source_location": null, + "weight": 1.0, + "_src": "prompt_engineering_guide_doc", + "_tgt": "prompt_engineering_guide_prompt_report", + "source": "prompt_engineering_guide_doc", + "target": "prompt_engineering_guide_prompt_report" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/a-guide-to-agents.md", + "source_location": null, + "weight": 1.0, + "_src": "a_guide_to_agents_doc", + "_tgt": "a_guide_to_agents_agents_md", + "source": "a_guide_to_agents_doc", + "target": "a_guide_to_agents_agents_md" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/a-guide-to-agents.md", + "source_location": null, + "weight": 1.0, + "_src": "a_guide_to_agents_doc", + "_tgt": "a_guide_to_agents_instruction_budget", + "source": "a_guide_to_agents_doc", + "target": "a_guide_to_agents_instruction_budget" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/a-guide-to-agents.md", + "source_location": null, + "weight": 1.0, + "_src": "a_guide_to_agents_doc", + "_tgt": "a_guide_to_agents_stale_documentation_poisoning", + "source": "a_guide_to_agents_doc", + "target": "a_guide_to_agents_stale_documentation_poisoning" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/a-guide-to-agents.md", + "source_location": null, + "weight": 1.0, + "_src": "a_guide_to_agents_doc", + "_tgt": "a_guide_to_agents_path_scoped_rules", + "source": "a_guide_to_agents_doc", + "target": "a_guide_to_agents_path_scoped_rules" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/persuasion-principles.md", + "source_location": null, + "weight": 1.0, + "_src": "persuasion_principles_doc", + "_tgt": "persuasion_principles_parahuman_effect", + "source": "persuasion_principles_doc", + "target": "persuasion_principles_parahuman_effect" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/persuasion-principles.md", + "source_location": null, + "weight": 1.0, + "_src": "persuasion_principles_doc", + "_tgt": "persuasion_principles_commitment_principle", + "source": "persuasion_principles_doc", + "target": "persuasion_principles_commitment_principle" + }, + { + "relation": "cites", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/persuasion-principles.md", + "source_location": null, + "weight": 1.0, + "_src": "persuasion_principles_doc", + "_tgt": "persuasion_principles_call_me_a_jerk_paper", + "source": "persuasion_principles_doc", + "target": "persuasion_principles_call_me_a_jerk_paper" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-whitespace-and-formatting.md", + "source_location": null, + "weight": 1.0, + "_src": "research_whitespace_and_formatting_doc", + "_tgt": "research_whitespace_and_formatting_structural_formatting", + "source": "research_whitespace_and_formatting_doc", + "target": "research_whitespace_and_formatting_structural_formatting" + }, + { + "relation": "cites", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-whitespace-and-formatting.md", + "source_location": null, + "weight": 1.0, + "_src": "research_whitespace_and_formatting_doc", + "_tgt": "prompt_engineering_guide_prompt_report", + "source": "research_whitespace_and_formatting_doc", + "target": "prompt_engineering_guide_prompt_report" + }, + { + "relation": "cites", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-whitespace-and-formatting.md", + "source_location": null, + "weight": 1.0, + "_src": "research_whitespace_and_formatting_doc", + "_tgt": "research_whitespace_and_formatting_pathpiece", + "source": "research_whitespace_and_formatting_doc", + "target": "research_whitespace_and_formatting_pathpiece" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-context-rot-and-management.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_rot_and_management_doc", + "_tgt": "research_context_rot_and_management_context_rot", + "source": "research_context_rot_and_management_doc", + "target": "research_context_rot_and_management_context_rot" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-context-rot-and-management.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_rot_and_management_doc", + "_tgt": "research_context_rot_and_management_attention_budget", + "source": "research_context_rot_and_management_doc", + "target": "research_context_rot_and_management_attention_budget" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-context-rot-and-management.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_rot_and_management_doc", + "_tgt": "research_context_rot_and_management_lost_in_the_middle", + "source": "research_context_rot_and_management_doc", + "target": "research_context_rot_and_management_lost_in_the_middle" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-context-rot-and-management.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_rot_and_management_doc", + "_tgt": "research_context_rot_and_management_progressive_disclosure", + "source": "research_context_rot_and_management_doc", + "target": "research_context_rot_and_management_progressive_disclosure" + }, + { + "relation": "cites", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-context-rot-and-management.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_rot_and_management_doc", + "_tgt": "research_context_rot_and_management_effective_context_engineering", + "source": "research_context_rot_and_management_doc", + "target": "research_context_rot_and_management_effective_context_engineering" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-multilingual-performance.md", + "source_location": null, + "weight": 1.0, + "_src": "research_multilingual_performance_doc", + "_tgt": "research_multilingual_performance_tokenization_tax", + "source": "research_multilingual_performance_doc", + "target": "research_multilingual_performance_tokenization_tax" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-multilingual-performance.md", + "source_location": null, + "weight": 1.0, + "_src": "research_multilingual_performance_doc", + "_tgt": "research_multilingual_performance_self_translate_strategy", + "source": "research_multilingual_performance_doc", + "target": "research_multilingual_performance_self_translate_strategy" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-multilingual-performance.md", + "source_location": null, + "weight": 1.0, + "_src": "research_multilingual_performance_doc", + "_tgt": "research_multilingual_performance_portuguese_models", + "source": "research_multilingual_performance_doc", + "target": "research_multilingual_performance_portuguese_models" + }, + { + "relation": "cites", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-multilingual-performance.md", + "source_location": null, + "weight": 1.0, + "_src": "research_multilingual_performance_doc", + "_tgt": "research_multilingual_performance_petrov_tokenization_unfairness", + "source": "research_multilingual_performance_doc", + "target": "research_multilingual_performance_petrov_tokenization_unfairness" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/Evaluating-AGENTS-paper.pdf", + "source_location": null, + "weight": 1.0, + "_src": "evaluating_agents_paper_doc", + "_tgt": "evaluating_agents_paper_agentbench", + "source": "evaluating_agents_paper_doc", + "target": "evaluating_agents_paper_agentbench" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/Evaluating-AGENTS-paper.pdf", + "source_location": null, + "weight": 1.0, + "_src": "evaluating_agents_paper_doc", + "_tgt": "evaluating_agents_paper_repository_context_files", + "source": "evaluating_agents_paper_doc", + "target": "evaluating_agents_paper_repository_context_files" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/Evaluating-AGENTS-paper.pdf", + "source_location": null, + "weight": 1.0, + "_src": "evaluating_agents_paper_doc", + "_tgt": "evaluating_agents_paper_minimal_context_files", + "source": "evaluating_agents_paper_doc", + "target": "evaluating_agents_paper_minimal_context_files" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skills-standard/README.md", + "source_location": null, + "weight": 1.0, + "_src": "agentskills_specification_doc", + "_tgt": "agentskills_readme_doc", + "source": "agentskills_specification_doc", + "target": "agentskills_readme_doc" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skills-standard/agentskills-specification.md", + "source_location": null, + "weight": 1.0, + "_src": "agentskills_specification_doc", + "_tgt": "agentskills_specification_skill_manifest", + "source": "agentskills_specification_doc", + "target": "agentskills_specification_skill_manifest" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skills-standard/README.md", + "source_location": null, + "weight": 1.0, + "_src": "agentskills_evaluating_skills_doc", + "_tgt": "agentskills_readme_doc", + "source": "agentskills_evaluating_skills_doc", + "target": "agentskills_readme_doc" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skills-standard/agentskills-evaluating-skills.md", + "source_location": null, + "weight": 1.0, + "_src": "agentskills_evaluating_skills_doc", + "_tgt": "agentskills_evaluating_skills_eval_driven_iteration", + "source": "agentskills_evaluating_skills_doc", + "target": "agentskills_evaluating_skills_eval_driven_iteration" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skills-standard/README.md", + "source_location": null, + "weight": 1.0, + "_src": "agentskills_best_practices_doc", + "_tgt": "agentskills_readme_doc", + "source": "agentskills_best_practices_doc", + "target": "agentskills_readme_doc" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skills-standard/agentskills-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "agentskills_best_practices_doc", + "_tgt": "agentskills_best_practices_gotchas_pattern", + "source": "agentskills_best_practices_doc", + "target": "agentskills_best_practices_gotchas_pattern" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skills-standard/agentskills-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "agentskills_best_practices_doc", + "_tgt": "agentskills_best_practices_validation_loop", + "source": "agentskills_best_practices_doc", + "target": "agentskills_best_practices_validation_loop" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skills-standard/README.md", + "source_location": null, + "weight": 1.0, + "_src": "agentskills_readme_doc", + "_tgt": "agentskills_readme_agent_skills_standard", + "source": "agentskills_readme_doc", + "target": "agentskills_readme_agent_skills_standard" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skills-standard/README.md", + "source_location": null, + "weight": 1.0, + "_src": "agentskills_readme_doc", + "_tgt": "agentskills_what_are_skills_doc", + "source": "agentskills_readme_doc", + "target": "agentskills_what_are_skills_doc" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skills-standard/README.md", + "source_location": null, + "weight": 1.0, + "_src": "agentskills_readme_doc", + "_tgt": "agentskills_using_scripts_doc", + "source": "agentskills_readme_doc", + "target": "agentskills_using_scripts_doc" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skills-standard/README.md", + "source_location": null, + "weight": 1.0, + "_src": "agentskills_readme_doc", + "_tgt": "agentskills_optimizing_descriptions_doc", + "source": "agentskills_readme_doc", + "target": "agentskills_optimizing_descriptions_doc" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skills-standard/agentskills-optimizing-descriptions.md", + "source_location": null, + "weight": 1.0, + "_src": "agentskills_optimizing_descriptions_doc", + "_tgt": "agentskills_optimizing_descriptions_skill_triggering", + "source": "agentskills_optimizing_descriptions_doc", + "target": "agentskills_optimizing_descriptions_skill_triggering" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skills-standard/agentskills-what-are-skills.md", + "source_location": null, + "weight": 1.0, + "_src": "agentskills_what_are_skills_doc", + "_tgt": "agentskills_what_are_skills_skill_directory", + "source": "agentskills_what_are_skills_doc", + "target": "agentskills_what_are_skills_skill_directory" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skills-standard/agentskills-what-are-skills.md", + "source_location": null, + "weight": 1.0, + "_src": "agentskills_what_are_skills_doc", + "_tgt": "agentskills_what_are_skills_progressive_discovery", + "source": "agentskills_what_are_skills_doc", + "target": "agentskills_what_are_skills_progressive_discovery" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skills-standard/agentskills-using-scripts.md", + "source_location": null, + "weight": 1.0, + "_src": "agentskills_using_scripts_doc", + "_tgt": "agentskills_using_scripts_self_contained_scripts", + "source": "agentskills_using_scripts_doc", + "target": "agentskills_using_scripts_self_contained_scripts" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agent-protocols/a2a-protocol-huggingface-space.md", + "source_location": null, + "weight": 1.0, + "_src": "a2a_protocol_huggingface_space_doc", + "_tgt": "a2a_protocol_huggingface_space_a2a_protocol", + "source": "a2a_protocol_huggingface_space_doc", + "target": "a2a_protocol_huggingface_space_a2a_protocol" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agent-protocols/a2a-protocol-huggingface-space.md", + "source_location": null, + "weight": 1.0, + "_src": "a2a_protocol_huggingface_space_doc", + "_tgt": "a2a_protocol_huggingface_space_agent_card", + "source": "a2a_protocol_huggingface_space_doc", + "target": "a2a_protocol_huggingface_space_agent_card" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agent-protocols/anthropic-engineer-2026-forecast-full-connectivity-mcp.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_engineer_2026_forecast_full_connectivity_mcp_doc", + "_tgt": "anthropic_engineer_2026_forecast_full_connectivity_mcp_full_connectivity_stack", + "source": "anthropic_engineer_2026_forecast_full_connectivity_mcp_doc", + "target": "anthropic_engineer_2026_forecast_full_connectivity_mcp_full_connectivity_stack" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agent-protocols/anthropic-engineer-2026-forecast-full-connectivity-mcp.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_engineer_2026_forecast_full_connectivity_mcp_doc", + "_tgt": "anthropic_engineer_2026_forecast_full_connectivity_mcp_programmatic_tool_composition", + "source": "anthropic_engineer_2026_forecast_full_connectivity_mcp_doc", + "target": "anthropic_engineer_2026_forecast_full_connectivity_mcp_programmatic_tool_composition" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agent-protocols/ai-agent-protocols-2026-guide.md", + "source_location": null, + "weight": 1.0, + "_src": "ai_agent_protocols_2026_guide_doc", + "_tgt": "ai_agent_protocols_2026_guide_mcp_protocol", + "source": "ai_agent_protocols_2026_guide_doc", + "target": "ai_agent_protocols_2026_guide_mcp_protocol" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agent-protocols/ai-agent-protocols-2026-guide.md", + "source_location": null, + "weight": 1.0, + "_src": "ai_agent_protocols_2026_guide_doc", + "_tgt": "ai_agent_protocols_2026_guide_layered_stack", + "source": "ai_agent_protocols_2026_guide_doc", + "target": "ai_agent_protocols_2026_guide_layered_stack" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agent-protocols/architectural-paradigms-advanced-agentic-systems.md", + "source_location": null, + "weight": 1.0, + "_src": "architectural_paradigms_advanced_agentic_systems_doc", + "_tgt": "architectural_paradigms_advanced_agentic_systems_ace_framework", + "source": "architectural_paradigms_advanced_agentic_systems_doc", + "target": "architectural_paradigms_advanced_agentic_systems_ace_framework" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agent-protocols/fluid-human-agent-collaboration-pmc.md", + "source_location": null, + "weight": 1.0, + "_src": "fluid_human_agent_collaboration_pmc_doc", + "_tgt": "fluid_human_agent_collaboration_pmc_fluid_collaboration", + "source": "fluid_human_agent_collaboration_pmc_doc", + "target": "fluid_human_agent_collaboration_pmc_fluid_collaboration" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agent-protocols/fluid-human-agent-collaboration-pmc.md", + "source_location": null, + "weight": 1.0, + "_src": "fluid_human_agent_collaboration_pmc_doc", + "_tgt": "fluid_human_agent_collaboration_pmc_dynamic_mentalizing", + "source": "fluid_human_agent_collaboration_pmc_doc", + "target": "fluid_human_agent_collaboration_pmc_dynamic_mentalizing" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agent-protocols/advancing-agentic-ai-communication-protocols.md", + "source_location": null, + "weight": 1.0, + "_src": "advancing_agentic_ai_communication_protocols_doc", + "_tgt": "advancing_agentic_ai_communication_protocols_anp_protocol", + "source": "advancing_agentic_ai_communication_protocols_doc", + "target": "advancing_agentic_ai_communication_protocols_anp_protocol" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/README.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_readme_doc", + "_tgt": "spec_driven_development_readme_spec_driven_development", + "source": "spec_driven_development_readme_doc", + "target": "spec_driven_development_readme_spec_driven_development" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/README.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_readme_doc", + "_tgt": "spec_driven_development_arxiv_doc", + "source": "spec_driven_development_readme_doc", + "target": "spec_driven_development_arxiv_doc" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/README.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_readme_doc", + "_tgt": "spec_driven_development_variant_doc", + "source": "spec_driven_development_readme_doc", + "target": "spec_driven_development_variant_doc" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-variant.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_variant_doc", + "_tgt": "spec_driven_development_variant_kiro", + "source": "spec_driven_development_variant_doc", + "target": "spec_driven_development_variant_kiro" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-variant.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_variant_doc", + "_tgt": "spec_driven_development_variant_spec_kit", + "source": "spec_driven_development_variant_doc", + "target": "spec_driven_development_variant_spec_kit" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-variant.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_variant_doc", + "_tgt": "spec_driven_development_variant_tessl", + "source": "spec_driven_development_variant_doc", + "target": "spec_driven_development_variant_tessl" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_arxiv_doc", + "_tgt": "spec_driven_development_arxiv_spec_first", + "source": "spec_driven_development_arxiv_doc", + "target": "spec_driven_development_arxiv_spec_first" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_arxiv_doc", + "_tgt": "spec_driven_development_arxiv_spec_anchored", + "source": "spec_driven_development_arxiv_doc", + "target": "spec_driven_development_arxiv_spec_anchored" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_arxiv_doc", + "_tgt": "spec_driven_development_arxiv_spec_as_source", + "source": "spec_driven_development_arxiv_doc", + "target": "spec_driven_development_arxiv_spec_as_source" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_arxiv_doc", + "_tgt": "spec_driven_development_arxiv_sdd_workflow", + "source": "spec_driven_development_arxiv_doc", + "target": "spec_driven_development_arxiv_sdd_workflow" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-context-engineering-comprehensive.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_engineering_comprehensive_context_engineering", + "_tgt": "research_context_engineering_comprehensive_high_signal_tokens", + "source": "research_context_engineering_comprehensive_context_engineering", + "target": "research_context_engineering_comprehensive_high_signal_tokens" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.86, + "source_file": "docs/agent-protocols/architectural-paradigms-advanced-agentic-systems.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_engineering_comprehensive_context_engineering", + "_tgt": "architectural_paradigms_advanced_agentic_systems_ace_framework", + "source": "research_context_engineering_comprehensive_context_engineering", + "target": "architectural_paradigms_advanced_agentic_systems_ace_framework" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.82, + "source_file": "docs/general-llm/research-context-rot-and-management.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_engineering_comprehensive_context_engineering", + "_tgt": "research_context_rot_and_management_effective_context_engineering", + "source": "research_context_engineering_comprehensive_context_engineering", + "target": "research_context_rot_and_management_effective_context_engineering" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.78, + "source_file": "docs/general-llm/research-multilingual-performance.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_engineering_comprehensive_context_engineering", + "_tgt": "research_multilingual_performance_tokenization_tax", + "source": "research_context_engineering_comprehensive_context_engineering", + "target": "research_multilingual_performance_tokenization_tax" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.78, + "source_file": "docs/general-llm/research-context-engineering-comprehensive.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_engineering_comprehensive_high_signal_tokens", + "_tgt": "research_multilingual_performance_tokenization_tax", + "source": "research_context_engineering_comprehensive_high_signal_tokens", + "target": "research_multilingual_performance_tokenization_tax" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.85, + "source_file": "docs/general-llm/research-whitespace-and-formatting.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_engineering_comprehensive_high_signal_tokens", + "_tgt": "research_whitespace_and_formatting_structural_formatting", + "source": "research_context_engineering_comprehensive_high_signal_tokens", + "target": "research_whitespace_and_formatting_structural_formatting" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.68, + "source_file": "docs/general-llm/research-multilingual-performance.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_engineering_comprehensive_high_signal_tokens", + "_tgt": "research_multilingual_performance_self_translate_strategy", + "source": "research_context_engineering_comprehensive_high_signal_tokens", + "target": "research_multilingual_performance_self_translate_strategy" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.8, + "source_file": "docs/general-llm/a-guide-to-agents.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_rot_and_management_context_rot", + "_tgt": "a_guide_to_agents_stale_documentation_poisoning", + "source": "research_context_rot_and_management_context_rot", + "target": "a_guide_to_agents_stale_documentation_poisoning" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-context-rot-and-management.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_rot_and_management_attention_budget", + "_tgt": "research_context_rot_and_management_progressive_disclosure", + "source": "research_context_rot_and_management_attention_budget", + "target": "research_context_rot_and_management_progressive_disclosure" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.76, + "source_file": "docs/general-llm/research-context-rot-and-management.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_rot_and_management_progressive_disclosure", + "_tgt": "a_guide_to_agents_path_scoped_rules", + "source": "research_context_rot_and_management_progressive_disclosure", + "target": "a_guide_to_agents_path_scoped_rules" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skills-standard/agentskills-what-are-skills.md", + "source_location": null, + "weight": 1.0, + "_src": "research_context_rot_and_management_progressive_disclosure", + "_tgt": "agentskills_what_are_skills_progressive_discovery", + "source": "research_context_rot_and_management_progressive_disclosure", + "target": "agentskills_what_are_skills_progressive_discovery" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-whitespace-and-formatting.md", + "source_location": null, + "weight": 1.0, + "_src": "research_whitespace_and_formatting_structural_formatting", + "_tgt": "research_whitespace_and_formatting_pathpiece", + "source": "research_whitespace_and_formatting_structural_formatting", + "target": "research_whitespace_and_formatting_pathpiece" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-whitespace-and-formatting.md", + "source_location": null, + "weight": 1.0, + "_src": "research_whitespace_and_formatting_structural_formatting", + "_tgt": "prompt_engineering_guide_prompt_report", + "source": "research_whitespace_and_formatting_structural_formatting", + "target": "prompt_engineering_guide_prompt_report" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-multilingual-performance.md", + "source_location": null, + "weight": 1.0, + "_src": "research_multilingual_performance_tokenization_tax", + "_tgt": "research_multilingual_performance_petrov_tokenization_unfairness", + "source": "research_multilingual_performance_tokenization_tax", + "target": "research_multilingual_performance_petrov_tokenization_unfairness" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.82, + "source_file": "docs/shared/skills-standard/agentskills-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_agent_workflows_and_patterns_explore_plan_code_verify", + "_tgt": "agentskills_best_practices_validation_loop", + "source": "research_agent_workflows_and_patterns_explore_plan_code_verify", + "target": "agentskills_best_practices_validation_loop" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.91, + "source_file": "docs/spec-driven-development/spec-driven-development-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "research_agent_workflows_and_patterns_spec_plan_execute", + "_tgt": "spec_driven_development_arxiv_sdd_workflow", + "source": "research_agent_workflows_and_patterns_spec_plan_execute", + "target": "spec_driven_development_arxiv_sdd_workflow" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.65, + "source_file": "docs/general-llm/research-agent-workflows-and-patterns.md", + "source_location": null, + "weight": 1.0, + "_src": "research_agent_workflows_and_patterns_two_agent_architecture", + "_tgt": "ai_agent_protocols_2026_guide_layered_stack", + "source": "research_agent_workflows_and_patterns_two_agent_architecture", + "target": "ai_agent_protocols_2026_guide_layered_stack" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.72, + "source_file": "docs/general-llm/prompt-engineering-guide.md", + "source_location": null, + "weight": 1.0, + "_src": "prompt_engineering_guide_react_pattern", + "_tgt": "anthropic_engineer_2026_forecast_full_connectivity_mcp_programmatic_tool_composition", + "source": "prompt_engineering_guide_react_pattern", + "target": "anthropic_engineer_2026_forecast_full_connectivity_mcp_programmatic_tool_composition" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.67, + "source_file": "docs/general-llm/prompt-engineering-guide.md", + "source_location": null, + "weight": 1.0, + "_src": "prompt_engineering_guide_structured_output_two_stage", + "_tgt": "agentskills_best_practices_validation_loop", + "source": "prompt_engineering_guide_structured_output_two_stage", + "target": "agentskills_best_practices_validation_loop" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/Evaluating-AGENTS-paper.pdf", + "source_location": null, + "weight": 1.0, + "_src": "a_guide_to_agents_agents_md", + "_tgt": "evaluating_agents_paper_repository_context_files", + "source": "a_guide_to_agents_agents_md", + "target": "evaluating_agents_paper_repository_context_files" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/a-guide-to-agents.md", + "source_location": null, + "weight": 1.0, + "_src": "a_guide_to_agents_instruction_budget", + "_tgt": "a_guide_to_agents_path_scoped_rules", + "source": "a_guide_to_agents_instruction_budget", + "target": "a_guide_to_agents_path_scoped_rules" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.75, + "source_file": "docs/general-llm/a-guide-to-agents.md", + "source_location": null, + "weight": 1.0, + "_src": "a_guide_to_agents_path_scoped_rules", + "_tgt": "agentskills_what_are_skills_progressive_discovery", + "source": "a_guide_to_agents_path_scoped_rules", + "target": "agentskills_what_are_skills_progressive_discovery" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.87, + "source_file": "docs/shared/skills-standard/agentskills-specification.md", + "source_location": null, + "weight": 1.0, + "_src": "agentskills_specification_skill_manifest", + "_tgt": "agentskills_optimizing_descriptions_skill_triggering", + "source": "agentskills_specification_skill_manifest", + "target": "agentskills_optimizing_descriptions_skill_triggering" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skills-standard/agentskills-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "agentskills_evaluating_skills_eval_driven_iteration", + "_tgt": "agentskills_best_practices_validation_loop", + "source": "agentskills_evaluating_skills_eval_driven_iteration", + "target": "agentskills_best_practices_validation_loop" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.63, + "source_file": "docs/shared/skills-standard/agentskills-using-scripts.md", + "source_location": null, + "weight": 1.0, + "_src": "agentskills_using_scripts_self_contained_scripts", + "_tgt": "anthropic_engineer_2026_forecast_full_connectivity_mcp_programmatic_tool_composition", + "source": "agentskills_using_scripts_self_contained_scripts", + "target": "anthropic_engineer_2026_forecast_full_connectivity_mcp_programmatic_tool_composition" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agent-protocols/a2a-protocol-huggingface-space.md", + "source_location": null, + "weight": 1.0, + "_src": "a2a_protocol_huggingface_space_a2a_protocol", + "_tgt": "ai_agent_protocols_2026_guide_mcp_protocol", + "source": "a2a_protocol_huggingface_space_a2a_protocol", + "target": "ai_agent_protocols_2026_guide_mcp_protocol" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agent-protocols/ai-agent-protocols-2026-guide.md", + "source_location": null, + "weight": 1.0, + "_src": "a2a_protocol_huggingface_space_a2a_protocol", + "_tgt": "ai_agent_protocols_2026_guide_layered_stack", + "source": "a2a_protocol_huggingface_space_a2a_protocol", + "target": "ai_agent_protocols_2026_guide_layered_stack" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.67, + "source_file": "docs/agent-protocols/advancing-agentic-ai-communication-protocols.md", + "source_location": null, + "weight": 1.0, + "_src": "a2a_protocol_huggingface_space_a2a_protocol", + "_tgt": "advancing_agentic_ai_communication_protocols_anp_protocol", + "source": "a2a_protocol_huggingface_space_a2a_protocol", + "target": "advancing_agentic_ai_communication_protocols_anp_protocol" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.86, + "source_file": "docs/agent-protocols/anthropic-engineer-2026-forecast-full-connectivity-mcp.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_engineer_2026_forecast_full_connectivity_mcp_full_connectivity_stack", + "_tgt": "ai_agent_protocols_2026_guide_layered_stack", + "source": "anthropic_engineer_2026_forecast_full_connectivity_mcp_full_connectivity_stack", + "target": "ai_agent_protocols_2026_guide_layered_stack" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agent-protocols/anthropic-engineer-2026-forecast-full-connectivity-mcp.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_engineer_2026_forecast_full_connectivity_mcp_programmatic_tool_composition", + "_tgt": "ai_agent_protocols_2026_guide_mcp_protocol", + "source": "anthropic_engineer_2026_forecast_full_connectivity_mcp_programmatic_tool_composition", + "target": "ai_agent_protocols_2026_guide_mcp_protocol" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agent-protocols/ai-agent-protocols-2026-guide.md", + "source_location": null, + "weight": 1.0, + "_src": "ai_agent_protocols_2026_guide_mcp_protocol", + "_tgt": "ai_agent_protocols_2026_guide_layered_stack", + "source": "ai_agent_protocols_2026_guide_mcp_protocol", + "target": "ai_agent_protocols_2026_guide_layered_stack" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/agent-protocols/fluid-human-agent-collaboration-pmc.md", + "source_location": null, + "weight": 1.0, + "_src": "fluid_human_agent_collaboration_pmc_fluid_collaboration", + "_tgt": "fluid_human_agent_collaboration_pmc_dynamic_mentalizing", + "source": "fluid_human_agent_collaboration_pmc_fluid_collaboration", + "target": "fluid_human_agent_collaboration_pmc_dynamic_mentalizing" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/README.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_readme_spec_driven_development", + "_tgt": "spec_driven_development_arxiv_spec_first", + "source": "spec_driven_development_readme_spec_driven_development", + "target": "spec_driven_development_arxiv_spec_first" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/README.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_readme_spec_driven_development", + "_tgt": "spec_driven_development_arxiv_spec_anchored", + "source": "spec_driven_development_readme_spec_driven_development", + "target": "spec_driven_development_arxiv_spec_anchored" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/README.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_readme_spec_driven_development", + "_tgt": "spec_driven_development_arxiv_spec_as_source", + "source": "spec_driven_development_readme_spec_driven_development", + "target": "spec_driven_development_arxiv_spec_as_source" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-variant.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_arxiv_spec_first", + "_tgt": "spec_driven_development_variant_kiro", + "source": "spec_driven_development_arxiv_spec_first", + "target": "spec_driven_development_variant_kiro" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-variant.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_arxiv_spec_first", + "_tgt": "spec_driven_development_variant_spec_kit", + "source": "spec_driven_development_arxiv_spec_first", + "target": "spec_driven_development_variant_spec_kit" + }, + { + "relation": "conceptually_related_to", + "confidence": "AMBIGUOUS", + "confidence_score": 0.25, + "source_file": "docs/spec-driven-development/spec-driven-development-variant.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_arxiv_spec_anchored", + "_tgt": "spec_driven_development_variant_spec_kit", + "source": "spec_driven_development_arxiv_spec_anchored", + "target": "spec_driven_development_variant_spec_kit" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-variant.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_arxiv_spec_anchored", + "_tgt": "spec_driven_development_variant_tessl", + "source": "spec_driven_development_arxiv_spec_anchored", + "target": "spec_driven_development_variant_tessl" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/spec-driven-development-variant.md", + "source_location": null, + "weight": 1.0, + "_src": "spec_driven_development_arxiv_spec_as_source", + "_tgt": "spec_driven_development_variant_tessl", + "source": "spec_driven_development_arxiv_spec_as_source", + "target": "spec_driven_development_variant_tessl" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/Evaluating-AGENTS-paper.pdf", + "source_location": null, + "weight": 1.0, + "_src": "evaluating_agents_paper_agentbench", + "_tgt": "evaluating_agents_paper_repository_context_files", + "source": "evaluating_agents_paper_agentbench", + "target": "evaluating_agents_paper_repository_context_files" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/Evaluating-AGENTS-paper.pdf", + "source_location": null, + "weight": 1.0, + "_src": "evaluating_agents_paper_repository_context_files", + "_tgt": "evaluating_agents_paper_minimal_context_files", + "source": "evaluating_agents_paper_repository_context_files", + "target": "evaluating_agents_paper_minimal_context_files" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/skills-over-mcp-meeting-notes-2248.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_meeting_notes_2248_prompts_as_skills", + "_tgt": "mcp_specification_prompts_primitive", + "source": "skills_over_mcp_meeting_notes_2248_prompts_as_skills", + "target": "mcp_specification_prompts_primitive" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/skills-over-mcp-meeting-notes-2248.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_meeting_notes_2248_prompts_as_skills", + "_tgt": "skills_over_mcp_meeting_notes_2248_skills_as_instructors", + "source": "skills_over_mcp_meeting_notes_2248_prompts_as_skills", + "target": "skills_over_mcp_meeting_notes_2248_skills_as_instructors" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/skills-over-mcp-meeting-notes-2248.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_meeting_notes_2248_prompts_as_skills", + "_tgt": "skills_over_mcp_meeting_notes_2248_skills_as_helpers", + "source": "skills_over_mcp_meeting_notes_2248_prompts_as_skills", + "target": "skills_over_mcp_meeting_notes_2248_skills_as_helpers" + }, + { + "relation": "semantically_similar_to", + "confidence": "INFERRED", + "confidence_score": 0.8, + "source_file": "docs/mcp/skills-vs-mcp-speakeasy.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_meeting_notes_2248_prompts_as_skills", + "_tgt": "skills_vs_mcp_speakeasy_skills_layer", + "source": "skills_over_mcp_meeting_notes_2248_prompts_as_skills", + "target": "skills_vs_mcp_speakeasy_skills_layer" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/skills-over-mcp-meeting-notes-2248.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_meeting_notes_2248_agents_ignore_skills", + "_tgt": "skills_over_mcp_meeting_notes_2248_discovery_metadata_rationale", + "source": "skills_over_mcp_meeting_notes_2248_agents_ignore_skills", + "target": "skills_over_mcp_meeting_notes_2248_discovery_metadata_rationale" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/skills-over-mcp-meeting-notes-2248.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_meeting_notes_2248_agents_ignore_skills", + "_tgt": "skills_over_mcp_meeting_notes_2248_skills_as_instructors", + "source": "skills_over_mcp_meeting_notes_2248_agents_ignore_skills", + "target": "skills_over_mcp_meeting_notes_2248_skills_as_instructors" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.67, + "source_file": "docs/mcp/skills-over-mcp-meeting-notes-2248.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_meeting_notes_2248_agents_ignore_skills", + "_tgt": "skills_over_mcp_office_hours_2460_lazy_loading_pattern", + "source": "skills_over_mcp_meeting_notes_2248_agents_ignore_skills", + "target": "skills_over_mcp_office_hours_2460_lazy_loading_pattern" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/skills-over-mcp-meeting-notes-2248.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_meeting_notes_2248_skills_as_instructors", + "_tgt": "skills_over_mcp_meeting_notes_2248_skills_as_helpers", + "source": "skills_over_mcp_meeting_notes_2248_skills_as_instructors", + "target": "skills_over_mcp_meeting_notes_2248_skills_as_helpers" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.72, + "source_file": "docs/mcp/skills-over-mcp-meeting-notes-2248.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_meeting_notes_2248_vendor_neutral_skill_schema", + "_tgt": "skills_vs_mcp_speakeasy_two_layer_architecture", + "source": "skills_over_mcp_meeting_notes_2248_vendor_neutral_skill_schema", + "target": "skills_vs_mcp_speakeasy_two_layer_architecture" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.68, + "source_file": "docs/mcp/skills-over-mcp-office-hours-2460.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_meeting_notes_2248_vendor_neutral_skill_schema", + "_tgt": "skills_over_mcp_office_hours_2460_resource_uris", + "source": "skills_over_mcp_meeting_notes_2248_vendor_neutral_skill_schema", + "target": "skills_over_mcp_office_hours_2460_resource_uris" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/skills-over-mcp-office-hours-2460.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_office_hours_2460_well_known_skill_index", + "_tgt": "skills_over_mcp_office_hours_2460_skill_scheme_rejection_rationale", + "source": "skills_over_mcp_office_hours_2460_well_known_skill_index", + "target": "skills_over_mcp_office_hours_2460_skill_scheme_rejection_rationale" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/skills-over-mcp-office-hours-2460.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_office_hours_2460_well_known_skill_index", + "_tgt": "skills_over_mcp_office_hours_2460_lazy_loading_pattern", + "source": "skills_over_mcp_office_hours_2460_well_known_skill_index", + "target": "skills_over_mcp_office_hours_2460_lazy_loading_pattern" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/skills-over-mcp-office-hours-2460.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_office_hours_2460_well_known_skill_index", + "_tgt": "mcp_specification_prompts_primitive", + "source": "skills_over_mcp_office_hours_2460_well_known_skill_index", + "target": "mcp_specification_prompts_primitive" + }, + { + "relation": "semantically_similar_to", + "confidence": "INFERRED", + "confidence_score": 0.86, + "source_file": "docs/mcp/skills-over-mcp-office-hours-2460.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_office_hours_2460_lazy_loading_pattern", + "_tgt": "tool_search_redefining_agent_tool_calling_epsilla_tool_search", + "source": "skills_over_mcp_office_hours_2460_lazy_loading_pattern", + "target": "tool_search_redefining_agent_tool_calling_epsilla_tool_search" + }, + { + "relation": "semantically_similar_to", + "confidence": "INFERRED", + "confidence_score": 0.82, + "source_file": "docs/tool-calling/programmatic-tool-calling-claude-api.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_office_hours_2460_lazy_loading_pattern", + "_tgt": "programmatic_tool_calling_claude_api_tool_search", + "source": "skills_over_mcp_office_hours_2460_lazy_loading_pattern", + "target": "programmatic_tool_calling_claude_api_tool_search" + }, + { + "relation": "semantically_similar_to", + "confidence": "INFERRED", + "confidence_score": 0.78, + "source_file": "docs/mcp/long-live-mcp-aqfer.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_office_hours_2460_lazy_loading_pattern", + "_tgt": "long_live_mcp_aqfer_progressive_tool_discovery", + "source": "skills_over_mcp_office_hours_2460_lazy_loading_pattern", + "target": "long_live_mcp_aqfer_progressive_tool_discovery" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.84, + "source_file": "docs/mcp/skills-over-mcp-office-hours-2460.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_office_hours_2460_script_bearing_skills", + "_tgt": "mcp_specification_security_principles", + "source": "skills_over_mcp_office_hours_2460_script_bearing_skills", + "target": "mcp_specification_security_principles" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.74, + "source_file": "docs/mcp/skills-over-mcp-office-hours-2460.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_office_hours_2460_script_bearing_skills", + "_tgt": "mcp_programmatic_tool_calling_opensandbox_sandboxed_execution", + "source": "skills_over_mcp_office_hours_2460_script_bearing_skills", + "target": "mcp_programmatic_tool_calling_opensandbox_sandboxed_execution" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/skills-over-mcp-office-hours-2460.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_office_hours_2460_resource_uris", + "_tgt": "mcp_specification_resources_primitive", + "source": "skills_over_mcp_office_hours_2460_resource_uris", + "target": "mcp_specification_resources_primitive" + }, + { + "relation": "conceptually_related_to", + "confidence": "AMBIGUOUS", + "confidence_score": 0.25, + "source_file": "docs/mcp/skills-over-mcp-office-hours-2460.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_over_mcp_office_hours_2460_resource_uris", + "_tgt": "about_mcp_github_docs_toolsets", + "source": "skills_over_mcp_office_hours_2460_resource_uris", + "target": "about_mcp_github_docs_toolsets" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/mcp-specification.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_specification_json_rpc_roles", + "_tgt": "mcp_specification_resources_primitive", + "source": "mcp_specification_json_rpc_roles", + "target": "mcp_specification_resources_primitive" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/mcp-specification.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_specification_json_rpc_roles", + "_tgt": "mcp_specification_prompts_primitive", + "source": "mcp_specification_json_rpc_roles", + "target": "mcp_specification_prompts_primitive" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/mcp-specification.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_specification_json_rpc_roles", + "_tgt": "mcp_specification_tools_primitive", + "source": "mcp_specification_json_rpc_roles", + "target": "mcp_specification_tools_primitive" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/skills-vs-mcp-speakeasy.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_specification_prompts_primitive", + "_tgt": "skills_vs_mcp_speakeasy_server_prompt_bridge", + "source": "mcp_specification_prompts_primitive", + "target": "skills_vs_mcp_speakeasy_server_prompt_bridge" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/about-mcp-github-docs.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_specification_prompts_primitive", + "_tgt": "about_mcp_github_docs_toolsets", + "source": "mcp_specification_prompts_primitive", + "target": "about_mcp_github_docs_toolsets" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/about-mcp-github-docs.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_specification_resources_primitive", + "_tgt": "about_mcp_github_docs_toolsets", + "source": "mcp_specification_resources_primitive", + "target": "about_mcp_github_docs_toolsets" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/mcp-specification.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_specification_tools_primitive", + "_tgt": "mcp_specification_security_principles", + "source": "mcp_specification_tools_primitive", + "target": "mcp_specification_security_principles" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.77, + "source_file": "docs/mcp/mcp-specification.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_specification_tools_primitive", + "_tgt": "programmatic_tool_calling_claude_api_server_tools", + "source": "mcp_specification_tools_primitive", + "target": "programmatic_tool_calling_claude_api_server_tools" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/about-mcp-github-docs.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_specification_tools_primitive", + "_tgt": "about_mcp_github_docs_toolsets", + "source": "mcp_specification_tools_primitive", + "target": "about_mcp_github_docs_toolsets" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.85, + "source_file": "docs/mcp/about-mcp-github-docs.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_specification_security_principles", + "_tgt": "about_mcp_github_docs_push_protection", + "source": "mcp_specification_security_principles", + "target": "about_mcp_github_docs_push_protection" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.81, + "source_file": "docs/tool-calling/mcp-programmatic-tool-calling-opensandbox-dev.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_specification_security_principles", + "_tgt": "mcp_programmatic_tool_calling_opensandbox_sandboxed_execution", + "source": "mcp_specification_security_principles", + "target": "mcp_programmatic_tool_calling_opensandbox_sandboxed_execution" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/skills-vs-mcp-speakeasy.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_vs_mcp_speakeasy_skills_layer", + "_tgt": "skills_vs_mcp_speakeasy_mcp_layer", + "source": "skills_vs_mcp_speakeasy_skills_layer", + "target": "skills_vs_mcp_speakeasy_mcp_layer" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/skills-vs-mcp-speakeasy.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_vs_mcp_speakeasy_skills_layer", + "_tgt": "skills_vs_mcp_speakeasy_two_layer_architecture", + "source": "skills_vs_mcp_speakeasy_skills_layer", + "target": "skills_vs_mcp_speakeasy_two_layer_architecture" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/skills-vs-mcp-speakeasy.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_vs_mcp_speakeasy_mcp_layer", + "_tgt": "skills_vs_mcp_speakeasy_two_layer_architecture", + "source": "skills_vs_mcp_speakeasy_mcp_layer", + "target": "skills_vs_mcp_speakeasy_two_layer_architecture" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.72, + "source_file": "docs/mcp/skills-vs-mcp-speakeasy.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_vs_mcp_speakeasy_mcp_layer", + "_tgt": "about_mcp_github_docs_github_mcp_server", + "source": "skills_vs_mcp_speakeasy_mcp_layer", + "target": "about_mcp_github_docs_github_mcp_server" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.66, + "source_file": "docs/mcp/skills-vs-mcp-speakeasy.md", + "source_location": null, + "weight": 1.0, + "_src": "skills_vs_mcp_speakeasy_two_layer_architecture", + "_tgt": "programmatic_tool_calling_claude_api_mcp_connector", + "source": "skills_vs_mcp_speakeasy_two_layer_architecture", + "target": "programmatic_tool_calling_claude_api_mcp_connector" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/about-mcp-github-docs.md", + "source_location": null, + "weight": 1.0, + "_src": "about_mcp_github_docs_copilot_surfaces", + "_tgt": "about_mcp_github_docs_github_mcp_server", + "source": "about_mcp_github_docs_copilot_surfaces", + "target": "about_mcp_github_docs_github_mcp_server" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/about-mcp-github-docs.md", + "source_location": null, + "weight": 1.0, + "_src": "about_mcp_github_docs_github_mcp_server", + "_tgt": "about_mcp_github_docs_toolsets", + "source": "about_mcp_github_docs_github_mcp_server", + "target": "about_mcp_github_docs_toolsets" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/about-mcp-github-docs.md", + "source_location": null, + "weight": 1.0, + "_src": "about_mcp_github_docs_github_mcp_server", + "_tgt": "about_mcp_github_docs_push_protection", + "source": "about_mcp_github_docs_github_mcp_server", + "target": "about_mcp_github_docs_push_protection" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.65, + "source_file": "docs/tool-calling/programmatic-tool-calling-claude-api.md", + "source_location": null, + "weight": 1.0, + "_src": "about_mcp_github_docs_github_mcp_server", + "_tgt": "programmatic_tool_calling_claude_api_mcp_connector", + "source": "about_mcp_github_docs_github_mcp_server", + "target": "programmatic_tool_calling_claude_api_mcp_connector" + }, + { + "relation": "semantically_similar_to", + "confidence": "INFERRED", + "confidence_score": 0.86, + "source_file": "docs/mcp/long-live-mcp-aqfer.md", + "source_location": null, + "weight": 1.0, + "_src": "long_live_mcp_aqfer_progressive_tool_discovery", + "_tgt": "tool_search_redefining_agent_tool_calling_epsilla_tool_search", + "source": "long_live_mcp_aqfer_progressive_tool_discovery", + "target": "tool_search_redefining_agent_tool_calling_epsilla_tool_search" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.82, + "source_file": "docs/mcp/long-live-mcp-aqfer.md", + "source_location": null, + "weight": 1.0, + "_src": "long_live_mcp_aqfer_progressive_tool_discovery", + "_tgt": "programmatic_tool_calling_claude_api_tool_search", + "source": "long_live_mcp_aqfer_progressive_tool_discovery", + "target": "programmatic_tool_calling_claude_api_tool_search" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.74, + "source_file": "docs/tool-calling/tool-search-redefining-agent-tool-calling-epsilla.md", + "source_location": null, + "weight": 1.0, + "_src": "long_live_mcp_aqfer_progressive_tool_discovery", + "_tgt": "tool_search_redefining_agent_tool_calling_epsilla_jitr", + "source": "long_live_mcp_aqfer_progressive_tool_discovery", + "target": "tool_search_redefining_agent_tool_calling_epsilla_jitr" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/mcp/long-live-mcp-aqfer.md", + "source_location": null, + "weight": 1.0, + "_src": "long_live_mcp_aqfer_cimd", + "_tgt": "long_live_mcp_aqfer_xaa", + "source": "long_live_mcp_aqfer_cimd", + "target": "long_live_mcp_aqfer_xaa" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-acl.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_acl_paper", + "_tgt": "lost_in_the_middle_acl_positional_bias", + "source": "lost_in_the_middle_acl_paper", + "target": "lost_in_the_middle_acl_positional_bias" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-acl.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_acl_paper", + "_tgt": "lost_in_the_middle_acl_u_shaped_curve", + "source": "lost_in_the_middle_acl_paper", + "target": "lost_in_the_middle_acl_u_shaped_curve" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-acl.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_acl_paper", + "_tgt": "lost_in_the_middle_acl_eval_protocols", + "source": "lost_in_the_middle_acl_paper", + "target": "lost_in_the_middle_acl_eval_protocols" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_acl_paper", + "_tgt": "lost_in_the_middle_arxiv_preprint", + "source": "lost_in_the_middle_acl_paper", + "target": "lost_in_the_middle_arxiv_preprint" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-arxiv.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_acl_paper", + "_tgt": "lost_in_the_middle_arxiv_publication_link", + "source": "lost_in_the_middle_acl_paper", + "target": "lost_in_the_middle_arxiv_publication_link" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-researchgate.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_acl_paper", + "_tgt": "lost_in_the_middle_researchgate_reference", + "source": "lost_in_the_middle_acl_paper", + "target": "lost_in_the_middle_researchgate_reference" + }, + { + "relation": "cites", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/README.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_acl_paper", + "_tgt": "long_context_research_readme_collection", + "source": "lost_in_the_middle_acl_paper", + "target": "long_context_research_readme_collection" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-acl.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_acl_positional_bias", + "_tgt": "lost_in_the_middle_acl_u_shaped_curve", + "source": "lost_in_the_middle_acl_positional_bias", + "target": "lost_in_the_middle_acl_u_shaped_curve" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.85, + "source_file": "docs/claude-code/claude-prompting-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_acl_positional_bias", + "_tgt": "claude_prompting_best_practices_long_context_prompting", + "source": "lost_in_the_middle_acl_positional_bias", + "target": "claude_prompting_best_practices_long_context_prompting" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-researchgate.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_arxiv_preprint", + "_tgt": "lost_in_the_middle_researchgate_reference", + "source": "lost_in_the_middle_arxiv_preprint", + "target": "lost_in_the_middle_researchgate_reference" + }, + { + "relation": "cites", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/README.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_arxiv_preprint", + "_tgt": "long_context_research_readme_collection", + "source": "lost_in_the_middle_arxiv_preprint", + "target": "long_context_research_readme_collection" + }, + { + "relation": "cites", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/README.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_researchgate_reference", + "_tgt": "long_context_research_readme_collection", + "source": "lost_in_the_middle_researchgate_reference", + "target": "long_context_research_readme_collection" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-researchgate.md", + "source_location": null, + "weight": 1.0, + "_src": "lost_in_the_middle_researchgate_reference", + "_tgt": "lost_in_the_middle_researchgate_access_note", + "source": "lost_in_the_middle_researchgate_reference", + "target": "lost_in_the_middle_researchgate_access_note" + }, + { + "relation": "cites", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/README.md", + "source_location": null, + "weight": 1.0, + "_src": "long_context_research_readme_collection", + "_tgt": "long_context_research_readme_multihop_followup", + "source": "long_context_research_readme_collection", + "target": "long_context_research_readme_multihop_followup" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/README.md", + "source_location": null, + "weight": 1.0, + "_src": "long_context_research_readme_multihop_followup", + "_tgt": "long_context_research_readme_kg_extraction_fragility", + "source": "long_context_research_readme_multihop_followup", + "target": "long_context_research_readme_kg_extraction_fragility" + }, + { + "relation": "rationale_for", + "confidence": "INFERRED", + "confidence_score": 0.81, + "source_file": "docs/long-context-research/README.md", + "source_location": null, + "weight": 1.0, + "_src": "long_context_research_readme_rag_edge_placement", + "_tgt": "claude_prompting_best_practices_long_context_prompting", + "source": "long_context_research_readme_rag_edge_placement", + "target": "claude_prompting_best_practices_long_context_prompting" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.7, + "source_file": "docs/claude-code/claude-prompting-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "long_context_research_readme_rag_edge_placement", + "_tgt": "claude_prompting_best_practices_quote_grounding", + "source": "long_context_research_readme_rag_edge_placement", + "target": "claude_prompting_best_practices_quote_grounding" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/mcp-programmatic-tool-calling-opensandbox-dev.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_programmatic_tool_calling_opensandbox_tool_search", + "_tgt": "mcp_programmatic_tool_calling_opensandbox_code_mode", + "source": "mcp_programmatic_tool_calling_opensandbox_tool_search", + "target": "mcp_programmatic_tool_calling_opensandbox_code_mode" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/mcp-programmatic-tool-calling-opensandbox-dev.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_programmatic_tool_calling_opensandbox_tool_search", + "_tgt": "programmatic_tool_calling_claude_api_tool_search", + "source": "mcp_programmatic_tool_calling_opensandbox_tool_search", + "target": "programmatic_tool_calling_claude_api_tool_search" + }, + { + "relation": "semantically_similar_to", + "confidence": "INFERRED", + "confidence_score": 0.87, + "source_file": "docs/tool-calling/mcp-programmatic-tool-calling-opensandbox-dev.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_programmatic_tool_calling_opensandbox_tool_search", + "_tgt": "tool_search_redefining_agent_tool_calling_epsilla_tool_search", + "source": "mcp_programmatic_tool_calling_opensandbox_tool_search", + "target": "tool_search_redefining_agent_tool_calling_epsilla_tool_search" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/mcp-programmatic-tool-calling-opensandbox-dev.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_programmatic_tool_calling_opensandbox_code_mode", + "_tgt": "mcp_programmatic_tool_calling_opensandbox_search_getschema_execute", + "source": "mcp_programmatic_tool_calling_opensandbox_code_mode", + "target": "mcp_programmatic_tool_calling_opensandbox_search_getschema_execute" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/mcp-programmatic-tool-calling-opensandbox-dev.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_programmatic_tool_calling_opensandbox_code_mode", + "_tgt": "programmatic_tool_calling_claude_api_code_execution", + "source": "mcp_programmatic_tool_calling_opensandbox_code_mode", + "target": "programmatic_tool_calling_claude_api_code_execution" + }, + { + "relation": "semantically_similar_to", + "confidence": "INFERRED", + "confidence_score": 0.84, + "source_file": "docs/tool-calling/cameronking4-programmatic-tool-calling-github.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_programmatic_tool_calling_opensandbox_code_mode", + "_tgt": "cameronking4_programmatic_tool_calling_programmatic_tool_calling", + "source": "mcp_programmatic_tool_calling_opensandbox_code_mode", + "target": "cameronking4_programmatic_tool_calling_programmatic_tool_calling" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/mcp-programmatic-tool-calling-opensandbox-dev.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_programmatic_tool_calling_opensandbox_sandboxed_execution", + "_tgt": "mcp_programmatic_tool_calling_opensandbox_sandbox_rationale", + "source": "mcp_programmatic_tool_calling_opensandbox_sandboxed_execution", + "target": "mcp_programmatic_tool_calling_opensandbox_sandbox_rationale" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.7, + "source_file": "docs/tool-calling/mcp-programmatic-tool-calling-opensandbox-dev.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_programmatic_tool_calling_opensandbox_search_getschema_execute", + "_tgt": "tool_search_redefining_agent_tool_calling_epsilla_jitr", + "source": "mcp_programmatic_tool_calling_opensandbox_search_getschema_execute", + "target": "tool_search_redefining_agent_tool_calling_epsilla_jitr" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.68, + "source_file": "docs/tool-calling/mcp-programmatic-tool-calling-opensandbox-dev.md", + "source_location": null, + "weight": 1.0, + "_src": "mcp_programmatic_tool_calling_opensandbox_search_getschema_execute", + "_tgt": "cameronking4_programmatic_tool_calling_code_execution_metatool", + "source": "mcp_programmatic_tool_calling_opensandbox_search_getschema_execute", + "target": "cameronking4_programmatic_tool_calling_code_execution_metatool" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/cameronking4-programmatic-tool-calling-github.md", + "source_location": null, + "weight": 1.0, + "_src": "cameronking4_programmatic_tool_calling_programmatic_tool_calling", + "_tgt": "cameronking4_programmatic_tool_calling_code_execution_metatool", + "source": "cameronking4_programmatic_tool_calling_programmatic_tool_calling", + "target": "cameronking4_programmatic_tool_calling_code_execution_metatool" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/cameronking4-programmatic-tool-calling-github.md", + "source_location": null, + "weight": 1.0, + "_src": "cameronking4_programmatic_tool_calling_programmatic_tool_calling", + "_tgt": "cameronking4_programmatic_tool_calling_mcp_bridge", + "source": "cameronking4_programmatic_tool_calling_programmatic_tool_calling", + "target": "cameronking4_programmatic_tool_calling_mcp_bridge" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.79, + "source_file": "docs/tool-calling/programmatic-tool-calling-claude-api.md", + "source_location": null, + "weight": 1.0, + "_src": "cameronking4_programmatic_tool_calling_programmatic_tool_calling", + "_tgt": "programmatic_tool_calling_claude_api_code_execution", + "source": "cameronking4_programmatic_tool_calling_programmatic_tool_calling", + "target": "programmatic_tool_calling_claude_api_code_execution" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/cameronking4-programmatic-tool-calling-github.md", + "source_location": null, + "weight": 1.0, + "_src": "cameronking4_programmatic_tool_calling_code_execution_metatool", + "_tgt": "programmatic_tool_calling_claude_api_code_execution", + "source": "cameronking4_programmatic_tool_calling_code_execution_metatool", + "target": "programmatic_tool_calling_claude_api_code_execution" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.76, + "source_file": "docs/tool-calling/cameronking4-programmatic-tool-calling-github.md", + "source_location": null, + "weight": 1.0, + "_src": "cameronking4_programmatic_tool_calling_mcp_bridge", + "_tgt": "programmatic_tool_calling_claude_api_mcp_connector", + "source": "cameronking4_programmatic_tool_calling_mcp_bridge", + "target": "programmatic_tool_calling_claude_api_mcp_connector" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/tool-search-redefining-agent-tool-calling-epsilla.md", + "source_location": null, + "weight": 1.0, + "_src": "tool_search_redefining_agent_tool_calling_epsilla_tool_search", + "_tgt": "tool_search_redefining_agent_tool_calling_epsilla_jitr", + "source": "tool_search_redefining_agent_tool_calling_epsilla_tool_search", + "target": "tool_search_redefining_agent_tool_calling_epsilla_jitr" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/tool-search-redefining-agent-tool-calling-epsilla.md", + "source_location": null, + "weight": 1.0, + "_src": "tool_search_redefining_agent_tool_calling_epsilla_tool_search", + "_tgt": "tool_search_redefining_agent_tool_calling_epsilla_prompt_caching_stability", + "source": "tool_search_redefining_agent_tool_calling_epsilla_tool_search", + "target": "tool_search_redefining_agent_tool_calling_epsilla_prompt_caching_stability" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/tool-search-redefining-agent-tool-calling-epsilla.md", + "source_location": null, + "weight": 1.0, + "_src": "tool_search_redefining_agent_tool_calling_epsilla_tool_search", + "_tgt": "programmatic_tool_calling_claude_api_tool_search", + "source": "tool_search_redefining_agent_tool_calling_epsilla_tool_search", + "target": "programmatic_tool_calling_claude_api_tool_search" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/programmatic-tool-calling-claude-api.md", + "source_location": null, + "weight": 1.0, + "_src": "programmatic_tool_calling_claude_api_server_tools", + "_tgt": "programmatic_tool_calling_claude_api_tool_search", + "source": "programmatic_tool_calling_claude_api_server_tools", + "target": "programmatic_tool_calling_claude_api_tool_search" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/programmatic-tool-calling-claude-api.md", + "source_location": null, + "weight": 1.0, + "_src": "programmatic_tool_calling_claude_api_server_tools", + "_tgt": "programmatic_tool_calling_claude_api_code_execution", + "source": "programmatic_tool_calling_claude_api_server_tools", + "target": "programmatic_tool_calling_claude_api_code_execution" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/programmatic-tool-calling-claude-api.md", + "source_location": null, + "weight": 1.0, + "_src": "programmatic_tool_calling_claude_api_server_tools", + "_tgt": "programmatic_tool_calling_claude_api_mcp_connector", + "source": "programmatic_tool_calling_claude_api_server_tools", + "target": "programmatic_tool_calling_claude_api_mcp_connector" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/tool-calling/programmatic-tool-calling-claude-api.md", + "source_location": null, + "weight": 1.0, + "_src": "programmatic_tool_calling_claude_api_tool_search", + "_tgt": "programmatic_tool_calling_claude_api_code_execution", + "source": "programmatic_tool_calling_claude_api_tool_search", + "target": "programmatic_tool_calling_claude_api_code_execution" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-implement-tool-use.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_implement_tool_use_tool_definition_quality", + "_tgt": "anthropic_implement_tool_use_input_examples", + "source": "anthropic_implement_tool_use_tool_definition_quality", + "target": "anthropic_implement_tool_use_input_examples" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-implement-tool-use.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_implement_tool_use_tool_definition_quality", + "_tgt": "anthropic_implement_tool_use_strict_tool_choice", + "source": "anthropic_implement_tool_use_tool_definition_quality", + "target": "anthropic_implement_tool_use_strict_tool_choice" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.71, + "source_file": "docs/structured-outputs/anthropic-implement-tool-use.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_implement_tool_use_input_examples", + "_tgt": "anthropic_increase_consistency_prompt_techniques", + "source": "anthropic_implement_tool_use_input_examples", + "target": "anthropic_increase_consistency_prompt_techniques" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/README.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_implement_tool_use_strict_tool_choice", + "_tgt": "structured_outputs_readme_strict_tool_use", + "source": "anthropic_implement_tool_use_strict_tool_choice", + "target": "structured_outputs_readme_strict_tool_use" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/README.md", + "source_location": null, + "weight": 1.0, + "_src": "structured_outputs_readme_structured_outputs", + "_tgt": "structured_outputs_readme_cfg_decoding", + "source": "structured_outputs_readme_structured_outputs", + "target": "structured_outputs_readme_cfg_decoding" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/README.md", + "source_location": null, + "weight": 1.0, + "_src": "structured_outputs_readme_structured_outputs", + "_tgt": "structured_outputs_readme_strict_tool_use", + "source": "structured_outputs_readme_structured_outputs", + "target": "structured_outputs_readme_strict_tool_use" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-increase-consistency.md", + "source_location": null, + "weight": 1.0, + "_src": "structured_outputs_readme_structured_outputs", + "_tgt": "anthropic_increase_consistency_schema_guarantee_rationale", + "source": "structured_outputs_readme_structured_outputs", + "target": "anthropic_increase_consistency_schema_guarantee_rationale" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.62, + "source_file": "docs/claude-code/claude-prompting-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "structured_outputs_readme_structured_outputs", + "_tgt": "claude_prompting_best_practices_xml_document_structure", + "source": "structured_outputs_readme_structured_outputs", + "target": "claude_prompting_best_practices_xml_document_structure" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-increase-consistency.md", + "source_location": null, + "weight": 1.0, + "_src": "structured_outputs_readme_strict_tool_use", + "_tgt": "anthropic_increase_consistency_schema_guarantee_rationale", + "source": "structured_outputs_readme_strict_tool_use", + "target": "anthropic_increase_consistency_schema_guarantee_rationale" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-increase-consistency.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_increase_consistency_prompt_techniques", + "_tgt": "anthropic_increase_consistency_retrieval_grounding", + "source": "anthropic_increase_consistency_prompt_techniques", + "target": "anthropic_increase_consistency_retrieval_grounding" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-increase-consistency.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_increase_consistency_prompt_techniques", + "_tgt": "anthropic_increase_consistency_prompt_chaining", + "source": "anthropic_increase_consistency_prompt_techniques", + "target": "anthropic_increase_consistency_prompt_chaining" + }, + { + "relation": "semantically_similar_to", + "confidence": "INFERRED", + "confidence_score": 0.77, + "source_file": "docs/claude-code/claude-prompting-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "anthropic_increase_consistency_retrieval_grounding", + "_tgt": "claude_prompting_best_practices_quote_grounding", + "source": "anthropic_increase_consistency_retrieval_grounding", + "target": "claude_prompting_best_practices_quote_grounding" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/hooks/claude-hook-reference-doc.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_hook_reference_doc_hook_lifecycle", + "_tgt": "claude_hook_reference_doc_decision_control", + "source": "claude_hook_reference_doc_hook_lifecycle", + "target": "claude_hook_reference_doc_decision_control" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/hooks/claude-hook-reference-doc.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_hook_reference_doc_hook_lifecycle", + "_tgt": "claude_hook_reference_doc_subagent_team_hooks", + "source": "claude_hook_reference_doc_hook_lifecycle", + "target": "claude_hook_reference_doc_subagent_team_hooks" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/hooks/automate-workflow-with-hooks.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_hook_reference_doc_hook_lifecycle", + "_tgt": "automate_workflow_with_hooks_deterministic_automation", + "source": "claude_hook_reference_doc_hook_lifecycle", + "target": "automate_workflow_with_hooks_deterministic_automation" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/skills/extend-claude-with-skills.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_hook_reference_doc_hook_lifecycle", + "_tgt": "extend_claude_with_skills_skill_frontmatter", + "source": "claude_hook_reference_doc_hook_lifecycle", + "target": "extend_claude_with_skills_skill_frontmatter" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/README.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_hook_reference_doc_hook_lifecycle", + "_tgt": "docs_readme_scope_partitioning", + "source": "claude_hook_reference_doc_hook_lifecycle", + "target": "docs_readme_scope_partitioning" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/hooks/automate-workflow-with-hooks.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_hook_reference_doc_decision_control", + "_tgt": "automate_workflow_with_hooks_hook_type_selection", + "source": "claude_hook_reference_doc_decision_control", + "target": "automate_workflow_with_hooks_hook_type_selection" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.84, + "source_file": "docs/claude-code/hooks/claude-hook-reference-doc.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_hook_reference_doc_subagent_team_hooks", + "_tgt": "creating_custom_subagents_custom_subagents", + "source": "claude_hook_reference_doc_subagent_team_hooks", + "target": "creating_custom_subagents_custom_subagents" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.77, + "source_file": "docs/claude-code/subagents/claude-orchestrate-of-claude-code-sessions.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_hook_reference_doc_subagent_team_hooks", + "_tgt": "claude_orchestrate_of_claude_code_sessions_agent_teams", + "source": "claude_hook_reference_doc_subagent_team_hooks", + "target": "claude_orchestrate_of_claude_code_sessions_agent_teams" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/subagents/creating-custom-subagents.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_hook_reference_doc_subagent_team_hooks", + "_tgt": "creating_custom_subagents_frontmatter_controls", + "source": "claude_hook_reference_doc_subagent_team_hooks", + "target": "creating_custom_subagents_frontmatter_controls" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/hooks/automate-workflow-with-hooks.md", + "source_location": null, + "weight": 1.0, + "_src": "automate_workflow_with_hooks_deterministic_automation", + "_tgt": "automate_workflow_with_hooks_hook_type_selection", + "source": "automate_workflow_with_hooks_deterministic_automation", + "target": "automate_workflow_with_hooks_hook_type_selection" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.65, + "source_file": "docs/claude-code/hooks/automate-workflow-with-hooks.md", + "source_location": null, + "weight": 1.0, + "_src": "automate_workflow_with_hooks_deterministic_automation", + "_tgt": "extend_claude_with_skills_reference_vs_task_content", + "source": "automate_workflow_with_hooks_deterministic_automation", + "target": "extend_claude_with_skills_reference_vs_task_content" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/skills/extend-claude-with-skills.md", + "source_location": null, + "weight": 1.0, + "_src": "extend_claude_with_skills_reference_vs_task_content", + "_tgt": "extend_claude_with_skills_skill_frontmatter", + "source": "extend_claude_with_skills_reference_vs_task_content", + "target": "extend_claude_with_skills_skill_frontmatter" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/skills/extend-claude-with-skills.md", + "source_location": null, + "weight": 1.0, + "_src": "extend_claude_with_skills_reference_vs_task_content", + "_tgt": "extend_claude_with_skills_context_fork", + "source": "extend_claude_with_skills_reference_vs_task_content", + "target": "extend_claude_with_skills_context_fork" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/README.md", + "source_location": null, + "weight": 1.0, + "_src": "extend_claude_with_skills_reference_vs_task_content", + "_tgt": "docs_readme_scope_partitioning", + "source": "extend_claude_with_skills_reference_vs_task_content", + "target": "docs_readme_scope_partitioning" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.69, + "source_file": "docs/claude-code/skills/extend-claude-with-skills.md", + "source_location": null, + "weight": 1.0, + "_src": "extend_claude_with_skills_skill_frontmatter", + "_tgt": "research_claude_code_skills_format_three_layer_extensibility", + "source": "extend_claude_with_skills_skill_frontmatter", + "target": "research_claude_code_skills_format_three_layer_extensibility" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.67, + "source_file": "docs/claude-code/skills/research-claude-code-skills-format.md", + "source_location": null, + "weight": 1.0, + "_src": "extend_claude_with_skills_skill_frontmatter", + "_tgt": "research_claude_code_skills_format_marketplace_format", + "source": "extend_claude_with_skills_skill_frontmatter", + "target": "research_claude_code_skills_format_marketplace_format" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/skills/extend-claude-with-skills.md", + "source_location": null, + "weight": 1.0, + "_src": "extend_claude_with_skills_context_fork", + "_tgt": "creating_custom_subagents_custom_subagents", + "source": "extend_claude_with_skills_context_fork", + "target": "creating_custom_subagents_custom_subagents" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/subagents/creating-custom-subagents.md", + "source_location": null, + "weight": 1.0, + "_src": "extend_claude_with_skills_context_fork", + "_tgt": "creating_custom_subagents_skills_preloading", + "source": "extend_claude_with_skills_context_fork", + "target": "creating_custom_subagents_skills_preloading" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.63, + "source_file": "docs/claude-code/subagents/claude-orchestrate-of-claude-code-sessions.md", + "source_location": null, + "weight": 1.0, + "_src": "extend_claude_with_skills_context_fork", + "_tgt": "claude_orchestrate_of_claude_code_sessions_subagents_vs_teams", + "source": "extend_claude_with_skills_context_fork", + "target": "claude_orchestrate_of_claude_code_sessions_subagents_vs_teams" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.7, + "source_file": "docs/claude-code/subagents/creating-custom-subagents.md", + "source_location": null, + "weight": 1.0, + "_src": "extend_claude_with_skills_context_fork", + "_tgt": "creating_custom_subagents_frontmatter_controls", + "source": "extend_claude_with_skills_context_fork", + "target": "creating_custom_subagents_frontmatter_controls" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/skills/research-claude-code-skills-format.md", + "source_location": null, + "weight": 1.0, + "_src": "research_claude_code_skills_format_three_layer_extensibility", + "_tgt": "research_claude_code_skills_format_progressive_disclosure", + "source": "research_claude_code_skills_format_three_layer_extensibility", + "target": "research_claude_code_skills_format_progressive_disclosure" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/skills/research-claude-code-skills-format.md", + "source_location": null, + "weight": 1.0, + "_src": "research_claude_code_skills_format_three_layer_extensibility", + "_tgt": "research_claude_code_skills_format_marketplace_format", + "source": "research_claude_code_skills_format_three_layer_extensibility", + "target": "research_claude_code_skills_format_marketplace_format" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.8, + "source_file": "docs/README.md", + "source_location": null, + "weight": 1.0, + "_src": "research_claude_code_skills_format_progressive_disclosure", + "_tgt": "docs_readme_scope_partitioning", + "source": "research_claude_code_skills_format_progressive_disclosure", + "target": "docs_readme_scope_partitioning" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.58, + "source_file": "docs/claude-code/subagents/creating-custom-subagents.md", + "source_location": null, + "weight": 1.0, + "_src": "research_claude_code_skills_format_marketplace_format", + "_tgt": "creating_custom_subagents_custom_subagents", + "source": "research_claude_code_skills_format_marketplace_format", + "target": "creating_custom_subagents_custom_subagents" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/subagents/claude-orchestrate-of-claude-code-sessions.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_orchestrate_of_claude_code_sessions_agent_teams", + "_tgt": "claude_orchestrate_of_claude_code_sessions_subagents_vs_teams", + "source": "claude_orchestrate_of_claude_code_sessions_agent_teams", + "target": "claude_orchestrate_of_claude_code_sessions_subagents_vs_teams" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/subagents/claude-orchestrate-of-claude-code-sessions.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_orchestrate_of_claude_code_sessions_subagents_vs_teams", + "_tgt": "creating_custom_subagents_custom_subagents", + "source": "claude_orchestrate_of_claude_code_sessions_subagents_vs_teams", + "target": "creating_custom_subagents_custom_subagents" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/subagents/creating-custom-subagents.md", + "source_location": null, + "weight": 1.0, + "_src": "creating_custom_subagents_custom_subagents", + "_tgt": "creating_custom_subagents_frontmatter_controls", + "source": "creating_custom_subagents_custom_subagents", + "target": "creating_custom_subagents_frontmatter_controls" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/subagents/creating-custom-subagents.md", + "source_location": null, + "weight": 1.0, + "_src": "creating_custom_subagents_custom_subagents", + "_tgt": "creating_custom_subagents_skills_preloading", + "source": "creating_custom_subagents_custom_subagents", + "target": "creating_custom_subagents_skills_preloading" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/README.md", + "source_location": null, + "weight": 1.0, + "_src": "creating_custom_subagents_custom_subagents", + "_tgt": "docs_readme_scope_partitioning", + "source": "creating_custom_subagents_custom_subagents", + "target": "docs_readme_scope_partitioning" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/README.md", + "source_location": null, + "weight": 1.0, + "_src": "docs_readme_scope_partitioning", + "_tgt": "docs_readme_task_reading_order", + "source": "docs_readme_scope_partitioning", + "target": "docs_readme_task_reading_order" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/README.md", + "source_location": null, + "weight": 1.0, + "_src": "docs_readme_task_reading_order", + "_tgt": "claude_prompting_best_practices_long_context_prompting", + "source": "docs_readme_task_reading_order", + "target": "claude_prompting_best_practices_long_context_prompting" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/claude-prompting-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_prompting_best_practices_long_context_prompting", + "_tgt": "claude_prompting_best_practices_xml_document_structure", + "source": "claude_prompting_best_practices_long_context_prompting", + "target": "claude_prompting_best_practices_xml_document_structure" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/claude-prompting-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_prompting_best_practices_long_context_prompting", + "_tgt": "claude_prompting_best_practices_quote_grounding", + "source": "claude_prompting_best_practices_long_context_prompting", + "target": "claude_prompting_best_practices_quote_grounding" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "weight": 1.0, + "_src": "how_claude_remembers_a_project_doc", + "_tgt": "how_claude_remembers_a_project_claude_md_files", + "source": "how_claude_remembers_a_project_doc", + "target": "how_claude_remembers_a_project_claude_md_files" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "weight": 1.0, + "_src": "how_claude_remembers_a_project_doc", + "_tgt": "how_claude_remembers_a_project_auto_memory", + "source": "how_claude_remembers_a_project_doc", + "target": "how_claude_remembers_a_project_auto_memory" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "weight": 1.0, + "_src": "how_claude_remembers_a_project_doc", + "_tgt": "how_claude_remembers_a_project_claude_rules", + "source": "how_claude_remembers_a_project_doc", + "target": "how_claude_remembers_a_project_claude_rules" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "weight": 1.0, + "_src": "how_claude_remembers_a_project_doc", + "_tgt": "how_claude_remembers_a_project_memory_command", + "source": "how_claude_remembers_a_project_doc", + "target": "how_claude_remembers_a_project_memory_command" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "weight": 1.0, + "_src": "how_claude_remembers_a_project_claude_md_files", + "_tgt": "how_claude_remembers_a_project_managed_claude_md", + "source": "how_claude_remembers_a_project_claude_md_files", + "target": "how_claude_remembers_a_project_managed_claude_md" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "weight": 1.0, + "_src": "how_claude_remembers_a_project_claude_md_files", + "_tgt": "how_claude_remembers_a_project_claude_md_excludes", + "source": "how_claude_remembers_a_project_claude_md_files", + "target": "how_claude_remembers_a_project_claude_md_excludes" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "weight": 1.0, + "_src": "how_claude_remembers_a_project_claude_md_files", + "_tgt": "how_claude_remembers_a_project_concise_specific_instructions", + "source": "how_claude_remembers_a_project_claude_md_files", + "target": "how_claude_remembers_a_project_concise_specific_instructions" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "how_claude_remembers_a_project_claude_md_files", + "_tgt": "research_subagent_best_practices_subagents", + "source": "how_claude_remembers_a_project_claude_md_files", + "target": "research_subagent_best_practices_subagents" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "weight": 1.0, + "_src": "how_claude_remembers_a_project_auto_memory", + "_tgt": "how_claude_remembers_a_project_memory_md_entrypoint", + "source": "how_claude_remembers_a_project_auto_memory", + "target": "how_claude_remembers_a_project_memory_md_entrypoint" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "weight": 1.0, + "_src": "how_claude_remembers_a_project_auto_memory", + "_tgt": "how_claude_remembers_a_project_memory_command", + "source": "how_claude_remembers_a_project_auto_memory", + "target": "how_claude_remembers_a_project_memory_command" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.86, + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "weight": 1.0, + "_src": "how_claude_remembers_a_project_auto_memory", + "_tgt": "research_subagent_best_practices_persistent_memory", + "source": "how_claude_remembers_a_project_auto_memory", + "target": "research_subagent_best_practices_persistent_memory" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "weight": 1.0, + "_src": "how_claude_remembers_a_project_claude_rules", + "_tgt": "how_claude_remembers_a_project_concise_specific_instructions", + "source": "how_claude_remembers_a_project_claude_rules", + "target": "how_claude_remembers_a_project_concise_specific_instructions" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "weight": 1.0, + "_src": "how_claude_remembers_a_project_claude_rules", + "_tgt": "how_claude_remembers_a_project_context_noise_reduction", + "source": "how_claude_remembers_a_project_claude_rules", + "target": "how_claude_remembers_a_project_context_noise_reduction" + }, + { + "relation": "semantically_similar_to", + "confidence": "INFERRED", + "confidence_score": 0.68, + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "weight": 1.0, + "_src": "how_claude_remembers_a_project_claude_rules", + "_tgt": "skill_authoring_best_practices_progressive_disclosure", + "source": "how_claude_remembers_a_project_claude_rules", + "target": "skill_authoring_best_practices_progressive_disclosure" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "weight": 1.0, + "_src": "how_claude_remembers_a_project_claude_md_excludes", + "_tgt": "how_claude_remembers_a_project_context_noise_reduction", + "source": "how_claude_remembers_a_project_claude_md_excludes", + "target": "how_claude_remembers_a_project_context_noise_reduction" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.84, + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md", + "source_location": null, + "weight": 1.0, + "_src": "how_claude_remembers_a_project_concise_specific_instructions", + "_tgt": "skill_authoring_best_practices_context_window_efficiency", + "source": "how_claude_remembers_a_project_concise_specific_instructions", + "target": "skill_authoring_best_practices_context_window_efficiency" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_create_plugin_doc_doc", + "_tgt": "claude_create_plugin_doc_plugin_manifest", + "source": "claude_create_plugin_doc_doc", + "target": "claude_create_plugin_doc_plugin_manifest" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_create_plugin_doc_doc", + "_tgt": "claude_create_plugin_doc_plugin_skills_directory", + "source": "claude_create_plugin_doc_doc", + "target": "claude_create_plugin_doc_plugin_skills_directory" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_create_plugin_doc_doc", + "_tgt": "claude_create_plugin_doc_skill_namespacing", + "source": "claude_create_plugin_doc_doc", + "target": "claude_create_plugin_doc_skill_namespacing" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_create_plugin_doc_doc", + "_tgt": "claude_create_plugin_doc_plugin_root_structure", + "source": "claude_create_plugin_doc_doc", + "target": "claude_create_plugin_doc_plugin_root_structure" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_create_plugin_doc_doc", + "_tgt": "claude_create_plugin_doc_plugin_dir_flag", + "source": "claude_create_plugin_doc_doc", + "target": "claude_create_plugin_doc_plugin_dir_flag" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_create_plugin_doc_doc", + "_tgt": "claude_create_plugin_doc_reload_plugins", + "source": "claude_create_plugin_doc_doc", + "target": "claude_create_plugin_doc_reload_plugins" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_create_plugin_doc_doc", + "_tgt": "claude_create_plugin_doc_plugin_settings_json", + "source": "claude_create_plugin_doc_doc", + "target": "claude_create_plugin_doc_plugin_settings_json" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_create_plugin_doc_doc", + "_tgt": "claude_create_plugin_doc_plugin_migration", + "source": "claude_create_plugin_doc_doc", + "target": "claude_create_plugin_doc_plugin_migration" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_create_plugin_doc_plugin_manifest", + "_tgt": "claude_create_plugin_doc_sharing_and_reuse", + "source": "claude_create_plugin_doc_plugin_manifest", + "target": "claude_create_plugin_doc_sharing_and_reuse" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_create_plugin_doc_plugin_manifest", + "_tgt": "claude_create_plugin_doc_plugin_root_structure", + "source": "claude_create_plugin_doc_plugin_manifest", + "target": "claude_create_plugin_doc_plugin_root_structure" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.77, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_create_plugin_doc_plugin_manifest", + "_tgt": "research_subagent_best_practices_plugin_shipped_agents", + "source": "claude_create_plugin_doc_plugin_manifest", + "target": "research_subagent_best_practices_plugin_shipped_agents" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.89, + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_create_plugin_doc_plugin_skills_directory", + "_tgt": "skill_authoring_best_practices_yaml_frontmatter", + "source": "claude_create_plugin_doc_plugin_skills_directory", + "target": "skill_authoring_best_practices_yaml_frontmatter" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_create_plugin_doc_skill_namespacing", + "_tgt": "claude_create_plugin_doc_conflict_prevention", + "source": "claude_create_plugin_doc_skill_namespacing", + "target": "claude_create_plugin_doc_conflict_prevention" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_create_plugin_doc_plugin_root_structure", + "_tgt": "research_subagent_best_practices_plugin_shipped_agents", + "source": "claude_create_plugin_doc_plugin_root_structure", + "target": "research_subagent_best_practices_plugin_shipped_agents" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.9, + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_create_plugin_doc_plugin_dir_flag", + "_tgt": "claude_create_plugin_doc_reload_plugins", + "source": "claude_create_plugin_doc_plugin_dir_flag", + "target": "claude_create_plugin_doc_reload_plugins" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.78, + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_create_plugin_doc_plugin_settings_json", + "_tgt": "research_subagent_best_practices_plugin_shipped_agents", + "source": "claude_create_plugin_doc_plugin_settings_json", + "target": "research_subagent_best_practices_plugin_shipped_agents" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md", + "source_location": null, + "weight": 1.0, + "_src": "claude_create_plugin_doc_plugin_migration", + "_tgt": "claude_create_plugin_doc_sharing_and_reuse", + "source": "claude_create_plugin_doc_plugin_migration", + "target": "claude_create_plugin_doc_sharing_and_reuse" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "skill_authoring_best_practices_doc", + "_tgt": "skill_authoring_best_practices_yaml_frontmatter", + "source": "skill_authoring_best_practices_doc", + "target": "skill_authoring_best_practices_yaml_frontmatter" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "skill_authoring_best_practices_doc", + "_tgt": "skill_authoring_best_practices_description_field", + "source": "skill_authoring_best_practices_doc", + "target": "skill_authoring_best_practices_description_field" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "skill_authoring_best_practices_doc", + "_tgt": "skill_authoring_best_practices_progressive_disclosure", + "source": "skill_authoring_best_practices_doc", + "target": "skill_authoring_best_practices_progressive_disclosure" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "skill_authoring_best_practices_doc", + "_tgt": "skill_authoring_best_practices_workflow_checklists", + "source": "skill_authoring_best_practices_doc", + "target": "skill_authoring_best_practices_workflow_checklists" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "skill_authoring_best_practices_doc", + "_tgt": "skill_authoring_best_practices_feedback_loops", + "source": "skill_authoring_best_practices_doc", + "target": "skill_authoring_best_practices_feedback_loops" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "skill_authoring_best_practices_doc", + "_tgt": "skill_authoring_best_practices_evaluation_driven_development", + "source": "skill_authoring_best_practices_doc", + "target": "skill_authoring_best_practices_evaluation_driven_development" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "skill_authoring_best_practices_doc", + "_tgt": "skill_authoring_best_practices_utility_scripts", + "source": "skill_authoring_best_practices_doc", + "target": "skill_authoring_best_practices_utility_scripts" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "skill_authoring_best_practices_doc", + "_tgt": "skill_authoring_best_practices_runtime_environment", + "source": "skill_authoring_best_practices_doc", + "target": "skill_authoring_best_practices_runtime_environment" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "skill_authoring_best_practices_yaml_frontmatter", + "_tgt": "skill_authoring_best_practices_description_field", + "source": "skill_authoring_best_practices_yaml_frontmatter", + "target": "skill_authoring_best_practices_description_field" + }, + { + "relation": "conceptually_related_to", + "confidence": "INFERRED", + "confidence_score": 0.86, + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "skill_authoring_best_practices_description_field", + "_tgt": "research_subagent_best_practices_actionable_descriptions", + "source": "skill_authoring_best_practices_description_field", + "target": "research_subagent_best_practices_actionable_descriptions" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "skill_authoring_best_practices_progressive_disclosure", + "_tgt": "skill_authoring_best_practices_context_window_efficiency", + "source": "skill_authoring_best_practices_progressive_disclosure", + "target": "skill_authoring_best_practices_context_window_efficiency" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "skill_authoring_best_practices_progressive_disclosure", + "_tgt": "skill_authoring_best_practices_runtime_environment", + "source": "skill_authoring_best_practices_progressive_disclosure", + "target": "skill_authoring_best_practices_runtime_environment" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "skill_authoring_best_practices_workflow_checklists", + "_tgt": "skill_authoring_best_practices_feedback_loops", + "source": "skill_authoring_best_practices_workflow_checklists", + "target": "skill_authoring_best_practices_feedback_loops" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "skill_authoring_best_practices_feedback_loops", + "_tgt": "skill_authoring_best_practices_machine_verifiability", + "source": "skill_authoring_best_practices_feedback_loops", + "target": "skill_authoring_best_practices_machine_verifiability" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "skill_authoring_best_practices_evaluation_driven_development", + "_tgt": "skill_authoring_best_practices_machine_verifiability", + "source": "skill_authoring_best_practices_evaluation_driven_development", + "target": "skill_authoring_best_practices_machine_verifiability" + }, + { + "relation": "conceptually_related_to", + "confidence": "AMBIGUOUS", + "confidence_score": 0.45, + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "skill_authoring_best_practices_evaluation_driven_development", + "_tgt": "research_subagent_best_practices_confidence_filtering", + "source": "skill_authoring_best_practices_evaluation_driven_development", + "target": "research_subagent_best_practices_confidence_filtering" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skill-authoring-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "skill_authoring_best_practices_utility_scripts", + "_tgt": "skill_authoring_best_practices_context_window_efficiency", + "source": "skill_authoring_best_practices_utility_scripts", + "target": "skill_authoring_best_practices_context_window_efficiency" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_doc", + "_tgt": "research_subagent_best_practices_subagents", + "source": "research_subagent_best_practices_doc", + "target": "research_subagent_best_practices_subagents" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_doc", + "_tgt": "research_subagent_best_practices_context_fork", + "source": "research_subagent_best_practices_doc", + "target": "research_subagent_best_practices_context_fork" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_doc", + "_tgt": "research_subagent_best_practices_tool_restrictions", + "source": "research_subagent_best_practices_doc", + "target": "research_subagent_best_practices_tool_restrictions" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_doc", + "_tgt": "research_subagent_best_practices_pretooluse_hooks", + "source": "research_subagent_best_practices_doc", + "target": "research_subagent_best_practices_pretooluse_hooks" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_doc", + "_tgt": "research_subagent_best_practices_persistent_memory", + "source": "research_subagent_best_practices_doc", + "target": "research_subagent_best_practices_persistent_memory" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_doc", + "_tgt": "research_subagent_best_practices_worktree_isolation", + "source": "research_subagent_best_practices_doc", + "target": "research_subagent_best_practices_worktree_isolation" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_doc", + "_tgt": "research_subagent_best_practices_confidence_filtering", + "source": "research_subagent_best_practices_doc", + "target": "research_subagent_best_practices_confidence_filtering" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_doc", + "_tgt": "research_subagent_best_practices_plugin_shipped_agents", + "source": "research_subagent_best_practices_doc", + "target": "research_subagent_best_practices_plugin_shipped_agents" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_doc", + "_tgt": "research_subagent_best_practices_agent_teams", + "source": "research_subagent_best_practices_doc", + "target": "research_subagent_best_practices_agent_teams" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_doc", + "_tgt": "research_subagent_best_practices_actionable_descriptions", + "source": "research_subagent_best_practices_doc", + "target": "research_subagent_best_practices_actionable_descriptions" + }, + { + "relation": "cites", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_doc", + "_tgt": "research_subagent_best_practices_everything_claude_code", + "source": "research_subagent_best_practices_doc", + "target": "research_subagent_best_practices_everything_claude_code" + }, + { + "relation": "cites", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_doc", + "_tgt": "research_subagent_best_practices_anthropic_subagents_docs", + "source": "research_subagent_best_practices_doc", + "target": "research_subagent_best_practices_anthropic_subagents_docs" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_subagents", + "_tgt": "research_subagent_best_practices_explore_agent", + "source": "research_subagent_best_practices_subagents", + "target": "research_subagent_best_practices_explore_agent" + }, + { + "relation": "implements", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_subagents", + "_tgt": "research_subagent_best_practices_context_fork", + "source": "research_subagent_best_practices_subagents", + "target": "research_subagent_best_practices_context_fork" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_subagents", + "_tgt": "research_subagent_best_practices_context_preservation", + "source": "research_subagent_best_practices_subagents", + "target": "research_subagent_best_practices_context_preservation" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_subagents", + "_tgt": "research_subagent_best_practices_agent_teams", + "source": "research_subagent_best_practices_subagents", + "target": "research_subagent_best_practices_agent_teams" + }, + { + "relation": "conceptually_related_to", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_explore_agent", + "_tgt": "research_subagent_best_practices_tool_restrictions", + "source": "research_subagent_best_practices_explore_agent", + "target": "research_subagent_best_practices_tool_restrictions" + }, + { + "relation": "references", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_context_fork", + "_tgt": "research_subagent_best_practices_agent_field", + "source": "research_subagent_best_practices_context_fork", + "target": "research_subagent_best_practices_agent_field" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_context_fork", + "_tgt": "research_subagent_best_practices_context_preservation", + "source": "research_subagent_best_practices_context_fork", + "target": "research_subagent_best_practices_context_preservation" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_tool_restrictions", + "_tgt": "research_subagent_best_practices_safety_guardrails", + "source": "research_subagent_best_practices_tool_restrictions", + "target": "research_subagent_best_practices_safety_guardrails" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_pretooluse_hooks", + "_tgt": "research_subagent_best_practices_safety_guardrails", + "source": "research_subagent_best_practices_pretooluse_hooks", + "target": "research_subagent_best_practices_safety_guardrails" + }, + { + "relation": "rationale_for", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md", + "source_location": null, + "weight": 1.0, + "_src": "research_subagent_best_practices_worktree_isolation", + "_tgt": "research_subagent_best_practices_safety_guardrails", + "source": "research_subagent_best_practices_worktree_isolation", + "target": "research_subagent_best_practices_safety_guardrails" + } + ], + "hyperedges": [ + { + "id": "anthropic_reliability_stack", + "label": "Anthropic Reliability Stack", + "nodes": [ + "anthropic_structured_outputs_structured_outputs", + "anthropic_structured_outputs_json_outputs", + "anthropic_strict_tool_use_strict_tool_use", + "anthropic_structured_outputs_constrained_decoding" + ], + "relation": "form", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/structured-outputs/anthropic-structured-outputs.md" + }, + { + "id": "humanlayer_runtime_stack", + "label": "HumanLayer Runtime Stack", + "nodes": [ + "humanlayer_repository_analysis_humanlayer_wui", + "humanlayer_repository_analysis_hlyr", + "humanlayer_repository_analysis_hld", + "humanlayer_repository_analysis_claudecode_go", + "humanlayer_repository_analysis_approval_loop" + ], + "relation": "participate_in", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/human-layer-project/humanlayer-repository-analysis.md" + }, + { + "id": "rpi_workflow_implementations", + "label": "RPI Workflow Implementations", + "nodes": [ + "advanced-context-engineering_rpi_workflow", + "research-plan-implement_humanlayer_rpi", + "rpir-tyler-burleigh_rpir_workflow", + "building-agent-harness_atelier" + ], + "relation": "implement", + "confidence": "EXTRACTED", + "confidence_score": 0.95, + "source_file": "docs/agentic-engineering/research-plan-implement-rpi.md" + }, + { + "id": "context_degradation_phenomena", + "label": "Context Degradation Phenomena", + "nodes": [ + "advanced-context-engineering_dumb_zone", + "progressive-disclosure_context_rot", + "shedding-dead-context_dead_context", + "context-stops-being-scarce_compaction_problem" + ], + "relation": "form", + "confidence": "INFERRED", + "confidence_score": 0.85, + "source_file": "docs/context-engineering/advanced-context-engineering-coding-agents-dev.md" + }, + { + "id": "mcp_ecosystem_components", + "label": "MCP Ecosystem Components", + "nodes": [ + "mcp-vs-a2a_mcp", + "mcp-typescript-sdk_typescript_sdk", + "mcp-servers_reference_servers", + "mcp-transport_mrtr_stateless", + "mcp-transport_server_cards", + "anthropic-mcp-topics_mcp_ecosystem" + ], + "relation": "participate_in", + "confidence": "EXTRACTED", + "confidence_score": 0.9, + "source_file": "docs/mcp/mcp-vs-a2a-dzone.md" + }, + { + "id": "long_context_mitigation_experiments", + "label": "Long-Context Mitigation Experiments", + "nodes": [ + "lost_in_the_middle_and_in_between_multi_hop_qa", + "lost_in_the_middle_and_in_between_chain_of_thought_prompting", + "lost_in_the_middle_and_in_between_knowledge_graph_triple_extraction", + "lost_in_the_middle_and_in_between_document_summarization" + ], + "relation": "participate_in", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md" + }, + { + "id": "context_engineering_scoped_bundle", + "label": "Context Engineering Scoped File Set", + "nodes": [ + "research_context_engineering_comprehensive_doc", + "research_context_rot_and_management_doc", + "research_whitespace_and_formatting_doc", + "research_multilingual_performance_doc", + "research_agent_workflows_and_patterns_doc" + ], + "relation": "form", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/general-llm/research-context-engineering-comprehensive.md" + }, + { + "id": "agent_skills_standard_docset", + "label": "Agent Skills Standard Document Set", + "nodes": [ + "agentskills_readme_doc", + "agentskills_what_are_skills_doc", + "agentskills_specification_doc", + "agentskills_best_practices_doc", + "agentskills_using_scripts_doc", + "agentskills_optimizing_descriptions_doc", + "agentskills_evaluating_skills_doc" + ], + "relation": "form", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/shared/skills-standard/README.md" + }, + { + "id": "spec_driven_development_tool_landscape", + "label": "Spec-Driven Development Tool Landscape", + "nodes": [ + "spec_driven_development_readme_doc", + "spec_driven_development_variant_kiro", + "spec_driven_development_variant_spec_kit", + "spec_driven_development_variant_tessl" + ], + "relation": "participate_in", + "confidence": "EXTRACTED", + "confidence_score": 1.0, + "source_file": "docs/spec-driven-development/README.md" + }, + { + "id": "skills_over_mcp_discovery_stack", + "label": "Skills over MCP Discovery Stack", + "nodes": [ + "skills_over_mcp_meeting_notes_2248_prompts_as_skills", + "skills_over_mcp_office_hours_2460_well_known_skill_index", + "skills_over_mcp_office_hours_2460_lazy_loading_pattern", + "mcp_specification_prompts_primitive", + "skills_vs_mcp_speakeasy_server_prompt_bridge" + ], + "relation": "form", + "confidence": "INFERRED", + "confidence_score": 0.86, + "source_file": "docs/mcp/skills-over-mcp-office-hours-2460.md" + }, + { + "id": "programmatic_tool_calling_stack", + "label": "Programmatic Tool Calling Stack", + "nodes": [ + "long_live_mcp_aqfer_progressive_tool_discovery", + "tool_search_redefining_agent_tool_calling_epsilla_tool_search", + "programmatic_tool_calling_claude_api_tool_search", + "mcp_programmatic_tool_calling_opensandbox_code_mode", + "cameronking4_programmatic_tool_calling_programmatic_tool_calling" + ], + "relation": "form", + "confidence": "INFERRED", + "confidence_score": 0.88, + "source_file": "docs/tool-calling/programmatic-tool-calling-claude-api.md" + }, + { + "id": "claude_extensibility_stack", + "label": "Claude Extensibility Stack", + "nodes": [ + "claude_hook_reference_doc_hook_lifecycle", + "extend_claude_with_skills_context_fork", + "creating_custom_subagents_custom_subagents", + "claude_orchestrate_of_claude_code_sessions_agent_teams", + "research_claude_code_skills_format_marketplace_format" + ], + "relation": "participate_in", + "confidence": "INFERRED", + "confidence_score": 0.82, + "source_file": "docs/README.md" + }, + { + "id": "project_memory_stack", + "label": "Project memory stack", + "nodes": [ + "how_claude_remembers_a_project_claude_md_files", + "how_claude_remembers_a_project_auto_memory", + "how_claude_remembers_a_project_memory_command" + ], + "relation": "form", + "confidence": "EXTRACTED", + "confidence_score": 0.95, + "source_file": "docs/claude-code/memory/how-claude-remembers-a-project.md" + }, + { + "id": "plugin_package_layout", + "label": "Plugin package layout", + "nodes": [ + "claude_create_plugin_doc_plugin_manifest", + "claude_create_plugin_doc_plugin_skills_directory", + "claude_create_plugin_doc_plugin_root_structure", + "claude_create_plugin_doc_plugin_settings_json" + ], + "relation": "form", + "confidence": "EXTRACTED", + "confidence_score": 0.96, + "source_file": "docs/claude-code/plugins/claude-create-plugin-doc.md" + }, + { + "id": "subagent_guardrail_bundle", + "label": "Subagent guardrail bundle", + "nodes": [ + "research_subagent_best_practices_tool_restrictions", + "research_subagent_best_practices_pretooluse_hooks", + "research_subagent_best_practices_worktree_isolation" + ], + "relation": "form", + "confidence": "INFERRED", + "confidence_score": 0.84, + "source_file": "docs/general-llm/subagents/research-subagent-best-practices.md" + } + ] +} \ No newline at end of file diff --git a/graphify-out/manifest.json b/graphify-out/manifest.json new file mode 100644 index 00000000..7f9f2b12 --- /dev/null +++ b/graphify-out/manifest.json @@ -0,0 +1,87 @@ +{ + "docs/README.md": 1776801179.765, + "docs/spec-driven-development/README.md": 1776802616.448999, + "docs/spec-driven-development/spec-driven-development-main.md": 1776802506.493999, + "docs/spec-driven-development/spec-driven-development-variant.md": 1776802587.901999, + "docs/structured-outputs/anthropic-implement-tool-use.md": 1776802701.055, + "docs/structured-outputs/anthropic-structured-outputs.md": 1776802637.308, + "docs/structured-outputs/README.md": 1776802737.737999, + "docs/structured-outputs/anthropic-increase-consistency.md": 1776802701.055, + "docs/structured-outputs/anthropic-strict-tool-use.md": 1776802637.308, + "docs/human-layer-project/humanlayer-repository-analysis.md": 1776821034.213, + "docs/long-context-research/lost-in-the-middle-acl.md": 1776802662.24, + "docs/agentic-engineering/building-agent-harness-martin-richards.md": 1776802490.828999, + "docs/agentic-engineering/research-plan-implement-rpi.md": 1776802490.828999, + "docs/agentic-engineering/research-plan-implement-review-tyler-burleigh.md": 1776802490.828999, + "docs/agentic-engineering/claude-cookbook-anthropic.md": 1776802490.828999, + "docs/agentic-engineering/agentic-software-modernization-markus-harrer.md": 1776802490.828999, + "docs/context-engineering/context-engineering-most-important-skill-dev.md": 1776802517.229, + "docs/context-engineering/advanced-context-engineering-coding-agents-dev.md": 1776802517.229, + "docs/context-engineering/context-stops-being-scarce-paddo.md": 1776802517.229, + "docs/context-engineering/progressive-disclosure-ai-agents.md": 1776802517.229, + "docs/context-engineering/shedding-dead-context-ryan-spletzer.md": 1776802517.229, + "docs/context-engineering/context-engineering-commercial-agents-jeremy-daly.md": 1776802517.229, + "docs/context-engineering/agents-md-is-a-liability-paddo.md": 1776802517.229, + "docs/context-engineering/pi-context-zone-github.md": 1776802517.229, + "docs/shared/skill-authoring-best-practices.md": 1776801081.217999, + "docs/shared/skills-standard/agentskills-specification.md": 1776801081.221999, + "docs/shared/skills-standard/agentskills-evaluating-skills.md": 1776801081.217999, + "docs/shared/skills-standard/agentskills-best-practices.md": 1776801081.217999, + "docs/shared/skills-standard/README.md": 1776801081.217999, + "docs/shared/skills-standard/agentskills-optimizing-descriptions.md": 1776801081.217999, + "docs/shared/skills-standard/agentskills-what-are-skills.md": 1776801081.221999, + "docs/shared/skills-standard/agentskills-using-scripts.md": 1776801081.221999, + "docs/mcp/mcp-vs-a2a-dzone.md": 1776802322.259, + "docs/mcp/mcp-http2-http3-reddit.md": 1776802322.259, + "docs/mcp/mcp-typescript-sdk-github.md": 1776802522.588, + "docs/mcp/skills-over-mcp-meeting-notes-2248.md": 1776802455.832999, + "docs/mcp/long-live-mcp-aqfer.md": 1776802208.763999, + "docs/mcp/skills-vs-mcp-speakeasy.md": 1776802455.832999, + "docs/mcp/mcp-specification.md": 1776802208.763999, + "docs/mcp/about-mcp-github-docs.md": 1776802208.763999, + "docs/mcp/long-live-mcp-aws.md": 1776802322.259, + "docs/mcp/modelcontextprotocol-servers-github.md": 1776802522.588, + "docs/mcp/a2a-protocol-huggingface.md": 1776802593.186, + "docs/mcp/skills-over-mcp-office-hours-2460.md": 1776802455.832999, + "docs/mcp/anthropic-mcp-github-topics.md": 1776802593.186, + "docs/claude-code/claude-prompting-best-practices.md": 1776801081.051, + "docs/claude-code/hooks/claude-hook-reference-doc.md": 1776801081.051, + "docs/claude-code/hooks/automate-workflow-with-hooks.md": 1776801081.051, + "docs/claude-code/memory/how-claude-remembers-a-project.md": 1776801081.055, + "docs/claude-code/subagents/claude-orchestrate-of-claude-code-sessions.md": 1776801081.055, + "docs/claude-code/subagents/creating-custom-subagents.md": 1776801081.055, + "docs/claude-code/plugins/claude-create-plugin-doc.md": 1776801081.055, + "docs/claude-code/skills/extend-claude-with-skills.md": 1776801081.055, + "docs/claude-code/skills/research-claude-code-skills-format.md": 1776801081.055, + "docs/harness-engineering/skill-issue-harness-engineering-for-coding-agents.md": 1776831578.391, + "docs/harness-engineering/harnessengineering-building-the-operating-system-for-autonomous-agents.md": 1776860631.3660908, + "docs/harness-engineering/harness-engineering.md": 1776834090.838999, + "docs/general-llm/research-agent-workflows-and-patterns.md": 1776801081.145999, + "docs/general-llm/research-context-engineering-comprehensive.md": 1776801081.145999, + "docs/general-llm/prompt-engineering-guide.md": 1776801081.145999, + "docs/general-llm/a-guide-to-agents.md": 1776801081.145999, + "docs/general-llm/persuasion-principles.md": 1776801081.145999, + "docs/general-llm/research-whitespace-and-formatting.md": 1776801081.145999, + "docs/general-llm/research-context-rot-and-management.md": 1776801081.145999, + "docs/general-llm/subagents/research-subagent-best-practices.md": 1776801081.145999, + "docs/agent-protocols/a2a-protocol-huggingface-space.md": 1776802273.339999, + "docs/agent-protocols/anthropic-engineer-2026-forecast-full-connectivity-mcp.md": 1776802273.339999, + "docs/agent-protocols/ai-agent-protocols-2026-guide.md": 1776802273.339999, + "docs/tool-calling/README.md": 1776802884.529, + "docs/tool-calling/mcp-programmatic-tool-calling-opensandbox-dev.md": 1776802859.23, + "docs/tool-calling/cameronking4-programmatic-tool-calling-github.md": 1776802859.23, + "docs/tool-calling/tool-search-redefining-agent-tool-calling-epsilla.md": 1776802859.23, + "docs/tool-calling/programmatic-tool-calling-claude-api.md": 1776802859.23, + "docs/spec-driven-development/spec-driven-development-arxiv.md": 1776802453.642999, + "docs/long-context-research/lost-in-the-middle-arxiv.md": 1776802662.24, + "docs/long-context-research/README.md": 1776802735.829999, + "docs/long-context-research/lost-in-the-middle-researchgate.md": 1776802662.24, + "docs/long-context-research/lost-in-the-middle-and-in-between-arxiv.md": 1776802704.721999, + "docs/general-llm/research-multilingual-performance.md": 1776801081.145999, + "docs/general-llm/Call_Me_A_Jerk_Persuading_AI_to_Comply_with_Objectionable_Requests.pdf": 1776801081.119999, + "docs/general-llm/Evaluating-AGENTS-paper.md": 1776801081.119999, + "docs/general-llm/Evaluating-AGENTS-paper.pdf": 1776801081.141999, + "docs/agent-protocols/architectural-paradigms-advanced-agentic-systems.md": 1776802273.339999, + "docs/agent-protocols/fluid-human-agent-collaboration-pmc.md": 1776802273.339999, + "docs/agent-protocols/advancing-agentic-ai-communication-protocols.md": 1776802273.339999 +} \ No newline at end of file diff --git a/plugins/prp-core/.claude-plugin/plugin.json b/plugins/prp-core/.claude-plugin/plugin.json index f0c38353..bcf88581 100644 --- a/plugins/prp-core/.claude-plugin/plugin.json +++ b/plugins/prp-core/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "prp-core", - "version": "3.1.2", + "version": "3.2.0", "description": "Complete PRP workflow automation tools and methodology - plan, implement, debug, review, commit, and PR skills with specialized agents", "author": { "name": "Rodrigo França", diff --git a/plugins/prp-core/CLAUDE.md b/plugins/prp-core/CLAUDE.md index c0acd6e2..773f233a 100644 --- a/plugins/prp-core/CLAUDE.md +++ b/plugins/prp-core/CLAUDE.md @@ -1,8 +1,9 @@ -# PRP Core Plugin — Claude Code plugin providing PRP workflow automation (v3.0.0). +# PRP Core Plugin — Claude Code plugin providing PRP workflow automation. ## Scope -- This directory is the installable plugin payload: `skills/`, `agents/`, `hooks/`, and `.claude-plugin/plugin.json`. +- This directory is the installable plugin payload: `skills/`, `agents/`, `hooks/`, `rules/`, `references/`, and `.claude-plugin/plugin.json`. - Root `.claude/` is a development mirror, not the package itself. - Keep shared plugin artifacts aligned between this directory and the root `.claude/` mirror. - Root-only extras such as `prp-core-runner`, `update-review-instructions`, and `gpui-researcher` do not ship in the plugin. +- Cross-cutting policy lives in `references/`. Skills and agents link to those files instead of duplicating policy inline. diff --git a/plugins/prp-core/README.md b/plugins/prp-core/README.md index 86eda5ec..3159469a 100644 --- a/plugins/prp-core/README.md +++ b/plugins/prp-core/README.md @@ -1,82 +1,228 @@ # PRP Core Plugin -`prp-core` is the installable Claude Code plugin in this repository. It packages the PRP workflow as skills, agents, and hooks. +`prp-core` is the installable Claude Code plugin in this repository. It packages the PRP (Product Requirements → Plan → Implement → Validate) workflow as skills, agents, rules, hooks, and reference files. ## What ships in the plugin -| Artifact | Count | Notes | +| Artifact type | Count | Location | | --- | --- | --- | -| Skills | 17 | The installable workflow surface in `skills/` | -| Agents | 12 | Specialized review and analysis agents in `agents/` | -| Rules | 1 | Path-scoped execution rules in `rules/` | -| Stop hooks | 2 | Registered from `hooks/hooks.json` | +| Skills | 17 | `skills/` | +| Agents | 12 | `agents/` | +| Rules | 2 | `rules/` | +| Stop hooks | 2 | `hooks/` (registered via `hooks/hooks.json`) | +| References | 5 | `references/` | + +## Workflow overview + +The PRP pipeline runs in four phases. Each phase requires a durable artifact before the next begins. + +``` +Feature idea / Issue / Question + ↓ +[Research] prp-prd / prp-codebase-question / prp-research-team / prp-debug + ↓ .prd.md / .issues/*.md +[Plan] prp-plan / prp-issue-investigate → prp-advisor (gate) + ↓ .plan.md +[Implement] prp-implement / prp-ralph / prp-issue-fix → prp-verification-before-completion (gate) + ↓ reports/*.md +[Ship] prp-commit → prp-pr +``` + +Parallel review runs independently: `prp-review` / `prp-review-agents` → `.claude/PRPs/reviews/*.md`. ## Skills ### Planning and execution -- `prp-prd` -- `prp-plan` -- `prp-implement` -- `prp-ralph` -- `prp-ralph-cancel` -- `prp-ralph-loop` +**`prp-prd`** — Research and produce a PRD from a feature idea. +- **Why**: Establishes goal, business value, user-visible behavior, and technical context before any planning or code begins. +- **Produces**: `.claude/PRPs/prds/.prd.md` +- **Connects to**: → `prp-plan` (next phase), ← `prp-advisor` (pre-work gate) + +**`prp-plan`** — Create an implementation plan from a PRD or direct request. +- **Why**: Turns research findings into actionable phases, task lists, and executable validation commands. Separates planning context from implementation context. +- **Produces**: `.claude/PRPs/plans/.plan.md` +- **Connects to**: ← `prp-prd` (upstream artifact), → `prp-implement` (next phase), → `plan-critic` agent (review gate) + +**`prp-implement`** — Execute a `.plan.md` file with per-step validation. +- **Why**: Mechanical execution in a clean, plan-scoped context. Delegates noisy work to subagents. Archives the plan on completion. +- **Produces**: `.claude/PRPs/reports/-report.md`; moves plan to `plans/completed/` +- **Connects to**: ← `prp-plan` (upstream artifact), → `prp-verification-before-completion` (gate), → `prp-commit`, → `prp-pr` + +**`prp-ralph`** — Start an autonomous iterative loop over a plan. +- **Why**: Enables fully autonomous multi-iteration execution. The stop hook is the control boundary — Ralph keeps running until all validations pass or the iteration limit is hit. +- **Produces**: `.claude/prp-ralph.state.md` (active loop state) +- **Connects to**: → `prp-ralph-stop.sh` hook (control boundary), ↔ `prp-ralph-loop` (iteration logic), → `prp-ralph-cancel` + +**`prp-ralph-cancel`** — Cancel an active Ralph loop. +- **Why**: Safety escape hatch when you need to interrupt an autonomous loop without losing partial work. +- **Produces**: Removes `.claude/prp-ralph.state.md`; preserves modified files and existing commits +- **Connects to**: ↔ `prp-ralph` (active loop state) + +**`prp-ralph-loop`** — Internal iteration logic for active Ralph loops. +- **Why**: Activated automatically when `.claude/prp-ralph.state.md` exists. Executes the next plan iteration and applies the completion protocol. +- **Produces**: Updated state file per iteration +- **Connects to**: ← `prp-ralph-stop.sh` hook (guards each iteration), ↔ `prp-ralph` (state file) ### Issues and debugging -- `prp-issue-investigate` -- `prp-issue-fix` -- `prp-debug` +**`prp-issue-investigate`** — Investigate a GitHub issue and produce an implementation plan artifact. +- **Why**: Structures analysis into a durable, reviewable artifact before any code changes. Separates investigation context from implementation context. +- **Produces**: `.claude/PRPs/issues/issue--.md` +- **Connects to**: ← GitHub issue (input), → `prp-issue-fix` (downstream executor), ← `prp-advisor` (pre-work gate) -### Research and review +**`prp-issue-fix`** — Implement a fix from an issue investigation artifact. +- **Why**: Clean implementation context after investigation is complete and documented. Ensures changes are grounded in the investigation artifact. +- **Produces**: Code changes; moves issue artifact to `issues/completed/` +- **Connects to**: ← `prp-issue-investigate` (upstream artifact), → `prp-verification-before-completion` (gate), → `prp-commit`, → `prp-pr` -- `prp-codebase-question` -- `prp-research-team` -- `prp-review` -- `prp-review-agents` -- `prp-verification-before-completion` +**`prp-debug`** — Root-cause analysis using the 5 Whys technique. +- **Why**: Prevents surface-level fixes by identifying the specific code or config that, if changed, prevents the issue from recurring. +- **Produces**: Analysis report with root cause and fix recommendation +- **Connects to**: → feeds findings into `prp-issue-investigate` or `prp-plan` -### Advisor and validation +### Research -- `prp-advisor` +**`prp-codebase-question`** — Answer codebase questions through systematic parallel exploration. +- **Why**: Runs exploration in isolated subagent contexts to prevent research noise from polluting the parent agent's context. Returns only a compact, structured answer. +- **Produces**: Condensed research summary (in-context; no artifact file) +- **Connects to**: → feeds into `prp-plan` or user decision points + +**`prp-research-team`** — Design a dynamic research team and plan. +- **Why**: Orchestrates multiple parallel specialist agents for complex cross-cutting research. The stop hook validates output structure so it is always complete. +- **Produces**: Structured research plan with 6 required sections +- **Connects to**: → `prp-research-team-stop.sh` hook (output validator), → feeds into `prp-prd` or `prp-plan` + +### Review + +**`prp-review`** — Comprehensive PR code review. +- **Why**: Validates changes against project patterns locally before posting. Documents findings in a durable artifact for audit and iteration. +- **Produces**: `.claude/PRPs/reviews/pr--review.md` +- **Connects to**: ← open PR (input), → `code-reviewer`, `silent-failure-hunter` agents + +**`prp-review-agents`** — Multi-aspect PR review using parallel specialized agents. +- **Why**: Runs independent specialist agents (tests, errors, types, docs, security) in isolated contexts and surfaces only high-signal findings. +- **Produces**: Structured findings per aspect +- **Connects to**: ← open PR (input), → `code-reviewer`, `pr-test-analyzer`, `silent-failure-hunter`, `type-design-analyzer`, `docs-impact-agent`, `comment-analyzer` + +**`prp-verification-before-completion`** — Final validation gate before claiming work is complete. +- **Why**: Required back-pressure mechanism. Forces deterministic proof (test output, build output) before any completion claim. Enforced by the `prp-workflow` rule. +- **Produces**: No artifact; blocks phase completion until validation evidence is produced +- **Connects to**: → required before `prp-commit` at each phase; enforced by `prp-workflow` rule + +### Advisor + +**`prp-advisor`** — Call the advisor before substantive work. +- **Why**: Catches design flaws at the highest-leverage moment — before code is written. Detects the native advisor tool if available; falls back to the `prp-advisor` subagent. +- **Produces**: Advisory critique (in-context; no artifact file) +- **Connects to**: → called before any `*.prd.md` or `*.plan.md` is created; ↔ `prp-advisor` agent (fallback backend) ### Git workflow -- `prp-commit` -- `prp-pr` +**`prp-commit`** — Create atomic commits by logical scope following Conventional Commits. +- **Why**: Keeps git history tied to plan phases. Groups changes by logical scope rather than bulk-staging everything at once. +- **Produces**: One commit per logical scope in the diff +- **Connects to**: ← `prp-implement`, `prp-issue-fix` (trigger points), → `prp-pr` + +**`prp-pr`** — Create a pull request from the current branch. +- **Why**: Standardizes PR creation with template usage, auto-detects base branch, and validates git state before pushing. +- **Produces**: GitHub pull request +- **Connects to**: ← `prp-commit` (must precede); hard gate enforced by `prp-workflow` rule: plan must be archived first ## Agents -- `code-reviewer` -- `code-simplifier` -- `codebase-analyst` -- `codebase-explorer` -- `comment-analyzer` -- `docs-impact-agent` -- `plan-critic` -- `pr-test-analyzer` -- `prp-advisor` -- `silent-failure-hunter` -- `type-design-analyzer` -- `web-researcher` +Agents are spawned by skills in isolated subagent contexts. Each receives a condensed brief, completes one job, and returns compact output. Every agent belongs to one component class defined in `references/harness-taxonomy.md`. + +| Agent | Class | Role | Invoked by | +| --- | --- | --- | --- | +| `code-reviewer` | Advisory | Reviews code for guideline compliance, bugs, and quality. High-confidence issues only. | `prp-review`, `prp-review-agents` | +| `code-simplifier` | Advisory | Identifies simplification opportunities in recently changed code. Advisory only. | `prp-review-agents` | +| `codebase-analyst` | Advisory | Traces implementation details and data flow with `file:line` references. | `prp-plan`, `prp-implement` | +| `codebase-explorer` | Advisory | Finds where code lives and shows how it is implemented. Combines search with pattern extraction. | `prp-plan`, `prp-codebase-question` | +| `comment-analyzer` | Advisory | Verifies that code comments match actual code behavior. Advisory only. | `prp-review-agents` | +| `docs-impact-agent` | Advisory | Identifies stale docs and missing entries for new user-facing features. Advisory only. | `prp-review-agents` | +| `plan-critic` | Advisory | Reviews PRP plans for completeness, coherence, and blind spots before generation. | `prp-plan`, `prp-implement` | +| `pr-test-analyzer` | Advisory | Analyzes PR test coverage for behavioral completeness, not line metrics. | `prp-review-agents` | +| `prp-advisor` | Advisory | Advisory subagent backing the `prp-advisor` skill when the native advisor tool is unavailable. | `prp-advisor` skill | +| `silent-failure-hunter` | Advisory | Hunts for swallowed errors, inadequate error handling, and inappropriate fallbacks. | `prp-review`, `prp-review-agents` | +| `type-design-analyzer` | Advisory | Analyzes type design for encapsulation, invariant expression, and enforcement quality. | `prp-review-agents` | +| `web-researcher` | Advisory | Searches for current documentation, APIs, and best practices beyond training data. | `prp-plan`, `prp-codebase-question`, `prp-research-team` | ## Rules -The plugin ships one path-scoped execution rule in `rules/`: +The plugin ships two path-scoped execution rules in `rules/`. + +**`prp-workflow`** — Enforces the full PRP lifecycle for `*.prd.md` and `*.plan.md` files. +- Fires when a PRD or plan file is open. +- Enforces: GitHub issue creation after PRD/plan creation, `prp-verification-before-completion` at each phase, `prp-commit` after each phase, and `prp-pr` only after plan is archived. +- **Why it ships**: End users executing PRP workflows need this lifecycle enforcement. Without it, phases get skipped and artifacts drift. -- **`prp-workflow`** — enforces the full PRP lifecycle for `*.prd.md` and `*.plan.md` files: advisor-first gate, GitHub issue creation and tracking, verification before each phase completion, atomic commits, and PR sequencing after plan archival. +**`harness-engineering`** — Enforces harness engineering principles during PRP execution. +- Fires when `.claude/**/*.md`, `*.prd.md`, or `*.plan.md` files are open. +- Enforces: Smart Zone context budget (three tiers), subagent isolation, artifact discipline, and progressive disclosure. +- **Why it ships**: Plugin users need the same smart-zone and compaction discipline as contributors. Context budget violations degrade reasoning quality for all users. ## Hooks -Plugin installs register both Stop hooks automatically through `hooks/hooks.json`: +Plugin installs register both Stop hooks automatically through `hooks/hooks.json`. They fire when Claude Code attempts to stop or exit. -- `${CLAUDE_PLUGIN_ROOT}/hooks/prp-ralph-stop.sh` -- `${CLAUDE_PLUGIN_ROOT}/hooks/prp-research-team-stop.sh` +**`prp-ralph-stop.sh`** — Controls Ralph loop continuation. +- Checks `.claude/prp-ralph.state.md` on each Stop event. +- Blocks exit while an active Ralph loop is running and has remaining iterations. +- **Why it ships**: Without this hook, Ralph cannot maintain autonomous execution across Stop events. The hook is the control boundary that keeps the loop alive. -`prp-ralph-stop.sh` keeps active Ralph loops running until they complete or hit the iteration limit. `prp-research-team-stop.sh` validates the generated research plan through `.claude/prp-research-team.state` and blocks exit until the required sections are present. +**`prp-research-team-stop.sh`** — Validates generated research plans. +- Checks `.claude/prp-research-team.state` on each Stop event. +- Blocks exit unless the research plan contains all 6 required sections: `Research Question`, `Research Question Decomposition`, `Team Composition`, `Research Tasks`, `Team Orchestration Guide`, `Acceptance Criteria`. +- **Why it ships**: Without this gate, the `prp-research-team` skill can silently produce incomplete plans. The hook is the structural completeness contract. If you install the plugin, you do **not** need to add those hooks manually to your settings. Manual hook setup only applies when you copy the repo-local `.claude/` mirror yourself. +## References + +The `references/` directory contains authoritative policy documents. Skills and agents link to these instead of duplicating policy inline. When two components disagree about a policy, the reference file wins. + +**`harness-taxonomy.md`** — Component class definitions: artifact / advisory / utility. +- **Why**: Every skill and agent belongs to exactly one class. Class membership determines context profile, brief size limits, and whether the component can modify files. +- **Used by**: All skills and agents to understand their execution contract. + +**`context-budget-policy.md`** — Smart-zone thresholds, brief size limits, and compaction rules. +- **Why**: Context bloat degrades reasoning quality. This policy is the single source of truth for when to compact, when to delegate, and what the brief size limits are. +- **Used by**: All skills at each phase handoff; governs when to fork a subagent. + +**`execution-policy.md`** — Four delegation modes: inline / subagent / parallel / harness. +- **Why**: Without clear delegation rules, skills do too much work inline and pollute context with exploration noise. This policy defines when work stays inline vs. goes to a subagent. +- **Used by**: Any skill or agent deciding whether to fork a subagent or stay inline. + +**`artifact-lifecycle.md`** — PRP artifact locations, naming conventions, and archival rules. +- **Why**: Artifact paths must be consistent across skills so downstream skills can find upstream artifacts. This is the authoritative path table. +- **Used by**: All artifact-producing skills (`prp-prd`, `prp-plan`, `prp-implement`, `prp-issue-investigate`, `prp-review`). + +**`agent-prompt-style.md`** — One-job CRITICAL declaration, DO NOT boundaries, output format contract. +- **Why**: Agents without explicit one-job boundaries expand scope and waste context. This style guide makes agent contracts testable and repeatable. +- **Used by**: Any new or modified agent prompt; enforced by the `harness-engineering` rule. + +## Artifacts written by the skills + +```text +.claude/PRPs/ +├── prds/ ← PRDs created by prp-prd +├── plans/ ← Plans created by prp-plan +│ └── completed/ ← Archived by prp-implement after execution +├── reports/ ← Execution reports from prp-implement +├── issues/ ← Investigation artifacts from prp-issue-investigate +│ └── completed/ ← Archived by prp-issue-fix after fix +└── reviews/ ← Review artifacts from prp-review +``` + +Runtime loop state (active only; not archived): + +```text +.claude/ +├── prp-ralph.state.md ← Active Ralph loop state (removed by prp-ralph-cancel) +└── prp-research-team.state ← Active research-team validation state +``` + ## Plugin vs. repo-local extras The root repository also contains a `.claude/` mirror for development. These assets are **not** part of the `prp-core` plugin package: @@ -160,19 +306,6 @@ Run the prp-review skill for PR 123 Run the prp-review-agents skill for PR 123 with all aspects ``` -## Artifacts written by the skills - -```text -.claude/PRPs/ -├── prds/ -├── plans/ -│ └── completed/ -├── reports/ -├── issues/ -│ └── completed/ -└── reviews/ -``` - ## Requirements - Claude Code with plugin support diff --git a/plugins/prp-core/agents/codebase-analyst.md b/plugins/prp-core/agents/codebase-analyst.md index 7cd2d72a..24f14241 100644 --- a/plugins/prp-core/agents/codebase-analyst.md +++ b/plugins/prp-core/agents/codebase-analyst.md @@ -9,9 +9,9 @@ maxTurns: 15 skills: [prp-advisor] --- -You are a code analysis specialist. You analyze implementation details, trace data flow, and explain technical workings with precise file:line references. +## CRITICAL: Your only job is to analyze existing code and document how it works -## CRITICAL: Document What Exists, Nothing More +> This agent follows the prompt structure in `plugins/prp-core/references/agent-prompt-style.md`. - **DO NOT** suggest improvements, propose enhancements, or critique implementations - **DO NOT** perform root cause analysis or comment on quality/performance/security @@ -19,7 +19,6 @@ You are a code analysis specialist. You analyze implementation details, trace da - **ONLY** describe what exists, how it works, and how components interact - **ALWAYS** Call `prp-advisor` skill BEFORE assuming work is done or building on an assumption. If the task requires orientation first (finding files, fetching a source, seeing what's there), do that, then call `prp-advisor` agent. Orientation is not substantive work. Writing, editing, and declaring an answer are. - You are a documentarian, not a critic or consultant. ## Core Responsibilities diff --git a/plugins/prp-core/agents/codebase-explorer.md b/plugins/prp-core/agents/codebase-explorer.md index 721331e0..b9e6ea45 100644 --- a/plugins/prp-core/agents/codebase-explorer.md +++ b/plugins/prp-core/agents/codebase-explorer.md @@ -8,9 +8,9 @@ maxTurns: 15 skills: [prp-advisor] --- -You are a codebase explorer. You find WHERE code lives and show HOW it's implemented with concrete examples, precise file:line references, and actual code patterns. +## CRITICAL: Your only job is to map where code lives and show how it is implemented -## CRITICAL: Document What Exists, Nothing More +> This agent follows the prompt structure in `plugins/prp-core/references/agent-prompt-style.md`. - **DO NOT** suggest improvements, critique implementations, or identify problems - **DO NOT** recommend refactoring, reorganization, or evaluate pattern quality diff --git a/plugins/prp-core/agents/plan-critic.md b/plugins/prp-core/agents/plan-critic.md index 2fd97d5c..8152cbe6 100644 --- a/plugins/prp-core/agents/plan-critic.md +++ b/plugins/prp-core/agents/plan-critic.md @@ -9,7 +9,9 @@ maxTurns: 10 You are a PRP artifact critic. You review consolidated findings and draft content for implementation plans and product requirement documents. Your goal is to catch gaps, blind spots, and weak assumptions BEFORE the artifact is written — when course corrections are still cheap. -## CRITICAL: Actionable Feedback Only +## CRITICAL: Your only job is to validate PRP artifacts for completeness, coherence, and blind spots + +> This agent follows the prompt structure in `plugins/prp-core/references/agent-prompt-style.md`. - **DO** identify specific gaps with concrete suggestions to fix them - **DO** challenge untested assumptions with evidence from the codebase diff --git a/plugins/prp-core/agents/prp-advisor.md b/plugins/prp-core/agents/prp-advisor.md index 8f7c1b3c..cd93cf4f 100644 --- a/plugins/prp-core/agents/prp-advisor.md +++ b/plugins/prp-core/agents/prp-advisor.md @@ -8,7 +8,9 @@ color: purple maxTurns: 10 --- -You are a senior technical advisor. Your only job is to review the current task state and give precise, actionable guidance. You receive the full conversation history — every tool call, every result. +## CRITICAL: Your only job is to review task state and give precise, actionable guidance + +> This agent follows the prompt structure in `plugins/prp-core/references/agent-prompt-style.md`. ## Constraints diff --git a/plugins/prp-core/agents/web-researcher.md b/plugins/prp-core/agents/web-researcher.md index 054d6253..60aa5d33 100644 --- a/plugins/prp-core/agents/web-researcher.md +++ b/plugins/prp-core/agents/web-researcher.md @@ -7,7 +7,19 @@ tools: [WebSearch, WebFetch, Bash] maxTurns: 10 --- -You are a web research specialist. You find accurate, relevant information from web sources and synthesize it into actionable knowledge with proper citations. +## CRITICAL: Your only job is to find and synthesize accurate information from web sources + +> This agent follows the prompt structure in `plugins/prp-core/references/agent-prompt-style.md`. + +- DO NOT speculate when you can search +- DO NOT present findings without source citations +- DO NOT claim expertise beyond what sources support + +## Core Responsibilities + +1. Find accurate, up-to-date information from authoritative web sources +2. Synthesize findings into actionable knowledge with proper citations +3. Flag conflicting information, outdated content, and coverage gaps ## Process diff --git a/plugins/prp-core/references/agent-prompt-style.md b/plugins/prp-core/references/agent-prompt-style.md new file mode 100644 index 00000000..c83a10d3 --- /dev/null +++ b/plugins/prp-core/references/agent-prompt-style.md @@ -0,0 +1,60 @@ +# Agent Prompt Style + +PRP agents follow a structured prompt style that keeps each agent's job explicit, its boundaries clear, and its output predictable. The pattern is: one declared job, explicit DO NOT boundaries, a short strategy, and a fixed output shape. + +## Required structure + +Every agent prompt must include these sections in this order: + +### 1. Critical role declaration + +State the agent's single job in the opening line. Use this form: + +``` +## CRITICAL: Your only job is to [specific role] +``` + +Follow with explicit `DO NOT` boundaries for the two or three most common scope violations. + +### 2. Core Responsibilities + +A numbered list of the concrete actions the agent takes. Keep it to five or fewer items. Each item should be specific enough that another agent could audit whether it was followed. + +### 3. Strategy + +A numbered stepwise approach the agent should follow. This is not a restatement of responsibilities — it is the execution order and any key decision rules. + +### 4. Output Format + +Define the required output shape. For agents with a repeatable task, the output format should be explicit: section names, order, and any required fields. + +## Hard boundaries + +- One job per agent. An agent that reviews code must not also modify files. An agent that explores must not also commit. +- Minimal tool surface. Grant only the tools the agent genuinely needs. Advisory agents typically need read-only access (grep, glob, view, bash). No file modification tools. +- `maxTurns` must be declared. Advisory agents: `maxTurns: 10`. Exploration agents: `maxTurns: 15`. + +## PRP-specific frontmatter + +Retain all PRP-specific frontmatter fields (`name`, `description`, `model`, `color`, `tools`, `maxTurns`, `skills`) unless a change is explicitly justified by the plugin's actual runtime needs. Do not silently drop fields when adapting prompt structure. + +Model selection: +- `haiku` — structured, fast tasks (explorer, comment-analyzer, docs-impact) +- `sonnet` — reasoning-heavy tasks (analyst, reviewer, type-design, advisor) + +## Adaptation guidance + +When normalizing an existing agent against this style: + +1. Add the critical role declaration if it is missing. +2. Add hard `DO NOT` boundaries if implicit. +3. Consolidate scattered responsibilities into numbered Core Responsibilities. +4. Add a Strategy section if the agent's approach is not stated. +5. Make the output format explicit if it is currently open-ended. +6. Do not remove PRP-specific frontmatter or behavioral capabilities without reviewing the plugin's runtime contract. + +## Related references + +- `harness-taxonomy.md` — advisory vs. utility component classification +- `execution-policy.md` — when an agent receives a subagent delegation +- `context-budget-policy.md` — brief size limits for agent output diff --git a/plugins/prp-core/references/artifact-lifecycle.md b/plugins/prp-core/references/artifact-lifecycle.md new file mode 100644 index 00000000..718b18b6 --- /dev/null +++ b/plugins/prp-core/references/artifact-lifecycle.md @@ -0,0 +1,42 @@ +# Artifact Lifecycle + +PRP artifacts are first-class outputs. Every artifact must declare where it lives, when it is archived, and who is responsible for it. + +## Artifact locations + +All PRP runtime artifacts live under `.claude/PRPs/` in the target project: + +| Type | Path | Created by | +|------|------|------------| +| PRDs | `.claude/PRPs/prds/` | `prp-prd` | +| Plans | `.claude/PRPs/plans/` | `prp-plan` | +| Completed plans | `.claude/PRPs/plans/completed/` | `prp-implement` (archives after execution) | +| Reports | `.claude/PRPs/reports/` | `prp-implement` | +| Issues | `.claude/PRPs/issues/` | `prp-issue-investigate` | +| Completed issues | `.claude/PRPs/issues/completed/` | `prp-issue-fix` | +| Reviews | `.claude/PRPs/reviews/` | `prp-review` | +| Ralph archives | `.claude/PRPs/ralph-archives/` | `prp-ralph` (on completion) | + +## Lifecycle rules + +1. **Every artifact must declare its location before writing.** Skills should state the output path in the first step of their workflow, not after the artifact is written. + +2. **Archive completed artifacts consistently.** A plan is not complete until it is moved to `plans/completed/`. An issue investigation is not resolved until the artifact is in `issues/completed/`. Archival is a workflow step, not optional cleanup. + +3. **Durable artifacts carry the workflow forward; transcripts do not.** Only the compact artifact (PRD, plan, report, issue plan) should be passed between phases. Raw conversation history, exploration transcripts, and verbose tool outputs are temporary and must not be treated as the handoff artifact. + +4. **Keep repo-only eval outputs outside the shipped plugin payload.** Prompt-evaluation fixtures, test case files, and CI scripts live in the repository's `tests/` and `scripts/` directories. They do not live in `plugins/prp-core/` and must not be referenced from shipped skills. + +## Artifact naming + +- PRDs: `.prd.md` +- Plans: `.plan.md` +- Reports: `-report.md` +- Issues: `issue--.md` +- Reviews: `pr--review.md` + +## Related references + +- `harness-taxonomy.md` — which skills own which artifact class +- `context-budget-policy.md` — brief size limits that govern handoff artifacts +- `execution-policy.md` — delegation modes that determine which agent writes the artifact diff --git a/plugins/prp-core/references/context-budget-policy.md b/plugins/prp-core/references/context-budget-policy.md new file mode 100644 index 00000000..46c33278 --- /dev/null +++ b/plugins/prp-core/references/context-budget-policy.md @@ -0,0 +1,39 @@ +# Context Budget Policy + +The PRP harness treats context as a budget. Every token must earn its place. Noise in the context window degrades reasoning quality measurably — 3,000 filler tokens drop simple math accuracy from 0.92 to 0.68 (Levy et al., ACL 2024). + +## Core rules + +1. **Keep orchestration in the smart zone by default.** Run the primary workflow in a context window that stays under 40% capacity. Above that threshold, compaction or a subagent handoff is required before the next phase starts. + +2. **Compact noisy outputs into bounded briefs before handoff.** Raw exploration transcripts, verbose tool outputs, and multi-file search dumps must be summarized before they pass to the next skill or phase. Brief size limits: + - Discovery brief: ≤ 50 lines + - Execution brief: ≤ 30 lines + - Validation brief: ≤ 20 lines + +3. **Prefer references over repeated inline policy text.** Cross-cutting policy belongs in shared reference files, not duplicated in each skill. Skills should link to the relevant reference instead of restating the same guidance. + +4. **Treat raw agent transcripts as temporary context.** Subagent work products belong in compact artifacts (plan files, reports, brief sections), not in the main conversation thread. Discard exploration transcripts once the artifact is written. + +5. **Progressive disclosure for instructions.** Load skill context, agents, and path-scoped rules on demand. Do not front-load all workflow instructions into every session. Always-loaded instructions must stay under 200 lines total. + +## Compaction checkpoints + +Each orchestration-heavy skill must define at least one compaction checkpoint — a point where noisy intermediate outputs are summarized before the next step proceeds. Typical checkpoints: + +- After initial codebase exploration → summarize into a discovery brief +- After running all review agents → summarize findings into a combined report +- After investigation → summarize into an issue plan before implementation + +## When to start a fresh context + +Start a fresh context or subagent when: +- The current context is over 40% capacity and more noisy work remains +- The next task is genuinely independent of the current exploration +- The task is a large-corpus search, multi-file read, or iterative test loop + +## Related references + +- `harness-taxonomy.md` — component classes and their context profiles +- `execution-policy.md` — when to delegate to a subagent vs. stay inline +- `artifact-lifecycle.md` — where durable outputs live diff --git a/plugins/prp-core/references/execution-policy.md b/plugins/prp-core/references/execution-policy.md new file mode 100644 index 00000000..259e7333 --- /dev/null +++ b/plugins/prp-core/references/execution-policy.md @@ -0,0 +1,69 @@ +# Execution Policy + +The PRP harness defines four delegation modes. Choose the right mode based on task size, isolation, and determinism. + +## Delegation modes + +### 1. Stay inline + +Use for short, deterministic, single-scope work that does not generate noisy intermediate output. + +Criteria: +- Task completes in a handful of tool calls +- Output is compact and directly consumed by the next step +- No heavy exploration or multi-file corpus search needed + +Examples: reading a single file, writing a short commit message, running a schema check. + +### 2. Use an isolated subagent + +Use for noisy exploration, large-corpus research, or analysis that would pollute the main context. + +Criteria: +- Task requires reading many files, searching broadly, or running iterative queries +- Output will be summarized into a compact brief before handoff +- The work is independent enough that the main context does not need to observe every step + +Examples: codebase exploration, external documentation research, review analysis. + +Return only the compact result (discovery brief, research summary, review report) to the main context. Discard the exploration transcript. + +### 3. Parallelize + +Use only when scopes are genuinely independent — no shared files, no shared state, no ordering dependency. + +Criteria: +- Two or more subagents can run without reading each other's outputs +- Merging their results does not require re-running either agent +- Parallelism does not create conflicting writes + +If in doubt, keep execution sequential. + +### 4. Use a documented repo-local evaluation harness + +Use for deterministic batch verification that would otherwise repeat the same manual checks in the main context. + +Criteria: +- A documented evaluation script already exists for this check +- The check is repeatable, schema-bound, or file-comparison-based +- The result is pass/fail, not a reasoning judgment + +Do not assume evaluation harness scripts ship in the plugin payload. They are repo-only tools. Do not name repo-only script paths inside shipped skills. + +## Parallelism guard + +Before parallelizing, answer: "Do these scopes share any file, state variable, or sequencing constraint?" If yes, keep them sequential. + +## Research-team policy + +When `prp-research-team` is used, the orchestrating agent must: +1. Decompose the question into genuinely independent sub-questions +2. Assign each sub-question to a specialist with a bounded scope +3. Collect compact summaries from each specialist +4. Synthesize into the required six-section research plan + +## Related references + +- `context-budget-policy.md` — brief size limits and compaction rules +- `harness-taxonomy.md` — which component class owns each decision +- `artifact-lifecycle.md` — where outputs land after delegation diff --git a/plugins/prp-core/references/harness-taxonomy.md b/plugins/prp-core/references/harness-taxonomy.md new file mode 100644 index 00000000..0e575c7f --- /dev/null +++ b/plugins/prp-core/references/harness-taxonomy.md @@ -0,0 +1,48 @@ +# Harness Taxonomy + +The PRP harness groups all shipped components into three classes. Every skill and agent belongs to exactly one class. + +## Component Classes + +### Sequential artifact skills + +Create durable PRP artifacts that later phases consume. Each skill owns one phase of the research → plan → implement → validate lifecycle and hands forward a compact artifact, not raw exploration output. + +- `prp-prd` — PRD from a feature request +- `prp-plan` — implementation plan from a PRD or brief +- `prp-implement` — execution from a plan +- `prp-issue-investigate` — investigation artifact from a GitHub issue +- `prp-issue-fix` — fix and PR from an investigation artifact +- `prp-research-team` — research plan from a question +- `prp-codebase-question` — scoped research answer + +### Advisory components + +Challenge, review, and refine decisions. Advisory components do not create primary workflow artifacts and do not modify files directly. + +- `prp-advisor` — pre-work and mid-work critique gate +- `prp-review` — PR review artifact +- `prp-review-agents` — multi-aspect specialized review +- `prp-prd` — also acts as an advisory scope-clarifier before planning +- `plan-critic` agent — targeted plan critique +- `prp-advisor` agent — advisor subagent for the skill +- `codebase-analyst` agent — deep implementation analysis +- `codebase-explorer` agent — file and structure exploration +- `web-researcher` agent — external documentation research + +### Utility components + +Execute one narrow, repeatable job as the final step of a flow. Utilities should not contain research or planning logic. + +- `prp-commit` — atomic commit from staged changes +- `prp-pr` — pull request from a branch +- `prp-verification-before-completion` — final validation gate before claiming done +- `prp-ralph` / `prp-ralph-cancel` / `prp-ralph-loop` — autonomous loop control +- `prp-debug` — root-cause analysis helper + +## Design intent + +- Each class has a different context profile. Artifact skills need bounded briefs and compaction checkpoints. Advisory components need read access and a fixed output shape. Utility components should keep their context footprint small. +- A skill that crosses two classes (for example, an artifact skill that also reviews) should be refactored or the review step should be delegated to an advisory agent. +- See `context-budget-policy.md` for compaction rules and brief size limits. +- See `execution-policy.md` for when to stay inline vs. delegate to a subagent. diff --git a/plugins/prp-core/rules/harness-engineering.md b/plugins/prp-core/rules/harness-engineering.md new file mode 100644 index 00000000..fdb89453 --- /dev/null +++ b/plugins/prp-core/rules/harness-engineering.md @@ -0,0 +1,38 @@ +--- +paths: + - ".claude/**/*.md" + - "**/*.prd.md" + - "**/*.plan.md" +--- +# Harness Engineering Conventions + +Authoritative sources: `references/context-budget-policy.md`, `references/harness-taxonomy.md`. + +## Smart Zone (Priority 1 — CRITICAL) + +- Context < 40% capacity: normal execution. +- Context 40–70%: compact aggressively. Write current state to an artifact; start next phase with a fresh window. +- Context > 70%: delegate remaining work to a subagent; pass only the compacted artifact, not raw context. +- 3,000 filler tokens drop math accuracy from 0.92 → 0.68 (Levy et al., ACL 2024). Every token must earn its place. + +## Harness Rails (Priority 2 — HIGH) + +- Deterministic validation gates precede completion claims. Run `prp-verification-before-completion` before declaring any phase done. +- A behavioral rule repeated more than twice signals a harness gap. Fix the harness, not the wording. + +## Subagent Isolation (Priority 3 — HIGH) + +- Research, analysis, and exploration run in forked subagent contexts. +- Return only condensed output (~1,000 tokens) to the parent agent. +- Never carry raw tool output or exploration transcripts across phase boundaries. + +## Artifact Discipline (Priority 4 — MEDIUM) + +- Each phase (research → plan → implement → review) must produce a durable artifact before the next phase begins. +- Component taxonomy: artifact / advisory / utility — see `references/harness-taxonomy.md`. +- Brief size limits: discovery ≤ 50 lines, execution ≤ 30 lines, validation ≤ 20 lines. + +## Progressive Disclosure (Priority 5 — MEDIUM) + +- Never duplicate policy inline in skills or agents — link to `references/` instead. +- Skills link to reference files; reference files are the single source of truth. diff --git a/plugins/prp-core/rules/prp-workflow.md b/plugins/prp-core/rules/prp-workflow.md index ed0ba776..c7daf80c 100644 --- a/plugins/prp-core/rules/prp-workflow.md +++ b/plugins/prp-core/rules/prp-workflow.md @@ -12,7 +12,7 @@ paths: ## After `*.prd.md` Is Created -- Create a GitHub issue using the skill's `references/issue-template.md`. +- Create a GitHub issue using the `prp-prd` skill's `references/issue-template.md`. - Write the created issue URL into the `` field in the PRD file. ## After `*.prd.md` Is Updated @@ -21,7 +21,7 @@ paths: ## After `*.plan.md` Is Created -- Create a GitHub issue using the skill's `references/issue-template.md`. +- Create a GitHub issue using the `prp-plan` skill's `references/issue-template.md`. - Write the parent PRD issue URL into `` in the plan file. - Write the created plan issue URL into `` in the plan file. diff --git a/plugins/prp-core/skills/prp-advisor/SKILL.md b/plugins/prp-core/skills/prp-advisor/SKILL.md index 5cd9836d..9bb53733 100644 --- a/plugins/prp-core/skills/prp-advisor/SKILL.md +++ b/plugins/prp-core/skills/prp-advisor/SKILL.md @@ -9,6 +9,8 @@ user-invocable: false ## Step 1: Detect Advisor Path +> **Policy**: This is an advisory component per `plugins/prp-core/references/harness-taxonomy.md`. It reviews and challenges decisions but does not create primary workflow artifacts. + Check whether the native `advisor` tool is available in your current tool set. - **`advisor` tool available** → proceed to [Native Advisor](#native-advisor) diff --git a/plugins/prp-core/skills/prp-codebase-question/SKILL.md b/plugins/prp-core/skills/prp-codebase-question/SKILL.md index 9613632c..a904ae84 100644 --- a/plugins/prp-core/skills/prp-codebase-question/SKILL.md +++ b/plugins/prp-core/skills/prp-codebase-question/SKILL.md @@ -11,6 +11,16 @@ Answer codebase questions by spawning parallel specialized agents, synthesizing **CRITICAL**: DO NOT suggest improvements, critique implementations, or propose changes. Only describe what exists, where it lives, and how it works. +## Context Contract + +Before starting, read `plugins/prp-core/references/context-budget-policy.md` and `plugins/prp-core/references/execution-policy.md`. + +- Compact noisy findings into a **bounded brief** before each handoff (discovery ≤ 50 lines, execution ≤ 30 lines, validation ≤ 20 lines). +- Parallelize only independent scopes; keep sequential when scopes share files or state. +- Isolate noisy exploration in subagents; return only the compact artifact to the main context. +- For deterministic batch verification, use the repo-local evaluation harness. +- All PRP artifacts go under `.claude/PRPs/` — see `artifact-lifecycle.md`. + ## Phase 1: PARSE 1. If user mentions specific files, read them FULLY first diff --git a/plugins/prp-core/skills/prp-commit/SKILL.md b/plugins/prp-core/skills/prp-commit/SKILL.md index 8d2f9df6..68379a0a 100644 --- a/plugins/prp-core/skills/prp-commit/SKILL.md +++ b/plugins/prp-core/skills/prp-commit/SKILL.md @@ -7,6 +7,8 @@ description: "Atomic commits by logical scope following Conventional Commits 1.0 ## Strict Rules +> **Policy**: This is a utility component per `plugins/prp-core/references/harness-taxonomy.md`. It does one narrow job: create atomic commits. It must not contain research or planning logic. + - NEVER use `git add .` or `git add -A` — stage files by explicit path only - NEVER add `Co-Authored-By` or any AI attribution footer - One commit per logical scope — never mix unrelated changes diff --git a/plugins/prp-core/skills/prp-implement/SKILL.md b/plugins/prp-core/skills/prp-implement/SKILL.md index 73181e5e..54e0e9da 100644 --- a/plugins/prp-core/skills/prp-implement/SKILL.md +++ b/plugins/prp-core/skills/prp-implement/SKILL.md @@ -11,6 +11,18 @@ Execute the plan end-to-end with rigorous self-validation. You are autonomous. --- +## Context Contract + +Before starting, read `plugins/prp-core/references/context-budget-policy.md` and `plugins/prp-core/references/execution-policy.md`. + +- Compact noisy findings into a **bounded brief** before each handoff (discovery ≤ 50 lines, execution ≤ 30 lines, validation ≤ 20 lines). +- Parallelize only independent scopes; keep sequential when scopes share files or state. +- Isolate noisy exploration in subagents; return only the compact artifact to the main context. +- For deterministic batch verification, use the repo-local evaluation harness. +- All PRP artifacts go under `.claude/PRPs/` — see `artifact-lifecycle.md`. + +--- + ## Phase 0: DETECT — Project Environment ### 0.1 Identify Package Manager diff --git a/plugins/prp-core/skills/prp-issue-fix/SKILL.md b/plugins/prp-core/skills/prp-issue-fix/SKILL.md index f12c0346..a73e32e5 100644 --- a/plugins/prp-core/skills/prp-issue-fix/SKILL.md +++ b/plugins/prp-core/skills/prp-issue-fix/SKILL.md @@ -21,6 +21,18 @@ Execute the implementation plan from the `prp-issue-investigate` skill: --- +## Context Contract + +Before starting, read `plugins/prp-core/references/context-budget-policy.md` and `plugins/prp-core/references/execution-policy.md`. + +- Compact noisy findings into a **bounded brief** before each handoff (discovery ≤ 50 lines, execution ≤ 30 lines, validation ≤ 20 lines). +- Parallelize only independent scopes; keep sequential when scopes share files or state. +- Isolate noisy exploration in subagents; return only the compact artifact to the main context. +- For deterministic batch verification, use the repo-local evaluation harness. +- All PRP artifacts go under `.claude/PRPs/` — see `artifact-lifecycle.md`. + +--- + ## Phase 0: DETECT — Base Branch Load the base branch detection protocol: diff --git a/plugins/prp-core/skills/prp-issue-investigate/SKILL.md b/plugins/prp-core/skills/prp-issue-investigate/SKILL.md index 46ea875e..447eaec9 100644 --- a/plugins/prp-core/skills/prp-issue-investigate/SKILL.md +++ b/plugins/prp-core/skills/prp-issue-investigate/SKILL.md @@ -17,6 +17,18 @@ Investigate the issue/problem and produce a comprehensive implementation plan th --- +## Context Contract + +Before starting, read `plugins/prp-core/references/context-budget-policy.md` and `plugins/prp-core/references/execution-policy.md`. + +- Compact noisy findings into a **bounded brief** before each handoff (discovery ≤ 50 lines, execution ≤ 30 lines, validation ≤ 20 lines). +- Parallelize only independent scopes; keep sequential when scopes share files or state. +- Isolate noisy exploration in subagents; return only the compact artifact to the main context. +- For deterministic batch verification, use the repo-local evaluation harness. +- All PRP artifacts go under `.claude/PRPs/` — see `artifact-lifecycle.md`. + +--- + ## Phase 1: PARSE — Understand Input ### 1.1 Determine Input Type diff --git a/plugins/prp-core/skills/prp-plan/SKILL.md b/plugins/prp-core/skills/prp-plan/SKILL.md index b188b3c2..06616c53 100644 --- a/plugins/prp-core/skills/prp-plan/SKILL.md +++ b/plugins/prp-core/skills/prp-plan/SKILL.md @@ -11,6 +11,18 @@ description: "Create a comprehensive feature implementation plan with deep codeb --- +## Context Contract + +Before starting, read `plugins/prp-core/references/context-budget-policy.md` and `plugins/prp-core/references/execution-policy.md`. + +- Compact noisy findings into a **bounded brief** before each handoff (discovery ≤ 50 lines, execution ≤ 30 lines, validation ≤ 20 lines). +- Parallelize only independent scopes; keep sequential when scopes share files or state. +- Isolate noisy exploration in subagents; return only the compact artifact to the main context. +- For deterministic batch verification, use the repo-local evaluation harness. +- All PRP artifacts go under `.claude/PRPs/` — see `artifact-lifecycle.md`. + +--- + ## Phase 0: DETECT — Input Type Resolution | Input Pattern | Type | Action | diff --git a/plugins/prp-core/skills/prp-prd/SKILL.md b/plugins/prp-core/skills/prp-prd/SKILL.md index a793e411..d2a3b7b9 100644 --- a/plugins/prp-core/skills/prp-prd/SKILL.md +++ b/plugins/prp-core/skills/prp-prd/SKILL.md @@ -17,6 +17,14 @@ INITIATE → FOUNDATION → GROUNDING → DEEP DIVE → FEASIBILITY → DECISION Each phase builds on previous answers. Grounding phases validate assumptions via agents. +## Context Contract + +Before starting, read `plugins/prp-core/references/context-budget-policy.md` and `plugins/prp-core/references/execution-policy.md`. + +- Compact noisy findings into a **bounded brief** before handing off to the plan phase. +- Parallelize only independent research scopes; keep sequential when scopes share state. +- All PRP artifacts go under `.claude/PRPs/prds/` — see `artifact-lifecycle.md`. + ## Phase 1: INITIATE **No input**: Ask "What do you want to build?" diff --git a/plugins/prp-core/skills/prp-research-team/SKILL.md b/plugins/prp-core/skills/prp-research-team/SKILL.md index a8175ddb..581fd3b8 100644 --- a/plugins/prp-core/skills/prp-research-team/SKILL.md +++ b/plugins/prp-core/skills/prp-research-team/SKILL.md @@ -13,6 +13,18 @@ Design a dynamic team of research agents and a structured research plan. Targets --- +## Context Contract + +Before starting, read `plugins/prp-core/references/context-budget-policy.md` and `plugins/prp-core/references/execution-policy.md`. + +- Compact noisy findings into a **bounded brief** before each handoff (discovery ≤ 50 lines, execution ≤ 30 lines, validation ≤ 20 lines). +- Parallelize only independent scopes; keep sequential when scopes share files or state. +- Isolate noisy exploration in subagents; return only the compact artifact to the main context. +- For deterministic batch verification, use the repo-local evaluation harness. +- All PRP artifacts go under `.claude/PRPs/` — see `artifact-lifecycle.md`. + +--- + ## Variables | Variable | Source | Default | diff --git a/plugins/prp-core/skills/prp-review-agents/SKILL.md b/plugins/prp-core/skills/prp-review-agents/SKILL.md index 4468d1f5..cb148153 100644 --- a/plugins/prp-core/skills/prp-review-agents/SKILL.md +++ b/plugins/prp-core/skills/prp-review-agents/SKILL.md @@ -7,6 +7,16 @@ description: "Comprehensive PR review using specialized agents. Runs code qualit Run a multi-agent review on a pull request, with each agent focusing on a specific aspect of code quality. +## Context Contract + +Before starting, read `plugins/prp-core/references/context-budget-policy.md` and `plugins/prp-core/references/execution-policy.md`. + +- Compact noisy findings into a **bounded brief** before each handoff (discovery ≤ 50 lines, execution ≤ 30 lines, validation ≤ 20 lines). +- Parallelize only independent scopes; keep sequential when scopes share files or state. +- Isolate noisy exploration in subagents; return only the compact artifact to the main context. +- For deterministic batch verification, use the repo-local evaluation harness. +- All PRP artifacts go under `.claude/PRPs/` — see `artifact-lifecycle.md`. + ## Pre-Review Setup 1. **Identify the PR** diff --git a/plugins/prp-core/skills/prp-review/SKILL.md b/plugins/prp-core/skills/prp-review/SKILL.md index eaa687d1..a7463dd4 100644 --- a/plugins/prp-core/skills/prp-review/SKILL.md +++ b/plugins/prp-core/skills/prp-review/SKILL.md @@ -11,6 +11,18 @@ Perform a thorough, senior-engineer-level code review. --- +## Context Contract + +Before starting, read `plugins/prp-core/references/context-budget-policy.md` and `plugins/prp-core/references/execution-policy.md`. + +- Compact noisy findings into a **bounded brief** before each handoff (discovery ≤ 50 lines, execution ≤ 30 lines, validation ≤ 20 lines). +- Parallelize only independent scopes; keep sequential when scopes share files or state. +- Isolate noisy exploration in subagents; return only the compact artifact to the main context. +- For deterministic batch verification, use the repo-local evaluation harness. +- All PRP artifacts go under `.claude/PRPs/` — see `artifact-lifecycle.md`. + +--- + ## Phase 1: FETCH — Get PR Context ### 1.1 Parse Input diff --git a/plugins/prp-core/skills/prp-verification-before-completion/SKILL.md b/plugins/prp-core/skills/prp-verification-before-completion/SKILL.md index 2856207c..6e766c59 100644 --- a/plugins/prp-core/skills/prp-verification-before-completion/SKILL.md +++ b/plugins/prp-core/skills/prp-verification-before-completion/SKILL.md @@ -7,6 +7,8 @@ description: Use when about to claim a plan or work is complete, fixed, or passi ## Overview +> **Policy**: This is a utility component per `plugins/prp-core/references/harness-taxonomy.md`. It does one narrow job: verify that work is complete before any completion claim. + Claiming work is complete without verification is dishonesty, not efficiency. **Core principle:** Evidence before claims, always. diff --git a/scripts/prp_core_prompt_eval.py b/scripts/prp_core_prompt_eval.py new file mode 100644 index 00000000..995f2bf0 --- /dev/null +++ b/scripts/prp_core_prompt_eval.py @@ -0,0 +1,140 @@ +"""Deterministic prompt-contract evaluator for prp-core. + +Checks shipped skills, agents, and hooks for required contract strings, +and verifies that shipped plugin files match their .claude/ mirror copies. + +Usage: + uv run python scripts/prp_core_prompt_eval.py --list + uv run python scripts/prp_core_prompt_eval.py --case plan + uv run python scripts/prp_core_prompt_eval.py --case implement + uv run python scripts/prp_core_prompt_eval.py --case review + uv run python scripts/prp_core_prompt_eval.py --case issue-fix + uv run python scripts/prp_core_prompt_eval.py --case mirror-parity + uv run python scripts/prp_core_prompt_eval.py --case research-team-hook-schema + uv run python scripts/prp_core_prompt_eval.py --all + +Exit code 0 means all checks pass. Exit code 1 means one or more checks failed. +""" + +from __future__ import annotations + +import argparse +import json +import sys +from pathlib import Path + +CASES_PATH = Path("tests/prp-core/prompt-cases.json") +GREEN = "\033[92m" +RED = "\033[91m" +YELLOW = "\033[93m" +RESET = "\033[0m" +BOLD = "\033[1m" + + +def load_cases(path: Path) -> list[dict]: + if not path.exists(): + print(f"{RED}ERROR: cases file not found: {path}{RESET}", file=sys.stderr) + sys.exit(1) + return json.loads(path.read_text()) + + +def evaluate_must_contain(case: dict) -> list[str]: + """Check that each target file contains all required strings.""" + failures: list[str] = [] + targets = case.get("targets", []) + needles = case.get("must_contain", []) + for target in targets: + p = Path(target) + if not p.exists(): + failures.append(f" {RED}MISSING FILE{RESET}: {target}") + continue + text = p.read_text() + for needle in needles: + if needle not in text: + failures.append( + f" {RED}MISSING CONTRACT{RESET}: '{needle}' not found in {target}" + ) + return failures + + +def evaluate_mirror_parity(case: dict) -> list[str]: + """Check that each (shipped, mirror) pair is identical.""" + failures: list[str] = [] + for pair in case.get("mirror_pairs", []): + left_path, right_path = Path(pair[0]), Path(pair[1]) + if not left_path.exists(): + failures.append(f" {RED}MISSING{RESET}: shipped file not found: {left_path}") + continue + if not right_path.exists(): + failures.append(f" {RED}MISSING{RESET}: mirror file not found: {right_path}") + continue + if left_path.read_text() != right_path.read_text(): + failures.append(f" {RED}DRIFT{RESET}: {left_path} != {right_path}") + return failures + + +def run_case(case: dict) -> tuple[int, int]: + """Run one case. Returns (pass_count, fail_count).""" + case_id = case["id"] + description = case.get("description", "") + print(f"\n{BOLD}Case: {case_id}{RESET} {description}") + + if case_id == "mirror-parity": + failures = evaluate_mirror_parity(case) + else: + failures = evaluate_must_contain(case) + + if failures: + for f in failures: + print(f) + print(f" {RED}FAIL{RESET} [{case_id}]: {len(failures)} check(s) failed") + return 0, len(failures) + else: + print(f" {GREEN}PASS{RESET} [{case_id}]") + return 1, 0 + + +def main() -> None: + parser = argparse.ArgumentParser( + description="Deterministic prompt-contract evaluator for prp-core." + ) + group = parser.add_mutually_exclusive_group(required=True) + group.add_argument("--list", action="store_true", help="List available case IDs") + group.add_argument("--case", metavar="ID", help="Run a specific case by ID") + group.add_argument("--all", action="store_true", help="Run all cases") + args = parser.parse_args() + + cases = load_cases(CASES_PATH) + + if args.list: + print(f"{BOLD}Available prompt-eval cases:{RESET}") + for case in cases: + print(f" {case['id']:30s} {case.get('description', '')}") + print(f"\nRun with: uv run python scripts/prp_core_prompt_eval.py --case ") + print(f"Docs: docs/prp-core-prompt-evals.md") + return + + if args.all: + total_pass = total_fail = 0 + for case in cases: + p, f = run_case(case) + total_pass += p + total_fail += f + print(f"\n{BOLD}Results:{RESET} {total_pass} passed, {total_fail} failed") + if total_fail > 0: + sys.exit(1) + return + + case_map = {c["id"]: c for c in cases} + if args.case not in case_map: + available = ", ".join(case_map) + print(f"{RED}ERROR{RESET}: unknown case '{args.case}'. Available: {available}", file=sys.stderr) + sys.exit(1) + + p, f = run_case(case_map[args.case]) + if f > 0: + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/tests/prp-core/README.md b/tests/prp-core/README.md new file mode 100644 index 00000000..dcb10fc7 --- /dev/null +++ b/tests/prp-core/README.md @@ -0,0 +1,83 @@ +# PRP Core Prompt Evaluation Harness + +This directory contains the repo-only prompt-contract evaluation harness for `prp-core`. It is not part of the shipped plugin payload. + +## Purpose + +The harness runs deterministic checks against shipped skill files, agents, and hooks to catch: + +- Missing contract language (bounded brief sizes, context-budget-policy references, execution-policy references) +- Mirror drift (shipped `plugins/prp-core/` files diverging from `.claude/` mirror copies) +- Hook schema alignment (required research plan sections present in the stop hook) + +## Structure + +``` +tests/prp-core/ +├── README.md # This file +├── prompt-cases.json # Test case definitions +└── fixtures/ + ├── plan-request.txt # Example prp-plan invocation + ├── implement-request.txt # Example prp-implement invocation + ├── review-request.txt # Example prp-review invocation + └── issue-fix-request.txt # Example prp-issue-fix invocation +``` + +## Running the harness + +```bash +# List available cases +uv run python scripts/prp_core_prompt_eval.py --list + +# Run a specific case +uv run python scripts/prp_core_prompt_eval.py --case plan +uv run python scripts/prp_core_prompt_eval.py --case implement +uv run python scripts/prp_core_prompt_eval.py --case review +uv run python scripts/prp_core_prompt_eval.py --case issue-fix +uv run python scripts/prp_core_prompt_eval.py --case mirror-parity +uv run python scripts/prp_core_prompt_eval.py --case research-team-hook-schema + +# Run all cases +uv run python scripts/prp_core_prompt_eval.py --all +``` + +Exit code 0 = all checks pass. Exit code 1 = one or more checks failed. + +## Case types + +### Contract checks (`must_contain`) + +Check that target skill or hook files contain specific required strings. These catch when a skill is updated but the shared contract language is removed or never added. + +### Mirror parity (`mirror-parity`) + +Check that shipped plugin files in `plugins/prp-core/` match their `.claude/` mirror copies line-for-line. Both trees must stay in sync after every skill or agent update. + +### Hook schema alignment (`research-team-hook-schema`) + +Check that `prp-research-team-stop.sh` validates all six required research plan sections. This ensures the hook, skill, and documentation stay aligned. + +## CI + +The harness runs on pull requests that touch `plugins/prp-core/`, `.claude/`, `scripts/prp_core_prompt_eval.py`, or `tests/prp-core/`. See `.github/workflows/prp-core-prompt-evals.yml`. + +## Adding a new case + +Add a new entry to `prompt-cases.json`: + +```json +{ + "id": "my-case", + "description": "What this case checks", + "must_contain": ["required string 1", "required string 2"], + "targets": ["plugins/prp-core/skills/my-skill/SKILL.md"] +} +``` + +For mirror parity cases, use `mirror_pairs` instead of `targets` + `must_contain`. See existing cases for the format. + +## See also + +- `docs/prp-core-prompt-evals.md` — full evaluation model rationale +- `plugins/prp-core/references/context-budget-policy.md` — contract strings being tested +- `plugins/prp-core/references/execution-policy.md` — delegation contract being tested diff --git a/tests/prp-core/fixtures/implement-request.txt b/tests/prp-core/fixtures/implement-request.txt new file mode 100644 index 00000000..1326bd2f --- /dev/null +++ b/tests/prp-core/fixtures/implement-request.txt @@ -0,0 +1 @@ +Run the prp-implement skill with .claude/PRPs/plans/add-pagination.plan.md. Compact noisy findings into a bounded brief before handing off to the next phase. diff --git a/tests/prp-core/fixtures/issue-fix-request.txt b/tests/prp-core/fixtures/issue-fix-request.txt new file mode 100644 index 00000000..d08147b1 --- /dev/null +++ b/tests/prp-core/fixtures/issue-fix-request.txt @@ -0,0 +1 @@ +Run the prp-issue-fix skill for issue 123. Follow the execution-policy: use isolated subagents for noisy work and stay inline for deterministic steps. diff --git a/tests/prp-core/fixtures/plan-request.txt b/tests/prp-core/fixtures/plan-request.txt new file mode 100644 index 00000000..49b27f31 --- /dev/null +++ b/tests/prp-core/fixtures/plan-request.txt @@ -0,0 +1 @@ +Run the prp-plan skill to create an implementation plan for adding paginated search to the API. The plan should use bounded brief summaries and respect the context-budget-policy. diff --git a/tests/prp-core/fixtures/review-request.txt b/tests/prp-core/fixtures/review-request.txt new file mode 100644 index 00000000..f15e9c57 --- /dev/null +++ b/tests/prp-core/fixtures/review-request.txt @@ -0,0 +1 @@ +Run the prp-review skill for PR 42. Use isolated subagents for review aspects that are independent. Summarize findings into a bounded brief before posting. diff --git a/tests/prp-core/prompt-cases.json b/tests/prp-core/prompt-cases.json new file mode 100644 index 00000000..ce4e42fe --- /dev/null +++ b/tests/prp-core/prompt-cases.json @@ -0,0 +1,64 @@ +[ + { + "id": "plan", + "description": "prp-plan skill must include context-budget and artifact compaction language", + "input_file": "tests/prp-core/fixtures/plan-request.txt", + "must_contain": ["bounded brief", "context-budget-policy", "artifact"], + "targets": ["plugins/prp-core/skills/prp-plan/SKILL.md"] + }, + { + "id": "implement", + "description": "prp-implement skill must include bounded brief size limits and compaction checkpoint language", + "input_file": "tests/prp-core/fixtures/implement-request.txt", + "must_contain": ["bounded brief", "context-budget-policy", "execution-policy"], + "targets": ["plugins/prp-core/skills/prp-implement/SKILL.md"] + }, + { + "id": "review", + "description": "prp-review skill must include bounded brief and delegation language", + "input_file": "tests/prp-core/fixtures/review-request.txt", + "must_contain": ["bounded brief", "context-budget-policy"], + "targets": ["plugins/prp-core/skills/prp-review/SKILL.md"] + }, + { + "id": "issue-fix", + "description": "prp-issue-fix skill must include execution-policy and bounded brief language", + "input_file": "tests/prp-core/fixtures/issue-fix-request.txt", + "must_contain": ["bounded brief", "execution-policy"], + "targets": ["plugins/prp-core/skills/prp-issue-fix/SKILL.md"] + }, + { + "id": "mirror-parity", + "description": "Shipped plugin skills must match their .claude/ mirror copies", + "mirror_pairs": [ + ["plugins/prp-core/skills/prp-plan/SKILL.md", ".claude/skills/prp-plan/SKILL.md"], + ["plugins/prp-core/skills/prp-implement/SKILL.md", ".claude/skills/prp-implement/SKILL.md"], + ["plugins/prp-core/skills/prp-review/SKILL.md", ".claude/skills/prp-review/SKILL.md"], + ["plugins/prp-core/skills/prp-issue-fix/SKILL.md", ".claude/skills/prp-issue-fix/SKILL.md"], + ["plugins/prp-core/skills/prp-issue-investigate/SKILL.md", ".claude/skills/prp-issue-investigate/SKILL.md"], + ["plugins/prp-core/skills/prp-codebase-question/SKILL.md", ".claude/skills/prp-codebase-question/SKILL.md"], + ["plugins/prp-core/skills/prp-research-team/SKILL.md", ".claude/skills/prp-research-team/SKILL.md"], + ["plugins/prp-core/skills/prp-advisor/SKILL.md", ".claude/skills/prp-advisor/SKILL.md"], + ["plugins/prp-core/skills/prp-commit/SKILL.md", ".claude/skills/prp-commit/SKILL.md"], + ["plugins/prp-core/skills/prp-verification-before-completion/SKILL.md", ".claude/skills/prp-verification-before-completion/SKILL.md"], + ["plugins/prp-core/references/harness-taxonomy.md", ".claude/references/harness-taxonomy.md"], + ["plugins/prp-core/references/context-budget-policy.md", ".claude/references/context-budget-policy.md"], + ["plugins/prp-core/references/execution-policy.md", ".claude/references/execution-policy.md"], + ["plugins/prp-core/references/artifact-lifecycle.md", ".claude/references/artifact-lifecycle.md"], + ["plugins/prp-core/references/agent-prompt-style.md", ".claude/references/agent-prompt-style.md"] + ] + }, + { + "id": "research-team-hook-schema", + "description": "prp-research-team-stop.sh hook must validate all six required research plan sections", + "must_contain": [ + "## Research Question", + "## Research Question Decomposition", + "## Team Composition", + "## Research Tasks", + "## Team Orchestration Guide", + "## Acceptance Criteria" + ], + "targets": ["plugins/prp-core/hooks/prp-research-team-stop.sh"] + } +] diff --git a/wiki/CLAUDE.md b/wiki/CLAUDE.md new file mode 100644 index 00000000..4505862c --- /dev/null +++ b/wiki/CLAUDE.md @@ -0,0 +1,82 @@ +# LLM Wiki + +A personal knowledge base maintained by Claude Code. +Based on Andrej Karpathy's LLM Wiki pattern. + +## Purpose + +Claude maintains the wiki. The human curates sources, asks questions, and guides the analysis. + +## Ingest workflow + +When the user adds a new source to `docs/` and asks you to ingest it: + +1. Read the full source document +2. Discuss key takeaways with the user before writing anything +3. Create a summary page in `wiki/knowledge` named after the source +4. Create or update concept pages for each major idea or entity +5. Add knowledge-links ([[page-name]]) to connect related pages +6. Update `wiki/knowledge/index.md` with new pages and one-line descriptions +7. Append an entry to `wiki/knowledge/log.md` with the date, source name, and what changed + +A single source may touch 10-15 wiki pages. That is normal. + +## Page format + +Every wiki knowledge page should follow this structure: + +```markdown +# Page Title + +**Summary**: One to two sentences describing this page. +**Sources**: List of raw source files this page draws from. +**Last updated**: Date of most recent update. +--- + +Main content goes here. Use clear headings and short paragraphs. + +Link to related concepts using [[knowledge-links]] throughout the text. + +## Related pages + +- [[related-concept-1]] +- [[related-concept-2]] +``` + +## Citation rules + +- Every factual claim should reference its source file +- Use the format (source: filename.pdf) after the claim +- If two sources disagree, note the contradiction explicitly +- If a claim has no source, mark it as needing verification + +## Question answering + +When the user asks a question: + +1. Read `wiki/knowledge/index.md` first to find relevant pages +2. Read those pages and synthesize an answer +3. Cite specific wiki knowledge pages in your response +4. If the answer is not in the wiki knowledge, say so clearly +5. If the answer is valuable, offer to save it as a new wiki knowledge page + +Good answers should be filed back into the wiki knowledge so they compound over time. + +## Lint + +When the user asks you to lint or audit the wiki: + +- Check for contradictions between pages +- Find orphan pages (no inbound links from other pages) +- Identify concepts mentioned in pages that lack their own page +- Flag claims that may be outdated based on newer sources +- Check that all pages follow the page format above +- Report findings as a numbered list with suggested fixes + +## Rules + +- Never modify anything in the `docs/` folder +- Always update `wiki/knowledge/index.md` and `wiki/knowledge/log.md` after changes +- Keep page names lowercase with hyphens (e.g. `machine-learning.md`) +- Write in clear, plain language +- When uncertain about how to categorize something, ask the user diff --git a/wiki/knowledge/a2a-protocol.md b/wiki/knowledge/a2a-protocol.md new file mode 100644 index 00000000..d3031207 --- /dev/null +++ b/wiki/knowledge/a2a-protocol.md @@ -0,0 +1,178 @@ +# A2A Protocol + +**Summary**: The Agent-to-Agent (A2A) protocol by Google is an open HTTP-based standard for communication and interoperability between AI agents from different vendors and frameworks, using Agent Cards, task lifecycles, and streaming interactions. +**Sources**: `docs/agent-protocols/a2a-protocol-huggingface-space.md` +**Last updated**: 2026-04-21 + +--- + +## What is A2A? + +A2A (Agent-to-Agent) is an open-source framework launched by Google to facilitate communication and interoperability among AI agents (source: a2a-protocol-huggingface-space.md). It provides a standardized collaboration method for agents regardless of their underlying frameworks or vendors, enabling agents to securely exchange information, coordinate actions, and operate across diverse enterprise platforms. + +**Core question A2A answers**: How can AI agents developed by different teams, using different technologies, and owned by different organizations effectively communicate and collaborate? (source: a2a-protocol-huggingface-space.md) + +## Why A2A is Needed + +As AI agents become increasingly specialized, the need for them to collaborate on complex tasks grows. Without a common communication protocol, integrating disparate agents requires custom point-to-point solutions, making systems difficult to scale, maintain, and extend (source: a2a-protocol-huggingface-space.md). + +**Example**: A user asks their primary agent to plan an international trip. This single request might involve coordinating multiple specialized agents: +1. An agent for flight bookings +2. An agent for hotel reservations +3. An agent for local tour recommendations +4. An agent handling currency conversion and travel advisories + +A2A provides the communication layer to orchestrate all four without custom connectors. + +## Application Scenarios + +**Enterprise Automation** (source: a2a-protocol-huggingface-space.md): A supply chain planning agent can coordinate with inventory management, logistics, and procurement agents — even if built by different vendors or on different frameworks. + +**Multi-Agent Collaboration**: A2A facilitates true multi-agent scenarios where agents can collaborate in natural, unstructured modes, even without shared memory, tools, or context. This goes beyond simply using one agent as a "tool" for another. See [[multi-agent-communication]] for orchestration patterns. + +**Cross-Platform Integration**: A2A allows agents to operate across an entire ecosystem of enterprise applications — CRM systems, knowledge bases, project management tools, and more. + +## Architecture + +A2A facilitates communication between **client agents** and **remote agents** (source: a2a-protocol-huggingface-space.md): + +- **User**: End user (human or automated service) initiating a request +- **A2A Client (Client Agent)**: Application or AI agent representing the user +- **A2A Server (Remote Agent)**: AI agent exposing an HTTP endpoint implementing A2A + +**Transport**: HTTP(S) with JSON-RPC 2.0 payloads (source: a2a-protocol-huggingface-space.md) + +**Authentication**: OAuth tokens, API keys, or mTLS passed via HTTP headers + +## Core Concepts + +### Agent Card + +A JSON metadata document discoverable via `/.well-known/agent.json` (source: a2a-protocol-huggingface-space.md). Describes: +- Agent identity, capabilities, and skills +- Endpoint URLs +- Authentication requirements +- Supported input/output media types + +Clients use the agent card to discover agents and learn how to interact securely. + +```typescript +interface AgentCard { + name: string; // e.g., "Recipe Agent" + description: string; + url: string; // Hosted URL + version: string; // e.g., "1.0.0" + capabilities: AgentCapabilities; + defaultInputModes: string[]; + defaultOutputModes: string[]; + skills: AgentSkill[]; + securitySchemes?: { [scheme: string]: SecurityScheme }; +} +``` + +### Task + +When a client sends a message, the agent may determine that fulfilling the request requires a task (source: a2a-protocol-huggingface-space.md). Each task has a unique ID and progresses through a defined lifecycle: + +``` +submitted → working → input-required → completed | failed +``` + +Tasks are stateful and may involve multiple exchanges. + +### Message + +Represents a single turn or unit of communication (source: a2a-protocol-huggingface-space.md). Has a `role` (user or agent) and contains one or more `Part` objects. Used to convey instructions, context, questions, answers, or status updates. + +### Part (Content Types) + +- `TextPart`: Plain text content +- `FilePart`: File transmitted as inline base64-encoded bytes or referenced via URI +- `DataPart`: Structured JSON data (forms, parameters, machine-readable information) + +### Artifact + +Represents output results generated by the remote agent (source: a2a-protocol-huggingface-space.md). Examples: generated documents, images, spreadsheets, structured data. Consists of one or more `Part` objects; can be streamed incrementally. + +## Interaction Mechanisms + +### Request/Response (Polling) + +Client sends request via `message/send` RPC method. Server may initially respond with `working` status; client polls via `tasks/get` until a terminal state (source: a2a-protocol-huggingface-space.md). + +### Streaming (SSE) + +Used for tasks producing results incrementally or providing real-time progress updates (source: a2a-protocol-huggingface-space.md). Client initiates with `message/stream`; server sends a stream of Server-Sent Events (SSE). Event types: `Task`, `Message`, `TaskStatusUpdateEvent`, `TaskArtifactUpdateEvent`. + +### Push Notifications + +For very long-running tasks or when persistent connections are impractical (source: a2a-protocol-huggingface-space.md). Client provides a webhook URL; server sends HTTP POST notifications on significant state changes. + +## Real-World Use Case: Employee Onboarding + +A new employee is hired. With A2A, each department exposes its own agent (source: a2a-protocol-huggingface-space.md): + +| Agent | Responsibilities | +|-------|-----------------| +| `hr-agent.company.com` | Create employee records, send documents | +| `it-agent.company.com` | Set up email accounts, order laptops | +| `facilities-agent.company.com` | Assign desks, print badges | + +The `OnboardingPro` orchestrator: +1. **Discovery**: Reads each agent's `.well-known/agent.json` +2. **Task Delegation**: Sends appropriate tasks to each specialized agent +3. **Continuous Updates**: Agents stream back progress via SSE +4. **Artifact Collection**: Final outputs returned as A2A artifacts +5. **Completion**: Notifies the hiring manager + +## A2A and MCP: Complementary, Not Competing + +(source: a2a-protocol-huggingface-space.md) + +- **[[mcp-specification]]**: Focuses on connecting individual AI models with external tools and data sources (Model ↔ Data/Tools). "If MCP is a socket wrench (for tools)..." +- **A2A**: Focuses on communication and collaboration between multiple AI agents (Agent ↔ Agent). "...then A2A is the conversation between mechanics (for collaboration)." + +> An agent application might use A2A to communicate with other agents, while internally, each agent employs MCP to interact with its specific tools and resources. + +See [[mcp-vs-a2a]] for a full protocol comparison and [[agent-protocol-standards]] for the broader standardization landscape. + +## Python SDK Quick Start + +```python +from a2a.server.apps import A2AStarletteApplication +from a2a.types import AgentCapabilities, AgentCard, AgentSkill + +# Install: uv add a2a-sdk uvicorn (Python 3.10+ required) + +skill = AgentSkill( + id='hello_world', + name='Returns hello world', + description='just returns hello world', + tags=['hello world'], + examples=['hi', 'hello world'], +) + +public_agent_card = AgentCard( + name='Hello World Agent', + description='Just a hello world agent', + url='http://0.0.0.0:9999/', + version='1.0.0', + defaultInputModes=['text'], + defaultOutputModes=['text'], + capabilities=AgentCapabilities(streaming=True), + skills=[skill], +) +``` + +Official Python SDK: https://github.com/google-a2a/a2a-python + +## Related pages + +- [[mcp-vs-a2a]] +- [[mcp-specification]] +- [[agent-protocol-standards]] +- [[multi-agent-communication]] +- [[agent-communication-protocols]] +- [[agent-workflows]] +- [[subagents]] +- [[agent-best-practices]] diff --git a/wiki/knowledge/agent-best-practices.md b/wiki/knowledge/agent-best-practices.md new file mode 100644 index 00000000..906b42ab --- /dev/null +++ b/wiki/knowledge/agent-best-practices.md @@ -0,0 +1,87 @@ +# Agent Best Practices + +**Summary**: Practical guidance for agent-assisted development from the current workflow and configuration docs — covering the harness model (instructions + tools + model), context management, plan-first execution, rule/skill differentiation, and code review rigor. +**Sources**: research-agent-workflows-and-patterns.md, a-guide-to-agents.md, `docs/harness-engineering/harness-engineering.md`, `docs/harness-engineering/harnessengineering-building-the-operating-system-for-autonomous-agents.md` +**Last updated**: 2026-04-22 + +--- + +## Agent Harness Model + +Every coding agent = **Instructions** + **Tools** + **Model** + +| Component | Purpose | Examples | +| ------------ | ------------------- | -------------------------------------- | +| Instructions | Guide behavior | CLAUDE.md, AGENTS.md, path-scoped rules | +| Tools | Extend capabilities | Terminal, browser, search, MCP servers | +| Model | Core reasoning | Claude, GPT, etc. | + +## Workflow Principles + +### 1. Plan Before Implementing + +- Start with an explicit planning phase before coding +- **Spec → Plan → Execute** separates thinking from doing +- Discrete loops prevent "getting over your skis" + +### 2. Context Management + +- Let the agent find files on demand — don't manually tag everything +- Fresh conversations when switching tasks (stale context poisons reasoning) +- After 2 failed corrections, `/clear` and restart +- Failed approaches accumulate and degrade performance + +### 3. Rules vs Skills + +| Aspect | Rules | Skills | +| ----------- | --------------------------- | ----------------------------------- | +| Loading | Always or glob-matched | On relevance or manual | +| Content | Static context, conventions | Dynamic workflows, domain knowledge | +| Size | Short, focused | Multi-phase, can reference files | +| When to add | Agent repeats same mistake | Reusable multi-step workflow | + +### 4. Specific Prompts + +"Write a test for auth.ts covering the logout edge case" beats "Write tests." + +### 5. Code Review Rigor + +Treat AI output like a junior developer's code review — verify, don't trust blindly. + +## Anti-Patterns + +| Anti-Pattern | Why It Fails | +| -------------------------------------- | ------------------------------------------------ | +| Copying entire style guides into rules | Bloats context; agent already knows conventions | +| Documenting every command | Agent knows npm, git, pytest | +| Adding rules for rare edge cases | Low signal-to-noise ratio | +| One-shotting complex projects | Models perform poorly without verification loops | +| Duplicating codebase knowledge | Agent can read the code itself | + +## Context Budget Rules + +- Keep config files under **200 lines** (~2,000–4,000 tokens) +- **~150–200 instructions** is the consistency limit for frontier LLMs +- Every token in always-loaded files loads on **every request** +- Test: "Would removing this cause mistakes?" If not, cut it + +## Parallel Execution + +- Route research and noisy verification to [[subagents]] +- Use isolated workspaces or worktrees when parallel tasks should not share file state +- Separate research, implementation, and review roles so each context stays narrow + +## Two-Agent Architecture (Long-Running Tasks) + +1. **Initializer** (first session) — creates structured artifacts (feature list, progress file) +2. **Coding Agent** (incremental) — picks up from artifacts, makes progress, updates state +3. Use **JSON for state** (model less likely to corrupt vs. Markdown) +4. Use **git history as state tracking** mechanism + +## Related pages + +- [[agent-workflows]] +- [[context-engineering]] +- [[progressive-disclosure]] +- [[agent-configuration-files]] +- [[evaluating-agents-paper]] diff --git a/wiki/knowledge/agent-communication-protocols.md b/wiki/knowledge/agent-communication-protocols.md new file mode 100644 index 00000000..3aba5e94 --- /dev/null +++ b/wiki/knowledge/agent-communication-protocols.md @@ -0,0 +1,76 @@ +# Agent Communication Protocols + +**Summary**: A structured academic overview of the four emerging communication standards for AI agents — MCP, ACP, A2A, and ANP — arguing that standardized protocols are essential for interoperability, tool discovery, and coordinated task execution across LLM-powered agents. +**Sources**: `docs/agent-protocols/advancing-agentic-ai-communication-protocols.md` +**Last updated**: 2026-04-21 + +--- + +## Overview + +Autonomous agents powered by Large Language Models require reliable and standardized frameworks to connect tools, exchange contextual information, and synchronize tasks across diverse systems (source: advancing-agentic-ai-communication-protocols.md). Without such standards, developers must manually create interfaces, handle authentication protocols, and navigate incompatible function-calling standards across platforms. + +This paper from the International Journal of Scientific Research in Science and Technology (IJSRST) explores four protocols that address this problem: [[mcp-specification]], A2A, ACP, and ANP (source: advancing-agentic-ai-communication-protocols.md). + +## The Four Protocols + +### Model Context Protocol (MCP) + +- **Architecture**: JSON-RPC based client-server (source: advancing-agentic-ai-communication-protocols.md) +- **Primary use**: Connecting agents to external tools, databases, and APIs +- **Key feature**: Secure tool execution with well-typed data transfer +- **Created by**: Anthropic (November 2024), donated to the Linux Foundation + +MCP solves the agent-to-tool connectivity problem. See [[mcp-specification]] for full detail and [[mcp-vs-a2a]] for a comparison. + +### Agent-to-Agent Protocol (A2A) + +- **Architecture**: HTTP-based, JSON-RPC with SSE streaming (source: advancing-agentic-ai-communication-protocols.md) +- **Primary use**: Peer-to-peer task delegation between agents +- **Key feature**: Capability-rich Agent Cards for agent discovery +- **Created by**: Google (April 2025), donated to the Linux Foundation + +A2A enables [[multi-agent-communication]] without requiring shared internal logic. See [[agent-to-agent-protocol]] for full detail. + +### Agent Communication Protocol (ACP) + +- **Architecture**: REST-compliant (source: advancing-agentic-ai-communication-protocols.md) +- **Primary use**: Asynchronous streaming, multimodal agent outputs +- **Key feature**: Support for multipart message formats +- **Created by**: IBM Research; merged into A2A in September 2025 + +ACP is now deprecated. Its capabilities have been absorbed into A2A under the Linux Foundation (source: advancing-agentic-ai-communication-protocols.md). + +### Agent Network Protocol (ANP) + +- **Architecture**: Decentralized, using Decentralized Identifiers (DIDs) and JSON-LD semantic graphs (source: advancing-agentic-ai-communication-protocols.md) +- **Primary use**: Agent discovery in open networks and secure collaboration +- **Key feature**: DIDs for decentralized identity +- **Created by**: Agent Network Protocol Contributors + +ANP addresses open-network discovery scenarios where agents from unknown organizations must find and authenticate each other without a central registry. + +## Why Standardization Matters + +Without standardized protocols, the integration surface between agents and tools grows as N×M — every agent must implement a custom connector for every tool. Standardization collapses this to M+N (source: advancing-agentic-ai-communication-protocols.md). This applies equally to [[multi-agent-communication]], where each agent pair would otherwise require bespoke negotiation logic. + +Standardized protocols also enable: +- **Interoperability**: Agents from different vendors can discover and communicate with each other +- **Tool discovery**: Agents can find and invoke tools without hard-coded connectors +- **Coordinated task execution**: Distributed workflows become composable rather than monolithic + +See [[agent-protocol-standards]] for a synthesized comparison across all four protocols and guidance on when to use each. + +## Historical Context + +Earlier multi-agent communication standards include KQML (1993) and FIPA (2000), which defined communicative act libraries for agent coordination (source: advancing-agentic-ai-communication-protocols.md). The current wave of protocols builds on those foundations but is designed for LLM-native, HTTP-first environments. + +## Related pages + +- [[agent-protocol-standards]] +- [[mcp-specification]] +- [[agent-to-agent-protocol]] +- [[mcp-vs-a2a]] +- [[multi-agent-communication]] +- [[context-engineering]] +- [[agent-workflows]] diff --git a/wiki/knowledge/agent-configuration-files.md b/wiki/knowledge/agent-configuration-files.md new file mode 100644 index 00000000..ec749d87 --- /dev/null +++ b/wiki/knowledge/agent-configuration-files.md @@ -0,0 +1,65 @@ +# Agent Configuration Files + +**Summary**: Repository-level context files such as AGENTS.md, CLAUDE.md, and Claude Code path-scoped rules provide persistent project instructions to coding agents — and work best when they stay minimal, specific, and non-redundant. +**Sources**: a-guide-to-agents.md, how-claude-remembers-a-project.md, Evaluating-AGENTS-paper.md +**Last updated**: 2026-04-22 + +--- + +## File Types by Platform + +| File | Platform | Purpose | Loaded | +| ----------------------- | -------------- | ------------------------------------ | ------------------------------------ | +| **CLAUDE.md** | Claude Code | Project-specific instructions | Every request (directory hierarchy) | +| **AGENTS.md** | Cross-platform | Portable agent instructions | Every request | +| **.claude/rules/*.md** | Claude Code | Path-scoped rules with glob patterns | When matching files touched | + +## CLAUDE.md Hierarchy + +Claude Code discovers CLAUDE.md files via directory traversal: + +1. **Managed policy** — `/etc/claude-code/CLAUDE.md` (Linux), `C:\Program Files\ClaudeCode\CLAUDE.md` (Windows) +2. **User** — `~/.claude/CLAUDE.md` +3. **Project** — `./CLAUDE.md` or `./.claude/CLAUDE.md` +4. **Subdirectory** — Nested CLAUDE.md files (lazy loading) + +**Load precedence**: Enterprise > Personal > Project + +### Import Syntax + +- `@README` — Import file content +- `@package.json` — Import project files +- `@~/.claude/my-instructions.md` — Cross-project imports +- Import depth limit: **5 hops maximum** + +## Line Budgets + +| Config File | Target | Maximum | +| ---------------- | --------------- | --------- | +| Root CLAUDE.md | 15–40 lines | 200 lines | +| Plugin CLAUDE.md | 10–30 lines | 200 lines | +| Rule files | 10–30 lines | 200 lines | + +## What Belongs in Config Files + +**Include** (unique value): + +- Build and test commands +- Project-specific conventions not inferable from code +- Non-obvious patterns and gotchas +- Package manager preferences + +**Exclude** (redundant or harmful): + +- Standard language conventions (agents already know these) +- Information inferable from the codebase +- Dense paragraphs (use headers and bullets) +- Personal preferences in project files +- Conflicting rules (model picks one arbitrarily) + +## Related pages + +- [[progressive-disclosure]] +- [[evaluating-agents-paper]] +- [[claude-code-memory]] +- [[context-engineering]] diff --git a/wiki/knowledge/agent-harness-design.md b/wiki/knowledge/agent-harness-design.md new file mode 100644 index 00000000..01bc8f83 --- /dev/null +++ b/wiki/knowledge/agent-harness-design.md @@ -0,0 +1,200 @@ +# Agent Harness Design + +**Summary**: Design patterns for building agent harnesses — the skills, workflows, and methodology layers that encode how an agent should think and build — covering isolation, context management, tool access, component taxonomy, and iterative evolution principles. +**Sources**: `docs/agentic-engineering/building-agent-harness-martin-richards.md`, `docs/agentic-engineering/research-plan-implement-rpi.md`, `docs/agentic-engineering/research-plan-implement-review-tyler-burleigh.md` +**Last updated**: 2026-04-22 + +--- + +## Overview + +An agent harness is the structured layer sitting between a raw LLM and a complex software task. It encodes discipline: how to research, how to plan, how to implement, and when to stop. See [[agent-harness]] for the practitioner-level case for building one. This page focuses on the design patterns that make harnesses effective. + +The core design principle is that **harness quality matters more than model choice** — LangChain improved their agent from 52.8% to 66.5% on Terminal Bench 2.0 by only changing the harness, keeping the model fixed (source: building-agent-harness-martin-richards.md). + +## Component Taxonomy + +Effective harnesses distinguish between three categories of components that serve different purposes and should not be designed the same way (source: building-agent-harness-martin-richards.md): + +### Sequential Artifact-Producing Components (e.g., `spec:` skills) + +These follow a rigid, ordered procedure and produce concrete documents or code: +- Invoked explicitly or triggered by a previous component +- Produce a durable artifact that downstream components consume +- Examples: research skill (→ `spec.md`), plan skill (→ `plan.json`), implement skill (→ code files) + +**Design rule**: Give these strict templates for artifact format. Ambiguous output format creates downstream parsing failures. + +### Advisory Context-Adaptive Components (e.g., `oracle:` skills) + +These adapt to context rather than following a rigid procedure: +- Examples: `oracle:architect` (applies DDD patterns, thinks about component responsibilities), `oracle:challenge` (pushes back on your approach, pokes holes in design) +- Should not produce required artifacts — they produce insights that inform human or agent decisions + +**Design rule**: Do not force these into artifact-producing molds. A thinking tool should adapt; forcing it into rigid procedure wastes its value (source: building-agent-harness-martin-richards.md). + +### Utility Components (e.g., `code:` skills) + +Simple, single-purpose tools invoked on demand: +- Examples: commit, review, format +- Should be stateless and idempotent where possible + +**Design rule**: Keep utilities narrow. Collapsing everything into one format loses the distinction between sequential, advisory, and utility behavior. + +## Context Isolation Patterns + +### Fresh Context Windows Per Phase + +The most critical harness design pattern: each major phase (research, plan, implement) must start with a fresh context window containing only the compacted artifact from the previous phase (source: research-plan-implement-rpi.md). + +**Why**: Context quality directly determines output quality. Carrying raw search results, failed attempts, or prior exploration into a new phase degrades all downstream reasoning. + +**Implementation**: Phase transitions are not just workflow steps — they are explicit context boundaries. The harness must enforce these boundaries, not just suggest them. + +### Subagent Isolation for Noisy Operations + +During the Research phase, [[subagents]] perform operations that generate significant context noise (glob, grep, file reads). These must run in isolated contexts and return only compacted summaries to the main context (source: research-plan-implement-rpi.md). + +**Pattern**: +``` +Main context (clean) + └── Subagent context (noisy) + ├── glob operations + ├── grep operations + └── file reads + → Returns: compacted summary only +``` + +This is the primary [[context-engineering]] mechanism for preventing research phase operations from polluting the planning context. + +### Active Context Compaction + +When an agent strays off path, the correct response is never to correct it in-context — that adds noise (source: agentic-software-modernization-markus-harrer.md). The harness design should support a compaction escape hatch: + +1. Agent summarizes current state into a compact file (`state.md`) +2. New context starts with only that summary +3. Fresh reasoning from a clean starting point + +Harnesses should make this path explicit and easy, not an exceptional manual intervention. + +## Artifact Design Patterns + +### Compacted Research Documents + +Research artifacts (~200 lines) should capture (source: research-plan-implement-rpi.md): + +``` +research_doc.md +├─ Problem Summary +├─ Relevant Files Identified +│ └─ file paths with roles +├─ Information Flow Analysis +│ └─ function/module call chains +├─ Key Findings +└─ Recommended Approach +``` + +**Design rule**: Optimize for correctness over completeness. A shorter artifact that is accurate is more valuable than a longer one with errors. + +### Compacted Implementation Plans + +Plan artifacts (~200 lines) should capture (source: research-plan-implement-rpi.md): + +``` +implementation_plan.md +├─ Step 1: [Description] +│ ├─ File: exact/path/to/file.ext +│ ├─ Function: function_name() +│ └─ Tests: test_file.ext +├─ Step 2: ... +└─ Step N: ... +``` + +**Design rule**: Specify exact file paths and function names. Ambiguous plans produce ambiguous code. + +### Progress Tracking for Long Tasks + +For implementations spanning multiple context resets, `progress.md` provides state continuity (source: research-plan-implement-rpi.md): + +``` +progress.md +├─ Goal: [Feature description] +├─ Completed Steps: [✓] step list +├─ Current Step: [→] active step +├─ Remaining Steps: [ ] future steps +└─ Current Issue: [blocker if any] +``` + +### Annotated Living Documents (Burleigh Pattern) + +Tyler Burleigh's annotation approach treats plan documents as shared mutable state between human and agent (source: research-plan-implement-review-tyler-burleigh.md): + +1. Agent produces initial plan +2. Human adds inline annotations directly into the document +3. Agent updates based on annotations +4. Repeat until aligned + +**Design rule**: Plan documents should be designed for annotation — use clear section headers, avoid dense prose, prefer lists that are easy to comment within. + +## Skill Routing and Auto-Loading + +Effective harnesses route to the right skill automatically based on context (source: building-agent-harness-martin-richards.md): + +- Saying "create a spec for user auth" → agent matches to `spec:research` +- Working in a FastAPI project → FastAPI-specific skill auto-loads +- Working in a Drizzle project → Drizzle-specific skill auto-loads + +**Design principle**: Skills should be discoverable by description, not just by exact name. The harness orchestrator component (e.g., `spec:orchestrator` in Atelier) handles this routing. + +## Backflow and Non-Waterfall Design + +Effective harnesses explicitly support backflow between phases — they are not waterfall (source: building-agent-harness-martin-richards.md): + +``` +Research ←→ Human annotation ←→ Research (repeat) + ↓ +Plan ←→ Human annotation ←→ Plan (repeat) + ↓ +Implement → finds gap → Plan (update and continue) + ↓ +Finish (validate + review) +``` + +**Design rule**: Implementation finding gaps in the plan and pushing back is expected and healthy. The harness should make going back to update the plan easy, not feel like a failure. + +## Multi-Model Specialization + +As harnesses scale, different components should use different models (source: research-plan-implement-review-tyler-burleigh.md): + +| Component Type | Model Recommendation | Reason | +|----------------|---------------------|---------| +| Research | Strongest available | Requires synthesis and judgment | +| Planning | Strongest available | Requires synthesis and judgment | +| Implementation | Faster/cheaper | Guided by plan, largely mechanical | +| Review | Different from implementer | Uncorrelated error profiles catch more bugs | + +**Design rule**: Model selection should be a harness configuration, not a per-session manual choice. + +## Iterative Harness Evolution + +Building a harness is itself an iterative process. Known failure patterns to avoid (source: building-agent-harness-martin-richards.md): + +**Avoid rigidity (v1 mistake):** Subagents chained in strict waterfall sequence work in narrow cases but can't handle the back-and-forth that real development requires. + +**Avoid over-ceremony:** Delta tracking with explicit `ADDED`/`MODIFIED`/`REMOVED` markers felt precise but became too ceremonial in practice. + +**Start minimal:** Begin with the research → plan → implement loop. Get it working well before adding advisory or utility layers. The discipline matters more than the sophistication. + +## Related pages + +- [[agent-harness]] +- [[harness-engineering]] +- [[agentic-engineering-workflow]] +- [[rpi-workflow]] +- [[rpir-workflow]] +- [[context-engineering]] +- [[subagents]] +- [[progressive-disclosure]] +- [[agent-workflows]] +- [[agent-best-practices]] +- [[agentic-software-modernization]] diff --git a/wiki/knowledge/agent-harness.md b/wiki/knowledge/agent-harness.md new file mode 100644 index 00000000..3f7b05b1 --- /dev/null +++ b/wiki/knowledge/agent-harness.md @@ -0,0 +1,115 @@ +# Agent Harness + +**Summary**: An agent harness is a set of skills, workflows, and methodology that teaches an AI agent how you think and how you build — and building one well matters more than which model you choose, as demonstrated by LangChain's 52.8% → 66.5% Terminal Bench improvement using the same model with a better harness. +**Sources**: `docs/agentic-engineering/building-agent-harness-martin-richards.md` +**Last updated**: 2026-04-22 + +--- + +## Overview + +Martin C. Richards argues that agent productivity depends less on which AI model you use and more on how you "harness" it. The agents that produce good work are the ones that know how their operator works — conventions, patterns, preferences. The ones that fail are the ones dropped into a codebase with no context and expected to read minds (source: building-agent-harness-martin-richards.md). + +The insight is blunt: complaining that an agent wrote bad code without giving it context is like calling a new hire incompetent on their first day because they didn't already know your codebase. + +For the broader synthesis of the newer harness-engineering corpus, see [[harness-engineering]]. + +For design patterns behind building harnesses, see [[agent-harness-design]]. For how harnesses implement [[rpi-workflow]] and [[rpir-workflow]], see those pages. + +## Harness > Model + +LangChain improved their coding agent from **52.8% to 66.5%** on Terminal Bench 2.0 by only changing the harness, keeping the model fixed. Same model, better harness, better results (source: building-agent-harness-martin-richards.md). + +Mitchell Hashimoto named this practice **harness engineering** in February 2026. OpenAI and Martin Fowler (Thoughtworks) subsequently adopted the term. + +- **Hashimoto's definition (reactive):** Anytime an agent makes a mistake, engineer a fix so it doesn't happen again. +- **Richards' definition (proactive):** A harness is a set of skills, workflows, and methodology that teaches your agent how you think and how you build. It makes success the more likely outcome rather than the lucky one. + +## The Core Pattern: Independent Discovery + +The research → plan → implement loop was independently discovered across multiple practitioners (source: building-agent-harness-martin-richards.md): + +- **[[rpi-workflow]] (HumanLayer)** — Research → Plan → Implement, with explicit context compaction +- **[[rpir-workflow]] (Superpowers, 40k+ stars)** — Same shape with review cycles +- **Atelier (Richards)** — Annotation cycle before code is written + +Boris Tane's formulation captures the core principle: + +> Never let Claude write code until you've reviewed and approved a written plan. + +His annotation cycle: research the codebase, write a plan in markdown, **annotate the plan back and forth until it's right**, then implement. The plan becomes shared mutable state between you and the agent. + +The convergence says more about the pattern than about any one implementation. **The harness doesn't need to be complicated. It needs to exist.** + +## Atelier: Richards' Implementation + +Atelier implements the research → plan → implement loop through four sequential skills (source: building-agent-harness-martin-richards.md): + +**`spec:research`** — Explores the codebase, reads existing patterns, surfaces relevant context, and produces `spec.md`. The research artifact everything downstream depends on. + +**`spec:plan`** — Takes the research and breaks it into tasks, producing `plan.json`. A good plan means the agent can commit to a specific approach and execute without constant course-correction. A bad plan means expensive mid-implementation back-and-forth. + +**`spec:implement`** — Executes the plan with TDD. Each task gets built, tested, verified. + +**`spec:finish`** — Validates the whole implementation and runs a review pass. + +**`spec:orchestrator`** — Handles skill routing, deciding which skill to load based on what you're doing. + +``` +graph LR + R[spec:research] --> P[spec:plan] + R -- "review & annotate" --> H[human] + H -- "update spec" --> R + P --> I[spec:implement] + I --> F[spec:finish] + I -- "gaps found" --> P +``` + +**This isn't waterfall.** Backflow is expected. You review the research, annotate it, and loop with the agent until the spec is right. Implementation can still find gaps in the plan and push back — that's what separates it from rigid spec-driven development that breaks when reality arrives (source: building-agent-harness-martin-richards.md). + +## Harness Component Types + +Atelier has 34 skills organized into three namespaces that reflect different types of harness components (source: building-agent-harness-martin-richards.md): + +**`spec:` skills — Sequential, artifact-producing.** They produce documents, plans, code. They're meant to be followed closely and invoked explicitly or triggered by a previous skill. + +**`oracle:` skills — Advisory, context-adaptive.** `oracle:architect` applies DDD patterns and thinks about component responsibilities. `oracle:challenge` pushes back on your approach and pokes holes in your design. These adapt to context rather than following a rigid procedure. + +**`code:` skills — Utilities.** Review, commit, the kind of thing you reach for when you need it. + +> A spec workflow needs structure and artifacts, a thinking tool should adapt, and a commit helper just needs to work when called. Collapsing everything into one format loses that distinction. + +Skills auto-load when relevant. Say "create a spec for user auth" and the agent matches that to `spec:research`. Language-specific skills (Drizzle, Fastify, FastAPI, SQLAlchemy) activate based on what you're working in (source: building-agent-harness-martin-richards.md). + +## Evolution of Atelier + +Understanding the evolution helps avoid known dead-ends (source: building-agent-harness-martin-richards.md): + +**August 2025 (v1):** Rigid. Subagents chained in sequence, waterfall-style. Context-reader feeds requirements-gatherer feeds spec-writer. Worked narrowly but couldn't handle the back-and-forth real development requires. + +**January 2026:** Moved to a living `spec.md` model with delta tracking for brownfield changes (`ADDED`, `MODIFIED`, `REMOVED` markers). Closer, but still too ceremonial. + +**Current version:** Dropped the ceremony. Skills changed the paradigm — before skills, you had agents, commands, and reference docs, and tried to chain them yourself. Skills let you capture knowledge in a form the agent can find on its own. Describe what you want to do, and the agent loads the relevant skill by context. + +## Building Your Own Harness + +Install Atelier if you want: `npx skills add martinffx/atelier`. Or grab specific skills: `npx skills add martinffx/atelier --skill spec:research`. + +But the real suggestion is to **build your own.** The value of Atelier isn't in the specific skills — it's in the idea that your agent's harness should be engineered with the same care as the software it produces (source: building-agent-harness-martin-richards.md). + +Start with the research → plan → implement loop. Get that working in your own way. Add skills for how you write code, how you test, how you think about design. + +> The discipline matters more than the tools. The harness just encodes that discipline so you don't have to remember it each time. + +## Related pages + +- [[agent-harness-design]] +- [[harness-engineering]] +- [[rpi-workflow]] +- [[rpir-workflow]] +- [[agentic-engineering-workflow]] +- [[agent-best-practices]] +- [[subagents]] +- [[context-engineering]] +- [[progressive-disclosure]] +- [[agent-workflows]] diff --git a/wiki/knowledge/agent-protocol-standards.md b/wiki/knowledge/agent-protocol-standards.md new file mode 100644 index 00000000..4864f661 --- /dev/null +++ b/wiki/knowledge/agent-protocol-standards.md @@ -0,0 +1,134 @@ +# Agent Protocol Standards + +**Summary**: An overview of the emerging standard protocols for AI agent communication — MCP, A2A, ACP, and ANP — covering their differences, complementarity, governance status, and guidance on when to use each. +**Sources**: `docs/agent-protocols/advancing-agentic-ai-communication-protocols.md`, `docs/agent-protocols/ai-agent-protocols-2026-guide.md`, `docs/agent-protocols/a2a-protocol-huggingface-space.md` +**Last updated**: 2026-04-21 + +--- + +## The Landscape in 2026 + +The agent protocol space has consolidated rapidly. Four protocols were competing as of mid-2025; by early 2026, the picture has simplified (source: ai-agent-protocols-2026-guide.md): + +- **MCP** (Model Context Protocol) — the de facto agent-to-tool standard +- **A2A** (Agent-to-Agent Protocol) — the emerging agent-to-agent standard +- **ACP** — merged into A2A in September 2025; deprecated +- **ANP** — active in open-network/decentralized identity scenarios + +The two dominant standards are **complementary, not competing**: MCP solves the agent-to-tool problem; A2A solves the agent-to-agent problem (source: a2a-protocol-huggingface-space.md). + +## MCP: Agent-to-Tool + +**Model Context Protocol** was created by Anthropic (November 2024) and donated to the Linux Foundation's Agentic AI Foundation (AAIF) in December 2025 (source: ai-agent-protocols-2026-guide.md). + +| Attribute | Value | +|-----------|-------| +| Primary purpose | Agent ↔ Tool/Data/API | +| Transport | JSON-RPC over Streamable HTTP, stdio | +| Authentication | OAuth 2.0, OIDC Discovery | +| Discovery | MCP Registry, server.json | +| Key abstraction | Tools, Resources, Prompts | +| SDK languages | Python, TypeScript, C#, Java | +| Spec maturity | Stable (2025-11-25) | +| Monthly SDK downloads | 97M+ | + +**What MCP does**: Provides a universal interface layer between AI agents and external systems. Every agent-tool integration formerly required a custom connector; MCP collapses M×N integrations to M+N (source: advancing-agentic-ai-communication-protocols.md). + +**What MCP does NOT do**: Handle agent-to-agent communication. That is out of scope by design. + +See [[mcp-specification]] for full documentation and [[anthropic-2026-full-connectivity]] for the philosophy behind agent-native MCP design. + +## A2A: Agent-to-Agent + +**Agent-to-Agent Protocol** was created by Google (April 2025) and donated to the Linux Foundation in June 2025 (source: ai-agent-protocols-2026-guide.md). The Technical Steering Committee includes Google, Microsoft, AWS, Cisco, Salesforce, ServiceNow, SAP, and IBM. + +| Attribute | Value | +|-----------|-------| +| Primary purpose | Agent ↔ Agent | +| Transport | HTTP + JSON-RPC + SSE | +| Authentication | OAuth 2.0, mTLS, JWTs | +| Discovery | Agent Cards (JSON at `/.well-known/agent.json`) | +| Key abstraction | Tasks, Agent Cards | +| SDK languages | Python, TypeScript, Go, Java | +| Spec maturity | Stable (v0.3, July 2025) | + +**What A2A does**: Enables agents from different vendors, frameworks, and organizations to collaborate on tasks without sharing internal logic. Each agent is treated as an opaque service (source: ai-agent-protocols-2026-guide.md). + +See [[agent-to-agent-protocol]] and [[a2a-protocol]] for full documentation and [[mcp-vs-a2a]] for a direct comparison. + +## ACP: Deprecated + +**Agent Communication Protocol** was IBM Research's REST-native approach to agent interoperability, launched March 2025. In September 2025, IBM announced ACP would merge with A2A under the Linux Foundation (source: ai-agent-protocols-2026-guide.md). + +- Do not build new systems on ACP +- Follow IBM's migration guide to A2A +- ACP's multipart/multimodal messaging capabilities now exist in A2A + +## ANP: Open Network Discovery + +**Agent Network Protocol** (by Agent Network Protocol Contributors) addresses scenarios where agents from unknown organizations must find and authenticate each other without a central registry (source: advancing-agentic-ai-communication-protocols.md). + +| Attribute | Value | +|-----------|-------| +| Architecture | Decentralized (DIDs + JSON-LD semantic graphs) | +| Primary use | Agent discovery in open networks | +| Key feature | Decentralized Identifiers (DIDs) for identity | + +ANP is relevant for public internet agent discovery scenarios where MCP and A2A's more centralized or enterprise-focused models may not apply. + +## Complementarity Model + +``` +┌─────────────────────────────────────────────────┐ +│ Enterprise Multi-Agent System │ +│ │ +│ Agent A ────── A2A ────── Agent B │ +│ │ │ │ +│ MCP MCP │ +│ │ │ │ +│ Tool 1, 2, 3 Tool 4, 5, 6 │ +└─────────────────────────────────────────────────┘ +``` + +Each agent uses MCP to connect to its own tools and data sources, while A2A handles the agent-to-agent coordination layer (source: a2a-protocol-huggingface-space.md). + +## Decision Framework + +| Need | Protocol | +|------|---------| +| Agent needs tool access (database, API, SaaS) | MCP | +| Agents from different orgs need to collaborate | A2A | +| Human-in-the-loop mid-workflow | MCP (elicitation feature) | +| Long-running async agent workflows | A2A (task lifecycle + push notifications) | +| Agent discovery in open/public networks | ANP | +| Both tool access AND peer collaboration | MCP + A2A | + +(source: ai-agent-protocols-2026-guide.md) + +## Governance and Open Standards + +Both MCP and A2A are now under the Linux Foundation umbrella (source: ai-agent-protocols-2026-guide.md): +- MCP → Agentic AI Foundation (AAIF), donated December 2025 +- A2A → LF AI & Data, donated June 2025 + +This governance structure provides vendor-neutral stewardship and long-term stability for production deployments. The convergence of both protocols under the same umbrella organization suggests further integration and alignment is likely. + +## Historical Predecessors + +Earlier multi-agent communication standards include (source: advancing-agentic-ai-communication-protocols.md): +- **KQML** (1993): Knowledge Query and Manipulation Language — early communicative act library +- **FIPA** (2000): Foundation for Intelligent Physical Agents communicative act specification + +Modern protocols build on these foundations but are designed for LLM-native, HTTP-first, cloud-scale environments. + +## Related pages + +- [[mcp-specification]] +- [[mcp-vs-a2a]] +- [[agent-to-agent-protocol]] +- [[a2a-protocol]] +- [[agent-communication-protocols]] +- [[ai-agent-protocols-2026]] +- [[multi-agent-communication]] +- [[agentic-systems-architectural-paradigms]] +- [[agent-best-practices]] diff --git a/wiki/knowledge/agent-skills-standard.md b/wiki/knowledge/agent-skills-standard.md new file mode 100644 index 00000000..c1bf02a6 --- /dev/null +++ b/wiki/knowledge/agent-skills-standard.md @@ -0,0 +1,91 @@ +# Agent Skills Standard + +**Summary**: The open specification for extending AI agents with portable, version-controlled skill packages — defining SKILL.md structure, YAML frontmatter requirements, progressive disclosure loading, bundled file conventions, and the directory format adopted by Claude Code, Cursor, Codex, and other compliant agents. +**Sources**: agentskills-specification.md, agentskills-what-are-skills.md, skills-standard/README.md +**Last updated**: 2026-04-22 + +--- + +## What Is a Skill? + +A lightweight, open format for extending AI agent capabilities with specialized knowledge and workflows. Skills are: + +- **Portable** — Just files; easy to edit, version, share +- **Self-documenting** — Human-readable SKILL.md +- **Extensible** — From text instructions to executable code + +Canonical reference: [agentskills.io](https://agentskills.io) + +## Directory Structure + +``` +skill-name/ +├── SKILL.md (required — metadata + instructions) +├── scripts/ (optional — executable code) +├── references/ (optional — documentation) +└── assets/ (optional — templates, resources) +``` + +## YAML Frontmatter + +| Field | Required | Constraints | +| --------------- | -------- | ------------------------------------------------------------------- | +| `name` | Yes | 1–64 chars, lowercase + hyphens, no `--`, must match directory name | +| `description` | Yes | 1–1024 chars, what + when | +| `license` | No | Short identifier or filename | +| `compatibility` | No | 1–500 chars, environment requirements | +| `metadata` | No | Key-value mapping (string → string) | +| `allowed-tools` | No | Space-delimited pre-approved tools | + +### Name Validation Rules + +- Unicode lowercase alphanumeric (a–z) + hyphens only +- Must NOT start or end with hyphen +- Must NOT contain consecutive hyphens (`--`) +- Must match parent directory name + +### Description Best Practices + +- Describe both WHAT the skill does and WHEN to use it +- Include keywords for agent discovery +- Good: "Extracts text and tables from PDF files, fills forms, merges PDFs. Use when working with PDFs." +- Bad: "Helps with PDFs." + +## Progressive Disclosure + +| Phase | Loaded When | Budget | +| -------------- | -------------------------- | ------------------------------------------ | +| **Discovery** | Session start | ~100 tokens (name + description only) | +| **Activation** | Task matches description | <5,000 tokens (full SKILL.md) | +| **Execution** | Explicitly loaded by skill | On-demand (references/, scripts/, assets/) | + +This three-phase loading keeps context efficient — only metadata loads at startup. + +## Bundled File References + +Use relative paths from skill root: + +- `references/api-errors.md` +- `scripts/extract.py` +- `assets/templates/output.md` + +One level deep from SKILL.md; avoid deeply nested chains. + +## Validation + +Use `skills-ref validate ./my-skill` to check frontmatter validity and naming conventions. + +## Platform Adoption + +| Platform | Skill Discovery Dirs | +| ------------- | ------------------------------------------ | +| Claude Code | `.claude/skills/`, plugin `skills/` | +| Cursor | `.agents/skills/`, `.cursor/skills/` | +| Codex | `.codex/skills/` | +| Compatibility | Cross-discovers from other platforms' dirs | + +## Related pages + +- [[skill-authoring]] +- [[claude-code-skills]] +- [[progressive-disclosure]] diff --git a/wiki/knowledge/agent-to-agent-protocol.md b/wiki/knowledge/agent-to-agent-protocol.md new file mode 100644 index 00000000..8f47488a --- /dev/null +++ b/wiki/knowledge/agent-to-agent-protocol.md @@ -0,0 +1,102 @@ +# Agent-to-Agent Protocol + +**Summary**: A2A (Agent-to-Agent Protocol) is Google's open standard for enabling autonomous software agents to communicate, coordinate, and collaborate regardless of vendor or framework — complementing MCP's agent-to-tool layer with an agent-to-agent orchestration layer. +**Sources**: a2a-protocol-huggingface.md, mcp-vs-a2a-dzone.md +**Last updated**: 2026-04-22 + +--- + +## What Is A2A? + +The Agent-to-Agent (A2A) Protocol is an open standard introduced by Google in early 2025 for multi-agent systems. Where [[mcp-specification]] standardizes how agents access tools and resources, A2A standardizes how agents communicate with each other (source: a2a-protocol-huggingface.md). + +A2A enables autonomous software agents to discover, delegate, negotiate, and share tasks or state asynchronously, regardless of framework or vendor (source: mcp-vs-a2a-dzone.md). + +## Core Concepts + +**Agent Cards** are JSON descriptors that advertise an agent's capabilities, authentication requirements, and supported modalities (text, audio, video, structured data). They are exposed at `/.well-known/agent.json` (source: a2a-protocol-huggingface.md). + +**Tasks** are the unit of work in A2A. Agents delegate tasks to one another; tasks can be synchronous or long-running/asynchronous (source: a2a-protocol-huggingface.md). + +**Artifacts** are the outputs of completed tasks. Agents pass artifacts to downstream agents in a multi-step [[agent-workflows]] pattern, enabling chained processing pipelines (source: a2a-protocol-huggingface.md). + +**Push Notifications** support real-time updates for long-running tasks via webhooks or Server-Sent Events (SSE) (source: a2a-protocol-huggingface.md). + +## How A2A Differs from MCP + +| Dimension | A2A | MCP | +|-----------|-----|-----| +| Full name | Agent-to-Agent Protocol | Model Context Protocol | +| Origin | Google (early 2025) | Anthropic (November 2024) | +| Primary purpose | Agent-to-agent coordination | Agent-to-tool/resource integration | +| Communication model | Peer-to-peer between agents | Client (agent) to server (tool) | +| State | Stateful tasks with lifecycle | Stateless-ish tool invocations | +| Discovery | Agent Cards at `/.well-known/agent.json` | MCP Registry / capability negotiation | +| Authentication | Per-agent auth, supports enterprise SSO | OAuth 2.1, CIMD, XAA | +| Modalities | Text, audio, video, structured data | Text, binary resources | +| Industry support | Google, broad ecosystem | Anthropic, 170+ AAIF members | + +(source: a2a-protocol-huggingface.md) + +## Use Cases Where A2A Shines + +1. **Research pipelines**: A coordinator agent (A2A) delegates to a web research specialist, a code execution specialist, and a summarizer — each using MCP for their specific tool access (source: a2a-protocol-huggingface.md). + +2. **Customer service orchestration**: A triage agent routes tickets to billing specialist, technical specialist, or escalation agent based on intent — each running independently with their own tools (source: a2a-protocol-huggingface.md). + +3. **Data pipeline automation**: A planning agent spawns data extraction agents in parallel (A2A), collects artifacts, then delegates to transformation and reporting agents (source: a2a-protocol-huggingface.md). + +These patterns align with [[subagents]] and [[agent-workflows]] best practices for multi-agent system design. + +## A2A + MCP Combined Architecture + +The two protocols are designed to be complementary. A2A handles the orchestration layer; MCP handles how each agent accesses its tools (source: a2a-protocol-huggingface.md): + +``` +┌─────────────────────────────────────────────────────────────────────┐ +│ ENTERPRISE AI PLATFORM │ +│ │ +│ ┌─────────────────────────────────────────────────────────────┐ │ +│ │ A2A ORCHESTRATION LAYER │ │ +│ │ Orchestrator Agent │ │ +│ │ │ (A2A delegation) │ │ +│ │ ┌────┴────────────────────────┐ │ │ +│ │ V V │ │ +│ │ Research Agent Writer Agent │ │ +│ │ (A2A peer) (A2A peer) │ │ +│ └───────┬─────────────────────────┬───────────────────────── ┘ │ +│ │ MCP tool access │ MCP tool access │ +│ ┌───────▼─────────────────────────▼───────────────────────────┐ │ +│ │ MCP TOOL ACCESS LAYER │ │ +│ │ Web Search │ Database │ File System │ APIs │ │ +│ └─────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────┘ +``` + +## Industry Adoption + +Companies exploring or adopting A2A (source: a2a-protocol-huggingface.md): +- **Google** (creator and primary advocate) +- **Salesforce** (AgentForce integration) +- **SAP** (Business AI) +- **ServiceNow** (Now Assist) +- **Workday** + +## Governance + +A2A and MCP operate on parallel tracks with separate governance bodies and separate specifications. No official bridge protocol exists yet, though community implementations use both. The industry consensus is that both protocols will coexist as complementary layers in enterprise AI architectures (source: a2a-protocol-huggingface.md). + +## Resources + +- [A2A Protocol GitHub](https://github.com/google-a2a/a2a-protocol) +- [A2A Protocol Specification](https://google-a2a.github.io/a2a-spec/) +- [Hugging Face: a2aprotocol organization](https://huggingface.co/a2aprotocol) + +## Related pages + +- [[mcp-vs-a2a]] +- [[mcp-specification]] +- [[agent-workflows]] +- [[subagents]] +- [[agent-best-practices]] +- [[mcp-programmatic-tool-calling]] diff --git a/wiki/knowledge/agent-workflows.md b/wiki/knowledge/agent-workflows.md new file mode 100644 index 00000000..0402fae1 --- /dev/null +++ b/wiki/knowledge/agent-workflows.md @@ -0,0 +1,79 @@ +# Agent Workflows + +**Summary**: Proven patterns for structuring LLM agent work — from the fundamental Explore → Plan → Code → Verify loop to multi-agent architectures and progressive context strategies that prevent one-shotting complex projects. +**Sources**: research-agent-workflows-and-patterns.md, a-guide-to-agents.md +**Last updated**: 2026-04-22 + +--- + +## The Fundamental Loop + +Claude performs dramatically better when it can verify its own work: + +``` +Explore → Plan → Code → Verify → (loop if needed) +``` + +Discrete loops prevent one-shotting complex projects. Planning separated from execution improves outcomes. + +## Five Core Patterns + +### 1. Spec → Plan → Execute + +Separate planning from execution entirely. Planning agent researches and produces a structured plan; execution agent follows it. Prevents "getting over your skis" on complex projects. + +### 2. Two-Agent Architecture (Long-Running Tasks) + +- **Initializer** (first session): Creates structured artifacts (JSON feature list, progress file, architecture docs) +- **Coding Agent** (incremental sessions): Picks up work from artifacts, makes progress, updates state +- State tracking via JSON (model less likely to inappropriately modify vs. Markdown) and git history + +### 3. Subagent Delegation + +Route complex research to [[subagents]] to keep the main context focused: + +- Subagents return only summaries (1,000–2,000 tokens) vs. exploration tokens (tens of thousands) +- Main agent stays within its token budget and focused on the current task + +### 4. Progressive Context Loading + +Implement [[progressive-disclosure]] for instructions: + +- Path-scoped rules trigger only when matching files are read +- Skills load full instructions only when invoked +- Subagent summaries replace full exploration traces + +### 5. Orchestrator Pattern + +Team lead coordinates multiple agents working in parallel: + +- Shared task list with claim/complete workflow +- Task dependencies prevent premature work +- Direct communication between teammates +- Available in Claude Code via Agent Teams (experimental) + +## Context Management During Workflows + +| Strategy | Complexity | Effectiveness | +| ------------------------------- | ---------- | ---------------------------------- | +| Auto-compaction (summarization) | Low | Matches sophisticated alternatives | +| JSON state files | Low | Reliable for structured data | +| Git-based state tracking | Medium | Natural for code-producing agents | +| External memory (CLAUDE.md) | Medium | Persistent across sessions | +| Subagent architecture | High | Best for parallel/complex work | + +Boris Cherny's principle applies: prefer simple, straightforward solutions over complex memory architectures. Auto-compaction via summarization (asking Claude to summarize its context) works as well as more sophisticated approaches. + +## Anti-Patterns + +- **One-shotting complex projects** — Fails predictably; use discrete loops +- **Accumulating failed approaches** — After 2 failed corrections, clear context and restart +- **Overly generic instructions** — Specific credentialed personas outperform generic "helpful assistant" labels +- **Aggressive delegation language** — Replace "CRITICAL: You MUST" with normal guidance "Use when..." + +## Related pages + +- [[subagents]] +- [[progressive-disclosure]] +- [[context-engineering]] +- [[claude-code-subagents]] diff --git a/wiki/knowledge/agentic-engineering-workflow.md b/wiki/knowledge/agentic-engineering-workflow.md new file mode 100644 index 00000000..c5a28b68 --- /dev/null +++ b/wiki/knowledge/agentic-engineering-workflow.md @@ -0,0 +1,140 @@ +# Agentic Engineering Workflow + +**Summary**: A synthesized overview of the Research-Plan-Implement (RPI) agentic engineering workflow pattern, showing how it independently emerged across multiple practitioners and organizations as the reliable answer to unstructured "vibe coding" with AI agents. +**Sources**: `docs/agentic-engineering/research-plan-implement-rpi.md`, `docs/agentic-engineering/research-plan-implement-review-tyler-burleigh.md`, `docs/agentic-engineering/building-agent-harness-martin-richards.md`, `docs/agentic-engineering/agentic-software-modernization-markus-harrer.md` +**Last updated**: 2026-04-21 + +--- + +## The Core Pattern + +Across independent practitioners, organizations, and use cases, one workflow pattern emerged consistently as the reliable foundation for AI-assisted software development: + +**Research → Plan → Implement** — with fresh context windows between each phase, human review at the highest-leverage checkpoints, and written artifacts as persistent sources of truth. + +The convergence across unrelated teams is the strongest signal that the core insight is sound: **the bottleneck in AI-assisted development is not code generation — it's ensuring the model understands what to build before it starts** (source: research-plan-implement-review-tyler-burleigh.md). + +## Why This Pattern Exists + +### The Context Window Constraint + +AI models are stateless. They only know what exists in their current context window. When that window becomes too full — above ~40% utilization per HumanLayer — the model enters a "Dumb Zone" where response quality degrades rapidly and hallucinations increase (source: agentic-software-modernization-markus-harrer.md). + +This is the root cause that drives the workflow design. Every structural choice in the RPI pattern — fresh sessions per phase, compacted artifacts, subagent isolation — exists to manage context quality. See [[context-engineering]] for the full treatment. + +### The Error Compounding Problem + +Without structure, errors compound: + +| Error Location | Downstream Impact | +|----------------|-------------------| +| Bad research | 1,000s of bad lines of code | +| Bad plan | 100s of bad lines of code | +| Bad code | 1 bad line of code | + +This leverage model explains why reviewing ~400 lines of specification artifacts (200 research + 200 plan) delivers more value than reviewing 2,000 lines of generated code (source: research-plan-implement-rpi.md). Shift human review left, to the phases where errors are still cheap. + +### The Vibe Coding Failure Mode + +Without structure, AI coding tends to go the same way: give the model a prompt, it produces something close but not quite right, and you spend the rest of the session correcting it. As complexity grows, the back-and-forth compounds — the model carries forward bad assumptions, the context window fills with failed attempts, and you end up doing most of the work yourself (source: research-plan-implement-review-tyler-burleigh.md). + +## Implementations of the Pattern + +The same core pattern appears under different names and with different emphases: + +| Implementation | Authors | Key Emphasis | Artifact Names | +|----------------|---------|--------------|----------------| +| [[rpi-workflow]] | Dex Horthy / HumanLayer | Context window management, FIC methodology | `research_doc.md`, `implementation_plan.md` | +| [[rpir-workflow]] | Tyler Burleigh | Explicit review sessions, multi-model scaling | `RESEARCH.md`, `PLAN.md`, `PLAN-CHECKLIST.md` | +| Atelier ([[agent-harness]]) | Martin C. Richards | Skill-based harness, annotation cycles | `spec.md`, `plan.json` | +| [[agentic-software-modernization]] | Markus Harrer | Legacy system constraints, codebase hygiene | `state.md`, traceability comments | + +All converge on the same observation: the discipline matters more than the tools. The harness, workflow, or methodology just encodes that discipline so you don't have to remember it each time (source: building-agent-harness-martin-richards.md). + +## Phase-by-Phase Structure + +### Research Phase + +**Goal**: Understand the codebase structure and information flow before touching any code. + +**Key practices**: +- Use [[subagents]] for noisy operations (glob, grep, file reads) in isolated contexts +- Start with a fresh context window containing only the problem definition +- Output a compacted artifact (~200 lines) capturing: relevant files, information flow, key findings, recommended approach + +**Human leverage**: Highest. A mistake here creates thousands of lines of incorrectly architected code downstream (source: research-plan-implement-rpi.md). + +**Anti-pattern**: Letting the agent jump directly to implementation. "Never let Claude write code until you've reviewed and approved a written plan" (source: building-agent-harness-martin-richards.md). + +### Plan Phase + +**Goal**: Convert research findings into a precise, ordered implementation blueprint. + +**Key practices**: +- Start with a fresh context window containing only the research artifact and problem definition +- Define numbered, sequential steps with exact file paths and function signatures +- Include testing and verification procedures +- For legacy systems, target rule-based change recipes over non-deterministic agentic workloads (source: agentic-software-modernization-markus-harrer.md) + +**Human leverage**: High. A mistake here creates hundreds of lines in wrong locations or wrong patterns. + +### Implement Phase + +**Goal**: Execute the plan with minimal deviation. + +**Key practices**: +- Start with a fresh context window containing only the implementation plan +- Run tests after each step +- Use `progress.md` to track state across context resets for complex tasks +- Apply [[progressive-disclosure]] — load only what the current step needs +- Active context compaction: when an agent strays, summarize to `state.md` and start a fresh chat rather than correcting in-context (source: agentic-software-modernization-markus-harrer.md) + +**Human leverage**: Lowest at this phase — but review loops with Critic Agents can catch issues mechanically (source: agentic-software-modernization-markus-harrer.md). + +## Key Supporting Practices + +### Fresh Context Windows Between Phases + +Every phase transition should begin a new context window. Carrying forward the previous phase's exploration noise contaminates the context and degrades the model's reasoning. This is especially critical when correcting mistakes — correcting in-context is almost always wrong (source: research-plan-implement-review-tyler-burleigh.md). + +### Written Artifacts as Shared Truth + +Artifacts persist across sessions and give the model a clear reference point. Without them, context is lost between sessions and the model has to re-infer your intent every time (source: research-plan-implement-review-tyler-burleigh.md). + +### Subagent Isolation + +During the Research phase, [[subagents]] perform noisy discovery operations in isolated contexts and return only compacted summaries to the main context. This is the primary [[context-engineering]] tool for preventing context pollution from file search operations (source: research-plan-implement-rpi.md). + +### Multi-Model Role Specialization + +As workflows scale, different models handle different roles. Research, planning, and review require synthesis and judgment — use your strongest model. Implementation guided by a detailed plan is largely mechanical — use a faster, cheaper model. Models with different architectures produce largely uncorrelated errors, so using different models for implement and review catches more bugs (source: research-plan-implement-review-tyler-burleigh.md). + +### Harness Engineering + +The workflow should be encoded in a [[agent-harness]] — a set of skills, workflows, and methodology — so the discipline doesn't have to be remembered each time. LangChain improved their agent from 52.8% to 66.5% on Terminal Bench 2.0 by only changing the harness, keeping the model fixed (source: building-agent-harness-martin-richards.md). + +## Autonomous Scaling + +The workflow scales from human-in-the-loop to near-fully-autonomous depending on plan detail (source: research-plan-implement-review-tyler-burleigh.md): + +| Plan detail | Safe autonomy level | +|-------------|---------------------| +| High-level goal | Human-in-the-loop for every step | +| Phased plan with architecture decisions made | Autonomous per phase, review between phases | +| Detailed plan with file paths and function signatures | Autonomous implementation, human reviews PR | +| Exact specifications with test cases | Fully autonomous with automated verification | + +## Related pages + +- [[rpi-workflow]] +- [[rpir-workflow]] +- [[agent-harness]] +- [[agent-harness-design]] +- [[agentic-software-modernization]] +- [[claude-cookbook]] +- [[context-engineering]] +- [[subagents]] +- [[progressive-disclosure]] +- [[agent-workflows]] +- [[agent-best-practices]] +- [[mcp-specification]] diff --git a/wiki/knowledge/agentic-software-modernization.md b/wiki/knowledge/agentic-software-modernization.md new file mode 100644 index 00000000..e323f351 --- /dev/null +++ b/wiki/knowledge/agentic-software-modernization.md @@ -0,0 +1,125 @@ +# Agentic Software Modernization + +**Summary**: Agentic AI can genuinely modernize legacy software (COBOL, RPG, etc.), but only with disciplined preparation — the [[rpi-workflow]], active context compaction, iterative critic-agent loops, codebase hygiene investment, and strict traceability links from new code back to original business logic. +**Sources**: `docs/agentic-engineering/agentic-software-modernization-markus-harrer.md` +**Last updated**: 2026-04-21 + +--- + +## Overview + +Markus Harrer's analysis draws on Stanford studies (120k+ developers) and leading AI engineering firms (OpenHands, HumanLayer) to argue that agentic AI can automate exhausting software modernization, but not the way most people imagine. Simply unleashing AI agents on an old codebase and hoping for a miracle is a recipe for disaster (source: agentic-software-modernization-markus-harrer.md). + +Successful Agentic Software Modernization requires a fundamental shift: **away from vibe coding towards disciplined preparation and execution.** + +## The Core Problem: The Context Bottleneck + +AI models (LLMs) are stateless — they only know what exists in their current context window. In complex legacy systems, it is impossible to fit the entire context (millions of lines of code, dependencies, business logic) into this window (source: agentic-software-modernization-markus-harrer.md). + +When the window becomes too full (above ~40% utilization per Dex Horthy of HumanLayer), the model enters the **'Dumb Zone'** where response quality degrades rapidly and hallucinations increase. See [[context-engineering]] for strategies to manage this constraint. + +A more fundamental problem identified by Stanford studies is **Entropy**: when AI agents work within existing low-quality codebases, the produced code mirrors the low standards of the existing environment — creating a death spiral that automates the creation of technical debt rather than eliminating it (source: agentic-software-modernization-markus-harrer.md). + +The art of Agentic Software Modernization is not about generating more legacy code faster, but about **surgically managing the AI agent's access to the right context** to understand and improve the system. + +## Emerging Practices + +### 1. Implement the RPI Workflow + +The biggest trap is letting the agent code immediately. Instead, divide the process into strict phases (source: agentic-software-modernization-markus-harrer.md): + +**Phase 1 — Research (Understanding):** The agent analyzes only the existing codebase to understand how a feature works. Output is not code but a summary explaining where relevant logic resides. Data-driven software analytics can help analyze large-scale systems. Enriching the codebase summary beforehand (signaling outdated parts, no-go areas, code that fits current system ideas) guides the agent more precisely. + +**Phase 2 — Plan (Intent Compression):** Based on the research, the agent creates a detailed plan of which files need to be changed and how. This plan represents the "compressed intent" of the modification. In the best case, scope down activities enough to switch from **agentic workloads** (non-deterministic) towards **rule-based search-and-replace workloads** — letting an agent craft change recipes and execute those deterministically. + +**Phase 3 — Implement (Coding):** Only now does the agent change or write code, based strictly on the approved plan or deterministic transformation rules. + +> If the plan is non-existent, wrong, or too vague, 1,000 lines of generated code are worthless and tedious to review. Invest human intelligence in reviewing research and planning steps early — not just reviewing final code. + +See [[rpi-workflow]] and [[rpir-workflow]] for detailed implementations of this pattern. + +### 2. Use Iterative Refinement with Critic Agents + +Never attempt a complex migration (e.g., COBOL to Java) in a single "One-Shot" prompt. The OpenHands team demonstrated this almost always leads to hallucinations (source: agentic-software-modernization-markus-harrer.md). + +Instead, use an iterative loop with specialized roles: + +- **Engineer Agent:** Attempts to solve the task (e.g., migrating code) +- **Critic Agent:** A separate agent that *only reads*. It analyzes generated code, runs tests, and provides harsh feedback with scores + +The process: Engineer delivers → Critic evaluates and sends feedback → Engineer improves → Repeat until quality standard is met. + +Leverage compilation errors, code duplication detection, and architectural violation checks as mechanical levers to guide the agent rather than relying solely on LLM judgment. This maps to the review phase in [[rpir-workflow]]. + +### 3. Invest in Codebase Hygiene First + +Stanford data (Yegor Denisov-Blanch) shows a clear correlation (source: agentic-software-modernization-markus-harrer.md): + +- **Clean environments** (high test coverage, good modularity, typing) → AI can autonomously drive a large share of sprint tasks +- **"Dirty" environments** (high entropy, technical debt) → AI struggles, produces more errors, and can accelerate technical debt (the "Rework" trap) + +High-leverage hygiene moves before scaling AI: +- Rename cryptic one-letter variables to reflect actual technical or business domain +- Build out higher-level concepts or refactor towards well-known patterns/idioms +- Focus on safe refactorings that don't break code (most structural renames qualify) + +### 4. Practice Active Context Compaction + +When an agent strays off the path, the human impulse is to correct it within the same chat. **This is a mistake.** Every failed attempt clutters the context window with noise (source: agentic-software-modernization-markus-harrer.md). + +The better approach — **active context compaction** (context reset): + +1. Have the agent summarize the current state and findings into a compact file (`state.md` or similar) +2. Start a completely new chat with a fresh context +3. Feed only the summary in as the starting point + +This keeps the agent in the "Smart Zone" of its context window. See [[context-engineering]] and [[progressive-disclosure]] for the broader patterns behind this approach. For models with limited context windows (DeepSeek, Minimax, Moonshot), active context compaction and rigid external state management is essential for good results. + +### 5. Maintain Traceability Links + +When migrating legacy code (e.g., COBOL to Java), the connection to the original business logic must never be lost. OpenHands recommends that the agent insert comments in the new code linking exactly to the line numbers of the old code where that logic originated (source: agentic-software-modernization-markus-harrer.md). + +When using graph analytics or creating flowcharts, ensure that line numbers, identifiers, or filenames are used in generated outputs so you can quickly verify they are not hallucinated. + +## Traps to Avoid + +### 1. Vibe Coding + +"Vibe Coding" describes back-and-forth chatting guided more by feelings than specifications ("Make that prettier," "No, that feels wrong"). This leads to bloated context windows and confused models. AI Engineering in legacy system environments requires precision, not vibes (source: agentic-software-modernization-markus-harrer.md). + +### 2. Underestimating Rework + +Stanford studies show that while AI tools increase output (more Pull Requests), they often dramatically increase rework — time developers spend repairing or rewriting AI-generated code. Measuring only speed/volume misses the massive cost of quality assurance (source: agentic-software-modernization-markus-harrer.md). + +### 3. Blind Line-by-Line Code Review + +In a world where an agent can generate 20,000 lines of TypeScript in minutes, traditional human line-by-line review is no longer scalable. Dex Horthy's **Hierarchy of Leverage** states: **1 Bad Line of Plan == 100 Bad Lines of Code.** Shift human review "left" to the research results and plan, before code is even written (source: agentic-software-modernization-markus-harrer.md). + +In the research stage, look for rule-based changes — most structural changes are equal over the completed codebase. Review change pattern by change pattern, not line by line. + +### 4. Expecting Magic in Niche Languages + +AI model performance depends heavily on training data. For popular languages (Python, Java, JavaScript) they work excellently. For niche languages or old dialects (specific COBOL variants, obscure DSLs, RPG), AI can actually **decrease productivity** — the agent hallucinates and humans spend all their time correcting it (source: agentic-software-modernization-markus-harrer.md). + +Check StackOverflow tags and the TIOBE programming popularity index. Every language not in the top 10 for the last few years needs a different approach: broader reverse engineering towards specs or tests, or a traditional transpiler approach to convert the niche language to a more popular language that an AI agent can then work with. + +## The Role Shift: From Coder to Architect of Intent + +Agentic Software Modernization works, but it requires discipline. The role of the human developer is shifting — we are becoming less writers of syntax and more the **architects of the intent** (source: agentic-software-modernization-markus-harrer.md). + +Think of AI agents in legacy systems as new senior developers that can do amazing things but need decent onboarding: step-by-step introduction to the system, background context on the existing code, and careful introduction to the nasty parts of the systems over time. + +> Those who just click "Refactor all this" will end up in chaos. + +## Related pages + +- [[rpi-workflow]] +- [[rpir-workflow]] +- [[agentic-engineering-workflow]] +- [[agent-harness]] +- [[agent-harness-design]] +- [[context-engineering]] +- [[progressive-disclosure]] +- [[agent-best-practices]] +- [[agent-workflows]] +- [[subagents]] diff --git a/wiki/knowledge/agentic-systems-architectural-paradigms.md b/wiki/knowledge/agentic-systems-architectural-paradigms.md new file mode 100644 index 00000000..6ac57023 --- /dev/null +++ b/wiki/knowledge/agentic-systems-architectural-paradigms.md @@ -0,0 +1,127 @@ +# Agentic Systems Architectural Paradigms + +**Summary**: A synthesis of research on the three foundational paradigms of advanced agentic AI — the dual symbolic/neural framework, context engineering as the primary quality lever, and programmatic execution replacing sequential tool-calling. +**Sources**: `docs/agent-protocols/architectural-paradigms-advanced-agentic-systems.md` +**Last updated**: 2026-04-21 + +--- + +## The Dual-Paradigm Framework + +A systematic review of 90 studies (2018–2025) introduces a dual-paradigm framework that categorizes all agentic AI systems into two distinct lineages (source: architectural-paradigms-advanced-agentic-systems.md): + +### Paradigm 1: Symbolic / Classical + +- **Foundation**: Algorithmic planning, explicit rules, persistent state memory +- **Strengths**: Predictability, traceability, deterministic behavior +- **Dominant domains**: Healthcare (medical diagnosis), safety-critical applications +- **Governance**: Well-understood accountability chains +- **Limitation**: Brittleness in dynamic environments; limited generalization + +### Paradigm 2: Neural / Generative + +- **Foundation**: LLMs, stochastic generation, prompt-driven orchestration +- **Strengths**: Adaptability, generalization, contextual reasoning +- **Dominant domains**: Finance (autonomous trading), robotics, dynamic real-world environments +- **Governance**: Significant challenge — requires new accountability frameworks +- **Limitation**: Unpredictability, hallucination risks, opacity + +**Key finding**: The choice of paradigm is strategic. Symbolic systems dominate safety-critical domains; neural systems prevail in adaptive, data-rich environments. The future lies in *intentional integration* — systems that are both adaptable and reliable (source: architectural-paradigms-advanced-agentic-systems.md). + +## Context Engineering + +[[context-engineering]] has emerged as the key discipline for advanced agentic systems — not just model reasoning, but the strategic construction of the entire contextual signal set (source: architectural-paradigms-advanced-agentic-systems.md). + +### What Context Engineering Addresses + +- System instructions and agent state +- Tool availability and external knowledge +- Prior conversation history and agent-generated strategies +- Real-time environmental information + +### Agentic Context Engineering (ACE) Framework + +The ACE framework (Zhang et al., 2025) treats context as an evolving "playbook" rather than a static prompt (source: architectural-paradigms-advanced-agentic-systems.md). It: + +- **Incrementally accumulates** user-provided and agent-generated strategies over time +- **Reflects and curates** context to prevent "brevity bias" (oversimplification of complex tasks) +- **Guards against "context collapse"** — loss of detail over many agent iterations +- Enables agents to self-improve at both offline and online stages + +**Key insight**: Context quality, not raw model reasoning power, is now the strategic bottleneck for agentic AI performance (source: architectural-paradigms-advanced-agentic-systems.md). + +## Protocol Standardization + +The companion paper (arXiv:2508.10146) provides systematic review of leading agentic frameworks: CrewAI, LangGraph, AutoGen, Semantic Kernel, Agno, Google ADK, and MetaGPT (source: architectural-paradigms-advanced-agentic-systems.md). + +### Agent Communication Protocols Analyzed + +- **Contract Net Protocol (CNP)**: Classic multi-agent negotiation protocol +- **[[agent-to-agent-protocol]] (A2A)**: Google's peer-to-peer task delegation standard +- **Agent Network Protocol (ANP)**: Decentralized identity-based discovery +- **Agora**: Emerging protocol for agent marketplace interactions + +### The Pre-MCP Integration Problem + +Before standardization, every agent-tool integration required custom implementation — creating an N×M combinatorial explosion of connectors (source: architectural-paradigms-advanced-agentic-systems.md). [[mcp-specification]] collapses this to M+N by providing a universal interface layer. + +### MCP Benefits for Protocol Standardization + +- **Interoperability**: Abstracts away integration complexity for heterogeneous agents and tools +- **Scalability**: Decouples model engineering from brittle, bespoke context "glue code" +- **Efficiency & Security**: Enables secure, programmatic access to real-time data and cross-application orchestration + +See [[agent-protocol-standards]] for a full comparison of standardization approaches. + +## Programmatic Execution Paradigm + +Modern agentic frameworks have shifted from sequential tool-calling to programmatic execution (source: architectural-paradigms-advanced-agentic-systems.md): + +### Sequential Tool-Calling (Legacy) + +``` +Agent calls Tool A → waits → calls Tool B → waits → calls Tool C +``` + +High latency, multiple network hops, cumbersome, slow, and prone to error. + +### Programmatic Tool Composition (Modern) + +``` +Agent writes a script composing A, B, C → executes in one operation +``` + +Low latency, single execution context, fast and efficient. This is the approach advocated in [[anthropic-2026-full-connectivity]]. + +### Key Capabilities of Modern Frameworks + +- **Persistent Memory Management** across sessions +- **Dynamic Protocol-Oriented Tool Use** with MCP +- **Modular Context Playbooks**: Swapping, stacking, and evolving contextual modules as tasks progress +- **Safety Guardrails & Governance**: Crucial for goal-directed multi-agent systems + +## Critical Research Gaps + +1. **Governance deficit for symbolic systems**: Lack of robust frameworks despite high safety requirements (source: architectural-paradigms-advanced-agentic-systems.md) +2. **Hybrid neuro-symbolic architectures**: Pressing need for systems combining the reliability of symbolic with the adaptability of neural approaches +3. **Explainability**: Neural systems lack the traceability needed for high-stakes decisions +4. **Alignment**: Growing challenge of ensuring agentic systems align with human values and societal norms + +## Strategic Roadmap + +1. Build hybrid architectures combining symbolic reliability with neural adaptability (source: architectural-paradigms-advanced-agentic-systems.md) +2. Adopt standardized protocols (MCP, A2A) to eliminate integration complexity +3. Invest in [[context-engineering]] as the primary quality lever +4. Shift from prompt engineering to programmatic execution patterns +5. Develop governance frameworks appropriate to each paradigm's risk profile + +## Related pages + +- [[context-engineering]] +- [[mcp-specification]] +- [[agent-to-agent-protocol]] +- [[agent-protocol-standards]] +- [[multi-agent-communication]] +- [[agent-workflows]] +- [[agent-best-practices]] +- [[anthropic-2026-full-connectivity]] diff --git a/wiki/knowledge/agents-md-liability.md b/wiki/knowledge/agents-md-liability.md new file mode 100644 index 00000000..1bedebe2 --- /dev/null +++ b/wiki/knowledge/agents-md-liability.md @@ -0,0 +1,94 @@ +# AGENTS.md as a Liability + +**Summary**: Research-backed analysis of why growing AGENTS.md (and CLAUDE.md) files beyond a certain size actively harm AI agent performance through attention dilution, primacy bias, and the dumb zone. +**Sources**: agents-md-is-a-liability-paddo.md, advanced-context-engineering-coding-agents-dev.md +**Last updated**: 2026-04-21 + +--- + +Every time an agent makes a mistake, you add a rule. Every edge case gets a line. Every convention gets documented. You're at 300 lines, then 500, then 1000. This makes performance worse, not better. See [[agent-configuration-files]] for the structural overview and [[context-rot]] for the underlying mechanisms. + +## The 500 Instruction Ceiling + +The IFScale benchmark (Distyl AI, NeurIPS 2025) tested 20 frontier models on instruction following at scale. Task: generate a business report including specific keywords, each keyword counting as one instruction, scaled from 10 to 500 (source: agents-md-is-a-liability-paddo.md). + +**The best frontier model scored 68% at 500 instructions. One in three instructions simply got skipped.** + +Three distinct degradation patterns emerged: + +| Pattern | Models | Behavior | +|---------|--------|----------| +| Threshold decay | o3, Gemini 2.5 Pro | Near-perfect until 100–250 instructions, then cliff | +| Linear decay | GPT-4.1, Claude Sonnet 4 | Steady, predictable decline from the start | +| Exponential decay | GPT-4o, LLaMA-4-Scout | Rapid collapse | + +Every model showed **primacy bias**: earlier instructions receive more attention than later ones. As instruction density increases, errors shift from *modification* (doing it wrong) to *omission* (not doing it at all). **The model doesn't misinterpret your 400th rule. It doesn't see it** (source: agents-md-is-a-liability-paddo.md). + +## Five Compounding Architectural Problems + +### 1. Lost in the Middle + +Stanford and Meta's 2023 research (TACL 2024) documented the U-shaped attention curve. Models attend strongly to the beginning and end of context; the middle is a dead zone. With 20 documents in context, placing the answer in the middle dropped accuracy by 20+ points. In one case, GPT-3.5-Turbo performed *worse with context than without it* — adding information actively hurt the model (source: agents-md-is-a-liability-paddo.md). + +### 2. The Dumb Zone + +Dex Horthy (HumanLayer) coined this after analyzing 100,000+ developer sessions. Past roughly 40% of context capacity, "the model starts drifting, hallucinating, and forgetting its own instructions." This is how attention works — token relationships scale quadratically. Each additional token makes every other token slightly harder to attend to (source: agents-md-is-a-liability-paddo.md). + +> The more you use the context window, the worse the outcomes you'll get. — Dex Horthy, HumanLayer + +### 3. Context Rot + +Chroma Research identified three compounding mechanisms: lost-in-the-middle positional bias, attention dilution (softmax spreads attention thinner as context grows), and distractor interference (semantically similar but irrelevant content causes hallucinations). **Critical finding: degradation happens at every context length increment, not just near the limit. A 1M token window still rots at 50K tokens** (source: agents-md-is-a-liability-paddo.md). See [[context-rot]]. + +### 4. Attention Sinks + +Xiao et al. (ICLR 2024) showed initial tokens receive disproportionate attention regardless of semantic content. Replace your first tokens with newline characters and the effect persists — it's positional, not semantic. **This is why instructions at the top of your AGENTS.md get followed more reliably: they're sitting on the attention sink** (source: agents-md-is-a-liability-paddo.md). + +### 5. Attention Dilution + +The transformer's softmax attention is a zero-sum game. Attention weights are positive and sum to 1. More tokens means less attention per token. **You can't add context without diluting attention to existing context.** Every line in your AGENTS.md competes with every other line for the model's focus (source: agents-md-is-a-liability-paddo.md). + +## The Repetition Hack (and Its Limits) + +Google Research (December 2025): repeating the prompt as `` instead of `` yielded 47 wins out of 70 benchmark-model combinations, 0 losses. Gemini 2.0 Flash-Lite jumped from 21% to 97% on middle-of-list retrieval (source: agents-md-is-a-liability-paddo.md). + +The mechanical explanation: autoregressive models process left to right; causal masking means each token can only attend to preceding tokens. When the model first reads instructions, it's blind to the following context. By repeating instructions after the context, the second copy can attend to everything — simulating bidirectional attention within a unidirectional architecture. + +This is already used in production: Claude Code injects `` tags throughout tool results; Claude.ai uses `` tags. The downside: repetition eats context, which triggers the very problems you're trying to solve (source: agents-md-is-a-liability-paddo.md). + +## What This Means for Your AGENTS.md + +Actionable principles from the research convergence (source: agents-md-is-a-liability-paddo.md): + +- **Shorter is better** — Every additional instruction dilutes attention to all others. A 200-line file with 5 critical rules is worse than a 50-line file with the same 5 rules and less noise +- **Front-load and back-load** — Primacy bias and recency bias are both real. Your most important rules belong at the very beginning and end. The middle is where rules go to die +- **Modularize** — Claude Code's Skills system, path-specific rules, and lazy-loaded context exist for a reason. Don't stuff everything into one file. Load domain-specific rules only when the domain is active. See [[progressive-disclosure]] +- **Prune aggressively** — Semantically related but irrelevant instructions are worse than unrelated ones (Chroma's distractor interference). Docker conventions don't help when the agent writes React — they actively hurt +- **Prefer positive instructions** — "Do NOT do X" fails more often than "always do Y" +- **The sweet spot: low hundreds** — Reasoning models maintain near-perfect performance through 100–250 instructions before degrading; non-reasoning models degrade earlier + +> Agent-generated context quickly turns into noise rather than useful information. — JetBrains Research, NeurIPS 2025 + +## Architectural Solutions over Prompt Engineering + +Every mitigation for context degradation — repetition, reminders, sub-agents, summarization — consumes more context, which makes the problem worse. You're fighting attention dilution by adding tokens that dilute attention (source: agents-md-is-a-liability-paddo.md). + +Architectural solutions matter more: +- **RPI loop** (Research → Plan → Implement with fresh context per phase) works because it doesn't try to fit everything into one window. See [[context-engineering]] for the Research–Plan–Implement workflow +- **Observation masking** — JetBrains' "Complexity Trap" research showed hiding tool output details cuts costs 50% without degrading task performance + +**Your AGENTS.md isn't a knowledge base. It's an attention budget. Spend it wisely** (source: agents-md-is-a-liability-paddo.md). + +## The Ball of Mud Anti-Pattern + +Agent misbehaves → you add a rule → it misbehaves differently → you add another rule → config becomes unmaintainable. Auto-generated files compound this: they start large and rules only accumulate. The fix is the **deletion test**: regularly review every instruction and ask "Would removing this cause the agent to make mistakes?" If not, cut it. See [[context-engineering]] for more on this anti-pattern. + +## Related pages + +- [[context-engineering]] +- [[context-rot]] +- [[progressive-disclosure]] +- [[agent-configuration-files]] +- [[agent-best-practices]] +- [[million-token-context-window]] +- [[dead-context]] diff --git a/wiki/knowledge/ai-agent-protocols-2026.md b/wiki/knowledge/ai-agent-protocols-2026.md new file mode 100644 index 00000000..8a0f3eb7 --- /dev/null +++ b/wiki/knowledge/ai-agent-protocols-2026.md @@ -0,0 +1,132 @@ +# AI Agent Protocols 2026 + +**Summary**: A complete technical guide to the AI agent protocol landscape in 2026, covering MCP and A2A as the two surviving standards (ACP having merged into A2A), with a decision framework and layered production architecture. +**Sources**: `docs/agent-protocols/ai-agent-protocols-2026-guide.md` +**Last updated**: 2026-04-21 + +--- + +## The Consolidation: Two Protocols, One Architecture + +As of 2026, the AI agent communication layer has consolidated around two complementary protocols (source: ai-agent-protocols-2026-guide.md): + +- **[[mcp-specification]]** (Model Context Protocol) — the agent-to-tool standard +- **[[agent-to-agent-protocol]]** (A2A) — the agent-to-agent standard + +ACP (IBM Research) officially merged into A2A under the Linux Foundation in September 2025, leaving a functionally two-protocol world (source: ai-agent-protocols-2026-guide.md). See [[mcp-vs-a2a]] for a direct comparison. + +## MCP: Agent-to-Tool Standard + +Model Context Protocol is Anthropic's open standard for connecting AI agents to external tools, data sources, and services. Before MCP, every AI application needed custom integrations for every data source — an M×N explosion. MCP collapses this to M+N (source: ai-agent-protocols-2026-guide.md). + +**Architecture**: Client-server over JSON-RPC. MCP Clients (AI agents) connect to MCP Servers that expose tools, resources, and prompts from external systems. + +**Key capabilities (spec 2025-11-25)** (source: ai-agent-protocols-2026-guide.md): +- **Tool exposure**: Servers declare callable functions with typed schemas +- **Resource access**: Structured data retrieval from any connected system +- **Streamable HTTP transport**: Replaced the original SSE-only approach for better scalability +- **Elicitation**: Servers can request structured input from users mid-workflow +- **Sampling**: Servers can request LLM completions through the client (with tool-calling support) +- **Tasks (experimental)**: Durable request tracking with polling and deferred results + +**2026 adoption** (source: ai-agent-protocols-2026-guide.md): +- 97 million monthly SDK downloads +- 10,000+ active MCP servers +- Adopted by OpenAI, Google DeepMind, Microsoft (Windows 11 + Copilot), AWS +- Donated to the Agentic AI Foundation (AAIF) under the Linux Foundation in December 2025 +- SDKs available in Python, TypeScript, C#, and Java + +MCP does **not** handle [[multi-agent-communication]] — that is by design. It solves agent-to-tool connectivity only. + +## A2A: Agent-to-Agent Standard + +A2A is Google's protocol for agent-to-agent collaboration. It treats each agent as an opaque service — agents can collaborate without sharing internal logic, framework details, or implementation (source: ai-agent-protocols-2026-guide.md). + +**Architecture**: HTTP-based with JSON-RPC messaging and Server-Sent Events (SSE) for streaming. + +**Key capabilities (v0.3, July 2025)** (source: ai-agent-protocols-2026-guide.md): +- **Agent Cards**: JSON metadata documents describing an agent's capabilities and auth requirements — like an OpenAPI spec for agents +- **Task management**: Structured lifecycle (submitted → working → completed/failed) with support for long-running operations +- **Streaming and push notifications**: Real-time updates via SSE, plus webhook-based push for async workflows +- **Enterprise-grade auth**: OAuth 2.0, mTLS, JWTs +- **Multimodal messaging**: Text, files, structured data, and rich media + +**Governance**: Donated to the Linux Foundation in June 2025. The Technical Steering Committee includes Google, Microsoft, AWS, Cisco, Salesforce, ServiceNow, SAP, and IBM (source: ai-agent-protocols-2026-guide.md). + +## Protocol Comparison + +| Feature | MCP | A2A | +|---------|-----|-----| +| Primary purpose | Agent ↔ Tool/Data | Agent ↔ Agent | +| Created by | Anthropic (Nov 2024) | Google (Apr 2025) | +| Governance | AAIF / Linux Foundation | LF AI & Data / Linux Foundation | +| Transport | JSON-RPC / Streamable HTTP | HTTP + JSON-RPC + SSE | +| Authentication | OAuth 2.0, OIDC | OAuth 2.0, mTLS, JWTs | +| Discovery | MCP Registry, server.json | Agent Cards | +| Spec maturity | Stable (2025-11-25) | Stable (v0.3) | + +(source: ai-agent-protocols-2026-guide.md) + +## Decision Framework + +**Use MCP when** (source: ai-agent-protocols-2026-guide.md): +- Agents need tool access (databases, APIs, SaaS platforms, file systems) +- Building single-agent applications (coding assistant, customer support bot) +- You want plug-and-play integrations from the 10,000+ existing MCP server ecosystem +- Human-in-the-loop workflows are needed (MCP's elicitation feature) + +**Use A2A when** (source: ai-agent-protocols-2026-guide.md): +- Multiple agents need to collaborate (planning → research → analysis pipeline) +- Operating across organizational boundaries +- Vendor-agnostic agent orchestration is required (LangChain + AutoGen, etc.) +- Long-running, asynchronous workflows are the norm + +**Use both when** (source: ai-agent-protocols-2026-guide.md): +- Building enterprise-grade multi-agent systems (the most common 2026 pattern) +- Agents need both tool access AND peer collaboration +- Designing for scale with separate tool-access and agent-coordination layers + +## Production Architecture: The Layered Stack + +``` +Layer 3 — Orchestration + A supervisory agent coordinates the multi-agent pipeline via A2A + and accesses its own tools (logging, monitoring) via MCP + +Layer 2 — Agent Coordination (A2A) + Agents discover each other via Agent Cards, + delegate tasks, and receive structured results + +Layer 1 — Tool Access (MCP) + Each agent connects to its tools via MCP servers + (CRM agent → Salesforce; analytics agent → data warehouse) +``` + +(source: ai-agent-protocols-2026-guide.md) + +## Future Directions + +- **Convergence accelerating**: More mergers and tighter MCP/A2A integration expected (source: ai-agent-protocols-2026-guide.md) +- **Registry problem resolved**: MCP community-driven server registry launched November 2025 +- **Authentication converging**: Both protocols now standardize on OAuth 2.0 / OIDC +- **Agent payments**: Google's Agent Payments Protocol (AP2, September 2025) signals autonomous financial transactions between agents + +## Architecture Recommendations + +1. Adopt MCP for all tool connectivity — it is the undisputed standard (source: ai-agent-protocols-2026-guide.md) +2. Adopt A2A for any multi-agent coordination workflows +3. Abstract your protocol layer so you can evolve with the protocols +4. Invest in observability — both protocols support structured logging hooks +5. Do not build on ACP — it is deprecated; follow IBM's migration guide to A2A +6. Do not build custom agent-to-agent protocols — the A2A ecosystem compounds fast + +## Related pages + +- [[agent-protocol-standards]] +- [[mcp-specification]] +- [[agent-to-agent-protocol]] +- [[mcp-vs-a2a]] +- [[multi-agent-communication]] +- [[agent-workflows]] +- [[context-engineering]] +- [[agent-best-practices]] diff --git a/wiki/knowledge/anthropic-2026-full-connectivity.md b/wiki/knowledge/anthropic-2026-full-connectivity.md new file mode 100644 index 00000000..05e4cb84 --- /dev/null +++ b/wiki/knowledge/anthropic-2026-full-connectivity.md @@ -0,0 +1,94 @@ +# Anthropic 2026 Full Connectivity Forecast + +**Summary**: Anthropic engineer David Soria Parra's forecast that 2026 will be the year of AI agent "full connectivity," combining computer use, CLI tools, and MCP — and his critique of naive REST-to-MCP wrapping in favor of programmatic, agent-native interface design. +**Sources**: `docs/agent-protocols/anthropic-engineer-2026-forecast-full-connectivity-mcp.md` +**Last updated**: 2026-04-21 + +--- + +## The 2026 Full Connectivity Vision + +David Soria Parra of Anthropic, speaking on the AI Engineer podcast, forecasts that 2026 will be defined by "full connectivity" for AI agents (source: anthropic-engineer-2026-forecast-full-connectivity-mcp.md): + +> "2026 I think is all about connectivity and the best agents use every available method." + +The future stack is a pragmatic combination of: + +- **Computer use** (GUI automation — direct screen control and visual understanding) +- **Command-line interfaces (CLIs)** — script execution and system-level access +- **[[mcp-specification]]** — agent-native design, programmatic calling, server discovery, and skills over MCP +- **Packaged Skills** — domain knowledge bundles with continuous updates + +The key shift is moving away from single-method dogma toward whichever connectivity method best fits each task (source: anthropic-engineer-2026-forecast-full-connectivity-mcp.md). + +## The "Cringe" Critique: Why REST Wrapping Fails + +A central theme is a blunt critique of a common industry practice: mechanically converting existing REST APIs into MCP servers (source: anthropic-engineer-2026-forecast-full-connectivity-mcp.md): + +> "We all need to stop taking REST APIs and put them one to one into an MCP server. Every time I see someone building another REST to MCP server conversion tool, I'm… it's a bit cringe because I think it just results in horrible things." + +**The problem**: REST APIs are designed for deterministic, step-by-step human orchestration. They often require multiple sequential calls to complete a single logical task. Forcing an agent to navigate this ignores the agent's unique strength: reasoning about an entire workflow at once. + +| Paradigm | How It Works | Agent Experience | +|----------|-------------|-----------------| +| Sequential REST Wrapping | Agent calls API A, waits, calls API B, waits, calls API C | Cumbersome, slow, prone to error | +| Programmatic Tool Calling | Agent writes a script composing A, B, C in one optimized operation | Fast, efficient, feels "intelligent" | + +(source: anthropic-engineer-2026-forecast-full-connectivity-mcp.md) + +This connects to the broader [[agentic-systems-architectural-paradigms]] shift from sequential tool-calling to programmatic execution. + +## Programmatic Orchestration: Agents as Micro-Programmers + +The alternative is programmatic tool calling — enabling the model to write a small script or program that composes multiple tools in a single operation (source: anthropic-engineer-2026-forecast-full-connectivity-mcp.md): + +> "You don't want the model to go call a tool, take the result, then go and call another tool… what you're effectively doing is you're letting the model orchestrate things together." + +This shift represents a move from agents as simple tool-users to agents as **micro-programmers**, capable of crafting bespoke solutions on the fly. It requires backend services designed to accept and safely execute agent-generated programs — a significant architectural departure from traditional API design. + +This is directly relevant to [[context-engineering]]: the quality of the agent's execution depends heavily on how context is structured to support programmatic reasoning. + +## MCP's New Infrastructure Features + +Two key innovations coming to the MCP ecosystem (source: anthropic-engineer-2026-forecast-full-connectivity-mcp.md): + +### Server Discovery + +Allows AI agents and crawlers to automatically detect MCP servers running on websites or local networks. This removes the need for manual configuration and registry listings, making tool discovery as easy as browsing to a webpage. + +### Skills over MCP + +Allows server authors to bundle updated domain knowledge, prompts, and capabilities directly within the MCP server itself: + +> "It allows you as a server author to continuously ship updated skills without having to rely on plug-in mechanisms and registries." + +This creates a **decentralized and dynamic update model** — an agent connecting to a server immediately gains access to its latest capabilities without any central approval process (source: anthropic-engineer-2026-forecast-full-connectivity-mcp.md). + +## MCP Adoption Metrics (as of 2026) + +(source: anthropic-engineer-2026-forecast-full-connectivity-mcp.md) + +- **97 million** monthly MCP SDK downloads +- All major cloud and AI vendors (OpenAI, Google, Microsoft, AWS) now ship MCP-compatible solutions +- **10,000+** active MCP servers in production +- SDKs available in Python, TypeScript, C#, and Java +- Registry-free skills and server discovery now standard features + +## Industry Implications + +The companies and developers who move beyond the "cringe" phase of simple API wrapping and invest in designing interfaces specifically for AI interaction will build the platforms for the first generation of truly useful general knowledge worker agents (source: anthropic-engineer-2026-forecast-full-connectivity-mcp.md). + +> "This transition marks the moment AI stops adapting to our digital world and begins forcing our digital world to adapt to it." + +See [[agent-best-practices]] for concrete recommendations derived from this philosophy, and [[multi-agent-communication]] for how A2A complements MCP in the full connectivity stack. + +## Related pages + +- [[mcp-specification]] +- [[mcp-vs-a2a]] +- [[agent-protocol-standards]] +- [[agentic-systems-architectural-paradigms]] +- [[context-engineering]] +- [[agent-workflows]] +- [[agent-best-practices]] +- [[subagents]] diff --git a/wiki/knowledge/anthropic-output-consistency.md b/wiki/knowledge/anthropic-output-consistency.md new file mode 100644 index 00000000..452fe93b --- /dev/null +++ b/wiki/knowledge/anthropic-output-consistency.md @@ -0,0 +1,115 @@ +# Anthropic Output Consistency Techniques + +**Summary**: Prompt engineering and workflow techniques for increasing Claude output consistency, with clear guidance on when to use these versus the guaranteed-compliance approach of Structured Outputs. +**Sources**: `docs/structured-outputs/anthropic-increase-consistency.md` +**Last updated**: 2026-04-21 + +--- + +> **For guaranteed JSON schema conformance**, use [[structured-outputs-anthropic]] (`output_config.format`) instead of the techniques below. The techniques on this page are useful for general output consistency or when you need flexibility beyond strict JSON schemas. (source: anthropic-increase-consistency.md) + +## When to Use Each Approach + +| Situation | Recommended Approach | +|-----------|---------------------| +| Guaranteed JSON schema compliance | **[[structured-outputs-anthropic]]** (`output_config.format`) | +| Guaranteed schema-valid tool inputs | **[[anthropic-strict-tool-use]]** (`strict: true`) | +| General JSON-like output with flexibility | Prompt engineering + prefilling | +| Consistent tone or character | Role system prompts + examples | +| Contextual consistency across a session | Retrieval-augmented prompting | +| Complex multi-step workflow | Prompt chaining | + +(source: anthropic-increase-consistency.md) + +## Technique 1: Specify the Desired Output Format + +Precisely define your desired output format in your system prompt or user message. Include: (source: anthropic-increase-consistency.md) +- The exact format (JSON, XML, Markdown, plain text) +- Required fields and their types +- Any ordering or nesting requirements +- An example output structure + +This is the baseline for all consistency work and pairs naturally with [[prompt-engineering]] patterns. + +## Technique 2: Prefill Claude's Response + +> **Not supported** on Claude Mythos Preview, Opus 4.7, Opus 4.6, and Sonnet 4.6. Use [[structured-outputs-anthropic]] or system prompt instructions instead. (source: anthropic-increase-consistency.md) + +Prefill the `Assistant` turn with the start of your desired format to bypass friendly preambles and lock in structure: + +```python +response = client.messages.create( + model="claude-haiku-4-5", + max_tokens=1024, + messages=[ + {"role": "user", "content": "Extract the name and email from: Alice Smith, alice@example.com"}, + {"role": "assistant", "content": '{"name": "'} # prefill starts the JSON + ] +) +``` + +By starting Claude's response with `{"name": "`, Claude is constrained to continue in JSON format. (source: anthropic-increase-consistency.md) + +## Technique 3: Constrain with Examples (Few-Shot Prompting) + +Provide concrete examples of your desired output. This trains Claude's understanding better than abstract instructions: (source: anthropic-increase-consistency.md) + +``` +System: You extract contact information and return it in this exact format: +Name: [full name] +Email: [email address] +Phone: [phone number or "N/A"] + +Example output: +Name: Jane Doe +Email: jane@example.com +Phone: 555-9876 + +User: Here is an email from Bob Jones at bob@acme.com, phone 555-1234. +``` + +Few-shot examples prime Claude to follow the same format reliably across varied inputs. + +## Technique 4: Use Retrieval for Contextual Consistency + +For tasks requiring consistent context (chatbots, knowledge bases), use retrieval to ground Claude's responses in a fixed information set. This prevents hallucination and drift from source material — especially important when output format must reference specific entities or values from a corpus. (source: anthropic-increase-consistency.md) + +This technique pairs naturally with [[context-engineering]] patterns. + +## Technique 5: Chain Prompts for Complex Tasks + +Break complex tasks into smaller, consistent subtasks. Each subtask gets Claude's full attention, reducing inconsistency errors at scale. (source: anthropic-increase-consistency.md) + +**Example pipeline:** +1. **Extract**: Raw data from the input document → unstructured text +2. **Parse**: Extracted text into a structured schema → JSON +3. **Validate**: Enrich the JSON with business logic → final output + +Chaining lets each step have its own clear format constraint, making failures easier to detect and handle per stage. This is a core pattern in [[agent-workflows]]. + +## Technique 6: Keep Claude in Character + +For role-based applications, consistent character requires deliberate prompting: (source: anthropic-increase-consistency.md) + +- **System prompts to set the role**: Define Claude's role, personality, background, and specific traits in detail. +- **Prepare for possible scenarios**: List common scenarios and expected responses to "train" Claude to handle diverse situations without breaking character. + +**Example system prompt:** +``` +You are Aria, a friendly customer support assistant for Acme Inc. +You always respond in English, use a warm and professional tone, +and never discuss competitors. If you don't know the answer, +say: "I'll need to check on that for you — let me get back to you shortly." +``` + +--- + +## Related pages + +- [[structured-outputs-anthropic]] +- [[anthropic-strict-tool-use]] +- [[prompt-engineering]] +- [[context-engineering]] +- [[agent-workflows]] +- [[agent-best-practices]] +- [[json-schema-for-ai]] diff --git a/wiki/knowledge/anthropic-strict-tool-use.md b/wiki/knowledge/anthropic-strict-tool-use.md new file mode 100644 index 00000000..126ab19e --- /dev/null +++ b/wiki/knowledge/anthropic-strict-tool-use.md @@ -0,0 +1,121 @@ +# Anthropic Strict Tool Use + +**Summary**: Setting `strict: true` on a Claude tool definition activates grammar-constrained sampling, guaranteeing tool inputs exactly match the provided JSON Schema — eliminating type mismatches and missing fields in agentic systems. +**Sources**: `docs/structured-outputs/anthropic-strict-tool-use.md`, `docs/structured-outputs/anthropic-implement-tool-use.md` +**Last updated**: 2026-04-21 + +--- + +Strict tool use is the tool-input counterpart to JSON outputs. Where [[structured-outputs-anthropic]] controls *what Claude says*, strict tool use controls *how Claude calls your functions*. (source: anthropic-strict-tool-use.md) + +## Why It Matters for Agents + +Without strict mode, Claude may provide `passengers: "two"` or `passengers: "2"` when your booking function expects `passengers: 2`. This causes runtime errors and requires defensive validation and retry logic throughout the pipeline. (source: anthropic-strict-tool-use.md) + +**With `strict: true`:** +- Functions always receive correctly-typed arguments +- No validation/retry loops for tool calls +- Production agents that work consistently at scale + +## Enabling Strict Mode + +Add `"strict": true` as a top-level property alongside `name`, `description`, and `input_schema`. Also add `"additionalProperties": false` to your schema objects — this is required for the grammar compiler. (source: anthropic-strict-tool-use.md) + +```json +{ + "name": "book_flight", + "description": "Book a flight for the given passenger count and route.", + "strict": true, + "input_schema": { + "type": "object", + "properties": { + "origin": { + "type": "string", + "description": "IATA airport code for the departure airport" + }, + "destination": { + "type": "string", + "description": "IATA airport code for the arrival airport" + }, + "passengers": { + "type": "integer", + "description": "Number of passengers" + } + }, + "required": ["origin", "destination", "passengers"], + "additionalProperties": false + } +} +``` + +**What is guaranteed:** +- The tool `input` block strictly follows `input_schema` +- The tool `name` is always valid (from the provided tool set) (source: anthropic-strict-tool-use.md) + +## Combining with `tool_choice` + +Pair `strict: true` with `tool_choice: {"type": "any"}` for the strongest guarantee: Claude *will* call a tool *and* its inputs will be schema-valid. (source: anthropic-strict-tool-use.md, anthropic-implement-tool-use.md) + +```python +response = client.messages.create( + model="claude-opus-4-5", + max_tokens=1024, + tools=[{"name": "book_flight", "strict": True, "input_schema": { ... }}], + tool_choice={"type": "any"}, + messages=[{"role": "user", "content": "Book a flight from JFK to LAX for 2 passengers"}] +) +``` + +See [[anthropic-tool-use]] for the full `tool_choice` options and compatibility constraints. + +## Common Use Cases + +| Use case | Why strict mode helps | +|----------|-----------------------| +| **Booking systems** | Guarantees integer passenger counts, valid enum seat classes, required date fields | +| **Database queries** | Ensures filter parameters have correct types, preventing malformed queries | +| **API integrations** | Guarantees function calls match the expected API contract exactly | +| **Multi-agent pipelines** | Ensures upstream Claude calls pass correctly-typed inputs to downstream agents | +| **Financial calculations** | Prevents string-typed amounts where integers or floats are required | + +(source: anthropic-strict-tool-use.md) + +## Schema Complexity Limits + +The same limits from [[structured-outputs-anthropic]] apply to strict tools (combined across all strict schemas in a request): (source: anthropic-strict-tool-use.md) + +| Limit | Value | +|-------|-------| +| Strict tools per request | **20** | +| Total optional parameters across all strict tool schemas | **24** | +| Total parameters with union types across all strict schemas | **16** | + +Exceeding these returns `400 "Schema is too complex for compilation"`. See [[json-schema-for-ai]] for tips on reducing complexity. + +## JSON Schema Limitations + +Strict tool use uses the same constrained JSON Schema subset as structured outputs. Some features (e.g., `minimum`, `maximum`, `minLength`, `maxLength`) are not supported. See the [Structured Outputs guide](./structured-outputs-anthropic.md) for the complete list. (source: anthropic-strict-tool-use.md) + +## Data Retention + +Tool `input_schema` definitions are compiled into grammars and **cached for up to 24 hours**. Prompts and responses qualify for Zero Data Retention (ZDR). (source: anthropic-strict-tool-use.md) + +**HIPAA eligible** — but **PHI must not appear in schema definitions**: +- Property names +- `enum` values +- `const` values +- `pattern` regular expressions + +PHI should only appear in message content (prompts and responses). (source: anthropic-strict-tool-use.md) + +--- + +## Related pages + +- [[structured-outputs-anthropic]] +- [[anthropic-tool-use]] +- [[json-schema-for-ai]] +- [[tool-use-patterns]] +- [[agent-workflows]] +- [[agent-best-practices]] +- [[mcp-programmatic-tool-calling]] diff --git a/wiki/knowledge/anthropic-tool-use.md b/wiki/knowledge/anthropic-tool-use.md new file mode 100644 index 00000000..10853539 --- /dev/null +++ b/wiki/knowledge/anthropic-tool-use.md @@ -0,0 +1,159 @@ +# Anthropic Tool Use — Implementation Guide + +**Summary**: How to implement tool use with the Claude API, covering tool definition best practices, `input_examples`, `tool_choice` control options, and how Claude responds when tools are available. +**Sources**: `docs/structured-outputs/anthropic-implement-tool-use.md` +**Last updated**: 2026-04-21 + +--- + +Tool use lets Claude call external functions. Tools are defined in the `tools` top-level parameter of the API request. Strict schema validation is layered on top via `strict: true` — see [[anthropic-strict-tool-use]]. (source: anthropic-implement-tool-use.md) + +## Choosing a Model + +- **Claude Opus 4.7**: Best for complex tools and ambiguous queries; handles multiple tools well and seeks clarification when needed. +- **Claude Haiku models**: Suitable for straightforward tools; may infer missing parameters. + +When using tool use with extended thinking, consult the extended thinking compatibility notes. (source: anthropic-implement-tool-use.md) + +## Tool Definition Parameters + +| Parameter | Required | Description | +|-----------|----------|-------------| +| `name` | Yes | Must match `^[a-zA-Z0-9_-]{1,64}$` | +| `description` | Yes | Detailed description of what the tool does, when to use it, and its limitations | +| `input_schema` | Yes | JSON Schema object defining expected parameters | +| `input_examples` | No | Array of example input objects | +| `strict` | No | Set `true` to enable grammar-constrained schema validation | +| `cache_control` | No | Prompt caching configuration | + +When you call the API with `tools`, Claude automatically receives a constructed system prompt explaining the available tools. (source: anthropic-implement-tool-use.md) + +## Best Practices for Tool Definitions + +**Provide extremely detailed descriptions** — this is the most important factor in tool performance. A good description covers: (source: anthropic-implement-tool-use.md) +- What the tool does +- When it should (and should not) be used +- What each parameter means and how it affects behavior +- Important caveats and limitations (aim for 3–4+ sentences per tool) + +**Consolidate related operations** — rather than `create_pr`, `update_pr`, `close_pr`, use a single tool with an `action` parameter. Fewer, more capable tools reduce selection ambiguity. (source: anthropic-implement-tool-use.md) + +**Good vs. poor description example:** + +```python +# Good +{ + "name": "get_stock_price", + "description": ( + "Retrieves the current stock price for a given ticker symbol. " + "The ticker symbol must be a valid symbol for a publicly traded company on a major US stock exchange like NYSE or NASDAQ. " + "The tool will return the closing price and is updated at the end of each trading day. " + "It does not provide real-time prices or data for OTC markets." + ), + "input_schema": { + "type": "object", + "properties": { + "ticker": {"type": "string", "description": "The stock ticker symbol, e.g. AAPL for Apple Inc."} + }, + "required": ["ticker"] + } +} + +# Poor +{ + "name": "get_stock_price", + "description": "Gets the stock price for a ticker.", + "input_schema": {"type": "object", "properties": {"ticker": {"type": "string"}}, "required": ["ticker"]} +} +``` + +See [[json-schema-for-ai]] for guidance on writing effective `input_schema` definitions. + +## Providing Tool Use Examples (`input_examples`) + +Concrete input examples help Claude understand how to use a tool, especially for complex schemas. (source: anthropic-implement-tool-use.md) + +```python +{ + "name": "book_flight", + "input_examples": [ + {"origin": "JFK", "destination": "LAX", "date": "2025-12-01", "passengers": 2}, + {"origin": "SFO", "destination": "ORD", "date": "2025-11-15", "passengers": 1} + ] +} +``` + +**Requirements and limitations:** +- Each example must be valid according to the tool's `input_schema` (invalid examples return `400`) +- Not supported for server-side tools +- Token cost: ~20–50 tokens for simple examples, ~100–200 for complex nested objects + +## Controlling Output with `tool_choice` + +| Value | Behavior | +|-------|----------| +| `auto` | Claude decides whether to call any tool (default when tools are provided) | +| `any` | Claude must use one of the provided tools (model's choice) | +| `tool` | Forces use of a specific named tool | +| `none` | Prevents tool use (default when no tools provided) | + +When `tool_choice` is `any` or `tool`, the API prefills the assistant message — Claude will not emit a natural language explanation before `tool_use` blocks. (source: anthropic-implement-tool-use.md) + +**Extended thinking compatibility**: only `auto` and `none` are compatible with extended thinking. **Claude Mythos Preview** does not support forced tool use — use `auto` and rely on prompting. (source: anthropic-implement-tool-use.md) + +### Combining `tool_choice` with `strict: true` + +Use `tool_choice: {"type": "any"}` + `strict: true` to guarantee both that a tool is called AND that its inputs conform to your schema: (source: anthropic-implement-tool-use.md) + +```python +response = client.messages.create( + model="claude-sonnet-4-5", + max_tokens=1024, + tools=[{ + "name": "save_contact", + "description": "Save a contact record.", + "strict": True, + "input_schema": { + "type": "object", + "properties": { + "name": {"type": "string"}, + "email": {"type": "string"} + }, + "required": ["name", "email"], + "additionalProperties": False + } + }], + tool_choice={"type": "any"}, + messages=[{"role": "user", "content": "Save Alice Smith, alice@example.com"}] +) +``` + +## Model Responses with Tools + +Claude often comments on what it's doing before invoking tools. Your code should treat these as normal assistant text and not rely on specific phrasing conventions. (source: anthropic-implement-tool-use.md) + +Example for "What's the weather in San Francisco right now, and what time is it there?": +``` +Claude: "I'll check the current weather and time in San Francisco for you!" +[tool_use: get_weather {"location": "San Francisco, CA"}] +[tool_use: get_time {"location": "San Francisco, CA"}] +``` + +## Production Recommendations + +- **Use meaningful namespacing** in tool names when spanning multiple services (e.g., `github_list_prs`, `slack_send_message`) +- **Return only high-signal information** from tool responses — semantic identifiers (slugs, UUIDs) rather than opaque references +- **Enable `strict: true`** for production agentic workflows — see [[anthropic-strict-tool-use]] + +--- + +## Related pages + +- [[anthropic-strict-tool-use]] +- [[structured-outputs-anthropic]] +- [[json-schema-for-ai]] +- [[tool-use-patterns]] +- [[agent-workflows]] +- [[agent-best-practices]] +- [[mcp-programmatic-tool-calling]] +- [[prompt-engineering]] diff --git a/wiki/knowledge/claude-code-hooks.md b/wiki/knowledge/claude-code-hooks.md new file mode 100644 index 00000000..64ba2ba4 --- /dev/null +++ b/wiki/knowledge/claude-code-hooks.md @@ -0,0 +1,270 @@ +# Claude Code Hooks + +**Summary**: Deterministic automation points in the Claude Code lifecycle that execute shell commands, HTTP requests, LLM prompts, or agent-based verification at specific events — enabling formatting, validation, auditing, and control flow without relying on the model's judgment. +**Sources**: automate-workflow-with-hooks.md, claude-hook-reference-doc.md +**Last updated**: 2026-04-22 + +--- + +## Why Hooks + +Hooks provide **deterministic control** — they run reliably regardless of the model's behavior. Use hooks to enforce rules that must always apply, not as suggestions the model might follow. For decisions requiring judgment rather than deterministic rules, use [[#Prompt-Based Hooks]] or [[#Agent-Based Hooks]]. + +## Hook Types + +| Type | Mechanism | Use Case | Default Timeout | +| ----------- | ---------------------------------------- | ------------------------------------------ | --------------- | +| **Command** | Shell script execution | Formatting, file protection, audit logging | 600s (10 min) | +| **HTTP** | POST to external endpoint | Notifications, CI triggers, webhooks | 30s | +| **Prompt** | Single-turn Claude evaluation | Conditional logic, policy decisions | 30s | +| **Agent** | Claude with tool access (up to 50 turns) | Complex verification, multi-step checks | 60s | + +All timeouts are configurable per hook via the `timeout` field (in seconds). + +## Complete Lifecycle Events + +Every event in the Claude Code lifecycle has a corresponding hook point. Events fire at specific times during a session — some fire once, others fire repeatedly inside the agentic loop. + +### Session Lifecycle Events + +| Event | Triggers When | Matcher Target | Can Block? | +| -------------------- | ----------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | ---------- | +| `SessionStart` | Session begins or resumes | Source: `startup`, `resume`, `clear`, `compact` | No | +| `InstructionsLoaded` | CLAUDE.md or `.claude/rules/*.md` loaded into context | Load reason: `session_start`, `nested_traversal`, `path_glob_match`, `include`, `compact` | No | +| `SessionEnd` | Session terminates | End reason: `clear`, `resume`, `logout`, `prompt_input_exit`, `bypass_permissions_disabled`, `other` | No | + +### User Interaction Events + +| Event | Triggers When | Matcher Target | Can Block? | +| ------------------ | ------------------------------------------------ | ------------------------------------------------------------------------------ | ---------------------------------------------------- | +| `UserPromptSubmit` | User sends a prompt (before Claude processes it) | No matcher support | Yes — blocks prompt processing and erases the prompt | +| `Notification` | Claude Code sends a notification | Type: `permission_prompt`, `idle_prompt`, `auth_success`, `elicitation_dialog` | No | + +### Tool Execution Events + +| Event | Triggers When | Matcher Target | Can Block? | +| -------------------- | ------------------------- | ----------------------------------------------------------------------------- | --------------------------- | +| `PreToolUse` | Before tool execution | Tool name: `Bash`, `Edit`, `Write`, `Read`, `Glob`, `Grep`, `Agent`, `mcp__*` | Yes — blocks the tool call | +| `PermissionRequest` | Permission dialog appears | Tool name | Yes — denies the permission | +| `PostToolUse` | After tool succeeds | Tool name | No (tool already ran) | +| `PostToolUseFailure` | After tool fails | Tool name | No (tool already failed) | + +### Subagent and Team Events + +| Event | Triggers When | Matcher Target | Can Block? | +| --------------- | ------------------------------------ | --------------------------------------------------- | ------------------------------------- | +| `SubagentStart` | Subagent spawned | Agent type: `Bash`, `Explore`, `Plan`, custom names | No | +| `SubagentStop` | Subagent finishes | Agent type (same as SubagentStart) | Yes — prevents subagent from stopping | +| `TeammateIdle` | Agent team teammate about to go idle | No matcher support | Yes — keeps teammate working | +| `TaskCompleted` | Task marked as completed | No matcher support | Yes — prevents completion | + +### Completion Events + +| Event | Triggers When | Matcher Target | Can Block? | +| ------------- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------- | +| `Stop` | Claude finishes responding | No matcher support | Yes — prevents stopping, continues conversation | +| `StopFailure` | Turn ends due to API error | Error type: `rate_limit`, `authentication_failed`, `billing_error`, `invalid_request`, `server_error`, `max_output_tokens`, `unknown` | No (output/exit code ignored) | + +### Context and Configuration Events + +| Event | Triggers When | Matcher Target | Can Block? | +| -------------- | ----------------------------------------- | ------------------------------------------------------------------------------------------ | ------------------------------ | +| `PreCompact` | Before context compaction | Trigger: `manual`, `auto` | No | +| `PostCompact` | After compaction completes | Trigger: `manual`, `auto` | No | +| `ConfigChange` | Configuration file changes during session | Source: `user_settings`, `project_settings`, `local_settings`, `policy_settings`, `skills` | Yes (except `policy_settings`) | + +### Worktree Events + +| Event | Triggers When | Matcher Target | Can Block? | +| ---------------- | ------------------------------------------------------------ | ------------------ | ---------------------------------- | +| `WorktreeCreate` | Worktree created via `--worktree` or `isolation: "worktree"` | No matcher support | Yes — non-zero exit fails creation | +| `WorktreeRemove` | Worktree removed at session exit or subagent finish | No matcher support | No | + +### MCP Elicitation Events + +| Event | Triggers When | Matcher Target | Can Block? | +| ------------------- | ----------------------------------------------- | --------------- | ---------------------------------------------- | +| `Elicitation` | MCP server requests user input during tool call | MCP server name | Yes — denies the elicitation | +| `ElicitationResult` | User responds to MCP elicitation | MCP server name | Yes — blocks response (action becomes decline) | + +## Configuration + +Hooks are defined in JSON at multiple levels: + +| Location | Scope | Shareable | +| ----------------------------- | ------------------------- | ------------------------- | +| `~/.claude/settings.json` | All your projects | No, local to machine | +| `.claude/settings.json` | Single project | Yes, committable | +| `.claude/settings.local.json` | Single project | No, gitignored | +| Managed policy settings | Organization-wide | Yes, admin-controlled | +| Plugin `hooks/hooks.json` | When plugin is enabled | Yes, bundled with plugin | +| Skill or agent frontmatter | While component is active | Yes, defined in component | + +Enterprise administrators can use `allowManagedHooksOnly` to block user, project, and plugin hooks. + +### Structure + +``` +hooks → event name → matcher group array → hooks array → handler +``` + +### Handler Fields (Common) + +| Field | Required | Description | +| --------------- | -------- | -------------------------------------------------------------------------- | +| `type` | Yes | `"command"`, `"http"`, `"prompt"`, or `"agent"` | +| `timeout` | No | Seconds before canceling (defaults: 600 command, 30 prompt/http, 60 agent) | +| `statusMessage` | No | Custom spinner message while hook runs | +| `once` | No | If `true`, runs only once per session (skills only) | + +## Exit Code Control + +| Code | Behavior | +| ----- | ----------------------------------------------------------------------------------------------------------------- | +| `0` | Proceed (allow). Stdout parsed for JSON output. For `UserPromptSubmit`/`SessionStart`, stdout is added as context | +| `2` | Block with stderr feedback to Claude. JSON on stdout is ignored | +| Other | Proceed with stderr logging (visible in verbose mode via `Ctrl+O`) | + +For structured control, return JSON on exit 0: `{hookSpecificOutput: {hookEventName: "...", decision: {...}}}` + +### Universal JSON Output Fields + +| Field | Default | Description | +| ---------------- | ------- | --------------------------------------------------------------------------------- | +| `continue` | `true` | If `false`, Claude stops processing entirely (overrides event-specific decisions) | +| `stopReason` | none | Message shown to user when `continue` is `false` | +| `suppressOutput` | `false` | If `true`, hides stdout from verbose mode | +| `systemMessage` | none | Warning message shown to the user | + +## Advanced Control Patterns + +### `updatedInput` — Pre-Execution Validation + +`PreToolUse` and `PermissionRequest` hooks can modify tool parameters before execution via the `updatedInput` field. Combine with `"allow"` to auto-approve with modifications, or `"ask"` to show modified input to the user: + +```json +{ + "hookSpecificOutput": { + "hookEventName": "PreToolUse", + "permissionDecision": "allow", + "updatedInput": { "command": "npm run lint" }, + "additionalContext": "Current environment: production. Proceed with caution." + } +} +``` + +### `updatedPermissions` — Programmatic Permission Control + +`PermissionRequest` hooks can modify session permissions when allowing an action. Use `setMode` to change the permission mode, or `addAllowRule`/`addDenyRule` to add persistent rules: + +```json +{ + "hookSpecificOutput": { + "hookEventName": "PermissionRequest", + "decision": { + "behavior": "allow", + "updatedPermissions": [ + { "type": "setMode", "mode": "acceptEdits", "destination": "session" } + ] + } + } +} +``` + +### `additionalContext` — Context Injection + +Multiple events support injecting text into Claude's context via the `additionalContext` field in JSON output: `SessionStart`, `UserPromptSubmit`, `PreToolUse`, `PostToolUse`, `PostToolUseFailure`, `Notification`, and `SubagentStart`. Multiple hooks' values are concatenated. For `SessionStart`, `CLAUDE_ENV_FILE` enables persisting environment variables for all subsequent Bash commands in the session. + +## Async Hooks + +Set `"async": true` on command hooks to run them in the background without blocking Claude. Async hooks: + +- Receive the same JSON input on stdin as synchronous hooks +- **Cannot block or control behavior** — `decision`, `permissionDecision`, and `continue` fields have no effect since the action already completed +- If the hook produces `systemMessage` or `additionalContext`, the content is delivered on the next conversation turn +- Only `type: "command"` hooks support `async` — prompt/agent/http hooks cannot run asynchronously +- Each execution creates a separate background process with no deduplication +- Use the same default 10-minute timeout as sync hooks (configurable via `timeout`) + +## Matchers + +Matchers narrow hook scope using regex patterns against event-specific fields: + +- `"Bash"` — Only Bash tool invocations +- `"Edit|Write"` — Edit OR Write tools +- `"mcp__.*"` — All MCP tools +- `"mcp__github__.*"` — All tools from the GitHub MCP server +- `"mcp__.*__write.*"` — Any write tool from any MCP server + +Events without matcher support (`UserPromptSubmit`, `Stop`, `TeammateIdle`, `TaskCompleted`, `WorktreeCreate`, `WorktreeRemove`) always fire on every occurrence. A `matcher` field on these events is silently ignored. + +**Keep matchers narrow** to avoid unintended matches. Matching on `".*"` or leaving matcher empty for `PermissionRequest` would auto-approve every permission prompt. + +## Security Patterns + +### File Protection with Path Traversal Validation + +Use `PreToolUse` hooks on `Edit|Write` to check target file paths against protected patterns. Exit 2 to block. Validate against path traversal by checking for `..` sequences and canonicalizing paths before comparison. + +### Auto-Format After Writes + +`PostToolUse` hooks on `Edit|Write` extract the edited file path from stdin JSON (via `jq -r '.tool_input.file_path'`) and pipe to Prettier, Black, or your formatter of choice. + +### `stop_hook_active` — Infinite Loop Prevention + +`Stop` and `SubagentStop` hooks receive a `stop_hook_active` boolean in their JSON input. When `true`, it means Claude Code is already continuing as a result of a prior stop hook. **Always check this field** and exit 0 early to prevent Claude from running indefinitely: + +```bash +INPUT=$(cat) +if [ "$(echo "$INPUT" | jq -r '.stop_hook_active')" = "true" ]; then + exit 0 +fi +``` + +## Prompt-Based Hooks + +`type: "prompt"` hooks send the hook input data plus your prompt to a Claude model (Haiku by default, configurable via `model`). The model returns a yes/no decision as JSON: + +- `"ok": true` — the action proceeds +- `"ok": false` — the action is blocked; `reason` is fed back to Claude + +Use for decisions requiring judgment rather than deterministic rules. + +## Agent-Based Hooks + +`type: "agent"` hooks spawn a subagent with tool access (Read, Grep, Glob, etc.) to verify conditions against the actual state of the codebase. Same `"ok"`/`"reason"` response format as prompt hooks, but supports up to 50 tool-use turns with a longer default timeout of 60 seconds. Use `$ARGUMENTS` as a placeholder for hook input JSON in the prompt. + +## Common Patterns + +- **Auto-format**: PostToolUse on `Edit|Write` → `jq -r '.tool_input.file_path' | xargs npx prettier --write` +- **File protection**: PreToolUse on `Edit|Write` → check against protected paths, exit 2 to block +- **Audit logging**: PostToolUse → `jq -c '{timestamp: now | todate, source: .source, file: .file_path}' >> audit.log` +- **Notification**: Notification event → `osascript` (macOS) or `notify-send` (Linux) +- **Permission auto-approval**: PermissionRequest with narrow matcher + JSON `behavior: "allow"` decision +- **Context re-injection**: SessionStart with `compact` matcher → echo critical context back after compaction +- **Config audit**: ConfigChange → log `{timestamp, source, file_path}` to compliance log +- **Completeness check**: Stop → prompt hook asking if all tasks are complete, blocks with `reason` if not +- **Test verification**: Stop → agent hook that runs test suite before allowing completion +- **MCP tool logging**: PreToolUse on `mcp__github__.*` → log GitHub API operations +- **Environment setup**: SessionStart → write `export` statements to `$CLAUDE_ENV_FILE` for persistent env vars + +## HTTP Hooks + +`type: "http"` hooks POST event JSON to an endpoint. Response handling differs from command hooks: + +- **2xx with empty body** → success (equivalent to exit 0 with no output) +- **2xx with JSON body** → parsed using same JSON output schema as command hooks +- **Non-2xx / connection failure / timeout** → non-blocking error, execution continues + +To block a tool call via HTTP, return a 2xx response with appropriate `hookSpecificOutput` fields. Status codes alone cannot block actions. Header values support env var interpolation via `$VAR_NAME` syntax, but only variables listed in `allowedEnvVars` are resolved. + +## Hooks in Skills and Agents + +Hooks can be defined directly in [[claude-code-skills]] and [[claude-code-subagents]] YAML frontmatter. These hooks are scoped to the component's lifecycle and cleaned up when it finishes. All hook events are supported. For subagents, `Stop` hooks are automatically converted to `SubagentStop`. + +## Related pages + +- [[claude-code-plugins]] +- [[claude-code-memory]] +- [[claude-code-subagents]] +- [[agent-workflows]] diff --git a/wiki/knowledge/claude-code-memory.md b/wiki/knowledge/claude-code-memory.md new file mode 100644 index 00000000..0ebfc46d --- /dev/null +++ b/wiki/knowledge/claude-code-memory.md @@ -0,0 +1,94 @@ +# Claude Code Memory + +**Summary**: The multi-layered system by which Claude Code maintains persistent project context across sessions — comprising CLAUDE.md file hierarchies, path-scoped rules in `.claude/rules/`, auto memory from corrections, and import-based composition. +**Sources**: how-claude-remembers-a-project.md +**Last updated**: 2026-04-22 + +--- + +## Memory Layers + +| Layer | Written By | Persistence | Scope | +| --------------------- | ------------------- | ------------------------------ | ---------------------- | +| **CLAUDE.md** | Human | Permanent (version-controlled) | Project, user, managed | +| **`.claude/rules/`** | Human | Permanent (version-controlled) | Path-specific | +| **Auto memory** | Claude | Persistent (local storage) | User or project | +| **Imports** (`@file`) | Human (referencing) | Derived from source files | Composable | + +## CLAUDE.md File Hierarchy + +Discovery follows directory traversal from the working directory upward: + +``` +/etc/claude-code/CLAUDE.md ← Managed policy (enterprise) +~/.claude/CLAUDE.md ← User preferences +./CLAUDE.md ← Project root +./.claude/CLAUDE.md ← Alternative project location +./src/CLAUDE.md ← Subdirectory (lazy loaded) +``` + +## Path-Scoped Rules + +Rules in `.claude/rules/` are the implementation of [[progressive-disclosure]] for instructions: + +```yaml +--- +paths: + - "src/api/**/*.ts" +--- +Use Zod for all API input validation. +Return standardized error responses with error codes. +``` + +Rules load **only when matching files are touched**, keeping the context budget clean. + +### Pattern Examples + +- `**/*.ts` — All TypeScript files +- `src/components/*.tsx` — React components +- `src/**/*` — Everything in src +- `*.md` — All markdown files + +## Import Syntax + +Compose instructions from existing files: + +- `@README` — Import README content +- `@package.json` — Import package manifest +- `@~/.claude/my-instructions.md` — Cross-project shared instructions +- **Depth limit**: 5 hops maximum +- **Symlinks**: Supported for sharing rules across projects (`ln -s ~/shared-claude-rules .claude/rules/shared`) + +## Line Budget + +| File | Target | Maximum | +| ---------------- | ----------- | --------- | +| Root CLAUDE.md | 15–40 lines | 200 lines | +| Plugin CLAUDE.md | 10–30 lines | — | +| Rule files | 10–30 lines | — | + +Every token loads on **every request**. The test: "Would removing this cause the agent to make mistakes?" If not, cut it. + +## Auto Memory + +Claude automatically stores learnings from corrections and patterns: + +- Triggered when users correct Claude's behavior +- Stored locally (not in version control) +- Scopes: user, project + +## Key Practices + +- Use **specific, concrete instructions** ("Use 2-space indentation" not "Format properly") +- Use **markdown headers and bullets**, not dense paragraphs +- Run `/init` to auto-generate a starter CLAUDE.md +- Commit project CLAUDE.md to version control +- Put personal preferences in user CLAUDE.md, not project +- Don't mix conflicting rules (Claude picks one arbitrarily) + +## Related pages + +- [[agent-configuration-files]] +- [[progressive-disclosure]] +- [[context-engineering]] +- [[claude-code-hooks]] diff --git a/wiki/knowledge/claude-code-plugins.md b/wiki/knowledge/claude-code-plugins.md new file mode 100644 index 00000000..c24c2a62 --- /dev/null +++ b/wiki/knowledge/claude-code-plugins.md @@ -0,0 +1,154 @@ +# Claude Code Plugins + +**Summary**: Distributable packages that bundle skills, agents, hooks, MCP/LSP servers, and commands into a single installable unit with namespace isolation — the primary mechanism for sharing Claude Code extensions across teams and the community. +**Sources**: claude-create-plugin-doc.md, research-claude-code-skills-format.md +**Last updated**: 2026-04-22 + +--- + +## Plugin Structure + +``` +my-plugin/ +├── .claude-plugin/ +│ └── plugin.json (manifest — required) +├── skills/ (Agent Skills) +├── agents/ (Custom subagents) +├── hooks/ +│ └── hooks.json (Hook definitions) +├── commands/ (Legacy markdown commands) +├── settings.json (Default configuration) +├── .mcp.json (MCP servers) +├── .lsp.json (LSP servers) +└── README.md (Documentation) +``` + +> **Critical**: Components go at the plugin root, **not** inside `.claude-plugin/`. + +## Plugin.json Manifest + +| Field | Required | Description | +| ------------- | -------- | ----------------------------------------------- | +| `name` | Yes | Unique identifier (determines namespace prefix) | +| `version` | Yes | Semantic version | +| `description` | Yes | What the plugin does | +| `author` | Yes | `{name, email}` | +| `skills` | No | Skill directory references | +| `agents` | No | Agent definitions | +| `hooks` | No | Hook configurations | +| `mcpServers` | No | MCP server definitions | +| `lspServers` | No | LSP server definitions | + +## Namespacing + +Plugin components are automatically namespaced using the `name` field from `plugin.json` to prevent conflicts when multiple plugins are installed: + +| Component | Invocation Pattern | Example | +| --------- | --------------------------- | ------------------------ | +| Skills | `/plugin-name:skill-name` | `/code-tools:lint-check` | +| Commands | `/plugin-name:command-name` | `/code-tools:format` | + +This means two plugins can safely have skills with the same name — each is scoped by its plugin prefix. The plugin name from `plugin.json` determines the prefix. + +## `$ARGUMENTS` Placeholder + +Skills in plugins can use `$ARGUMENTS` to capture text the user provides after the skill name: + +``` +/my-plugin:hello Alex +``` + +In this example, `"Alex"` becomes `$ARGUMENTS` inside the SKILL.md body. This enables parameterized skill invocation. + +## Distribution + +### Three-Layer Extensibility + +1. **Standalone skills** — Individual SKILL.md directories +2. **Plugins** — Bundled packages of skills, agents, hooks +3. **Marketplaces** — Catalogs of plugins (`.claude-plugin/marketplace.json`) + +### Installation Sources + +- Relative file path +- GitHub repository (e.g., `owner/repo`) +- Git URL with optional subdirectory +- npm package + +### Marketplace Format + +The marketplace manifest catalogs plugins available for discovery and installation: + +```json +{ + "name": "my-marketplace", + "owner": {"name": "...", "email": "..."}, + "plugins": [ + { + "name": "my-plugin", + "source": {"source": "github", "repo": "owner/my-plugin"} + } + ] +} +``` + +**Note**: The `source` field is a marketplace-manifest-only field — it does not appear in per-plugin `plugin.json` manifests. + +Submission: `claude.ai/settings/plugins/submit` or `platform.claude.com/plugins/submit`. + +## Environment Variables + +| Variable | Scope | Description | +| ----------------------- | ----------------------- | ----------------------------------------------------------------- | +| `${CLAUDE_PLUGIN_ROOT}` | Plugin runtime | Plugin installation directory (use for referencing bundled files) | +| `${CLAUDE_PLUGIN_DATA}` | Plugin runtime | Persistent data directory for plugin state | +| `$CLAUDE_PROJECT_DIR` | Hooks | Project root directory | +| `$CLAUDE_ENV_FILE` | SessionStart hooks only | Path to write `export` statements for persistent env vars | + +## Plugin Security Constraints + +Agents bundled in plugins operate in a restricted security context: + +| Frontmatter Field | Behavior in Plugin Context | +| ----------------- | -------------------------- | +| `hooks` | **Silently ignored** | +| `mcpServers` | **Silently ignored** | +| `permissionMode` | **Silently ignored** | + +These restrictions prevent published plugins from escalating privileges. **Workaround**: Copy the agent file to `.claude/agents/` or `~/.claude/agents/` to use these fields outside the plugin sandbox. + +## Development Workflow + +### Local Testing + +1. Create plugin structure with manifest +2. Test locally: `claude --plugin-dir ./my-plugin` +3. Reload without restarting: `/reload-plugins` +4. Use `--plugin-dir` multiple times for multi-plugin testing + +### Override Behavior + +A local plugin (loaded via `--plugin-dir`) overrides a marketplace plugin with the same name — except for managed force-enabled plugins set by organization admins. + +### Debugging + +- Check plugin structure matches the expected layout +- Test components individually (skills, hooks, agents) +- Verify namespace doesn't conflict with other installed plugins +- Check `/plugins` output to see loaded plugin list + +## Key Practices + +- Use semantic versioning for releases +- Include README.md with installation and usage instructions +- Test with other plugins to ensure no namespace conflicts +- Use skills directory (not commands) for new capabilities +- Keep plugin `name` short and descriptive — it becomes the namespace prefix +- Test all components together via `--plugin-dir` before publishing + +## Related pages + +- [[claude-code-skills]] +- [[claude-code-hooks]] +- [[claude-code-subagents]] +- [[agent-skills-standard]] diff --git a/wiki/knowledge/claude-code-skills.md b/wiki/knowledge/claude-code-skills.md new file mode 100644 index 00000000..413cd888 --- /dev/null +++ b/wiki/knowledge/claude-code-skills.md @@ -0,0 +1,81 @@ +# Claude Code Skills + +**Summary**: Custom instruction packages that extend Claude Code's capabilities through SKILL.md files with YAML frontmatter — following the Agent Skills open standard with Claude-specific extensions for model selection, tool restriction, and context forking. +**Sources**: extend-claude-with-skills.md, research-claude-code-skills-format.md +**Last updated**: 2026-04-22 + +--- + +## Skill Format + +Every skill is a directory containing at minimum a `SKILL.md` file: + +``` +my-skill/ +├── SKILL.md (required — YAML frontmatter + instructions) +├── references/ (optional — detailed reference material) +├── assets/ (optional — templates, examples, data) +└── scripts/ (optional — executable helpers) +``` + +### YAML Frontmatter + +| Field | Required | Description | +| -------------------------- | -------- | ------------------------------------------ | +| `name` | Yes | 1–64 chars, lowercase, hyphens only | +| `description` | Yes | Max 1024 chars, third person, what + when | +| `disable-model-invocation` | No | `true` = manual only (for destructive ops) | +| `user-invocable` | No | `false` = background knowledge only | +| `allowed-tools` | No | Restrict available tools | +| `model` | No | sonnet, opus, haiku, full ID | +| `effort` | No | low, medium, high, max (Opus 4.6 only) | +| `context` | No | `fork` = isolated subagent execution | +| `agent` | No | Explore, Plan, general-purpose | +| `argument-hint` | No | Placeholder text for arguments | +| `hooks` | No | Lifecycle hook definitions | + +### String Substitutions + +- `$ARGUMENTS` / `$ARGUMENTS[N]` / `$N` — User-provided arguments +- `${CLAUDE_SESSION_ID}` — Current session identifier +- `${CLAUDE_SKILL_DIR}` — Skill directory path +- `` !`` `` — Dynamic context (runs shell command before sending to Claude) + +## Skill Locations + +1. **Plugin skills** — `/skills/` (namespaced: `plugin-name:skill-name`) +2. **Project skills** — `.claude/skills/` +3. **Personal skills** — `~/.claude/skills/` + +## Progressive Disclosure + +| Tier | Loaded When | Budget | +| -------------------------------- | ----------------- | ------------- | +| Metadata (name, description) | Session start | ~100 tokens | +| Instructions (SKILL.md body) | Skill activated | <5,000 tokens | +| Resources (references/, assets/) | Explicitly loaded | On-demand | + +## Bundled Skills + +Claude Code ships with: `/batch`, `/claude-api`, `/debug`, `/loop`, `/simplify` + +## Invocation + +- **Manual**: Type `/skill-name` in chat +- **Auto**: Claude decides based on description match +- **Plugin**: `/plugin-name:skill-name [arguments]` + +## Key Practices + +- Keep SKILL.md under **500 lines** (~5,000 tokens) +- Use `disable-model-invocation: true` for side-effect tasks (deploy, commit) +- Move large API docs to `references/` files, referenced from SKILL.md +- Test skills both ways: auto-invocation and manual with `/` +- Don't use skills for settings — use CLAUDE.md instead + +## Related pages + +- [[agent-skills-standard]] +- [[skill-authoring]] +- [[claude-code-plugins]] +- [[progressive-disclosure]] diff --git a/wiki/knowledge/claude-code-subagents.md b/wiki/knowledge/claude-code-subagents.md new file mode 100644 index 00000000..d8016d4c --- /dev/null +++ b/wiki/knowledge/claude-code-subagents.md @@ -0,0 +1,172 @@ +# Claude Code Subagents + +**Summary**: Task-specific assistants defined as Markdown files with YAML frontmatter that run in isolated context windows within Claude Code sessions — supporting tool restriction, model selection, permission modes, persistent memory, and worktree isolation. +**Sources**: creating-custom-subagents.md, claude-orchestrate-of-claude-code-sessions.md, research-subagent-best-practices.md +**Last updated**: 2026-04-22 + +--- + +## Definition Format + +```yaml +--- +name: code-reviewer +description: Reviews code for bugs, security issues, and style +tools: Read, Grep, Glob +model: haiku +maxTurns: 15 +--- +You are a senior code reviewer. Focus on correctness, security, and maintainability. +Report only issues with >80% confidence. +``` + +## Frontmatter Fields + +| Field | Required | Default | Description | +| ----------------- | -------- | ------- | --------------------------------------------- | +| `name` | Yes | — | Identifier | +| `description` | Yes | — | When to invoke | +| `tools` | No | All | Comma-separated allowlist | +| `disallowedTools` | No | None | Denylist | +| `model` | No | inherit | sonnet, opus, haiku, full ID | +| `maxTurns` | No | — | Cost control (15 exploration, 20 evaluation) | +| `permissionMode` | No | default | acceptEdits, dontAsk, bypassPermissions, plan | +| `memory` | No | — | user, project, local (cross-session learning) | +| `isolation` | No | — | `worktree` for git worktree isolation | +| `background` | No | false | Run asynchronously | +| `effort` | No | — | low, medium, high, max | +| `skills` | No | — | Available skills for the subagent | +| `mcpServers` | No | — | Scoped MCP server access | +| `hooks` | No | — | Subagent-specific hooks | + +## Locations (Priority Order) + +1. **CLI flag**: `--agents` with JSON definition +2. **Project**: `.claude/agents/` +3. **User**: `~/.claude/agents/` +4. **Plugin**: `agents/` directory + +## Built-in Subagents + +| Name | Model | Tools | Purpose | +| --------------- | ----- | ---------------- | --------------------------- | +| Explore | Haiku | Read, Grep, Glob | Fast, cheap codebase search | +| Plan | — | Read-only | Research and planning | +| general-purpose | — | All | Full capability | +| Bash | — | Terminal | Shell execution | + +## Effort Levels + +The `effort` frontmatter field controls how hard the model works: + +| Level | Behavior | Availability | +| -------- | ----------------------- | ------------- | +| `low` | Quick, shallow analysis | All models | +| `medium` | Standard depth | All models | +| `high` | Thorough analysis | All models | +| `max` | Deepest reasoning | Opus 4.6 only | + +## Session-Scoped Hooks for Subagents + +[[claude-code-hooks]] can be defined directly in subagent YAML frontmatter, scoped to the subagent's lifecycle: + +```yaml +--- +name: secure-writer +tools: Edit, Write +hooks: + PreToolUse: + - matcher: "Edit|Write" + hooks: + - type: command + command: "validate-paths.sh" + Stop: + - hooks: + - type: command + command: "run-tests.sh" +--- +``` + +All hook events are supported. Key behaviors: + +- `Stop` hooks defined in subagent frontmatter are **automatically converted** to `SubagentStop` events +- Hooks are cleaned up when the subagent finishes +- Project-level hooks for `SubagentStart`/`SubagentStop` also fire for custom subagents + +## Agent Teams (Experimental) + +Multiple Claude Code instances coordinating as a team. Enable with `CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1`. Requires Claude Code v2.1.32+. + +### Architecture + +| Component | Role | +| ------------- | ------------------------------------------------------------------------------- | +| **Team lead** | Creates team, breaks work into tasks, coordinates progress, synthesizes results | +| **Teammates** | Independent Claude Code instances, each with own context window | +| **Task list** | Shared state — teammates claim tasks, mark complete | +| **Mailbox** | Direct messaging between lead and teammates, or between teammates | + +### Task Workflow + +Tasks flow through states: **pending** → **in progress** → **completed**. Task dependencies are supported — a task won't become available until its dependencies are complete. File locking prevents race conditions when teammates access shared resources. + +### Communication + +- **Lead ↔ Teammate**: Direct messages via task list updates and mailbox +- **Teammate ↔ Teammate**: Broadcast messages for coordination +- Display modes: **in-process** (cycle with `Shift+Down`) or **split-panes** (tmux/iTerm2) + +### Team-Specific Hooks + +Two hook events exist specifically for agent teams: + +- `TeammateIdle` — fires when a teammate is about to go idle; block to keep it working +- `TaskCompleted` — fires when a task is marked complete; block to prevent completion + +### Sizing Guidelines + +| Guideline | Recommendation | +| ------------------ | ------------------------------------------------------------------------------- | +| Team size | **3–5 teammates** for most workflows | +| Tasks per teammate | **5–6 tasks** each keeps everyone productive | +| Scaling rule | Add teammates only when work genuinely benefits from parallelism | +| Task granularity | Self-contained units producing clear deliverables (function, test file, review) | + +Three focused teammates often outperform five scattered ones. Token costs scale linearly with teammate count, and coordination overhead increases with team size. + +### Team Use Cases + +- Parallel research from different angles +- Cross-layer coordination (frontend, backend, tests simultaneously) +- Model comparison on same task +- Large PR reviews split by concern area + +### Team Anti-Patterns + +- Sequential, tightly-coupled tasks (use single session) +- Same-file edits across teammates (causes overwrites) +- Simple tasks that don't justify coordination overhead (higher token cost) +- Running unattended too long (increases risk of wasted effort) + +### Current Limitations + +- No session resumption with in-process teammates (`/resume` and `/rewind` don't restore them) +- Task status can lag — teammates sometimes fail to mark tasks complete +- One team per session; no nested teams +- Lead is fixed for the session lifetime +- All teammates start with the lead's permission mode +- Split panes require tmux or iTerm2 (not supported in VS Code terminal, Windows Terminal, or Ghostty) + +## Key Constraint + +**Subagents cannot spawn other subagents** — this prevents infinite nesting. Use the `Agent(worker, researcher)` tool syntax to restrict which named subagents can be spawned from the parent context. + +**Plugin security**: Agents bundled in [[claude-code-plugins]] cannot use `hooks`, `mcpServers`, or `permissionMode` frontmatter fields — these are silently ignored when loading from a plugin context. To use these fields, copy the agent file to `.claude/agents/` or `~/.claude/agents/`. + +## Related pages + +- [[subagents]] +- [[claude-code-skills]] +- [[claude-code-plugins]] +- [[claude-code-hooks]] +- [[agent-workflows]] diff --git a/wiki/knowledge/claude-cookbook.md b/wiki/knowledge/claude-cookbook.md new file mode 100644 index 00000000..6b5353e9 --- /dev/null +++ b/wiki/knowledge/claude-cookbook.md @@ -0,0 +1,97 @@ +# Claude Cookbook + +**Summary**: The Anthropic Claude Cookbook is an official, community-driven collection of copy-paste code examples and guides covering core Claude capabilities, tool use, multimodal input, third-party integrations, and advanced agentic techniques including sub-agents and automated evaluations. +**Sources**: `docs/agentic-engineering/claude-cookbook-anthropic.md` +**Last updated**: 2026-04-21 + +--- + +## Overview + +The Claude Cookbook provides code and guides designed to help developers build with Claude. All examples are designed as copy-paste snippets adaptable to any project. While examples are primarily written in Python, the concepts can be adapted to any language that supports the Claude API (source: claude-cookbook-anthropic.md). + +**Prerequisites**: A Claude API key (sign up at anthropic.com). New to the Claude API? The [Claude API Fundamentals course](https://github.com/anthropics/courses/tree/master/anthropic_api_fundamentals) provides a solid foundation. + +The cookbook is hosted at: https://github.com/anthropics/anthropic-cookbook + +## Recipe Categories + +### Core Capabilities + +These examples cover foundational Claude abilities relevant to building any agentic system (source: claude-cookbook-anthropic.md): + +- **Classification** — Techniques for text and data classification using Claude. Relevant to [[agent-best-practices]] routing patterns. +- **Retrieval Augmented Generation (RAG)** — Enhancing Claude's responses with external knowledge sources. A key pattern for [[context-engineering]] when codebase knowledge exceeds the context window. +- **Summarization** — Techniques for effective text summarization. Directly applicable to [[rpi-workflow]] artifact compaction. + +### Tool Use and Integration + +- **Tool use** — Integrating Claude with external tools and functions to extend its capabilities (source: claude-cookbook-anthropic.md): + - Customer service agent patterns + - Calculator integration + - SQL query generation + +This maps to [[mcp-specification]] patterns and [[agent-workflows]] for structured tool invocation. + +### Third-Party Integrations + +- **RAG with vector databases** — Pinecone integration for semantic search +- **Wikipedia search** — Web knowledge retrieval +- **Web page reading** — Processing live web content +- **Embeddings with Voyage AI** — Creating vector embeddings + +(source: claude-cookbook-anthropic.md) + +### Multimodal Capabilities + +- **Vision** — Getting started with images, best practices for vision, interpreting charts and graphs, extracting content from forms +- **Image generation** — Using Claude with Stable Diffusion for image generation + +(source: claude-cookbook-anthropic.md) + +### Advanced Techniques + +These are the most relevant to [[agentic-engineering-workflow]] patterns (source: claude-cookbook-anthropic.md): + +- **[[subagents]]** — Using Haiku as a sub-agent in combination with Opus. Demonstrates the model-specialization pattern where cheaper/faster models handle mechanical subtasks while stronger models handle judgment-intensive phases — the same pattern used in [[rpir-workflow]] multi-agent scaling. +- **PDF upload and summarization** — Parsing and passing PDFs as text to Claude +- **Automated evaluations** — Using Claude to automate the prompt evaluation process (a form of Critic Agent, as described in [[agentic-software-modernization]]) +- **JSON mode** — Ensuring consistent JSON output from Claude +- **Content moderation** — Building a moderation filter +- **Prompt caching** — Efficient prompt caching techniques + +## Connection to Agentic Engineering Patterns + +The cookbook's advanced techniques section directly supports the agentic engineering patterns documented across this wiki: + +| Cookbook Recipe | Agentic Engineering Application | +|-----------------|----------------------------------| +| Sub-agents (Haiku + Opus) | [[rpi-workflow]] subagent isolation for research phase | +| RAG patterns | [[context-engineering]] for large codebases | +| Automated evaluations | Critic agent loop in [[agentic-software-modernization]] | +| Prompt caching | Cost reduction in multi-phase [[rpir-workflow]] | +| JSON mode | Structured plan artifacts in [[agent-harness]] workflows | + +## Additional Resources + +- [Anthropic developer documentation](https://docs.claude.com/claude/docs/guide-to-anthropics-prompt-engineering-resources) +- [Anthropic support docs](https://support.anthropic.com) +- [Anthropic Discord community](https://www.anthropic.com/discord) +- [Anthropic on AWS](https://github.com/aws-samples/anthropic-on-aws) — examples for Claude on AWS infrastructure + +(source: claude-cookbook-anthropic.md) + +## Contributing + +The Claude Cookbook thrives on community contributions. Review existing issues and pull requests before contributing. Share ideas for new examples on the [issues page](https://github.com/anthropics/anthropic-cookbook/issues) (source: claude-cookbook-anthropic.md). + +## Related pages + +- [[agent-workflows]] +- [[subagents]] +- [[context-engineering]] +- [[mcp-specification]] +- [[agent-best-practices]] +- [[rpi-workflow]] +- [[rpir-workflow]] +- [[agentic-software-modernization]] diff --git a/wiki/knowledge/code-to-contract.md b/wiki/knowledge/code-to-contract.md new file mode 100644 index 00000000..dbe13940 --- /dev/null +++ b/wiki/knowledge/code-to-contract.md @@ -0,0 +1,166 @@ +# Code to Contract + +**Summary**: Contract-based thinking in AI-assisted development — how specifications function as executable contracts between intent and implementation, and why treating specs as authoritative contracts (not advisory documents) is the shift that makes SDD work. +**Sources**: `docs/spec-driven-development/spec-driven-development-arxiv.md`, `docs/spec-driven-development/spec-driven-development-main.md`, `docs/spec-driven-development/spec-driven-development-variant.md` +**Last updated**: 2026-04-21 + +--- + +## The Traditional Problem: Specs as Advisory Documents + +For decades, code has been the de facto source of truth in software development (source: spec-driven-development-arxiv.md): +- Requirements documents exist but drift +- Design diagrams are drawn but rot +- Tests are written after the fact +- When a developer asks "what should this function do?" the answer is "read the code" + +Traditional design documents — HLD, LLD, SRS — are specifications in name but not in force. They drift from reality because **nothing enforces alignment between spec and code** (source: spec-driven-development-arxiv.md). + +This is the problem that spec-driven development solves: making the specification a **contract** that both humans and machines must honor, rather than a document that advisorily suggests what code might do. See [[spec-first-ai-development]] for the paradigm, and [[spec-driven-development]] for the full framework. + +--- + +## What Makes a Specification a Contract + +A specification becomes a contract when (source: spec-driven-development-arxiv.md): + +1. **It is executable** — BDD scenarios, API contract tests, or model simulations fail if code doesn't match +2. **It is enforced** — spec validation is embedded in CI/CD; drift causes immediate build failures +3. **It is authoritative** — when spec and code disagree, the spec wins; code must be fixed, not the spec ignored + +The shift is conceptual: from "the spec describes what we intended to build" to "the spec defines what the software must do — period." + +As Bryan Finster observed: "SDD is not a revolution… it's just BDD with branding." The branding reminds practitioners that specs should be authoritative, not advisory (source: spec-driven-development-arxiv.md). + +--- + +## Contracts in API Development + +API-first development is the most mature example of spec-as-contract in practice (source: spec-driven-development-arxiv.md): + +### OpenAPI (REST) + +Define REST APIs with complete endpoint specifications. The OpenAPI document is the contract: +- Generates server stubs that clients depend on +- Generates client SDKs +- Powers automated contract tests (via Specmatic, Pact) +- A build fails if the implementation deviates from the spec + +**Financial services case study**: An API-first microservices team using OpenAPI + Specmatic achieved a 75% reduction in integration cycle time. Mock servers generated from specs enabled frontend and backend parallel development; any deviation caused the build to fail, preventing drift (source: spec-driven-development-arxiv.md). + +### GraphQL SDL + +The GraphQL schema definition is a contract between frontend and backend. Type safety enforces it at the tooling level. + +### Protocol Buffers / gRPC + +Service interfaces defined in `.proto` files are strict contracts — automatic generation of strongly-typed client and server code makes deviation impossible. + +### Pact and Specmatic + +Contract testing tools that verify live implementations match their specs. These make the contract enforcement automated and continuous (source: spec-driven-development-arxiv.md). + +--- + +## Contracts in BDD + +Behavior-Driven Development frameworks are the direct ancestor of modern SDD and implement the contract model at the feature level (source: spec-driven-development-arxiv.md). + +Gherkin scenarios are executable contracts in natural language: + +```gherkin +Feature: Shopping Cart + Scenario: Adding item to empty cart + Given the cart is empty + When I add item "Widget" to the cart + Then the cart should contain 1 item + And the item should be "Widget" +``` + +A feature is "done" only when all scenarios pass. The scenario is not documentation of what was built — it is the contract that determines whether the feature exists at all (source: spec-driven-development-arxiv.md). + +**Enterprise BDD case study**: Product managers wrote Gherkin scenarios; a feature was "done" only when all scenarios passed; reduced requirement ambiguity (source: spec-driven-development-arxiv.md). + +--- + +## Contracts in Embedded and Safety-Critical Systems + +The spec-as-source model is already standard in safety-critical domains (source: spec-driven-development-arxiv.md): + +**Automotive (Simulink / ISO 26262)**: +- Engineers model control algorithms in Simulink +- Behavior is verified at the model level (the contract) +- Certified C code is auto-generated — nobody hand-edits it +- ISO 26262 certification applies to the model, not the generated code + +This is the fullest expression of code-to-contract thinking: **code is not the artifact, the contract is**. Code is a derivative that must be regenerated from the contract to remain valid. + +--- + +## Contracts as AI Inputs + +When AI coding assistants receive a specification as their primary input, the spec functions as an **unambiguous, executable contract** that constrains the model's output space (source: spec-driven-development-arxiv.md). Controlled studies show error reductions of up to 50% when human-refined specs are used as AI inputs. + +The contrast is stark (source: spec-driven-development-main.md): + +**No contract (vibe coding):** +> "Build a rate limiter middleware for Express." +→ Model fills thousands of unstated decisions from training data + +**With contract (spec-first):** +> "Implement the rate limiter defined in `.spec/features/rate-limiter.md`, which specifies a sliding window algorithm, 100 requests per minute per API key, 429 responses with Retry-After headers, and Redis-backed state for horizontal scaling." +→ Model generates code aligned with your actual requirements + +The contract also enables [[agent-workflows]] patterns like parallel agent execution: with specs partitioned at the contract level, multiple agents can implement different components simultaneously without interfering with each other (source: spec-driven-development-arxiv.md). + +--- + +## The Three Contract Levels + +Contracts vary in their enforcement strength (source: spec-driven-development-arxiv.md): + +| Level | Enforcement | Example | +|-------|-------------|---------| +| **Spec-first** | Human review only | Markdown spec reviewed before implementation | +| **Spec-anchored** | Automated tests in CI/CD | BDD scenarios, OpenAPI + Specmatic | +| **Spec-as-source** | Code regenerated from contract | Simulink, Tessl, OpenAPI server stubs | + +Each higher level eliminates a category of drift: spec-anchored eliminates undetected drift; spec-as-source eliminates drift by construction. + +--- + +## What SDD Actually Adds Over Traditional Design Documents + +Three concrete additions that transform specs from documentation into contracts (source: spec-driven-development-arxiv.md): + +1. **Executable specifications** — BDD scenarios, API contract tests, or model simulations fail if code doesn't match; the spec is enforced, not merely suggested +2. **CI/CD integration** — spec validation embedded in continuous integration catches drift immediately; breaking the contract breaks the build +3. **AI consumption** — SDD specs are structured so AI coding assistants can consume them, generating code from contracts rather than guessing from vague prompts + +--- + +## Contract Enforcement Pitfalls + +Even with executable contracts, several failure modes persist (source: spec-driven-development-arxiv.md): + +- **False confidence**: a passing contract test guarantees the software matches the spec, not that the spec itself is correct — if the contract is wrong, the implementation will be wrong too +- **Over-specification**: contracts that read like pseudo-code constrain implementation unnecessarily and become brittle +- **Spec rot / [[context-rot]]**: contracts drift from reality when not updated; automated enforcement is the only reliable prevention + +And from hands-on tool evaluation (source: spec-driven-development-variant.md): +- **AI non-determinism**: even with a detailed contract, agents sometimes ignore directives or over-interpret them; the contract improves consistency but does not eliminate non-determinism +- **Separation confusion**: keeping functional contracts (what the software does) cleanly separated from technical constraints (how it does it) is harder in practice than in theory + +--- + +## Related pages + +- [[spec-driven-development]] +- [[spec-first-ai-development]] +- [[spec-driven-development-practice]] +- [[spec-driven-development-critique]] +- [[context-engineering]] +- [[agent-workflows]] +- [[agentic-engineering-workflow]] +- [[agent-best-practices]] +- [[context-rot]] diff --git a/wiki/knowledge/commercial-agent-context.md b/wiki/knowledge/commercial-agent-context.md new file mode 100644 index 00000000..d7352440 --- /dev/null +++ b/wiki/knowledge/commercial-agent-context.md @@ -0,0 +1,137 @@ +# Commercial Agent Context Engineering + +**Summary**: Context engineering as infrastructure for multi-tenant, production agent systems — memory taxonomy, truth vs. acceleration layers, the full context engine loop, and why architectural solutions matter more than prompt engineering at scale. +**Sources**: context-engineering-commercial-agents-jeremy-daly.md +**Last updated**: 2026-04-21 + +--- + +Building agents for a single user on a laptop: almost anything works. Building commercial multi-tenant agents serving enterprises: almost nothing accidental survives. The difference is treating context as infrastructure — not an implementation detail. See [[context-engineering]] for the foundational concepts. + +## Context Is Infrastructure + +Models improve. APIs standardize. Tool use matures. But in commercial multi-tenant systems, these are not the determining factors. **Context is** (source: context-engineering-commercial-agents-jeremy-daly.md). + +Everyone has access to the same frontier models. Claude. GPT. Gemini. APIs are public, prices are falling, capabilities converging. What differentiates systems is no longer the model — **it's the context.** + +### The Three Non-Negotiables + +1. **Structural Isolation** — Tenant boundaries enforced at the infrastructure layer, not the model layer +2. **Deterministic Replay** — Ability to reconstruct exactly what the agent knew at decision time +3. **Economic Predictability** — Ability to predict and bound cost per run + +If your system cannot enforce tenant boundaries structurally, it will eventually leak. If it cannot run deterministic replays, you cannot debug or evolve it safely. If it cannot predict cost per run, it cannot scale sustainably (source: context-engineering-commercial-agents-jeremy-daly.md). + +### The Failure Mode + +Systems that optimize for *demo velocity* — indexing raw transcripts, reusing context wholesale, blurring truth and acceleration — discover cost, drift, and cross-tenant risk too late. Getting context wrong doesn't just produce worse answers. **You get silent corruption** (source: context-engineering-commercial-agents-jeremy-daly.md). + +## Memory as a Scoped, Typed System + +A robust commercial agent system classifies memory along two dimensions (source: context-engineering-commercial-agents-jeremy-daly.md): + +1. **Scope**: who can see it (a security boundary) +2. **Type**: what kind of memory it is (semantic role) + +### Memory Scopes (Security Boundaries) + +These are structural isolation layers enforced at the storage and routing layers — **never delegated to the model:** + +| Scope | Contents | Mutability | +|-------|----------|------------| +| **Global** | Safety rules, tool contracts, product ontology, agent "constitution" | Immutable at runtime; deployment-controlled; never writable by agents | +| **Tenant** | Organization policies, knowledge bases, playbooks, connector configurations | Shared within tenant; policy-gated promotion | +| **User** | Preferences, working style, personal notes, user-specific entitlements | Visible only to user; promotion-gated; TTL-based | +| **Session** | Tool outputs, intermediate plans, scratch buffers, temporary retrieval results | Short-lived; aggressive garbage collection; not durable unless promoted | + +### Memory Types (Semantic Role) + +- **Policy memory** — Normative rules and constraints. Global or tenant-scoped. Versioned and tightly controlled +- **Preference memory** — Stable personalization parameters. Usually user-scoped +- **Fact memory** — Durable assertions the agent may reuse. Must include provenance +- **Episodic memory** — Structured summaries of completed work. "Case resolved." "Migration completed." Reusable artifacts extracted from traces +- **Trace memory** — Raw, append-only execution events. Your flight recorder + +> Memory without scope is exposure. Memory without type is entropy. (source: context-engineering-commercial-agents-jeremy-daly.md) + +### Memory Layer Summary + +| Layer | Scope | Typical Contents | Retention | Write Policy | +|-------|-------|-----------------|-----------|-------------| +| Constitution | Global | Safety rules, tool contracts, ontologies | Versioned | Write-locked | +| Org memory | Tenant | Playbooks, knowledge base, connectors | Policy-based | Gated promotion | +| Personal memory | User | Preferences, working style, drafts | TTL-based + user controls | Gated promotion | +| Runtime state | Session | Tool outputs, scratch space, plans | Hours–days | Auto GC | +| Episodes | User/Tenant | "Case resolved," derived summaries | Months+ | Explicit promotion | +| Traces | Tenant | Events, retrievals, tool calls, approvals | Policy-based (often long-lived) | Append-only | + +## Truth vs. Acceleration + +The core distinction to preserve: **separate truth from acceleration** (source: context-engineering-commercial-agents-jeremy-daly.md). + +### Canonical Stores (Truth) + +**Canonical Event Log (Append-Only)** — Your flight recorder. Every agent run emits events: context retrieved, policies evaluated, tool calls made, approvals routed, outputs generated, memory promoted. Answers: What did the agent know at decision time? Which policy version applied? Why was this exception granted? + +**Canonical Structured Memory Store** — Durable memory state: facts, preferences, episodic summaries, approved overrides, tenant-level knowledge. Every record must include: scope, class, provenance, retention policy, and sensitivity classification. + +> If your vector store becomes your truth layer, you will eventually lose structural integrity. + +### Derived Stores (Projection) + +- **Retrieval Index (Vector/Hybrid Search)** — Your serving layer for recall. Must be rebuildable from canonical sources +- **Object Store** — Large payloads, attachments, extraction outputs. Content-addressed + +**Hybrid search** (lexical + semantic) provides stronger precision and filtering than vector-only retrieval. The index must NOT be built directly from raw transcripts — they are noisy, redundant, and context-fragmented (source: context-engineering-commercial-agents-jeremy-daly.md). + +## The Context Engine Loop + +Every agent run should follow a predictable sequence — skip steps and you get drift (source: context-engineering-commercial-agents-jeremy-daly.md): + +1. **Ingest** — Establish identity, scope, constraints, privacy mode. You DO NOT ask the model to filter data. You filter in the data plane +2. **Plan context needs** — Before retrieving anything, the agent plans what kind of context it needs. Prevents: "Retrieve everything and let the model figure it out" +3. **Retrieve** (isolation enforced here) — Filtering before ranking, not after. Every query must include: Tenant ID, scope visibility constraint, expiration checks, sensitivity boundaries +4. **Assemble working set** — Layered by priority: constitution → tenant policies → user preferences → retrieved facts/episodes → session state. Without layering and budgets, context windows become dumping grounds +5. **Semantic stabilization** — Before shrinking context, stabilize meaning: collapse verbose tool traces into structured summaries, extract typed episodic artifacts, convert dialogue into structured facts, normalize references, mark low-confidence artifacts, attach provenance metadata +6. **Agentic garbage collection** — After meaning is stabilized, optimize: enforce token budgets by layer, deduplicate redundant artifacts, drop stale session state, remove low-confidence provisional memory. **Agentic GC is not optimization — it is drift control** +7. **Infer and act** — Model + tools + policy enforcement + optional human approval +8. **Promotion gate** — Decide what becomes durable memory +9. **Emit trace envelope** — Record retrievals, actions, policies, versions, and cost surfaces +10. **Lifecycle garbage collection** — Expire session buffers, enforce retention, invalidate derived projections + +## Key Architectural Principles + +From field experience building multi-tenant production systems (source: context-engineering-commercial-agents-jeremy-daly.md): + +- **Context boundaries are infrastructure, not application logic** +- **Isolation** is structural, not prompt-based +- **Replay** is the baseline for trust +- **Economics** must be predictable per run +- **Truth is rebuildable, acceleration is disposable** +- **What you exclude from context is becoming as important as what you include** + +## Ecosystem Convergence + +Production systems — Claude Code, Cursor, Letta, AWS AgentCore — all converge on the same architectural patterns under production pressure (source: context-engineering-commercial-agents-jeremy-daly.md): + +- Typed, scoped memory layers +- Separation of event logs from structured state +- Explicit promotion gates +- Context budgets and priority ordering +- Aggressive pruning between turns +- Cryptographic isolation at projection layers + +The same pressures produce the same load-bearing patterns. + +## Relationship to Single-User Context Engineering + +The principles of [[context-engineering]] — signal-to-noise, position effects, [[progressive-disclosure]], compaction — all apply in commercial systems, but with additional dimensions: tenant isolation as a security property, replay as an audit requirement, and cost control as a contractual obligation rather than a preference. Multi-tenant failure modes include silent cross-tenant data leakage, not just degraded reasoning. See [[context-rot]] for the reasoning degradation mechanisms that apply across both contexts. + +## Related pages + +- [[context-engineering]] +- [[context-rot]] +- [[progressive-disclosure]] +- [[agent-best-practices]] +- [[agent-workflows]] +- [[prompt-engineering]] diff --git a/wiki/knowledge/context-engineering.md b/wiki/knowledge/context-engineering.md new file mode 100644 index 00000000..f25a2888 --- /dev/null +++ b/wiki/knowledge/context-engineering.md @@ -0,0 +1,149 @@ +# Context Engineering + +**Summary**: The discipline of managing all context fed to an LLM — instructions, examples, retrieved data, tool outputs — to maximize signal density per token and minimize degradation from noise, staleness, and position effects. +**Sources**: research-context-engineering-comprehensive.md, research-context-rot-and-management.md, a-guide-to-agents.md, research-whitespace-and-formatting.md, context-engineering-most-important-skill-dev.md, advanced-context-engineering-coding-agents-dev.md, context-engineering-commercial-agents-jeremy-daly.md +**Last updated**: 2026-04-21 + +--- + +Context engineering has replaced "prompt engineering" as the dominant framing for working with LLMs. Where prompt engineering focuses on crafting a single input, context engineering manages the entire context window across turns, tools, and sessions. + +## Core Principle + +Every token must earn its place. The test: "Would removing this cause the agent to make mistakes?" If not, cut it. Irrelevant content actively harms reasoning — simple math accuracy drops from **0.92 → 0.68** with just 3,000 filler tokens (source: Levy et al., ACL 2024). + +## The Token Budget + +- Frontier LLMs follow ~**150–200 instructions** with consistency; smaller models handle fewer (source: Humanlayer research) +- Adding instruction N interacts with all existing instructions via **n² pairwise attention relationships** — context growth is superlinear in cognitive load +- Configuration files (CLAUDE.md, AGENTS.md) load on **every request** — bloat directly wastes budget +- Target under 200 lines per config file (~2,000–4,000 tokens) +- Comprehensive prompt compression achieves 4–20× reduction; naive minification destroys the structure compression preserves +- Practical threshold: reasoning quality degrades around **3,000 filler tokens** (Levy et al., ACL 2024) + +## Position Effects (Lost in the Middle) + +Performance follows a U-shaped curve across the context window: + +| Position | Recall Quality | Recommendation | +| -------- | ----------------------------------- | ------------------------------------- | +| Start | High (primacy effect) | Critical instructions, system prompts | +| Middle | 10–20% drop (Liu et al., TACL 2023) | Reference data, documents | +| End | High (recency effect) | Queries, final instructions | + +Place long documents at the top and queries at the bottom — this exploits the primacy/recency effect and improves performance by ~30%. + +### Deep Dive: The U-Shaped Performance Curve + +Liu et al. (2024, TACL) formally established the [[u-shaped-attention-curve]] in a systematic study across multi-document QA (NaturalQuestions-Open, 20 documents) and synthetic key-value retrieval. Key quantitative findings (source: lost-in-the-middle-acl.md): + +- Performance at the **center** (position 10 of 20) drops **over 20 percentage points** vs. the edges. +- The curve holds across every tested model — GPT-3.5-Turbo, Claude, Llama-2, MPT — with **no model immune**. +- Even models explicitly marketed as "long-context" retain the U-shaped bias. +- The effect is not dataset-specific: the same curve appears on synthetic key-value retrieval data. + +For the full paper treatment, see [[lost-in-the-middle-paper]]. + +### The In-Between Compound Effect (Multi-Hop) + +Baker et al. (2024, arXiv 2412.10079) extended the finding to multi-hop QA and discovered a second degradation axis: the *relative distance between* multiple evidence documents (source: lost-in-the-middle-and-in-between-arxiv.md): + +- When two required evidence documents are **adjacent**, performance is consistently higher than when they are separated by distractor documents — **even when both are at the same absolute position**. +- This creates a 2D degradation surface: distance-from-edge × distance-between-evidence. +- The combinatorial explosion of position permutations (190 for 2-hop, 4,845 for 4-hop) makes re-ranking-based mitigations impractical for multi-hop settings. +- Chain-of-thought prompting improves document *identification* but does not resolve positional bias. + +This "in-between" effect matters for [[agent-workflows]]: agentic multi-hop reasoning over retrieved documents faces both axes of degradation simultaneously. For the full paper treatment, see [[lost-in-the-middle-in-between]]. + +### Mitigation Strategies + +The most effective mitigations, in order of practical utility (source: lost-in-the-middle-acl.md, lost-in-the-middle-and-in-between-arxiv.md): + +1. **Position-aware placement** — Always place the most critical document first and the query last. Free; no additional compute; 20–30% improvement from ordering alone. +2. **Query-Aware Contextualization** — Place the query both before and after the document list. Highly effective for key-value retrieval; limited for complex QA. +3. **[[progressive-disclosure]]** — Load context in tiers; prevent large "middles" from forming architecturally. +4. **Document re-ranking** — Effective for single-hop; combinatorially intractable for multi-hop. +5. **Permutation self-consistency** — N× inference cost; intractable at 3+ hops. + +For the full mitigation landscape, see [[long-context-mitigation]]. + +## Four Strategies for Context Management + +1. **[[progressive-disclosure]]** — Load instructions conditionally (skills, path-scoped rules, subagent summaries) instead of dumping everything into the main prompt +2. **Compaction** — Summarize accumulated context when nearing the window limit. The lightest-touch approach is **tool result clearing** (replacing verbose tool outputs with summaries). For conversations, ask the model to summarize. The simplest approach matches sophisticated alternatives. +3. **Structured note-taking** — Agent writes notes persisted outside the context window (JSON state files, progress.txt, git commits). The agent can reload specific notes on demand rather than carrying all history in context. This is [[prompt-engineering]]'s state tracking applied to context management. +4. **[[subagents]]** — Route complex research to isolated contexts; return only summaries (1,000–2,000 tokens) instead of tens of thousands of exploration tokens. JIT documentation is a variant: let the agent generate docs during planning, then use those docs (not the raw exploration) in execution. + +### Choosing a Strategy + +| Symptom | Strategy | +| ----------------------------------------- | ---------------------- | +| Context grows large across many turns | Compaction | +| Agent forgets earlier decisions | Structured note-taking | +| Single task needs deep exploration | Subagents | +| Instructions compete with data for tokens | Progressive disclosure | + +## The 'Ball of Mud' Anti-Pattern + +A common failure mode (documented in Anthropic's agent guide): the agent misbehaves → you add a rule → it misbehaves differently → you add another rule → the config becomes an unmaintainable mess. Auto-generated files compound this problem because they start large and rules only accumulate. + +The fix is the **deletion test**: regularly review every instruction and ask "Would removing this cause the agent to make mistakes?" If removing it has no effect, it was noise. This connects directly to the [[evaluating-agents-paper]]'s finding that LLM-generated context files hurt performance. + +## Formatting: Structure Over Minimalism + +Whitespace is nearly free. A blank line costs 1 token (same as 4× blank lines). Structural formatting measurably improves output: + +| Format | Token Cost vs. Plain Text | Value | +| ---------------- | ------------------------- | ------------------------- | +| Markdown headers | +81% | Clear section boundaries | +| XML tags | +165% | Unambiguous delimiters | +| YAML vs. JSON | YAML saves ~30% | Lower overhead for config | + +Well-formatted 1,000-token prompt beats a wall-of-text 900-token prompt. Cut content, not formatting. + +## The Five Pillars (Production Systems) + +From production experience building real AI applications, context engineering has five operational pillars (source: context-engineering-most-important-skill-dev.md): + +1. **Context Selection** — Every token must earn its place. Semantic retrieval with relevance filtering beats dumping everything in. Dynamic tool loading (4 tools vs. 50 reduces ~7,400 tokens of definitions) applies the same principle +2. **Context Structuring** — Same information structured differently produces dramatically different results. Exploit primacy and recency effects: critical instructions at start and end, less critical reference data in the middle +3. **Memory Architecture** — Three-tier model: working memory (context window), short-term memory (session store, 10–50 compressed entries), long-term memory (persistent store, effectively unlimited). See [[commercial-agent-context]] for the full taxonomy in multi-tenant systems +4. **Context Compression** — Summarization, semantic chunking, progressive summarization, key-value extraction. Makes more fit without losing signal +5. **Context Validation** — Validate token counts before sending, log what was included/excluded, monitor for context-related failures, A/B test context strategies + +## The Research–Plan–Implement Workflow + +A systematic approach to compaction at every stage (source: advanced-context-engineering-coding-agents-dev.md): + +| Phase | Goal | Output | +|-------|------|--------| +| **Research** | Understand how the system *actually* works; identify authoritative files | Short research artifact; validated findings | +| **Plan** | List exact steps with concrete files/snippets and validation after each change | Solid plan that constrains agent behavior | +| **Implement** | Mechanical execution | Small context, high reliability | + +Key insight: **Planning is the highest-leverage activity.** A solid plan dramatically constrains agent behavior. Bad plans produce dozens of bad lines. Bad research produces hundreds (source: advanced-context-engineering-coding-agents-dev.md). + +This workflow works because it doesn't try to fit everything into one window — each phase gets a fresh, appropriately-scoped context. + +## Context as Infrastructure (Commercial Systems) + +In commercial multi-tenant agent systems, context engineering becomes infrastructure with three non-negotiables: structural tenant isolation, deterministic replay, and economic predictability per run (source: context-engineering-commercial-agents-jeremy-daly.md). See [[commercial-agent-context]] for the full framework. + +The system prompt as a "desk" analogy: every component (system instructions, retrieved knowledge, tool definitions, conversation history, current query) competes for the same finite space. Add too much conversation history and you crowd out retrieved knowledge. Load too many tool definitions and you leave no room for examples (source: context-engineering-most-important-skill-dev.md). + +## Related pages + +- [[context-rot]] +- [[progressive-disclosure]] +- [[prompt-engineering]] +- [[whitespace-and-formatting]] +- [[multilingual-performance]] +- [[million-token-context-window]] +- [[dead-context]] +- [[agents-md-liability]] +- [[commercial-agent-context]] +- [[lost-in-the-middle-paper]] +- [[lost-in-the-middle-in-between]] +- [[u-shaped-attention-curve]] +- [[long-context-mitigation]] +- [[agent-workflows]] diff --git a/wiki/knowledge/context-rot.md b/wiki/knowledge/context-rot.md new file mode 100644 index 00000000..18ac1a08 --- /dev/null +++ b/wiki/knowledge/context-rot.md @@ -0,0 +1,114 @@ +# Context Rot + +**Summary**: The empirically validated phenomenon where LLM performance degrades as tokens accumulate in the context window, caused by attention scaling, training distribution mismatch, and position interpolation effects. +**Sources**: research-context-rot-and-management.md, research-context-engineering-comprehensive.md, agents-md-is-a-liability-paddo.md, pi-context-zone-github.md +**Last updated**: 2026-04-21 + +--- + +Context rot is not theoretical — it is measured across all 18 tested LLMs (Chroma Research, 2025). Even trivial tasks (word replication, basic retrieval) degrade as context grows. + +## Three Architectural Causes + +1. **n² scaling of self-attention** — Every new token attends to every previous token; compute grows quadratically +2. **Training distribution mismatch** — Models trained primarily on shorter sequences; long contexts are out-of-distribution +3. **Position interpolation degradation** — Extended context techniques (RoPE scaling, ALiBi) introduce approximation errors at extreme positions + +## Key Measurements + +| Metric | Value | Source | +| ---------------------------- | ------------------ | ------------------------- | +| Accuracy at 3K filler tokens | 0.92 → 0.68 | Levy et al., ACL 2024 | +| Middle-position recall loss | 10–20% | Liu et al., TACL 2023 | +| All 18 models degrade | Consistent pattern | Chroma Research, 2025 | +| Critical threshold | ~3,000 tokens | Four failure modes emerge | + +## Four Failure Modes at Threshold + +When context exceeds ~3,000 tokens of accumulated noise: + +1. **Refusal** — Model declines to answer, citing insufficient information +2. **Label bias** — Model defaults to most common training label +3. **CoT breakdown** — Chain-of-thought reasoning produces fluent but illogical steps +4. **Coverage loss** — Model addresses only a subset of provided information + +## Context Poisoning + +- **Failed approach accumulation**: After 2 failed corrections, better to `/clear` and restart than continue +- **Contradictory instructions**: Model picks one rule arbitrarily — no predictable resolution +- **Stale documentation**: Worse than no documentation; changing file paths invalidate documented locations +- **Counter-intuitive finding**: Shuffled haystacks *improve* performance vs. coherent text (disrupted narrative forces token-level attention) + +## Management Strategies + +| Strategy | Mechanism | When to Use | +| -------------------------- | ------------------------------- | -------------------------------------- | +| Compaction (summarization) | Ask Claude to summarize context | Multi-turn sessions, ~95% capacity | +| External memory | JSON state files, git history | Long-running agents, cross-session | +| [[subagents]] | Isolated context windows | Complex research, parallel exploration | +| [[progressive-disclosure]] | Load-on-demand via skills/rules | Configuration, instructions | +| Clear and restart | `/clear` command | After 2+ failed corrections | + +## Compression Research + +- LLMLingua: up to **20× compression** with minimal loss +- LongLLMLingua: **+21.4% performance** with ~4× fewer tokens on NaturalQuestions; **94% cost reduction** on LooGLE +- Contextual Retrieval: **35%** failure reduction with contextual embeddings; **67%** with reranking + +## The 500-Instruction Ceiling + +The IFScale benchmark (Distyl AI, NeurIPS 2025) tested 20 frontier models against instruction-following at scale: the best frontier model scored 68% at 500 instructions — **one in three instructions simply got skipped** (source: agents-md-is-a-liability-paddo.md). + +Key finding: as instruction density increases, errors shift from *modification* (doing it wrong) to *omission* (not doing it at all). The model doesn't misinterpret the 400th rule — it doesn't see it. Three degradation patterns emerge by model type: + +| Pattern | Models | Behavior | +|---------|--------|----------| +| Threshold decay | o3, Gemini 2.5 Pro | Near-perfect until 100–250 instructions, then cliff | +| Linear decay | GPT-4.1, Claude Sonnet 4 | Steady decline from start | +| Exponential decay | GPT-4o, LLaMA-4-Scout | Rapid collapse | + +Every model shows **primacy bias**: earlier instructions receive more attention. See [[agents-md-liability]] for the full analysis and actionable principles. + +## The Smart / Warm / Dumb Zone Framework + +The dumb zone — coined by Dex Horthy after analyzing 100,000+ developer sessions — is an operational framework for understanding context rot's impact on reasoning quality (source: pi-context-zone-github.md): + +| Zone | Context Used | What Happens | +|------|-------------|--------------| +| 🧠 Smart | 0–40% | Peak reasoning. Follows instructions, catches edge cases, accurate tool selection | +| ⚠️ Warm | 40–70% | Degrading. F1 scores drop ~45%. Instruction drift, shallow pattern matching, relies on pre-training over actual context | +| 🧟 Dumb | 70%+ | Broken. Hallucination rates spike to 40%. Infinite debug loops. Confidently wrong. Auto-compaction triggers but is lossy | + +### Model-Specific Smart Zone Boundaries (MRCR v2, 8-needle, March 2026) + +| Model | Window | MRCR @ 128K | MRCR @ 256K | MRCR @ 1M | Smart Zone | +|-------|--------|-------------|-------------|-----------|-----------| +| Claude Opus 4.6 | 1M | ~94% | 93% | **78%** | ~70% (most resilient) | +| Claude Sonnet 4.6 | 1M | — | — | 65% | ~50–60% | +| GPT-5.4 | 1M | 86% | 79% | 37% | ~30–40% | +| Gemini 3.1 Pro | 2M | 85% | ~50% | 26% | ~25–30% | +| DeepSeek V3 | 128K | **95%** | N/A | N/A | Near 100% within window | + +Claude Opus 4.6's Context Compaction architecture resists context rot better than any other model — 78% at 1M where GPT-5.4 drops to 37% and Gemini to 26%. But even Opus degrades. The 40% rule is conservative and works as a universal default (source: pi-context-zone-github.md). See [[million-token-context-window]] for extended analysis. + +### What Causes Context Rot (Expanded) + +Four compounding mechanisms beyond the three architectural causes (source: pi-context-zone-github.md): + +1. **Attention dilution** — Transformer attention is a fixed budget. More tokens = less focus per token +2. **Lost in the middle** — Models remember the beginning and end of context but forget the middle (U-shaped curve) +3. **Trajectory poison** — Conversation history full of the model's mistakes and your corrections. The model learns to predict more mistakes +4. **KV cache compression** — At high utilization, models compress older context, losing the "why" behind decisions + +## Attention Sinks + +Xiao et al. (ICLR 2024): initial tokens receive disproportionate attention regardless of their semantic content. Replace your first tokens with newline characters and the effect persists — it's positional, not semantic. This is why instructions at the top of AGENTS.md get followed more reliably: they're sitting on the attention sink (source: agents-md-is-a-liability-paddo.md). + +## Related pages + +- [[context-engineering]] +- [[progressive-disclosure]] +- [[prompt-engineering]] +- [[agents-md-liability]] +- [[million-token-context-window]] +- [[dead-context]] diff --git a/wiki/knowledge/context-scarcity-end.md b/wiki/knowledge/context-scarcity-end.md new file mode 100644 index 00000000..343d50d3 --- /dev/null +++ b/wiki/knowledge/context-scarcity-end.md @@ -0,0 +1,82 @@ +# Context Scarcity End + +**Summary**: Analysis of what happens when context stops being scarce — how flat-rate 1M context pricing changes workflows, removes economic pressure for discipline, and shifts the bottleneck from capacity to relevance. +**Sources**: context-stops-being-scarce-paddo.md, million-token-context-window-syntackle.md +**Last updated**: 2026-04-21 + +--- + +For three years, context scarcity shaped how developers worked with AI coding agents: be concise, front-load, accept that the agent will forget, and re-explain regularly. These were workarounds for a constraint, not good practices. When that constraint lifts, workflows can genuinely change — but not in the way the marketing suggests. See [[million-token-context-window]] for the technical limits that remain. + +## The Constraint That Shaped Behavior + +Context windows went from 4K (GPT-3.5) to 1M across multiple providers in three years. Before flat-rate 1M pricing, the economics discouraged filling the window. Sonnet 4.6 had 1M in public beta with a 2× input price multiplier above 200K tokens. Opus had no 1M access at all (source: context-stops-being-scarce-paddo.md). + +**When a capability stops costing extra, it stops being a feature and starts being assumed. That's when workflows actually change.** + +## What the Removal of the Tax Changes + +The pricing shift for Claude Opus/Sonnet 4.6 to flat rates across 1M removes the premium: + +- No `anthropic-beta: long-context-2025-01-01` header required +- Same per-token rate whether you send 9K or 900K tokens +- Media limits bumped 6× (up to 600 images or PDF pages per request) +- Max, Team, and Enterprise Claude Code users get 1M by default (source: context-stops-being-scarce-paddo.md) + +The competitive landscape now shows everyone at 1M capacity; **the differentiator is pricing structure**, not the number (source: context-stops-being-scarce-paddo.md). + +## The Compaction Problem and How Context Abundance Helps + +Claude Code compacts conversation history when context fills — a lossy compression that keeps the gist but loses specifics: exact error messages, architectural decisions from hours ago, the nuance of why you chose approach A over B (source: context-stops-being-scarce-paddo.md). + +**Compaction degrades non-linearly.** One compaction is tolerable. Three in a session means the agent works from a summary of a summary of a summary. + +At 200K effective context, compaction triggered regularly during long coding sessions. At 1M, the math shifts dramatically: +- Claude Code usable window before first compaction: ~134K at 200K → ~802K at 1M +- **That's not 5× more context — it's 5× longer before the first lossy compression** (source: context-stops-being-scarce-paddo.md) + +Jon Bell (Anthropic CPO) reported a 15% decrease in compaction events. The real impact: preserving fidelity of everything that came before the first compaction, enabling longer autonomous work stretches without human correction. + +## How Workflows Shift + +With abundant context, the specific habits that scarcity forced can be relaxed (source: context-stops-being-scarce-paddo.md): + +- **Longer uninterrupted sessions** — An agent that remembers your morning's architectural discussion when you implement something in the afternoon. No re-explaining +- **Bigger codebases in context** — At 1M tokens, ~15,000 lines of code with room for conversation; the constraint shifts from "what fits" to "what's relevant" +- **Fewer manual interventions** — Each compaction event is a moment where the human must correct the agent's compressed understanding; fewer compactions means fewer correction opportunities needed + +## The Discipline Paradox + +The tension is explicit in Anthropic's own guidance: they give you 1M tokens, but recommend finding **"the smallest set of high-signal tokens that maximize the likelihood of your desired outcome."** More isn't better. Less, but right, is better (source: context-stops-being-scarce-paddo.md). + +The benchmark data confirms why: +- Opus 4.6 scores 78.3% on MRCR v2 at 1M — best in industry +- That means **1 in 4 multi-needle retrievals fail** at that scale +- At 256K, Opus scores 92–93% + +The degradation curve is real even for the best model. See [[context-rot]] for the underlying mechanisms. + +### The Bigger-Window Trap + +A bigger context window is analogous to more RAM on a machine with a memory leak — it delays symptoms without fixing the cause. And it removes pressure to be disciplined. With 200K you're forced to be thoughtful; with 1M you can be sloppy and it *appears* to work, until failures become harder to diagnose (source: context-stops-being-scarce-paddo.md). + +Research finding from 2025 EMNLP: context length *alone* hurts performance even when extra context is relevant. **More capacity does not mean better reasoning. Discipline is a better bet than hope.** + +## The Trend Line and What Comes Next + +Context windows: 4K (GPT-3.5) → 1M (multiple providers) in three years. Growth has slowed — hardware constraints, diminishing returns on raw window size, and the shift toward better retrieval within existing windows (source: context-stops-being-scarce-paddo.md). + +**The next unlock probably isn't 10M tokens. It's better attention over the tokens you already have.** + +Anthropic's move to flat pricing signals they've solved the economics of long context at current scale. That shifts the engineering challenge from "fitting things in" to knowing what belongs at all. See [[context-engineering]] for the framework to address that challenge, and [[progressive-disclosure]] for tiered loading strategies. + +> Context engineering > context stuffing (source: context-stops-being-scarce-paddo.md) + +## Related pages + +- [[million-token-context-window]] +- [[context-engineering]] +- [[context-rot]] +- [[progressive-disclosure]] +- [[dead-context]] +- [[agents-md-liability]] diff --git a/wiki/knowledge/dead-context.md b/wiki/knowledge/dead-context.md new file mode 100644 index 00000000..f8aaafd8 --- /dev/null +++ b/wiki/knowledge/dead-context.md @@ -0,0 +1,107 @@ +# Dead Context + +**Summary**: Instructions, tool definitions, and metadata occupying the context window that aren't contributing to the current task — how dead context accumulates, what it costs, and strategies to shed it. +**Sources**: shedding-dead-context-ryan-spletzer.md, agents-md-is-a-liability-paddo.md +**Last updated**: 2026-04-21 + +--- + +Dead context is the accumulated weight of everything you've added to make your AI coding setup "better" — plugins installed, skills enabled, CLAUDE.md lines written — that consumes context window space on every session without contributing to the task at hand. It is the output of good intentions accumulating without discipline. See [[agents-md-liability]] for the research on why this degrades performance. + +## How Dead Context Accumulates + +Every plugin, every skill, every line in your `CLAUDE.md` occupies space in the context window. Even "lazy-loaded" tools that aren't fully active still carry a footprint — the model needs to keep a manifest of what's available, their descriptions, their trigger conditions, their tool schemas. None of that is free (source: shedding-dead-context-ryan-spletzer.md). + +The pattern: install every promising MCP server, enable a bunch of skills, add a global CLAUDE.md packed with instructions, bolt on project-level configs on top. Each felt like a small addition. Together they become a tax paid on every single session. + +**Observed impact:** Opening a fresh Claude Code session, typing one simple prompt, and watching 40% of a 200K context window — roughly 2 of 5 status bar segments — vanish before any real work began (source: shedding-dead-context-ryan-spletzer.md). + +## Your Context Window Is RAM + +The analogy: the context window is like RAM — a finite working space where the model holds everything it needs to reason about your problem. Plugins are like running programs (source: shedding-dead-context-ryan-spletzer.md). + +One VS Code window with all extensions is fine alone. Open five or six with their own extension hosts and language servers, add five or six Claude Code terminals, and a browser with 100+ tabs — and suddenly your machine is sluggish, spending more resources managing tools than doing work. + +The same happens to an LLM. Every plugin and instruction competes for the same finite resource the model needs to actually think about your code. + +**If 40% of your context is consumed by dead weight before you start, you're beginning every session in the "dumb zone"** — the region where model reasoning degrades (source: shedding-dead-context-ryan-spletzer.md). See [[million-token-context-window]] for zone definitions and benchmarks. + +## Beyond Waste: Attention Corruption + +Dead context doesn't just sit there inertly. **It dilutes the signal.** The model's attention spreads across everything in the window, so the more irrelevant context you pack in, the less weight the relevant context carries. Instructions get misapplied, tool descriptions bleed into each other, and the model confidently acts on the wrong context without indication that something went sideways (source: shedding-dead-context-ryan-spletzer.md). + +The analogy extends to rowhammer attacks on non-ECC memory: repeatedly accessing one row causes electrical interference that flips bits in adjacent rows. Data doesn't disappear — it *corrupts*. Similarly, overloaded context doesn't just forget things — it produces wrong answers that look plausible, ignored instructions, agents that know the right answer but say the wrong thing because the correct instruction was diluted by everything around it. + +**It's not just forgetting. It's degradation** — and like a bad DIMM, you might not realize it's happening until the output is already wrong (source: shedding-dead-context-ryan-spletzer.md). + +## The Bigger-Window Trap + +A bigger context window is analogous to more RAM on a machine with a memory leak — it delays symptoms without fixing the cause. And it removes the pressure to be disciplined (source: shedding-dead-context-ryan-spletzer.md). + +With 200K you're forced to be thoughtful about what you load. With 1M you can be sloppy, and it *appears* to work — until it doesn't. When it fails with a million tokens of context, failures are harder to diagnose because you can't easily pinpoint which tokens caused the corruption. See [[context-scarcity-end]] for analysis of how flat-rate 1M pricing changes this dynamic. + +## Three Layers of the Same Problem + +The problem appears simultaneously at three levels (source: shedding-dead-context-ryan-spletzer.md): + +**The editor:** VS Code loaded with dozens of extensions — linters, formatters, language packs, themes, tools tried once and forgotten. Each one adds overhead. + +**The AI harness:** A global CLAUDE.md grown into a sprawling instruction manual. Plugins enabled globally that only matter for specific projects. Skills accumulated without pruning. + +**The human:** Running five Claude Code instances simultaneously, five VS Code windows open, a browser with a mountain of tabs — doing to your own brain exactly what you're doing to the model: overloading your own context window with competing demands. + +## The Plugin Paradox + +Not all plugins are dead context. Some farm out what would otherwise be a token-intensive task to deterministic tools; some actually reduce overall token consumption by fetching *precisely* what the model needs without dumping everything into the window (source: shedding-dead-context-ryan-spletzer.md): + +- A plugin that `cat`'s an entire file into context: **expensive** +- A plugin that searches for the relevant function and returns just that: **saves context** + +The question isn't "how many plugins do I have?" It's "what's the ROI of each one?" Some plugins add small overhead but prevent the model from doing expensive, wasteful exploration on its own — those are keepers. The ones occupying space for a capability you use once a month? Dead context. + +## Memory Management Strategies + +If the context window is RAM, optimizing it is memory management (source: shedding-dead-context-ryan-spletzer.md): + +### Scope your editor per project +Launch VS Code with only extensions listed in a project's `.vscode/extensions.json`. Everything else disabled for that session. Each project only loads what it actually needs. + +### Audit your CLAUDE.md +Global and project-level CLAUDE.md files are prime candidates for dead context. Instructions that made sense three months ago may be irrelevant now. + +Ask Claude to audit your config: *"Review my CLAUDE.md and identify anything that could be removed, consolidated, or moved to a skill that only loads when needed."* + +Skills are the equivalent of swapping to disk — full instructions load only when invoked, instead of sitting in memory on every session. See [[progressive-disclosure]] for the tiered loading framework. + +### Scope plugins per project +Not every project needs every MCP server. Set project-level `.claude/settings.json` files that enable only the plugins relevant to that codebase. A Jekyll blog doesn't need a database plugin. An API project doesn't need a Playwright plugin. + +### Revisit regularly +This is not a one-time cleanup. Context cruft accumulates the way clutter accumulates in a house — you install a new skill, try a new MCP server, add a line to your CLAUDE.md. Each one is small. Over time they add up. + +## Use AI to Fix Your AI + +The best tool for shedding dead context is the very AI you're trying to optimize (source: shedding-dead-context-ryan-spletzer.md): + +- Ask Claude to generate your `.vscode/extensions.json` for a project +- Ask it to review your CLAUDE.md and suggest what to cut, optimize, or push into a skill +- Ask it to set up a scoped `.claude/settings.json` with only the plugins you need +- Ask it to create skills for instructions that don't need to load every session + +There's a nice recursion to it: the model, operating within its own constrained context window, helping you make that context window less constrained for the next session. It won't tell you to install fewer things — you have to bring the philosophy. + +## The Compaction Dread + +If you've used Claude Code in a long session, you know the feeling: watching the status bar tick down, doing mental math about whether you can fit one more big ask in before the session compacts. Every token of dead context is a token stolen from that budget. All those plugin manifests and stale instructions crowd out the space needed for the actual back-and-forth of getting work done (source: shedding-dead-context-ryan-spletzer.md). + +**The irony: the dead context was supposed to help.** + +## Related pages + +- [[agents-md-liability]] +- [[context-rot]] +- [[context-engineering]] +- [[progressive-disclosure]] +- [[million-token-context-window]] +- [[context-scarcity-end]] +- [[agent-configuration-files]] diff --git a/wiki/knowledge/dynamic-tool-discovery.md b/wiki/knowledge/dynamic-tool-discovery.md new file mode 100644 index 00000000..6f12f4a5 --- /dev/null +++ b/wiki/knowledge/dynamic-tool-discovery.md @@ -0,0 +1,135 @@ +# Dynamic Tool Discovery + +**Summary**: How AI agents find the right tool at runtime using semantic search instead of static pre-loaded lists — the Tool Search pattern, its scalability advantages, and its relationship to context engineering. +**Sources**: `docs/tool-calling/tool-search-redefining-agent-tool-calling-epsilla.md`, `docs/tool-calling/programmatic-tool-calling-claude-api.md`, `docs/tool-calling/mcp-programmatic-tool-calling-opensandbox-dev.md`, `docs/tool-calling/README.md` +**Last updated**: 2026-04-21 + +--- + +## Static vs. Dynamic Tool Loading + +### Static (traditional) approach + +All tool definitions are injected into the model context before any conversation begins. For a 5-server MCP setup (source: mcp-programmatic-tool-calling-opensandbox-dev.md): + +| Server | Tools | Tokens | +|--------|-------|--------| +| GitHub | 35 | ~26K | +| Slack | 11 | ~21K | +| Jira | 20 | ~17K | +| Sentry | 5 | ~3K | +| Grafana | 5 | ~3K | + +Total: **~70–134K tokens** consumed before the first user message. This is *context bloat*. + +### Dynamic (Tool Search) approach + +The model starts with only a **lightweight search stub (~500 tokens)**. When a capability is needed, it queries the search tool and receives only the relevant schema just-in-time (source: tool-search-redefining-agent-tool-calling-epsilla.md): + +``` +Initial context: System Prompt + tool_search stub (~500 tokens) + ↓ +User: "Create a PR on GitHub" + ↓ +Model → tool_search("github pull request") + ↓ +Discovers github.createPullRequest + ↓ +Injects only that definition (~800 tokens) + ↓ +Invokes tool + +Total: ~1.3K tokens (vs. ~72K traditional) +``` + +--- + +## Why This Is Like RAG + +Tool Search applies the same principle as Retrieval-Augmented Generation, but to tool schemas instead of knowledge documents (source: tool-search-redefining-agent-tool-calling-epsilla.md): + +- **RAG**: query → retrieve knowledge chunks → inject into context → answer +- **Tool Search**: query → retrieve tool definition → inject into context → invoke tool + +Both are instances of **Just-in-Time Retrieval (JITR)** — a core context engineering principle: do not preemptively load all potentially useful information; retrieve and inject it precisely when required (source: tool-search-redefining-agent-tool-calling-epsilla.md). + +See [[context-engineering]] for the broader JITR strategy in agentic systems. + +--- + +## The MCP Code Mode Three-Tool Pattern + +In MCP-based implementations, dynamic discovery uses three sequential tools rather than a single search stub (source: mcp-programmatic-tool-calling-opensandbox-dev.md): + +1. **`search`** — discovers tools matching a query, returning names and brief descriptions. +2. **`get_schema`** — retrieves the full parameter schema for specific tool names. +3. **`execute`** — runs generated code in a sandbox that invokes the discovered tool. + +This three-step pattern is the `.NET/C# MCP Code Mode` approach, as opposed to Anthropic's single `tool_search` server tool (source: mcp-programmatic-tool-calling-opensandbox-dev.md). + +See [[mcp-programmatic-tool-calling]] for implementation details and [[tool-calling-patterns]] for a comparison of all approaches. + +--- + +## Accuracy vs. Scale Trade-off + +Static loading hurts tool-selection accuracy even when context limits aren't reached — information overload degrades reasoning (source: tool-search-redefining-agent-tool-calling-epsilla.md): + +| Model | Accuracy (static) | Accuracy (Tool Search) | +|-------|-------------------|------------------------| +| Claude Opus 4 | 49% | 74% | +| Claude Opus 4.5 | 79.5% | 88.1% | + +Dynamic discovery reduces the decision space presented to the model at any given moment, improving both accuracy and efficiency. + +--- + +## Prompt Cache Implications + +Static tool lists that change across requests destroy prompt cache hits. Dynamic Tool Search fixes this by (source: tool-search-redefining-agent-tool-calling-epsilla.md): + +- Keeping the stable prefix (system prompt + search stub) **unchanged across turns**. +- Injecting discovered schemas only at the **end** of the context. + +Result: dramatically lower TTFT (Time to First Token) and fewer cache misses. See [[context-engineering]] for full prompt cache strategy. + +--- + +## When to Use Dynamic Discovery + +Dynamic tool discovery is recommended when (source: tool-search-redefining-agent-tool-calling-epsilla.md, mcp-programmatic-tool-calling-opensandbox-dev.md): + +- The agent has **10+ tools** available. +- Tool definitions span **multiple MCP servers**. +- Combined tool schemas exceed **10K tokens**. +- The application must scale to **hundreds of APIs** (enterprise environments). + +For smaller tool sets (fewer than 10 tools, < 10K token schemas), static loading remains simpler and sufficient. See [[tool-calling-patterns]] for guidance on when each pattern applies. + +--- + +## Implementations at a Glance + +| Platform | Mechanism | +|----------|-----------| +| Anthropic Claude (Sonnet 4+) | `tool_search` server tool | +| OpenAI GPT-5.4 | `tool_search` (namespace-level) | +| Spring AI | `ToolSearchToolCallAdvisor` | +| .NET MCP (Code Mode) | `search` + `get_schema` + `execute` | +| Vercel AI SDK | `withProgrammaticCalling()` — injects `code_execution` meta-tool | + +(sources: tool-search-redefining-agent-tool-calling-epsilla.md, mcp-programmatic-tool-calling-opensandbox-dev.md, cameronking4-programmatic-tool-calling-github.md) + +--- + +## Related pages + +- [[tool-search-epsilla]] +- [[programmatic-tool-calling]] +- [[programmatic-tool-calling-sdk]] +- [[mcp-programmatic-tool-calling]] +- [[tool-calling-patterns]] +- [[context-engineering]] +- [[agent-workflows]] +- [[anthropic-tool-use]] +- [[json-schema-for-ai]] diff --git a/wiki/knowledge/evaluating-agents-paper.md b/wiki/knowledge/evaluating-agents-paper.md new file mode 100644 index 00000000..3b9ee463 --- /dev/null +++ b/wiki/knowledge/evaluating-agents-paper.md @@ -0,0 +1,91 @@ +# Evaluating AGENTS.md Paper + +**Summary**: ETH Zurich study (February 2026, arXiv:2602.11988v1) empirically demonstrating that LLM-generated context files reduce task success by ~3% while increasing costs by 20%+, and that developer-provided context files show only marginal improvement — the foundational evidence for minimal, high-quality agent configuration. +**Sources**: Evaluating-AGENTS-paper.md +**Last updated**: 2026-04-22 + +--- + +## Key Finding + +**LLM-generated context files hurt more than they help.** Developer-written files provide marginal benefit at significant cost. The study directly challenges the practice of auto-generating comprehensive AGENTS.md files. + +## Results + +| Configuration | Success Rate Change | Cost Change | +| -------------------------- | ------------------- | ----------- | +| No context (baseline) | — | — | +| LLM-generated context | **−0.5% to −3%** | **+20–23%** | +| Developer-provided context | **+4% average** | **+19%** | + +### Per-Model Behavior + +| Model | Agent | No Context | LLM-Generated | Developer | Steps (No Context) | +| ------------ | ----------- | ------------------ | --------------------- | -------------- | ------------------ | +| Sonnet-4.5 | Claude Code | Strongest baseline | Slight degradation | Marginal gain | Fewest steps | +| GPT-5.2 | Codex | Strong baseline | Degradation | Marginal gain | Moderate | +| GPT-5.1 Mini | Codex | Moderate baseline | **Worst degradation** | No improvement | **Most steps** | +| Qwen3-30B | Qwen Code | Weakest baseline | Slight degradation | Small gain | Most variable | + +GPT-5.1 Mini showed **pathological behavior**: issuing multiple commands to find files already in context, reading them repeatedly, and generating incomplete outputs. Only **36%** of its generated files contained overviews vs. **95–100%** for other models. + +### Tool Mention Effects + +Context files that mention specific tools cause dramatically different usage patterns: + +| Tool | Usage When Mentioned | Usage When Not Mentioned | Ratio | +| ------------------- | --------------------- | ------------------------ | ----- | +| `uv` | **1.6× per instance** | <0.01 per instance | 160× | +| Repo-specific tools | **2.5× per instance** | <0.05 per instance | 50× | + +This demonstrates that agents **follow tool suggestions** in context files almost unconditionally — making the content of those files high-leverage for both benefit and harm. + +## Why Context Files Fail + +1. **Redundancy**: Context files work best when they are the **only** documentation — when other docs exist, they duplicate information +2. **Broader but undirected exploration**: Agents explore more (14–22% more reasoning tokens) but don't find relevant files faster +3. **Cost amplification**: 2.45 to 3.92 additional steps per task with context files +4. **Coverage of test modifications**: 75% average — agents still miss 25% of needed test changes + +## Methodology + +- **Benchmark**: AGENTBENCH (138 instances, 12 niche repos) + SWE-BENCH LITE (300 instances, popular repos) +- **Agents**: Claude Code, Codex, Qwen Code +- **Models**: Sonnet-4.5, GPT-5.2, GPT-5.1 Mini, Qwen3-30B +- **Settings**: No context vs. LLM-generated vs. developer-provided + +### AGENTBENCH Construction (5-Stage Pipeline) + +1. **Find repos**: GitHub search for Python projects with test suites and 400+ pull requests +2. **Filter PRs**: Rule-based + LLM filtering to select meaningful code changes (from 5,694 PRs down to 138 instances) +3. **Set up environments**: Reproducible Docker environments per repo +4. **Generate task descriptions**: LLM-authored problem statements from PR diffs +5. **Generate unit tests**: Automated test generation to verify correct resolution + +The benchmark intentionally selects **niche repositories** (12 repos) where context files could theoretically provide the most value — making the negative finding stronger. SWE-BENCH LITE provides the complementary signal from popular, well-documented repos. + +## Practical Implications + +These findings are the evidence base for [[progressive-disclosure]]: + +1. **Omit LLM-generated context files** — they cost more and deliver less +2. **Keep developer-written configs minimal** — only essential, non-redundant information +3. **Avoid redundancy** — if documentation exists elsewhere, don't duplicate it in config files +4. **Focus on unique value** — build commands, project-specific conventions, and non-obvious patterns are where config files add value +5. **Measure cost** — context files increase inference costs by 20%+ even when they don't improve outcomes + +## Connection to This Project + +This paper is the foundational evidence for the agent-engineering-toolkit's design approach: + +- Root CLAUDE.md targets 15–40 lines (not comprehensive documentation) +- Scope files target 10–30 lines (not exhaustive rule sets) +- No generated file exceeds 200 lines +- Every instruction must pass: "Would removing this cause the agent to make mistakes?" + +## Related pages + +- [[progressive-disclosure]] +- [[context-engineering]] +- [[context-rot]] +- [[agent-configuration-files]] diff --git a/wiki/knowledge/harness-engineering.md b/wiki/knowledge/harness-engineering.md new file mode 100644 index 00000000..6df2405d --- /dev/null +++ b/wiki/knowledge/harness-engineering.md @@ -0,0 +1,72 @@ +# Harness Engineering + +**Summary**: Harness engineering treats the model as only one part of an autonomous system and focuses on the surrounding control layer — context management, deterministic validation, external state, specialized roles, and controlled execution environments. +**Sources**: `docs/harness-engineering/harness-engineering.md`, `docs/harness-engineering/harnessengineering-building-the-operating-system-for-autonomous-agents.md`, `docs/harness-engineering/skill-issue-harness-engineering-for-coding-agents.md` +**Last updated**: 2026-04-22 + +--- + +Harness engineering is the shift from "just prompting" an LLM toward building the operating system around it. The model remains the reasoning engine, but reliability comes from the harness that constrains how it sees context, uses tools, stores state, and proves completion. + +## Core Shift + +| Vibe-style interaction | Harness engineering | +|---|---| +| Keeps failed attempts and logs in the same chat | Resets context aggressively and carries forward only compacted artifacts | +| Corrects mistakes by adding more chat instructions | Adds deterministic rails such as tests, linters, hooks, and budgets | +| Treats code generation as the main output | Treats specs, plans, progress files, and review artifacts as first-class outputs | + +## What the Harness Contains + +The current docs consistently describe the harness as the layer around the model: + +- **Context controls** — progressive disclosure, compaction, and fresh windows between phases +- **Deterministic rails** — tests, type checks, linters, schema validation, and stop/review gates +- **External state** — durable research notes, plans, progress files, and git history +- **Specialized roles** — subagents or teammate roles with narrow responsibilities +- **Controlled environments** — sandboxes, worktrees, permission boundaries, and reproducible execution surfaces + +## Design Principles + +### 1. Keep the model in the smart zone + +Long tasks degrade when the context fills with noise. Harness engineering responds with intentional compaction: write down the state, start the next phase with a fresh window, and avoid carrying raw exploration traces farther than necessary. + +### 2. Prefer back-pressure over corrective prompting + +If an agent can claim success without tests, artifacts, or review, the system is underspecified. A good harness forces the agent to prove completion through verification steps rather than relying on self-reported confidence. + +### 3. Use subagents as context firewalls + +Research-heavy or high-latency work should happen in isolated contexts. The parent agent gets back only the condensed result, not the entire search transcript. + +### 4. Make workflow artifacts first-class + +Research, planning, implementation, and review each need durable outputs. These artifacts let later phases start cleanly and give humans something compact to inspect before code changes snowball. + +## Workflow Shape + +The harness-engineering docs align with the [[agentic-engineering-workflow]] pattern: + +1. **Research** — find definitions, data flow, and constraints +2. **Plan** — turn findings into explicit steps and validation checks +3. **Implement** — execute in small increments with verification +4. **Review** — validate the combined result, often with a different role or model + +The point is not ceremony. The point is preventing confusion from compounding across a long autonomous run. + +## Operational Heuristics + +- Success logs should stay short; failure output should be detailed +- Repeated mistakes should drive harness improvements, not just stronger wording +- Parallelism helps only when tasks are genuinely isolated +- A better harness can outperform a model upgrade on the same task + +## Related pages + +- [[agent-harness]] +- [[agent-harness-design]] +- [[agentic-engineering-workflow]] +- [[context-engineering]] +- [[subagents]] +- [[mcp-transport]] diff --git a/wiki/knowledge/human-agent-collaboration.md b/wiki/knowledge/human-agent-collaboration.md new file mode 100644 index 00000000..5c2713d1 --- /dev/null +++ b/wiki/knowledge/human-agent-collaboration.md @@ -0,0 +1,130 @@ +# Human-Agent Collaboration + +**Summary**: Research from Frontiers in Robotics and AI introducing "fluid collaboration" (FC) — a mode of adaptive human-agent interaction requiring action-oriented Theory of Mind reasoning — and its implications for the design of AI agents that coordinate with humans in real-time. +**Sources**: `docs/agent-protocols/fluid-human-agent-collaboration-pmc.md` +**Last updated**: 2026-04-21 + +--- + +## What is Fluid Collaboration? + +Schröder, Heinrich, and Kopp (2025) introduce the concept of **fluid collaboration (FC)** — a mode of interaction marked by frequent, dynamic changes in tasks and roles in response to varying environmental demands (source: fluid-human-agent-collaboration-pmc.md). + +Consider working together in the kitchen to prepare a meal: +- Both partners share a common goal, but subtasks vary considerably +- Who is in charge of what is rarely determined at the outset +- Rapid changes in task assignments occur in response to arising needs or events +- Adaptations can be reactive *or* proactive +- Coordination is signaled implicitly (via behavioral changes) or explicitly (via communication) + +This is FC: natural and intuitive for humans, but **beyond the capabilities of current AI-based collaborative agents** (source: fluid-human-agent-collaboration-pmc.md). + +## Why FC Matters for AI + +FC would constitute a leap for human-agent interaction in settings where (source: fluid-human-agent-collaboration-pmc.md): +- Humans and AI differ considerably in skills and abilities +- Explicit negotiation and predetermination of roles is hardly feasible +- Humans identify and coordinate tasks as they arise in response to dynamic changes + +This is distinct from the [[agent-workflows]] and [[multi-agent-communication]] scenarios typical in agentic AI, which largely assume pre-defined roles and explicit task routing. + +## Theoretical Foundations + +### Core Competencies for Effective Teams + +Three competencies drive effective team success (Salas et al., 2018; source: fluid-human-agent-collaboration-pmc.md): +1. **Coordination**: Distributing tasks between team members effortlessly +2. **Communication**: Sharing information, especially proactive communication of future goals +3. **Adaptability**: Adjusting to changing circumstances + +**Team cognition** encompasses the collective knowledge structures that support coordinated performance, including shared mental models, mentalizing (Theory of Mind), and communication mechanisms. + +### Theory of Mind (ToM) + +**Theory of Mind** — also called *mentalizing* — is the ability to infer the mental states of others, such as intentions, goals, desires, or emotions (source: fluid-human-agent-collaboration-pmc.md). + +Traditionally studied as offline inference from passive observation. However, FC requires: +- **Active engagement** and online participation +- **Continuous inferences** of partners' intentions or beliefs about collaboration patterns +- **Concurrent** inference and task-oriented action + +FC demands **online ToM reasoning** that proceeds rapidly, operates in service of collaborative action, and runs concurrently with task execution — forms not yet successfully modeled computationally (source: fluid-human-agent-collaboration-pmc.md). + +## Defining and Measuring FC + +### Core Properties + +FC is characterized by (source: fluid-human-agent-collaboration-pmc.md): +- **Frequent task changes**: Assignments of tasks and resources change often +- **Dynamic patterns**: Collaboration patterns must be initiated, recognized, and coordinated +- **Minimal explicit communication**: Partners coordinate without extensive verbal negotiation +- **Efficient mentalizing**: Partners infer each other's intentions through behavioral observation + +### Empirical Metrics + +Two key metrics for measuring FC (source: fluid-human-agent-collaboration-pmc.md): +1. **Intertwinement**: How interleaved are the task contributions of different agents +2. **Fluidity**: How frequently and smoothly do task/resource assignments change + +## The Cooperative Cuisine Research Environment + +The authors introduce **Cooperative Cuisine (CoCu)**, an interactive environment inspired by the game *Overcooked!* that facilitates human-human and human-agent collaboration in dynamic settings (source: fluid-human-agent-collaboration-pmc.md). + +### Key Findings from the Human-Human Study + +1. Humans **naturally engage** in dynamic collaboration patterns +2. These patterns emerge **with minimal explicit communication** +3. Humans rely on **efficient mentalizing** to coordinate +4. FC can be **measured empirically** using objective metrics +5. High-performing pairs show more fluid task transitions + +## Requirements for FC-Capable AI Agents + +(source: fluid-human-agent-collaboration-pmc.md) + +1. **Dynamic mentalizing**: Ability to infer partners' intentions in real-time +2. **Resource-rational ToM**: Efficient inference under computational constraints +3. **Action-driven reasoning**: ToM integrated with action planning, not a separate offline module +4. **Rapid adaptation**: Fast enough to keep up with human coordination pace + +### Current AI Approaches and Their Limitations + +| Approach | Strengths | Limitations for FC | +|----------|-----------|-------------------| +| Multi-Agent Planning | Explicit plan creation | Requires domain knowledge; too slow for real-time FC | +| Multi-Agent RL | Fast inference | Sample-inefficient; high variance in collaborative settings | +| LLM-based coordination | Flexible, language-based | Not real-time; high latency | +| PACT model | Predictable behavior | Patterns fixed after optimization; not fluid | + +(source: fluid-human-agent-collaboration-pmc.md) + +## Design Principles for FC-Capable AI Agents + +(source: fluid-human-agent-collaboration-pmc.md) + +1. **Integrate** perception, ToM, planning, communication, and acting in a unified framework +2. **Model dynamic, action-oriented mentalizing** rather than static belief inference +3. **Enable proactive behavior**: Anticipate when another agent requires assistance +4. **Support implicit coordination**: Go beyond explicit language-based negotiation +5. **Design for real-time operation**: Inference must be fast enough for concurrent action + +These principles have direct implications for [[agent-best-practices]] and the design of [[subagents]] that work alongside humans. + +## Relationship to Protocol-Based Communication + +The FC paradigm is largely orthogonal to [[agent-protocol-standards]] such as [[mcp-specification]] and [[agent-to-agent-protocol]]. Those protocols define *how* agents exchange messages; fluid collaboration theory defines *what* the agent must reason about to participate effectively in dynamic, human-driven coordination. + +For AI agents operating in human-centric workflows, FC capabilities must sit on top of the communication layer — agents need both reliable [[multi-agent-communication]] infrastructure and the cognitive models to reason about human intentions in real-time. + +## Full Citation + +Schröder F, Heinrich F and Kopp S (2025). Towards fluid human-agent collaboration: From dynamic collaboration patterns to models of theory of mind reasoning. *Front. Robot. AI* 12:1532693. DOI: 10.3389/frobt.2025.1532693. PMCID: PMC12353729. + +## Related pages + +- [[multi-agent-communication]] +- [[agent-workflows]] +- [[subagents]] +- [[agent-best-practices]] +- [[context-engineering]] +- [[agent-protocol-standards]] diff --git a/wiki/knowledge/humanlayer-repository-analysis.md b/wiki/knowledge/humanlayer-repository-analysis.md new file mode 100644 index 00000000..c940bb9a --- /dev/null +++ b/wiki/knowledge/humanlayer-repository-analysis.md @@ -0,0 +1,112 @@ +# HumanLayer Repository Analysis + +**Summary**: A repository-level analysis of `humanlayer/`, showing that the codebase centers on a local agent runtime (`hlyr` + `hld`) and a primary desktop UI (`humanlayer-wui` / CodeLayer), with shared contracts, shared database layers, and several prototype or transitional surfaces. +**Sources**: `docs/human-layer-project/humanlayer-repository-analysis.md` +**Last updated**: 2026-04-22 + +--- + +The `humanlayer/` repository is best understood as **CodeLayer built on top of HumanLayer runtime infrastructure**. The root product story has shifted toward CodeLayer as an open-source IDE for orchestrating AI coding agents, while much of the runtime and package naming still reflects the earlier HumanLayer identity (source: `humanlayer-repository-analysis.md`). + +## Core architecture + +The center of gravity is the local runtime, not the docs site or the prototype apps. The key split is: + +- `hlyr/` as the CLI entrypoint and the stdio approvals MCP server +- `hld/` as the daemon that launches Claude Code sessions, injects MCP configuration, persists session state, and manages approvals +- `humanlayer-wui/` as the main user-facing desktop/web UI, branded as CodeLayer + +This makes the repo a practical example of [[agentic-engineering-workflow]] implemented as product infrastructure rather than as a loose collection of prompts and scripts (source: `humanlayer-repository-analysis.md`). + +## Runtime model + +The most important execution path is: + +1. A user launches work from the CLI or the CodeLayer UI. +2. `hld` starts or manages a Claude Code session. +3. The daemon injects a session-scoped MCP subprocess that resolves back to `hlyr mcp claude_approvals`. +4. Approval requests are stored, correlated to tool calls, and surfaced back to the user through daemon-facing clients. + +This is a concrete, productized form of local orchestration, session state management, and tool mediation. One detail matters: the stdio MCP path and the HTTP MCP path do not expose the same tool contract (`request_permission` vs `request_approval`). It overlaps with ideas in [[mcp-specification]], [[multi-agent-communication]], and [[claude-code-subagents]], but applies them as application architecture rather than as generic patterns (source: `humanlayer-repository-analysis.md`). + +## Primary versus secondary surfaces + +The repo has three visible maturity levels: + +### Shipping path + +- `humanlayer-wui/` +- `hlyr/` +- `hld/` +- `claudecode-go/` + +These form the product path that matters most to a reader trying to understand what the project actually ships (source: `humanlayer-repository-analysis.md`). + +### Shared layers + +- `packages/contracts/` +- `packages/database/` + +These packages support typed contracts, schema definition, and newer app surfaces. They are important, but they are not the repo's primary runtime center (source: `humanlayer-repository-analysis.md`). + +### Prototype or scaffold areas + +- `apps/react/` +- `apps/daemon/` + +These read as exploratory or scaffold layers rather than as the main delivered product. The repo keeps them alongside the production path, which is useful for experimentation but adds interpretation overhead for first-time readers (source: `humanlayer-repository-analysis.md`). + +## Tooling and operations + +The repo uses a hybrid orchestration model: + +- Bun + Turbo for top-level JS tasks +- Make as the real cross-project control plane +- Go for the daemon and Claude Code SDK +- Tauri/Rust for the desktop shell and packaging + +This is a good example of [[context-engineering]] applied to repo operations: each subsystem keeps its native tooling, while the root `Makefile` acts as the unifying control surface instead of forcing one artificial monorepo abstraction over everything (source: `humanlayer-repository-analysis.md`). + +The operational posture is strong: + +- CI separates checks and tests +- release automation handles macOS artifacts and Homebrew updates +- local development supports nightly builds, ticket-scoped environments, and worktree helpers + +That gives the repo unusually strong ergonomics for an agent product under active development (source: `humanlayer-repository-analysis.md`). + +## Main tensions in the codebase + +Three tensions define the repo's current shape: + +### Branding transition + +The project has clearly moved toward CodeLayer, but HumanLayer naming remains in packages, docs, and legacy material. This is the most visible sign of transition (source: `humanlayer-repository-analysis.md`). + +### Uneven monorepo boundaries + +Root JS workspaces cover only `apps/*` and `packages/*`, while major components like `humanlayer-wui/`, `hlyr/`, `hld/`, and `claudecode-go/` sit outside that workspace boundary and are orchestrated through Make. The repo is coherent, but not fully captured by the workspace model alone (source: `humanlayer-repository-analysis.md`). + +### Documentation lag + +The code and product direction are clearer than some documentation surfaces. Protocol docs, navigation, and older material do not always describe every active runtime surface or current product boundary (source: `humanlayer-repository-analysis.md`). + +## Why this repo matters + +This repository is a concrete example of how a local-first agent product can combine: + +- a daemon-owned orchestration model +- session-scoped MCP injection +- approval-aware execution +- persistent event/state tracking +- a desktop UI over the same runtime + +That makes it relevant to several existing wiki themes: [[agentic-engineering-workflow]], [[mcp-specification]], [[multi-agent-communication]], and [[context-engineering]]. It shows what those ideas look like when they are assembled into a working product repo instead of discussed as isolated concepts (source: `humanlayer-repository-analysis.md`). + +## Related pages + +- [[agentic-engineering-workflow]] +- [[context-engineering]] +- [[mcp-specification]] +- [[multi-agent-communication]] +- [[claude-code-subagents]] diff --git a/wiki/knowledge/index.md b/wiki/knowledge/index.md new file mode 100644 index 00000000..befe63dd --- /dev/null +++ b/wiki/knowledge/index.md @@ -0,0 +1,147 @@ +# Wiki Index + +Total content pages: **71** + +--- + +## Foundational Concepts + +| Page | Summary | +| -------------------------- | ------------------------------------------------------------------------- | +| [[context-engineering]] | Token budget management, position effects, four context strategies | +| [[context-rot]] | Empirical degradation from 0.92→0.68 accuracy, three architectural causes | +| [[progressive-disclosure]] | Tiered loading (always/on-demand/invoked), ETH Zurich evidence | +| [[prompt-engineering]] | Paradigm inversion for reasoning models, CoT/ToT/ReAct techniques | + +## Agent Architecture + +| Page | Summary | +| ----------------------------- | ---------------------------------------------------------------------- | +| [[evaluating-agents-paper]] | ETH Zurich 2026 study: minimal configs outperform comprehensive ones | +| [[agent-workflows]] | Fundamental loop, five core patterns, orchestration strategies | +| [[subagents]] | Isolated task specialists, tool restriction patterns, and prompt design | +| [[agent-configuration-files]] | AGENTS.md, CLAUDE.md, and path-scoped rule patterns for persistent instructions | +| [[agent-best-practices]] | Plan-first workflow guidance, context discipline, and common anti-patterns | +| [[harness-engineering]] | Control-layer view of autonomous agents: context, rails, state, and execution environments | + +## Claude Code Platform + +| Page | Summary | +| ------------------------- | ------------------------------------------------------------- | +| [[claude-code-skills]] | SKILL.md format, frontmatter, string substitutions, locations | +| [[claude-code-hooks]] | Lifecycle events, hook types, exit codes, matchers | +| [[claude-code-plugins]] | Plugin structure, manifest, distribution, namespacing | +| [[claude-code-memory]] | CLAUDE.md hierarchy, path-scoped rules, imports, auto memory | +| [[claude-code-subagents]] | Definition format, frontmatter fields, agent teams | + + +## Agent Skills Standard + +| Page | Summary | +| ------------------------- | ---------------------------------------------------------------- | +| [[agent-skills-standard]] | Open specification, frontmatter, progressive disclosure loading | +| [[skill-authoring]] | Eval-driven iteration, description optimization, script bundling | + +## Research + +| Page | Summary | +| ----------------------------- | --------------------------------------------------------------- | +| [[persuasion-in-ai]] | Seven persuasion principles with quantitative effect sizes | +| [[multilingual-performance]] | Tokenization disparities, English-thinking, Portuguese analysis | +| [[whitespace-and-formatting]] | Formatting costs (1 token), structural quality improvements | + +## MCP (Model Context Protocol) + +| Page | Summary | +|------|---------| +| [[mcp-specification]] | The MCP open protocol: JSON-RPC 2.0 architecture with Hosts, Clients, and Servers exposing Resources, Prompts, and Tools. Includes security principles and governance under AAIF/Linux Foundation. | +| [[mcp-vs-a2a]] | Comparison of MCP (agent-to-tool) and A2A (agent-to-agent) protocols — key differences, when to use each, and the combined enterprise architecture using both layers. | +| [[mcp-dev-summit]] | Recap of MCP Dev Summit NA 2026 (April 2–3, New York): June 2026 roadmap, MRTR stateless transport, CIMD/XAA auth, SDK V2, and MCP Apps. | +| [[mcp-skills-vs-mcp]] | Skills (institutional knowledge, the *how*) vs. MCP (authenticated capability access, the *ability*) — complementary layers, not competing technologies. | +| [[mcp-typescript-sdk]] | Official TypeScript SDK (`@modelcontextprotocol/sdk`): client/server APIs, STDIO and Streamable HTTP transports, Zod validation, OAuth 2.1 auth helpers, Express/Hono middleware. | +| [[mcp-servers]] | Reference servers (Fetch, Filesystem, Git, Memory, etc.), multi-language frameworks, MCP-compatible clients, and community ecosystem patterns (security proxies, skill libraries, grounding frameworks). | +| [[mcp-skills-interest-group]] | Skills Over MCP working group (est. Feb 2025): `/.well-known/agent-skills/index.json` discovery, lazy loading, skill enforcement modes, file-system dependency solutions, and script-bearing skill trust model. | +| [[agent-to-agent-protocol]] | Google's A2A protocol: Agent Cards, Tasks, Artifacts, and Push Notifications for peer-to-peer agent coordination — complementing MCP's agent-to-tool layer. | +| [[mcp-transport]] | STDIO (local) and Streamable HTTP (remote) transports, HTTP/2 requirements, current stateful limitations, and the June 2026 MRTR roadmap for serverless compatibility. | +| [[mcp-programmatic-tool-calling]] | Code-mode tool calling: the model writes a server-side program chaining multiple tool calls in sequence — planned 2026 roadmap item reducing round-trips and context window pressure. | + +## Context Engineering (New Sources) + +| Page | Summary | +|------|---------| +| [[million-token-context-window]] | What large context windows actually mean in practice — benchmark degradation curves, the compaction problem, and why discipline outperforms capacity | +| [[agents-md-liability]] | Why growing AGENTS.md files beyond a certain size actively harm AI agent performance through attention dilution, primacy bias, and the dumb zone | +| [[context-scarcity-end]] | How flat-rate 1M context pricing changes workflows, removes economic pressure for discipline, and shifts the bottleneck from capacity to relevance | +| [[dead-context]] | Instructions, tool definitions, and metadata occupying the context window without contributing to the current task — how it accumulates and strategies to shed it | +| [[commercial-agent-context]] | Context engineering as infrastructure for multi-tenant production agent systems — memory taxonomy, truth vs. acceleration layers, and the full context engine loop | + +## Agent Protocols & Communication + +| Page | Summary | +|------|---------| +| [[agent-communication-protocols]] | Academic overview of MCP, ACP, A2A, and ANP — four emerging communication standards for AI agents (IJSRST paper) | +| [[ai-agent-protocols-2026]] | Complete 2026 guide to the consolidated two-protocol world (MCP + A2A), with decision framework and production architecture | +| [[agentic-systems-architectural-paradigms]] | Dual symbolic/neural paradigm framework, context engineering as the primary quality lever, and programmatic execution patterns | +| [[anthropic-2026-full-connectivity]] | Anthropic engineer's forecast for 2026 "full connectivity" and critique of REST-wrapping anti-pattern | +| [[a2a-protocol]] | Google's Agent-to-Agent protocol: Agent Cards, task lifecycle, streaming, and Python SDK quick start | +| [[human-agent-collaboration]] | Fluid collaboration theory and Theory of Mind reasoning requirements for AI agents collaborating with humans in real-time | +| [[agent-protocol-standards]] | Synthesized overview of MCP, A2A, ACP, and ANP: differences, complementarity, governance, and decision guidance | +| [[multi-agent-communication]] | How agents communicate: message-passing patterns, orchestration models, task lifecycle, discovery, and authentication | + +## Structured Outputs (Anthropic) + +| Page | Summary | +|------|---------| +| [[structured-outputs-anthropic]] | Anthropic's constrained-decoding feature guaranteeing schema-compliant JSON via JSON outputs and strict tool use | +| [[anthropic-tool-use]] | Implementing tool use with the Claude API: definitions, best practices, input_examples, tool_choice | +| [[anthropic-strict-tool-use]] | Setting strict: true for grammar-constrained tool inputs, guaranteeing type-safe function calls | +| [[anthropic-output-consistency]] | Prompt engineering techniques for output consistency, with guidance on when to use structured outputs instead | +| [[json-schema-for-ai]] | Using JSON Schema to constrain LLM outputs: supported features, complexity limits, best practices | +| [[tool-use-patterns]] | Patterns for defining and using tools with LLMs: forced use, multi-tool, strict mode, namespacing | + +## Agentic Engineering + +| Page | Summary | +|------|---------| +| [[rpi-workflow]] | Three-phase Research → Plan → Implement workflow by Dex Horthy / HumanLayer; each phase produces a compacted artifact and starts with a fresh context window to maximize LLM output quality. | +| [[rpir-workflow]] | Tyler Burleigh's Research → Plan → Implement → Review extension; adds explicit review sessions after every phase, written artifact persistence, and a multi-model scaling model. | +| [[agent-harness]] | Martin C. Richards' case for building an agent harness — a set of skills, workflows, and methodology encoding how your agent thinks and builds — and why harness quality outweighs model choice. | +| [[claude-cookbook]] | Anthropic's official collection of copy-paste Claude code examples covering classification, RAG, tool use, multimodal capabilities, sub-agents, automated evaluations, and prompt caching. | +| [[agentic-software-modernization]] | Markus Harrer's analysis of using agentic AI for legacy software (COBOL, RPG) modernization: RPI workflow, critic-agent loops, codebase hygiene, context compaction, and traceability links. | +| [[agentic-engineering-workflow]] | Synthesized overview of the RPI/RPIR agentic engineering workflow pattern — how it independently emerged across multiple practitioners and why it is the reliable answer to vibe coding. | +| [[agent-harness-design]] | Design patterns for building agent harnesses: component taxonomy (sequential/advisory/utility), context isolation, artifact design, skill routing, backflow support, and multi-model specialization. | + +## Repository Analyses + +| Page | Summary | +|------|---------| +| [[humanlayer-repository-analysis]] | Analysis of the `humanlayer/` repo: CodeLayer product positioning, `hlyr` + `hld` local runtime architecture, primary WUI surface, shared packages, tooling model, and consolidation risks. | + +## Spec-Driven Development + +| Page | Summary | +|------|---------| +| [[spec-driven-development]] | Academic overview of SDD — treating specifications as the primary artifact of software development, with three rigor levels, four-phase workflow, AI error reduction evidence, tool survey, and case studies. | +| [[spec-driven-development-practice]] | Practitioner guide contrasting vibe coding with spec-first; step-by-step Spec Kit walkthrough, real prompting examples, and honest assessment of where SDD breaks down. | +| [[spec-driven-development-critique]] | Critical hands-on analysis of Kiro, Spec Kit, and Tessl from the Martin Fowler team; surfaces concrete failure modes: review overhead, false control, one-size-fits-all workflows, and AI non-determinism. | +| [[spec-first-ai-development]] | The spec-first paradigm — why writing specs before code matters for AI-assisted development, and how it shifts the developer's role to spec author and AI orchestrator. | +| [[code-to-contract]] | Contract-based thinking in AI-assisted development — how specifications become executable contracts enforced through CI/CD, BDD frameworks, and API contract testing tools. | + +## Long Context Research + +| Page | Summary | +|------|---------| +| [[lost-in-the-middle-paper]] | Full summary of Liu et al. (2024, TACL) — U-shaped performance curve, multi-doc QA and key-value retrieval experiments, 20%+ accuracy drop at center position, mitigation strategies. | +| [[lost-in-the-middle-in-between]] | Baker et al. (2024, arXiv) follow-up — multi-hop QA over long contexts, the "in-between" effect (relative distance between evidence documents), combinatorial explosion of position permutations. | +| [[u-shaped-attention-curve]] | Concept page — primacy effect, recency effect, middle neglect, quantitative evidence, practical placement guidelines for any LLM context window. | +| [[long-context-mitigation]] | Strategies for mitigating positional bias — position-aware placement, reranking, permutation self-consistency, CoT, knowledge graph extraction, summarization, progressive disclosure. | + +## Tool Calling + +| Page | Summary | +|------|---------| +| [[programmatic-tool-calling]] | Claude API programmatic tool calling: client vs. server tools, agentic loop, forced tool use, pricing, and the advanced code-execution pattern that collapses N round-trips into 2. | +| [[tool-search-epsilla]] | Epsilla's analysis of Tool Search as the most significant AI agent infrastructure shift of 2025-2026: semantic discovery, 85%+ token savings, accuracy benchmarks, prompt cache protection, and FAQ. | +| [[programmatic-tool-calling-sdk]] | cameronking4's Vercel AI SDK implementation of Programmatic Tool Calling: architecture, efficiency tables, cost analysis, and real usage examples across 100+ LLM providers. | +| [[dynamic-tool-discovery]] | Dynamic/semantic tool search vs. static tool lists: how agents find the right tool at runtime, the Tool Search pattern, JITR principle, and scalability guidance. | +| [[tool-calling-patterns]] | Synthesis of all tool calling patterns — forced, auto, parallel, programmatic, code-mode, MCP-based — with a decision guide for when to use each. | diff --git a/wiki/knowledge/json-schema-for-ai.md b/wiki/knowledge/json-schema-for-ai.md new file mode 100644 index 00000000..7ccac996 --- /dev/null +++ b/wiki/knowledge/json-schema-for-ai.md @@ -0,0 +1,167 @@ +# JSON Schema for AI — Constraining LLM Outputs + +**Summary**: Patterns, validation best practices, and limitations for using JSON Schema to constrain LLM outputs, focused on the Anthropic/Claude structured outputs and strict tool use implementations. +**Sources**: `docs/structured-outputs/anthropic-structured-outputs.md`, `docs/structured-outputs/anthropic-strict-tool-use.md`, `docs/structured-outputs/anthropic-implement-tool-use.md` +**Last updated**: 2026-04-21 + +--- + +JSON Schema is the foundation of [[structured-outputs-anthropic]] and [[anthropic-strict-tool-use]]. Both features compile your schema into a grammar that drives constrained sampling — meaning the model's token probabilities are filtered to only produce tokens that can lead to a valid schema-conformant output. (source: anthropic-structured-outputs.md) + +## Why Schema Quality Matters + +A poorly designed schema can: +- Exceed complexity limits and trigger `400` errors +- Slow down first-request latency due to grammar compilation time +- Produce semantically correct but practically useless outputs (e.g., wrong property ordering) +- Make it impossible to mark critical fields as required + +A well-designed schema acts as a contract between your application and the model, enabling reliable downstream parsing without defensive error handling. (source: anthropic-structured-outputs.md) + +## Supported JSON Schema Features + +Both JSON outputs and strict tool use support standard JSON Schema with some limitations. The Python, TypeScript, Ruby, and PHP SDKs **automatically transform** schemas with unsupported features before sending them to the API. (source: anthropic-structured-outputs.md) + +### Unsupported Constraints (auto-handled by SDKs) + +The following constraints are removed from the schema sent to Claude (SDKs inject them into field descriptions instead): +- `minimum` / `maximum` +- `minLength` / `maxLength` +- `minItems` / `maxItems` +- Non-supported string `format` values + +### Always Required for Strict Mode + +When using `strict: true` on a tool, add `"additionalProperties": false` to every object in your schema. This is required for the grammar compiler. (source: anthropic-strict-tool-use.md) + +## Schema Complexity Limits + +Schemas compile into grammars. The combined cost of all strict schemas in a single request must stay within: (source: anthropic-structured-outputs.md) + +| Limit | Value | Why it matters | +|-------|-------|----------------| +| Strict tools per request | **20** | Each tool adds to grammar size | +| Optional parameters (total) | **24** | Each optional parameter roughly doubles grammar state space | +| Parameters with union types (total) | **16** | `anyOf` / type arrays create exponential compilation cost | + +Exceeding these returns `400 "Schema is too complex for compilation"`. A 180-second compilation timeout applies as a final hard stop. + +## Best Practices + +### 1. Mark Required Fields Explicitly + +Always use the `required` array. Every field not in `required` counts toward the 24-optional-parameter limit and roughly doubles a portion of the grammar state space: (source: anthropic-structured-outputs.md) + +```json +{ + "type": "object", + "properties": { + "name": {"type": "string"}, + "email": {"type": "string"} + }, + "required": ["name", "email"], + "additionalProperties": false +} +``` + +### 2. Avoid Union Types Where Possible + +Union types (`anyOf`, `"type": ["string", "null"]`) each count toward the 16-union-parameter limit and create exponential grammar size. Prefer explicit types with clear semantics: (source: anthropic-structured-outputs.md) + +```json +// Prefer this (explicit required field) +"status": {"type": "string", "enum": ["active", "inactive"]} + +// Over this (nullable union — counts toward union limit) +"status": {"type": ["string", "null"]} +``` + +### 3. Write Descriptive Property Descriptions + +Since constraints like `minimum` are removed from the schema and injected into descriptions by the SDK, write clear human-readable descriptions that also inform Claude of constraints: (source: anthropic-structured-outputs.md) + +```json +"passengers": { + "type": "integer", + "description": "Number of passengers. Must be between 1 and 9." +} +``` + +### 4. Flatten Nested Structures + +Deeply nested objects with optional fields compound complexity multiplicatively. Flatten where semantically reasonable: (source: anthropic-structured-outputs.md) + +```json +// Flatter (preferred for strict schemas) +{ + "billing_street": {"type": "string"}, + "billing_city": {"type": "string"} +} + +// Nested (avoid in strict schemas with many optional fields) +{ + "billing": { + "type": "object", + "properties": { + "street": {"type": "string"}, + "city": {"type": "string"} + } + } +} +``` + +### 5. Use Enums for Constrained Values + +Enums are fully supported and help Claude pick valid values without guessing: (source: anthropic-strict-tool-use.md) + +```json +"seat_class": { + "type": "string", + "enum": ["economy", "business", "first"] +} +``` + +### 6. Property Ordering Behavior + +When using structured outputs, **required properties appear first in Claude's output**, followed by optional properties. Schema order is preserved within each group. If property order matters to your application, mark all properties as required. (source: anthropic-structured-outputs.md) + +## Grammar Caching + +The compiled grammar is cached for **24 hours** from last use. Cache invalidation is triggered by: +- Changing schema structure +- Changing the tool set in a request + +Changing only `name` or `description` fields does **not** invalidate the cache — this is important for prompt iteration without latency penalties. (source: anthropic-structured-outputs.md) + +## HIPAA Considerations + +JSON schemas are cached separately from message content. **PHI must not appear in schema definitions**: (source: anthropic-strict-tool-use.md) +- Property names +- `enum` values +- `const` values +- `pattern` regular expressions + +PHI should only appear in message content (prompts and responses). + +## SDK Transformation Pipeline + +For Python, TypeScript, Ruby, and PHP, the SDK transformation pipeline is: (source: anthropic-structured-outputs.md) +1. Remove unsupported constraints +2. Update field descriptions with constraint info +3. Add `additionalProperties: false` to all objects +4. Filter string formats to supported list only +5. Validate the API response against your **original** schema (with all constraints) + +This means Claude receives a simplified schema, but your code still enforces all constraints through post-response validation. + +--- + +## Related pages + +- [[structured-outputs-anthropic]] +- [[anthropic-strict-tool-use]] +- [[anthropic-tool-use]] +- [[tool-use-patterns]] +- [[prompt-engineering]] +- [[context-engineering]] +- [[agent-best-practices]] diff --git a/wiki/knowledge/log.md b/wiki/knowledge/log.md new file mode 100644 index 00000000..13d0e8d8 --- /dev/null +++ b/wiki/knowledge/log.md @@ -0,0 +1,162 @@ +# Wiki Change Log + +Append-only operation log. Most recent entries at top. + +--- + +## 2026-04-22 — Wiki sync against current docs corpus + +| File | Action | Description | +|------|--------|-------------| +| `harness-engineering.md` | created | Added a concise synthesis page for the `docs/harness-engineering/` corpus, covering the control-layer view of autonomous agents, deterministic rails, external state, and context firewalls. | +| `index.md` | updated | Removed stale compliance-routing entries, added `harness-engineering`, refreshed summaries, and corrected the total page count. | +| `log.md` | updated | Recorded the current docs-to-wiki sync. | +| `compliance-routing.md` | deleted | Pruned stale page tied to removed `docs/compliance/` sources. | +| `validation-routing-claude.md` | deleted | Pruned stale page tied to removed `docs/compliance/` sources. | +| Multiple knowledge pages | updated | Removed nonexistent `analysis-*` sources, corrected stale source paths, and pruned broken navigation links to missing wiki pages. | + +Sources: `docs/` excluding `docs/superpowers/` + +--- + +## 2026-04-21 — HumanLayer repository analysis + +| Date | Page | Type | Sources | +|------|------|------|---------| +| 2026-04-21 | [[humanlayer-repository-analysis]] | Source summary | `docs/humanlayer-repository-analysis.md` | + +Changes: Added a repository-analysis page for the local `humanlayer/` clone, covering CodeLayer positioning, the `hlyr` + `hld` runtime path, the primary `humanlayer-wui` surface, shared contract/database layers, tooling/release workflows, and the main consolidation risks. + +--- + +## 2026-04-21 — Long Context Research batch + +| File | Action | Description | +|------|--------|-------------| +| `lost-in-the-middle-paper.md` | created | Full wiki summary of Liu et al. (2024, TACL): U-shaped curve, experiments, mitigations, RAG implications. | +| `lost-in-the-middle-in-between.md` | created | Wiki summary of Baker et al. (2024, arXiv): in-between effect, multi-hop QA, combinatorial limits of mitigations. | +| `u-shaped-attention-curve.md` | created | Concept page for the U-shaped attention/performance curve: primacy, recency, middle neglect, placement guidelines. | +| `long-context-mitigation.md` | created | Comprehensive mitigation strategy page: 8 strategies with single-hop vs. multi-hop effectiveness ratings. | +| `context-engineering.md` | updated | Extended "Position Effects" section with deep-dive subsections on U-curve evidence and in-between compound effect; added 5 new cross-links in Related pages. | + +Sources: `docs/long-context-research/` + +--- + +## 2026-04-21 — Tool Calling batch + +| Date | Page | Type | Sources | +|------|------|------|---------| +| 2026-04-21 | [[programmatic-tool-calling]] | Source summary | programmatic-tool-calling-claude-api.md, README.md | +| 2026-04-21 | [[tool-search-epsilla]] | Source summary | tool-search-redefining-agent-tool-calling-epsilla.md, README.md | +| 2026-04-21 | [[programmatic-tool-calling-sdk]] | Source summary | cameronking4-programmatic-tool-calling-github.md, README.md | +| 2026-04-21 | [[dynamic-tool-discovery]] | Concept | tool-search-redefining-agent-tool-calling-epsilla.md, programmatic-tool-calling-claude-api.md, mcp-programmatic-tool-calling-opensandbox-dev.md, README.md | +| 2026-04-21 | [[tool-calling-patterns]] | Concept | All four source files + README.md | + +Sources: `docs/tool-calling/` + +--- + +## 2026-04-21 — Spec-Driven Development cluster + +Created 5 wiki pages from 3 source documents in `docs/spec-driven-development/`: + +- **spec-driven-development.md** — Source summary of arXiv:2602.00180 (Piskala, AIWare 2026); academic treatment of SDD principles, three rigor levels, four-phase workflow, AI coding agent benefits, tool survey (BDD, OpenAPI, Spec Kit, Kiro, Tessl), case studies (financial services, enterprise BDD, automotive embedded), pitfalls, and decision framework. +- **spec-driven-development-practice.md** — Source summary of DEV Community practitioner guide; vibe coding vs. spec-first contrast, Spec Kit step-by-step walkthrough, real prompting examples, three-tool comparison, honest failure modes (review overhead, exploratory work, AI non-determinism). +- **spec-driven-development-critique.md** — Source summary of martinfowler.com "Exploring Gen AI" critical analysis; hands-on evaluation of Kiro, Spec Kit, Tessl; concrete failure modes: markdown review fatigue, false sense of control, agent non-compliance, functional/technical spec confusion. +- **spec-first-ai-development.md** — Concept page on the spec-first paradigm; AI mind-reading problem, developer role transformation, specs as super-prompts for agent workflows, self-spec methods, when spec-first works and doesn't. +- **code-to-contract.md** — Concept page on executable contract thinking; API contracts (OpenAPI, gRPC, Pact), BDD as contract enforcement, embedded spec-as-source, contracts as AI inputs, three contract enforcement levels. + +Sources: `docs/spec-driven-development/` + +--- + +## 2026-04-21 — Agentic Engineering batch + +Created 7 wiki pages from 5 source documents in `docs/agentic-engineering/`. + +| File | Type | Source | +|------|------|--------| +| `rpi-workflow.md` | Source summary | `research-plan-implement-rpi.md` | +| `rpir-workflow.md` | Source summary | `research-plan-implement-review-tyler-burleigh.md` | +| `agent-harness.md` | Source summary | `building-agent-harness-martin-richards.md` | +| `claude-cookbook.md` | Source summary | `claude-cookbook-anthropic.md` | +| `agentic-software-modernization.md` | Source summary | `agentic-software-modernization-markus-harrer.md` | +| `agentic-engineering-workflow.md` | Concept | All 4 source docs above | +| `agent-harness-design.md` | Concept | `building-agent-harness-martin-richards.md`, `research-plan-implement-rpi.md`, `research-plan-implement-review-tyler-burleigh.md` | + +Sources: `docs/agentic-engineering/` + +--- + +## 2026-04-21 — Structured Outputs (Anthropic) batch + +**Sources read**: 4 files from `docs/structured-outputs/` + +**Pages created**: 6 knowledge pages + +| File | Type | Notes | +|------|------|-------| +| `structured-outputs-anthropic.md` | Source summary | Main structured outputs guide: JSON outputs, strict tool use, SDK helpers, schema limits, caching, HIPAA | +| `anthropic-tool-use.md` | Source summary | Tool definition best practices, input_examples, tool_choice options, model response handling | +| `anthropic-strict-tool-use.md` | Source summary | strict: true mechanics, guarantees, use cases, complexity limits, HIPAA rules | +| `anthropic-output-consistency.md` | Source summary | Six consistency techniques: format spec, prefilling, few-shot, retrieval, chaining, character prompts | +| `json-schema-for-ai.md` | Concept | Schema design patterns, supported features, complexity limits, SDK transformation pipeline | +| `tool-use-patterns.md` | Concept | Eight tool use patterns: forced, any, strict, auto, consolidated, namespaced, examples, combined | + +Sources: `docs/structured-outputs/` (Anthropic API only) + +--- + +## 2026-04-21 — Agent Protocols & Communication batch + +| File | Type | Source document | +|------|------|----------------| +| `agent-communication-protocols.md` | Source summary | `advancing-agentic-ai-communication-protocols.md` | +| `ai-agent-protocols-2026.md` | Source summary | `ai-agent-protocols-2026-guide.md` | +| `agentic-systems-architectural-paradigms.md` | Source summary | `architectural-paradigms-advanced-agentic-systems.md` | +| `anthropic-2026-full-connectivity.md` | Source summary | `anthropic-engineer-2026-forecast-full-connectivity-mcp.md` | +| `a2a-protocol.md` | Source summary | `a2a-protocol-huggingface-space.md` | +| `human-agent-collaboration.md` | Source summary | `fluid-human-agent-collaboration-pmc.md` | +| `agent-protocol-standards.md` | Concept | synthesized from multiple sources | +| `multi-agent-communication.md` | Concept | synthesized from multiple sources | + +Sources: `docs/agent-protocols/` + +--- + +## 2026-04-21 — Context Engineering docs ingested + +Sources: 9 files from `docs/context-engineering/` + +Pages created: +- `million-token-context-window.md` +- `agents-md-liability.md` +- `context-scarcity-end.md` +- `dead-context.md` +- `commercial-agent-context.md` + +Pages updated: +- `context-engineering.md` — added 3 new sources, Five Pillars section, Research–Plan–Implement workflow section, Context as Infrastructure section, and 4 new cross-links +- `progressive-disclosure.md` — added Four Implementation Patterns section, Context Trigger System section, Context Rot Connection section, Multi-Agent section, and When Not to Use section +- `context-rot.md` — added 2 new sources, 500-Instruction Ceiling section, Smart/Warm/Dumb Zone framework with model benchmarks, Attention Sinks section, and 3 new cross-links + +--- + +## 2026-04-21 — MCP docs ingested + +Sources: 13 files from `docs/mcp/` + +Pages created: +- `mcp-specification.md` +- `mcp-vs-a2a.md` +- `mcp-dev-summit.md` (combines long-live-mcp-aqfer.md + long-live-mcp-aws.md) +- `mcp-skills-vs-mcp.md` +- `mcp-typescript-sdk.md` +- `mcp-servers.md` (combines modelcontextprotocol-servers-github.md + anthropic-mcp-github-topics.md) +- `mcp-skills-interest-group.md` (combines skills-over-mcp-meeting-notes-2248.md + skills-over-mcp-office-hours-2460.md) +- `agent-to-agent-protocol.md` +- `mcp-transport.md` +- `mcp-programmatic-tool-calling.md` + +Changes: Added 10 substantive wiki knowledge pages covering the MCP ecosystem — protocol specification, transport mechanisms, TypeScript SDK, server ecosystem, Skills vs MCP architecture, the Skills Over MCP working group standards process, A2A protocol comparison, MCP Dev Summit 2026 recap, and programmatic tool calling. All pages cross-link to existing wiki pages and to each other. diff --git a/wiki/knowledge/long-context-mitigation.md b/wiki/knowledge/long-context-mitigation.md new file mode 100644 index 00000000..8a01240a --- /dev/null +++ b/wiki/knowledge/long-context-mitigation.md @@ -0,0 +1,134 @@ +# Long Context Mitigation + +**Summary**: Strategies for mitigating the "lost in the middle" positional bias in long-context LLMs — covering position-aware placement, reranking, permutation self-consistency, chain-of-thought prompting, context reduction, and progressive disclosure, with empirical evidence for each approach's effectiveness and limits. +**Sources**: lost-in-the-middle-acl.md, lost-in-the-middle-and-in-between-arxiv.md, `docs/long-context-research/README.md` +**Last updated**: 2026-04-22 + +--- + +Mitigating the [[u-shaped-attention-curve]] and "lost in the middle" phenomenon requires strategies at multiple levels: where content is placed, how it is selected, how context is structured, and how retrieval pipelines are designed. No single strategy eliminates the bias entirely; practical systems combine several approaches. + +## Strategy Overview + +| Strategy | Effectiveness (Single-Hop) | Effectiveness (Multi-Hop) | Notes | +|----------|---------------------------|--------------------------|-------| +| Position-aware placement | Always applicable | Always applicable | Free; no additional compute | +| Query-Aware Contextualization | High (key-value retrieval) | Limited (multi-doc QA) | Liu et al. primary mitigation | +| Document re-ranking | High | Impractical | Combinatorial explosion at 3+ hops | +| Permutation self-consistency | High | Intractable | 190–4,845 permutations for 2–4 hops | +| Chain-of-Thought prompting | Moderate | Partial | Does not resolve positional bias | +| Knowledge graph extraction | Moderate | Fragile | Reasoning chains fail at multi-hop | +| Document summarization | Moderate | Fragile | Loses critical details | +| Progressive disclosure | High | High | Architectural; prevents large middles | + +(source: README.md, lost-in-the-middle-and-in-between-arxiv.md) + +## Strategy 1: Position-Aware Placement + +The simplest and most universally effective mitigation is to deliberately place the most critical content at the beginning or end of the context window, exploiting the primacy and recency effects described in [[u-shaped-attention-curve]]. + +**Guidelines:** +- Place the most relevant retrieved documents **first** in the document list. +- Place the query or task specification **last** in the context. +- Reserve the middle of the context for less critical background material. + +**Effectiveness:** A 20–30% performance improvement from ordering alone, with no additional compute (source: lost-in-the-middle-acl.md). + +This is the foundational recommendation from [[lost-in-the-middle-paper]]. + +## Strategy 2: Query-Aware Contextualization + +Place the query **both before and after** the document list — at the very start and very end of the context. This exploits both primacy and recency effects simultaneously, anchoring the model's attention on the task from both ends. + +**From Liu et al. (2024) (source: lost-in-the-middle-and-in-between-arxiv.md):** +- Highly effective for synthetic key-value retrieval tasks. +- Limited effectiveness for complex multi-document QA, where the reasoning task is harder to anchor via position alone. + +## Strategy 3: Document Re-Ranking + +Before inserting retrieved documents into the context, rerank them by estimated relevance to the query, then place the highest-ranked documents at the edges (start and end) and lower-ranked documents in the middle. + +**Effectiveness for single-hop QA:** High — ensures the most relevant document is at an edge position, minimizing the probability of middle neglect. + +**Effectiveness for multi-hop QA:** Poor. Re-ranking for multi-hop settings requires knowing *which* documents are evidence for *which* reasoning step and optimizing both absolute position and relative distance between evidence pairs. The combinatorial explosion makes this impractical (source: lost-in-the-middle-and-in-between-arxiv.md): + +| Reasoning Hops | Evidence Position Permutations | +|---------------|-------------------------------| +| 2-hop | 190 | +| 3-hop | 1,140 | +| 4-hop | 4,845 | + +## Strategy 4: Permutation Self-Consistency + +Run the same query multiple times with different document orderings, then aggregate results (majority vote or ensemble). This averages out positional bias by sampling from the full positional distribution. + +**Effectiveness for single-hop QA:** High — eliminates the bias at the cost of N× inference compute. + +**Effectiveness for multi-hop QA:** Intractable for 3+ hops due to the same combinatorial explosion that undermines re-ranking (source: README.md). + +## Strategy 5: Chain-of-Thought Prompting + +Prompt the model to reason step by step before producing its final answer. CoT helps the model identify *which documents* are relevant but does not resolve the fundamental issue that positionally-disadvantaged documents are harder to access. + +**Finding from Baker et al. (2024):** CoT improves document identification — the model correctly identifies which documents matter more often — but does not eliminate the performance gap between edge-positioned and middle-positioned evidence (source: lost-in-the-middle-and-in-between-arxiv.md). + +**Best used when:** Combined with position-aware placement. Use CoT to identify relevant documents; use position-aware placement to ensure they are at context edges. + +## Strategy 6: Context Reduction via Knowledge Graph Extraction + +Extract structured knowledge graph triples from each document using an auxiliary model (e.g., LLaMA 2-7B). Replace verbose documents with concise triple representations, reducing total context size. + +**Benefit:** Reduces context length, potentially bringing more content within the primacy/recency zones. + +**Limitation:** The extracted reasoning chains are **fragile** — small errors in triple extraction cascade into multi-hop reasoning failures. Insufficient for reliable multi-hop QA (source: lost-in-the-middle-and-in-between-arxiv.md). + +## Strategy 7: Document Summarization + +Summarize each retrieved document to a fixed budget (e.g., ≤50 tokens using BART-large-CNN). Smaller context = less middle neglect. + +**Benefit:** Dramatically reduces context size; evidence documents are brought closer to context edges. + +**Limitation:** Summarization loses critical details required for multi-hop inference. The precision needed to resolve multi-hop QA often cannot survive aggressive summarization (source: lost-in-the-middle-and-in-between-arxiv.md). + +## Strategy 8: Progressive Disclosure + +Rather than loading all retrieved documents at once, use [[progressive-disclosure]] — load context in tiers based on relevance. In agentic settings, start with a small, highly relevant context; load additional documents only if the model signals uncertainty or requests more information. + +This keeps critical content near the edges of each context request while preventing the total context from becoming dominated by a large "middle zone." It is architecturally the most robust mitigation because it prevents the problem from arising rather than working around it after the fact. + +## Relationship to Context Engineering + +These mitigation strategies are core tools in the [[context-engineering]] toolkit. The U-shaped curve and these mitigations reinforce several [[context-engineering]] principles: + +1. **Every token must earn its place.** If a document can be placed in the middle and ignored, it should either be removed or relocated to an edge. +2. **Position is a first-class design decision**, not an afterthought. +3. **Context rot accelerates with length.** In very long contexts, the middle grows proportionally, degrading a larger fraction of the content. See [[context-rot]]. +4. **[[dead-context]] accumulates in the middle.** The "lost zone" is the primary accumulation point for dead context — content the model has access to but never effectively uses. + +## Implications for Agentic Systems + +In [[agent-workflows]], mitigation is especially challenging because: + +- Multi-hop reasoning (evidence gathered across multiple tool calls) maps directly to the multi-hop QA setting where strategies scale poorly. +- Agent context grows across turns, pushing earlier content toward the middle and into the "lost zone." +- Re-ranking becomes impractical as the number of reasoning steps increases. + +Recommended approach for agents: + +1. Use subagents to isolate reasoning steps, each with a fresh short context. +2. Apply [[progressive-disclosure]] — load only the documents needed for the current reasoning step. +3. Compress intermediate results before adding to shared context (compaction). + +See [[lost-in-the-middle-in-between]] for the research basis, and [[million-token-context-window]] for implications in very large context scenarios. + +## Related pages + +- [[u-shaped-attention-curve]] +- [[lost-in-the-middle-paper]] +- [[lost-in-the-middle-in-between]] +- [[context-engineering]] +- [[context-rot]] +- [[dead-context]] +- [[progressive-disclosure]] +- [[million-token-context-window]] +- [[agent-workflows]] diff --git a/wiki/knowledge/lost-in-the-middle-in-between.md b/wiki/knowledge/lost-in-the-middle-in-between.md new file mode 100644 index 00000000..05d69fca --- /dev/null +++ b/wiki/knowledge/lost-in-the-middle-in-between.md @@ -0,0 +1,112 @@ +# Lost in the Middle, and In-Between Paper + +**Summary**: Summary of Baker et al. (2024, arXiv) — extending "lost in the middle" to multi-hop QA and establishing that performance degrades not only with absolute position but also with the relative distance between evidence documents: the "in-between" effect. +**Sources**: lost-in-the-middle-and-in-between-arxiv.md, `docs/long-context-research/README.md` +**Last updated**: 2026-04-22 + +--- + +"Lost in the Middle, and In-Between: Enhancing Language Models' Ability to Reason Over Long Contexts in Multi-Hop QA" (Baker et al., arXiv 2412.10079, December 2024) extends the original [[lost-in-the-middle-paper]] findings to the multi-hop question answering (MHQA) setting. The key contribution is demonstrating an additional "in-between" effect: performance degrades not only with absolute position (distance from context edges) but also with the *relative distance between* multiple evidence documents within the same context. + +## Authors and Publication + +**Authors:** George Arthur Baker, Ankush Raut, Sagi Shaier, Lawrence E Hunter, Katharina von der Wense +**Affiliations:** University of Colorado Boulder; University of Chicago; Johannes Gutenberg University Mainz +**Published:** arXiv, December 2024 ([2412.10079](https://arxiv.org/abs/2412.10079)) +**Code:** [github.com/Spongeorge/long-context-multihop](https://github.com/Spongeorge/long-context-multihop) + +(source: lost-in-the-middle-and-in-between-arxiv.md) + +## The In-Between Effect + +The original [[lost-in-the-middle-paper]] established that performance is lowest when a single evidence document is in the center of the context. Baker et al. discover that in multi-hop QA, a second degradation axis exists: + +> When two required pieces of evidence are **adjacent** in the context, performance is consistently **higher** than when they are separated by distractor documents — even when both are at the same absolute position (source: lost-in-the-middle-and-in-between-arxiv.md). + +This "in-between" effect compounds with the original "lost in the middle" effect: + +- Evidence too far from edges → degradation (original finding, [[u-shaped-attention-curve]]) +- Evidence documents too far from each other → additional degradation (new finding) + +## Datasets Evaluated + +Three established multi-hop QA benchmarks were used (source: lost-in-the-middle-and-in-between-arxiv.md): + +| Dataset | Hops Required | Notes | +|---------|--------------|-------| +| HotpotQA | 2 | Cross-document reasoning | +| 2WikiMultiHopQA | 2–4 | Multi-step inference chains | +| MuSiQue | 2–4 | Compositional questions | + +Because official test sets are private, the authors split existing validation sets: first half for validation, second half for test. + +## Models Evaluated + +Results generalize across model families (source: lost-in-the-middle-and-in-between-arxiv.md): + +- **MPT-7b-8k-instruct** — instruction-tuned with ALiBi positional embeddings replacing traditional positional embeddings +- **Llama-2-7b-longlora-8k-ft** — fine-tuned for 8k context without instruction tuning +- **GPT-3.5-turbo-1106** — 16k context window; closed-source OpenAI model + +## Why Multi-Hop Compounds the Problem + +Single-hop QA (one evidence document needed) is already affected by the [[u-shaped-attention-curve]]. Multi-hop QA multiplies the problem: + +1. Each evidence document is individually susceptible to positional neglect. +2. The relative distance between evidence documents creates an additional degradation axis. +3. The combinatorial explosion of position orderings makes mitigation via re-ranking impractical: + +| Reasoning Hops | Evidence Position Permutations | +|---------------|-------------------------------| +| 2-hop | 190 | +| 3-hop | 1,140 | +| 4-hop | 4,845 | + +(source: lost-in-the-middle-and-in-between-arxiv.md) + +## Mitigation Strategies Evaluated + +### Chain-of-Thought Prompting + +CoT prompting **helps identify relevant documents** but does not resolve positional bias. The model becomes better at locating which documents matter, but its ability to use those documents still degrades based on where they sit in the context. CoT is an incomplete solution for the in-between problem (source: lost-in-the-middle-and-in-between-arxiv.md). + +### Knowledge Graph Triple Extraction + +Uses LLaMA 2-7B to extract structured triples from each document, condensing content to factual relationships. This reduces context size but produces reasoning chains that are **too fragile** for reliable multi-hop QA — losing the nuance required for intermediate reasoning steps (source: lost-in-the-middle-and-in-between-arxiv.md). + +### Document Summarization + +BART-large-CNN (fine-tuned on CNN/Daily Mail) generates ≤50-token summaries per document. Context size is reduced, but critical details required for multi-hop inference are often lost (source: lost-in-the-middle-and-in-between-arxiv.md). + +All tested mitigation strategies have significant limitations in the multi-hop setting. See [[long-context-mitigation]] for the broader landscape of strategies. + +## Distinction from Related Work + +This study differs from Levy et al. (2024), which also examines LLM performance degradation (source: lost-in-the-middle-and-in-between-arxiv.md): + +| Dimension | Baker et al. (2024) | Levy et al. (2024) | +|-----------|---------------------|---------------------| +| Focus | Document position within fixed context | Overall input size | +| Datasets | HotpotQA, 2WikiMultiHopQA, MuSiQue | Custom FLenQA (true/false) | +| Hops | Up to 4 | Up to 2 | + +## Implications for Agent Workflows + +In [[agent-workflows]], multi-hop reasoning over retrieved documents is common — an agent may need to gather evidence from multiple sources and synthesize across reasoning steps. The in-between effect means: + +- Simply placing both evidence pieces "near the edges" is insufficient if they are far apart from each other. +- [[long-context-mitigation]] strategies that work for single-hop RAG may not scale to multi-hop agentic reasoning. +- Context window design must account for **evidence clustering** — keeping related documents adjacent — not only document-to-edge distance. + +This connects to [[context-rot]]: as context windows grow longer, the distance between interdependent evidence documents tends to increase, compounding both effects simultaneously. In [[million-token-context-window]] scenarios, the problem becomes severe. + +## Related pages + +- [[lost-in-the-middle-paper]] +- [[u-shaped-attention-curve]] +- [[long-context-mitigation]] +- [[context-engineering]] +- [[context-rot]] +- [[dead-context]] +- [[agent-workflows]] +- [[million-token-context-window]] diff --git a/wiki/knowledge/lost-in-the-middle-paper.md b/wiki/knowledge/lost-in-the-middle-paper.md new file mode 100644 index 00000000..49091d93 --- /dev/null +++ b/wiki/knowledge/lost-in-the-middle-paper.md @@ -0,0 +1,91 @@ +# Lost in the Middle Paper + +**Summary**: Full summary of Liu et al. (2024, TACL) — the landmark paper establishing that LLMs underperform when relevant information is in the middle of long input contexts, exhibiting a characteristic U-shaped performance curve. +**Sources**: lost-in-the-middle-acl.md, lost-in-the-middle-arxiv.md, lost-in-the-middle-researchgate.md +**Last updated**: 2026-04-21 + +--- + +"Lost in the Middle: How Language Models Use Long Contexts" (Liu et al., 2024) is the foundational paper establishing the [[u-shaped-attention-curve]] phenomenon in long-context LLMs. Published in Transactions of the Association for Computational Linguistics (TACL), Volume 12, pp. 157–173 (DOI: 10.1162/tacl_a_00638), it demonstrates that performance is highest when relevant information appears at the beginning or end of the context window and degrades significantly when it appears in the middle — even for models explicitly designed for long contexts. + +## Authors and Publication + +**Authors:** Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang (Stanford University / Meta AI) +**Published:** TACL 2024 (arXiv preprint: July 2023) +**arXiv:** [2307.03172](https://arxiv.org/abs/2307.03172) — 18 pages, 16 figures +**ACL Anthology:** [2024.tacl-1.9](https://aclanthology.org/2024.tacl-1.9) + +(source: lost-in-the-middle-acl.md, lost-in-the-middle-arxiv.md) + +## Core Finding + +Performance can degrade by **over 20 percentage points** in multi-document QA when the answer document is positioned in the center of a 20-document context vs. the edges. This finding holds across all tested models — ChatGPT, Claude, and others — with no model immune to the bias (source: lost-in-the-middle-researchgate.md). + +The pattern is consistent: accuracy is highest at document position 1 and position 20, with the sharpest dip around the midpoint (position 10). This forms the characteristic **U-shaped performance curve** described in [[u-shaped-attention-curve]]. + +## Experiments + +### Task 1: Multi-Document Question Answering + +- **Dataset:** NaturalQuestions-Open +- **Setup:** A correct answer document is inserted at one of 20 positions; 19 distractor documents fill the rest. +- **Result:** Performance follows the U-curve; placing the answer in the center of the 20-document context produces the lowest accuracy (source: lost-in-the-middle-acl.md). + +### Task 2: Key-Value Retrieval + +- **Setup:** Synthetic JSON objects contain 75–140 key-value pairs; the model must retrieve the value for a specific key. +- **Result:** The same U-shaped positional bias emerges on synthetic data, confirming the effect is not dataset-specific (source: lost-in-the-middle-acl.md). + +## Models Evaluated + +All examined models — including those explicitly marketed as long-context capable — exhibited positional bias. The effect was observed on: + +- GPT-3.5-Turbo (ChatGPT) +- Claude (Anthropic) +- Llama-2 variants +- MPT-7b + +No model could reliably process relevant information equally well across all positions (source: lost-in-the-middle-acl.md). + +## Mitigation Strategies Proposed + +### Query-Aware Contextualization + +The paper's primary proposed mitigation: place the query **both before and after** the document list — at the start and end of the context simultaneously. This exploits both primacy and recency effects. + +- **Effective for:** Key-value retrieval tasks. +- **Limited for:** Multi-document QA — less effective due to the complexity of real NLP tasks (source: lost-in-the-middle-and-in-between-arxiv.md). + +### New Evaluation Protocols + +The paper introduces position-aware evaluation protocols for long-context LLMs — measuring performance at each position in the context window rather than aggregating across positions. This influenced subsequent benchmarking work and is now a standard approach in long-context evaluation. + +## Practical Implications for RAG Systems + +For [[context-engineering]] practitioners building RAG systems: + +1. **Most relevant documents should be placed at the beginning or end** of the context — not in the middle. +2. **Document ordering in retrieved sets matters** as much as retrieval quality itself. +3. **Increasing context window size alone does not resolve positional bias** — even the largest long-context models retain the U-shaped degradation. +4. **The [[u-shaped-attention-curve]] must be accounted for** when designing production retrieval pipelines. + +These findings directly inform [[long-context-mitigation]] strategies used in modern RAG architectures. + +## Connection to Context Rot + +The "lost in the middle" effect is one of the primary mechanisms behind [[context-rot]]: as context windows grow, information in the middle becomes increasingly invisible to the model — creating a form of dead content that takes up tokens without contributing to reasoning. See also [[dead-context]]. + +## Impact and Follow-On Work + +This paper is among the most-cited in LLM context research. It catalyzed Baker et al. (2024) — see [[lost-in-the-middle-in-between]] — which extends the findings to multi-hop QA and introduces the "in-between" effect: performance degrades not only with absolute position but also with the relative distance *between* evidence documents. + +## Related pages + +- [[u-shaped-attention-curve]] +- [[lost-in-the-middle-in-between]] +- [[long-context-mitigation]] +- [[context-engineering]] +- [[context-rot]] +- [[dead-context]] +- [[million-token-context-window]] +- [[agent-workflows]] diff --git a/wiki/knowledge/mcp-dev-summit.md b/wiki/knowledge/mcp-dev-summit.md new file mode 100644 index 00000000..af84afaf --- /dev/null +++ b/wiki/knowledge/mcp-dev-summit.md @@ -0,0 +1,91 @@ +# MCP Dev Summit + +**Summary**: Recap of the MCP Dev Summit North America 2026 (April 2-3, New York), where 1,200+ attendees confirmed MCP's production maturity and the June 2026 spec release was announced — bringing stateless transport, hardened async tasks, and enterprise auth via XAA. +**Sources**: long-live-mcp-aqfer.md, long-live-mcp-aws.md +**Last updated**: 2026-04-21 + +--- + +## Event Overview + +The MCP Dev Summit North America 2026 was held April 2–3 at the New York Marriott Marquis, drawing roughly 1,200 attendees — double the prior year's numbers (source: long-live-mcp-aws.md). It was the first summit under the new **Agentic AI Foundation (AAIF)** at the Linux Foundation, after MCP was donated there in December 2025 (source: long-live-mcp-aqfer.md). + +AAIF Director Jim Zemlin drew the CNCF comparison directly from the opening stage: CNCF took about 13 months to become infrastructure currency for cloud-native. MCP did the same in about 13 weeks (source: long-live-mcp-aqfer.md). + +## "MCP Is Not Dead" + +A central theme was refuting the online "MCP is dead" narrative. AWS's **James Hood** delivered a keynote titled "MCP @ Amazon Scale" with a Mark Twain quote: "the reports of my death have been greatly exaggerated." He stated: "There's a flurry of social media posts or articles proclaiming the death of MCP. I can tell you at Amazon, that is not true." (source: long-live-mcp-aqfer.md). + +The enterprise keynote roster confirmed production use at scale: Uber, Duolingo, Datadog, Docker, Nordstrom, Bloomberg, PwC, WorkOS, Workato (source: long-live-mcp-aqfer.md). + +OpenAI's **Nick Cooper** framed MCP's role: "MCP is an API for AI" — specifically designed assuming the consumer is a model, not a human developer. On CLIs versus MCP: "you clearly want both. The most powerful systems combine them." (source: long-live-mcp-aqfer.md). + +As of the summit, MCP exceeds 97 million SDK downloads per month with 170+ AAIF member organizations (source: long-live-mcp-aws.md). + +## June 2026 Roadmap + +MCP co-creator **David Soria Parra (DSP)** laid out three features landing in the June 2026 spec release (source: long-live-mcp-aqfer.md): + +1. **Stateless transport by default** — designed for serverless runtimes like AWS Lambda and Cloudflare Workers (see [[mcp-transport]]) +2. **Hardened long-running tasks** — SEP-1686, for async jobs that take minutes or hours +3. **Enterprise auth via Cross-App Access (XAA)** — moves authentication off bearer tokens to workforce IdP integration + +Beyond June, DSP flagged: Triggers (webhooks for MCP), Native streaming (replacing all-or-nothing tool results), **Skills over MCP** (ship skill libraries alongside MCP servers — see [[mcp-skills-interest-group]]), Interceptors (hooks for observability, policy, and telemetry), and Composability through code (see [[mcp-programmatic-tool-calling]]) (source: long-live-mcp-aqfer.md). + +## The MRTR Breakthrough: Stateless Transport + +**MRTR** (Multi Round-Trip Requests, SEP-2322) is the key change enabling serverless deployments (source: long-live-mcp-aqfer.md): + +> **Today**: A tool call is like a phone call — the client and server stay connected the whole time. This is structurally incompatible with serverless (AWS Lambda, Cloudflare Workers). +> +> **MRTR**: Turns the phone call into an email thread. Each message carries full prior context. When the server needs to ask a question, it closes the thread; the client returns later with a new message including the full prior exchange. Any server can resume — no held connections, no sticky routing, no shared memory. + +This moves features like elicitation, sampling, and long-running tasks from "impossible on serverless" to "just a normal email thread" (source: long-live-mcp-aqfer.md). + +**Cornelia Davis** at Temporal noted: "Doing async over a stateful transport protocol is really tricky. I was over the moon excited this morning when I heard we want to work on a transport protocol that is stateless." (source: long-live-mcp-aqfer.md). + +## SDK Roadmap + +**Max Isbey** at Anthropic announced the SDK roadmap (source: long-live-mcp-aqfer.md): +- TypeScript V2 alpha is available now +- Python V2 beta is Q2 2026 +- Both stable releases ship alongside the June spec + +The architectural headline: a **dispatcher pattern** that cleanly separates MCP semantics from wire format and transport, making pluggable transports practical for the first time. TypeScript V2 runs natively on Cloudflare Workers. **Go** joined TypeScript, Python, and C# in the tier-1 SDK list (source: long-live-mcp-aqfer.md). + +## Authentication: CIMD and XAA + +Two auth mechanisms were announced (source: long-live-mcp-aqfer.md): + +**CIMD** (Client ID Metadata Documents) replaces Dynamic Client Registration (DCR) with DNS-rooted trust. A client hosts a JSON metadata document at a well-known URL — that URL *is* the client ID. No registration table, no silent expiry, no impersonation risk. Claude Code shipped CIMD support two weeks before the summit. + +**Cross-App Access (XAA)** is the enterprise security story. Demoed live across Claude Code and Cursor into a Figma MCP server with zero consent screens visible to the user — XAA lets the workforce IdP handle authorization centrally, ending per-app OAuth prompt fatigue. + +**Aaron Parecki** (co-author of OAuth 2.1) summarized: "Most of OAuth works fine for MCP. It's three specific gaps." Client identification (CIMD). Bootstrap discovery from one URL (Protected Resource Metadata, RFC 9728). Enterprise consent fatigue (XAA) (source: long-live-mcp-aqfer.md). + +## MCP Apps + +MCP Apps is the first official MCP extension, shipped January 26. Servers return tool results pointing at HTML/JS/CSS bundles; the host renders them in a sandboxed iframe; UI and host communicate bidirectionally over postMessage. Within four months, every major host adopted it: Claude, ChatGPT, VS Code with Copilot, Cursor, Goose, Postman (source: long-live-mcp-aqfer.md). + +## Context Bloat: A Client Problem + +DSP addressed the concern that connecting an MCP server with a thousand tools blows out the context window. His answer: "that's a client problem, not a protocol problem." Claude Code already solved it with progressive tool discovery — loading tool definitions only when the model needs them — achieving about 85% reduction in token usage on real workloads (source: long-live-mcp-aqfer.md). See [[context-engineering]] for broader context management principles. + +## Key Themes + +1. **Interoperability**: MCP as "the API for AI" — the backbone of cross-vendor, cross-cloud agentic architectures (source: long-live-mcp-aws.md) +2. **Gateway Pattern**: Nearly all enterprise deployments rely on agent gateways and registries to govern access, security, and scalability (source: long-live-mcp-aws.md) +3. **Open Standards Stack**: AAIF's scope remains focused on agent-to-resource integration; other functions are addressed by companion standards (source: long-live-mcp-aws.md) + +The next North American AAIF event is AGNTCon + MCPCon North America in San Jose on October 22–23 (source: long-live-mcp-aqfer.md). + +## Related pages + +- [[mcp-specification]] +- [[mcp-transport]] +- [[mcp-skills-interest-group]] +- [[mcp-programmatic-tool-calling]] +- [[mcp-typescript-sdk]] +- [[mcp-servers]] +- [[context-engineering]] +- [[agent-best-practices]] diff --git a/wiki/knowledge/mcp-programmatic-tool-calling.md b/wiki/knowledge/mcp-programmatic-tool-calling.md new file mode 100644 index 00000000..a8340fc5 --- /dev/null +++ b/wiki/knowledge/mcp-programmatic-tool-calling.md @@ -0,0 +1,94 @@ +# MCP Programmatic Tool Calling + +**Summary**: MCP's programmatic (code-mode) tool calling allows a model to write a small program that chains multiple tool calls in sequence on the server side, reducing round-trips, improving efficiency, and fitting naturally with the stateless transport work described in [[mcp-transport]]. +**Sources**: long-live-mcp-aqfer.md, mcp-typescript-sdk-github.md, mcp-specification.md +**Last updated**: 2026-04-22 + +--- + +## What Is Programmatic Tool Calling? + +Programmatic tool calling (also called "composability through code") is a planned MCP capability where the model writes a small program that chains several tool calls in sequence **on the server side**, rather than making one tool call at a time over the network (source: long-live-mcp-aqfer.md). + +MCP co-creator David Soria Parra flagged this at the [[mcp-dev-summit]] as part of the 2026 roadmap beyond the June spec release: "Composability through code — lets the model write a small program that chains several tool calls in sequence on the server side." (source: long-live-mcp-aqfer.md) + +## Why It Matters + +The current MCP tool-calling model is sequential and stateful: the client calls a tool, waits for a result, decides what to call next, and repeats. This creates several inefficiencies (source: mcp-http2-http3-reddit.md): + +- **Round-trip latency**: Each tool call requires a network round-trip between the LLM client and the MCP server +- **Context window pressure**: Intermediate tool results must be returned to the LLM context, consuming tokens +- **Stateful connection requirement**: The server must hold the connection open across multiple exchanges + +Programmatic tool calling addresses all three by moving the chaining logic to the server, executing multiple tools locally, and returning only the final result to the client. + +## Relationship to Sampling + +MCP already supports **sampling** — a server-initiated capability where the server can request the LLM to perform a completion mid-execution (source: mcp-specification.md): + +```typescript +// Client-side: request LLM completion from the host +const result = await server.requestSampling({ + messages: [{ role: "user", content: { type: "text", text: "What is 2+2?" } }], + maxTokens: 100, +}); +``` + +Programmatic tool calling inverts this: instead of the server asking the LLM for a completion, the LLM gives the server a program to execute across multiple tools. + +## Relationship to MRTR (Stateless Transport) + +Programmatic tool calling complements [[mcp-transport]]. MRTR makes stateful interactions work on serverless infrastructure by turning "phone calls" into "email threads." Programmatic tool calling reduces the number of turns needed by batching tool invocations on the server side — together they enable powerful multi-step [[agent-workflows]] without holding connections open (source: long-live-mcp-aqfer.md). + +## Context Window Management Connection + +At the [[mcp-dev-summit]], DSP addressed context bloat — the problem that connecting an MCP server with thousands of tools blows out the context window. His answer: "that's a client problem, not a protocol problem." Claude Code already solved it with progressive tool discovery (loading tool definitions only when needed), achieving about 85% reduction in token usage on real workloads (source: long-live-mcp-aqfer.md). + +Programmatic tool calling extends this: once the model has decided on a multi-step action, it can delegate the entire sequence to the server rather than pulling each intermediate result back into context. This is a direct application of [[context-engineering]] principles — only include in context what the model actually needs at each step. + +## Skills Integration + +Programmatic tool calling has a natural relationship with [[mcp-skills-vs-mcp]]. A skill can define a multi-step procedure, and the model can implement that procedure as a server-side program that chains the relevant tools. The skill provides the *what* and *how*; programmatic tool calling provides the *execution efficiency* (source: long-live-mcp-aqfer.md). + +This is also related to the [[mcp-skills-interest-group]] discussions around "script-bearing skills" — skills with embedded code that needs to execute on the server: + +```json +{ + "requires": { + "language": "python3", + "packages": ["requests"], + "min_version": "3.11" + } +} +``` + +## Current Status (April 2026) + +Programmatic tool calling is a **planned feature** on the MCP roadmap beyond the June 2026 spec release. It is not yet available in production (source: long-live-mcp-aqfer.md). + +The [[mcp-typescript-sdk]] already provides the foundation via the dispatcher pattern in V2, which cleanly separates MCP semantics from transport and makes server-side composition more tractable (source: long-live-mcp-aqfer.md). + +For current multi-step tool orchestration, the standard approach is: +1. Use MCP's Sampling primitive for server-to-model interactions +2. Use [[agent-workflows]] patterns with multiple sequential tool calls +3. Structure [[subagents]] to each handle narrowly scoped tasks + +## Security Considerations + +Programmatic tool calling will require careful security design (source: mcp-specification.md): + +- Server-side programs represent a form of code execution on behalf of the model +- Trust model must ensure the LLM-generated program cannot exceed the permissions of the tools it calls +- Users should retain visibility and control — similar to how [[mcp-specification]] requires explicit user consent before invoking any tool + +## Related pages + +- [[mcp-specification]] +- [[mcp-transport]] +- [[mcp-dev-summit]] +- [[mcp-skills-vs-mcp]] +- [[mcp-skills-interest-group]] +- [[mcp-typescript-sdk]] +- [[agent-workflows]] +- [[subagents]] +- [[context-engineering]] diff --git a/wiki/knowledge/mcp-servers.md b/wiki/knowledge/mcp-servers.md new file mode 100644 index 00000000..ef4eaba6 --- /dev/null +++ b/wiki/knowledge/mcp-servers.md @@ -0,0 +1,124 @@ +# MCP Servers + +**Summary**: The MCP servers ecosystem includes official reference implementations maintained by Anthropic, a rich set of community frameworks and clients across multiple languages, and growing enterprise tooling for governance, security, and developer experience. +**Sources**: modelcontextprotocol-servers-github.md, anthropic-mcp-github-topics.md +**Last updated**: 2026-04-22 + +--- + +## Reference Servers + +The `modelcontextprotocol/servers` GitHub repository houses reference MCP server implementations maintained by Anthropic. These demonstrate canonical patterns and serve as the test bed for the [[mcp-specification]] (source: modelcontextprotocol-servers-github.md): + +| Server | Package | Purpose | +|--------|---------|---------| +| **Everything** | `@modelcontextprotocol/server-everything` | Exercises all MCP features — intended for client testing, not production | +| **Fetch** | `@modelcontextprotocol/server-fetch` | Web content fetching via headless Chrome; enables LLMs to retrieve and process web content | +| **Filesystem** | `@modelcontextprotocol/server-filesystem` | Secure file operations with configurable access controls (read/write, create/list/delete directories, search, metadata) | +| **Git** | `@modelcontextprotocol/server-git` | Git repository interaction and automation (file contents, branches, commits) | +| **Memory** | `@modelcontextprotocol/server-memory` | Local knowledge graph backed by a JSON file; stores memories as entities and relations | +| **Sequential Thinking** | `@modelcontextprotocol/server-sequential-thinking` | Dynamic, reflective problem-solving through structured step-by-step thinking | +| **Time** | `@modelcontextprotocol/server-time` | Time and timezone conversions | + +Each reference server follows MCP's core security principle: servers should request only the permissions they need (source: modelcontextprotocol-servers-github.md). + +## Server Frameworks + +Build your own MCP server using these community-maintained frameworks (source: modelcontextprotocol-servers-github.md): + +| Name | Language | Description | +|------|----------|-------------| +| FastMCP | Python | Fast, Pythonic MCP server building with simple API | +| FastMCP | TypeScript | High-level TypeScript framework | +| Foxy Contexts | Golang | Declarative MCP servers in Go | +| Genkit MCP | TypeScript | Genkit ↔ MCP integration | +| LiteMCP | TypeScript | Lightweight MCP server framework | +| mark3labs/mcp-go | Golang | Golang MCP implementation | +| mcp-framework | TypeScript | MCP framework for building servers | +| mcp-proxy | TypeScript | Wraps stdio servers with an SSE server | +| mcp-rs-template | Rust | MCP CLI server template for Rust | +| Quarkus MCP Server | Java | Quarkus framework integration | +| Spring AI MCP | Java | Java+Spring AI MCP client and server | + +## Client Frameworks + +Frameworks that consume MCP servers as clients (source: modelcontextprotocol-servers-github.md): + +| Name | Language | Description | +|------|----------|-------------| +| LangChain MCP | Python | LangChain MCP adapters | +| LangChain.js MCP | TypeScript | LangChain MCP adapters for JavaScript | +| mastra | TypeScript | TypeScript agent framework with MCP support | +| mcp-agent | Python | Framework to build AI agents | +| Semantic Kernel MCP | C# | Semantic Kernel integration | +| smolagents | Python | HuggingFace smolagents with MCP tooling | +| strands | Python | AWS Strands agent framework | +| vercel-ai-sdk | TypeScript | Vercel AI SDK with MCP toolset support | + +## MCP Clients (Applications) + +Applications with native MCP support (source: modelcontextprotocol-servers-github.md): + +- **Claude Desktop** — Anthropic's desktop client +- **Claude.ai** — Web interface with remote MCP server support +- **Cline** — Open source AI coding assistant (VS Code extension) +- **Continue** — Open-source AI code assistant +- **Cursor** — AI-powered code editor +- **Windsurf** — AI coding environment by Codeium +- **Zed** — Code editor with built-in MCP support +- **GitHub Copilot** — with MCP extension points + +## SDKs Across Languages + +The MCP ecosystem provides official SDKs in multiple languages (source: modelcontextprotocol-servers-github.md): + +TypeScript, Python, Java, Kotlin, C#, **Go** (added to tier-1 at [[mcp-dev-summit]]), Rust, Swift. + +See [[mcp-typescript-sdk]] for the TypeScript SDK reference. + +## Community Projects: Patterns and Innovation + +The `anthropic-mcp` GitHub topic surfaces notable community projects that illustrate how the ecosystem is extending beyond the official reference implementations (source: anthropic-mcp-github-topics.md): + +| Project | Pattern | +|---------|---------| +| **McpVanguard** | Security proxy: policy enforcement, rate limiting, audit logging sitting between MCP clients and servers | +| **mcpruntime** | Runtime framework: process manager for multiple MCP servers with health checks and restart policies | +| **dexterity** | Skill composition layer: DSL for composing multi-step skills from MCP tools with versioning | +| **mcp-explorer** | Interactive browser: connect to any MCP server, browse tools/resources/prompts visually, test in real time | +| **grounding-ai** | Grounding framework: uses MCP resource access to inject authoritative data into LLM context, reducing hallucination | +| **paki-curator** | Full agentic workflow: scheduled curation using MCP for both tool access and skill delivery | +| **claudeskills** | Community skills library: 50+ curated skills implemented as MCP prompts, installable via a companion server | + +(source: anthropic-mcp-github-topics.md) + +## Emerging Ecosystem Patterns + +Observations from the community (source: anthropic-mcp-github-topics.md): + +1. **Security is a top concern** — proxies, audit logs, and policy enforcement are being built independently because they're not yet standard in the spec +2. **Skills are gaining traction** — multiple repos use the ideas discussed in [[mcp-skills-vs-mcp]] as a first-class concept beyond MCP's "prompts" primitive +3. **Developer tooling is maturing** — the ecosystem has moved from "how do I build an MCP server?" to "how do I operate, debug, and govern multiple MCP servers in production?" +4. **Grounding as a pattern** — using MCP resource access for factual grounding is being pioneered by the community + +## Utilities + +| Tool | Description | +|------|-------------| +| MCP Inspector | Interactive debugging tool for MCP servers | +| mcptools | CLI for MCP server management | +| mcp-get | CLI for installing and managing MCP servers | +| mcp-manager | CLI for managing MCP server configurations | + +(source: modelcontextprotocol-servers-github.md) + +## Related pages + +- [[mcp-specification]] +- [[mcp-typescript-sdk]] +- [[mcp-transport]] +- [[mcp-skills-vs-mcp]] +- [[mcp-skills-interest-group]] +- [[mcp-dev-summit]] +- [[agent-best-practices]] +- [[claude-code-skills]] diff --git a/wiki/knowledge/mcp-skills-interest-group.md b/wiki/knowledge/mcp-skills-interest-group.md new file mode 100644 index 00000000..5c8c1f0b --- /dev/null +++ b/wiki/knowledge/mcp-skills-interest-group.md @@ -0,0 +1,165 @@ +# MCP Skills Interest Group + +**Summary**: The Skills Over MCP Interest Group (est. February 2025) is a cross-vendor working group formalizing standards for delivering skills (reusable agent instructions) over the MCP Prompts primitive, addressing agent ignoring of skills, discovery via `/.well-known/agent-skills/index.json`, lazy loading, and file-system dependencies. +**Sources**: skills-over-mcp-meeting-notes-2248.md, skills-over-mcp-office-hours-2460.md +**Last updated**: 2026-04-21 + +--- + +## Background and Mandate + +Anthropic's MCP 1.0 specification defines "Prompts" as a primitive — but the name is misleading and the feature has been underexplored. In practice, MCP prompts are best understood as **Skills**: reusable, named, structured instructions for how an agent should perform specific tasks using one or more tools (source: skills-over-mcp-meeting-notes-2248.md). + +The group's mandate (source: skills-over-mcp-meeting-notes-2248.md): +1. Define conventions, URI schemes, discovery mechanisms, lifecycle policies, and agent interaction patterns for skills over MCP +2. Address the cross-vendor interoperability gap for skill definitions +3. Solve the agent-ignoring-skills problem observed in production deployments + +## Inaugural Meeting (February 13, 2025) + +Attendees from Anthropic, AWS, Databricks, Stacklok, Astronomer, Vercel, Magic, and independent contributors (source: skills-over-mcp-meeting-notes-2248.md). + +### Core Problem: Agents Ignore Skills + +Production deployments show that when given both skill definitions and tools, most agents (LLMs) skip the skills and attempt to accomplish tasks by chaining raw tool calls directly. This produces three problems (source: skills-over-mcp-meeting-notes-2248.md): + +- **Correctness**: Agents make tool-use mistakes that skills were designed to prevent +- **Efficiency**: Agents take more steps than necessary +- **Predictability**: Results vary across runs and models + +Root causes identified: +1. No instruction to check skills before tool use in most system prompts +2. Skills appear as long text blobs with no structured metadata, making them "invisible" to attention mechanisms +3. No mechanism for agents to "discover" which skills apply to a given task + +### Skills-as-Instructors vs. Skills-as-Helpers + +The meeting's most substantive discussion produced two mental models (source: skills-over-mcp-meeting-notes-2248.md): + +**Skills-as-Instructors**: A skill is a *standing instruction* the agent must follow. Skills define policies, guardrails, workflows, and standard operating procedures. The skill is "in charge." + +**Skills-as-Helpers**: A skill is a *reference document* the agent can consult but is not required to follow. Skills provide context, tips, and best practices. The skill is "advisory." + +Working group consensus: both models are valid. Skills over MCP should support both modes via metadata flags — e.g., `"enforcement": "required"` vs. `"enforcement": "advisory"` (source: skills-over-mcp-meeting-notes-2248.md). + +This distinction maps to the broader [[agent-best-practices]] question of how much autonomy agents should have. + +### Cross-Vendor Interoperability + +Participants flagged the risk of skills becoming a "vendor moat" if each LLM vendor (Anthropic, AWS, Google) interprets skill definitions differently. The group agreed to draft an interoperability requirements document and a shared, vendor-neutral JSON schema (source: skills-over-mcp-meeting-notes-2248.md). + +### Skill Versioning + +Skills need version identifiers (semver preferred). Clients that cache old skill definitions must know when content is stale — ETag-style invalidation was proposed but not decided (source: skills-over-mcp-meeting-notes-2248.md). + +### Extending Prompts vs. New Primitive + +Working group leaning: extend the existing Prompts primitive via convention (metadata, naming, discovery) — *not* a new MCP primitive — to maintain spec compatibility without requiring server updates (source: skills-over-mcp-meeting-notes-2248.md). + +## Office Hours (March 24, 2025) + +Second meeting: open community office hours covering four agenda items (source: skills-over-mcp-office-hours-2460.md). + +### Decision: URI Scheme for Discovery + +The group adopted **`/.well-known/agent-skills/index.json`** as the primary discovery mechanism, following established HTTP well-known conventions (RFC 8615) (source: skills-over-mcp-office-hours-2460.md). + +**Index format** — a lightweight array of objects: +```json +[ + { + "id": "deploy-to-staging", + "name": "Deploy Service to Staging", + "description": "Deploy any microservice to the staging environment", + "version": "1.2.0", + "href": "/skills/deploy-to-staging.md", + "ttl": 300 + } +] +``` + +**Why not `skill://` custom URIs?** Requires all clients to implement custom resolution logic, raising the adoption barrier (source: skills-over-mcp-office-hours-2460.md). + +**For STDIO servers**: expose skills via `prompts/list` RPC with a convention that entries may include `"uri"` metadata pointing to a local file path (source: skills-over-mcp-office-hours-2460.md). + +### Lazy Loading Pattern + +Large MCP deployments may expose 100+ skills. The adopted lazy loading pattern (source: skills-over-mcp-office-hours-2460.md): + +``` +Client connects to MCP server + ↓ +Fetch /.well-known/agent-skills/index.json (lightweight: id + name + description) + ↓ +Agent reads task / user intent + ↓ +Agent searches index to identify relevant skills + ↓ +Agent fetches full skill documents for top candidates (on demand) + ↓ +Agent follows skill instructions when executing task +``` + +Key design decisions: +- **Index TTL**: include a `"ttl"` field (seconds); default suggested 300 seconds +- **Full skill caching**: use ETag headers (HTTP) or version identifiers (MCP RPC) +- **Search**: keyword/substring matching initially; vector/semantic search is out of scope for v1 +- **Limitation**: lazy loading does not automatically solve agents ignoring skills — system prompt instructions to check skills are still required (source: skills-over-mcp-office-hours-2460.md) + +This addresses [[context-engineering]] concerns: loading all skills at once is expensive in tokens, latency, and context window pressure. + +### Script-Bearing Skills and Trust + +Skills with embedded code (bash, Python, etc.) must declare a `"requires"` block (source: skills-over-mcp-office-hours-2460.md): + +```json +{ + "requires": { + "language": "python3", + "packages": ["requests", "pydantic"], + "min_version": "3.11" + } +} +``` + +Trust model decisions: +1. **Pure text skills** are the default and most common case +2. **Script-bearing skills** from untrusted servers must not execute without explicit user confirmation +3. **Sandboxing** is required — containers or virtual environments; the agent framework is responsible (source: skills-over-mcp-office-hours-2460.md) + +### File System Dependency Problem + +Many skills reference local files (e.g., `./docs/conventions.md`). When served remotely, these break. Three failure modes identified (source: skills-over-mcp-office-hours-2460.md): + +1. **Absolute paths**: hardcoded paths like `/Users/jack/project/README.md` fail for every other user +2. **Relative paths**: ambiguous — relative to the server? the client? the agent's working directory? +3. **Silent degradation**: skills designed for local context fail silently when run from a cloud MCP server + +**Adopted approach** (source: skills-over-mcp-office-hours-2460.md): +- **Long-term**: use MCP Resource URIs (e.g., `mcp://server-name/resources/schema.json`) instead of file system paths — servers expose files as resources, making them accessible remotely +- **Short-term**: skill scope declaration — `"local-only"`, `"remote-safe"`, or `"conditional"` — so clients can filter appropriately + +See [[mcp-specification]] for the Resources primitive that enables this pattern. + +## Open Questions Carried Forward + +- Should skill enforcement be opt-in or opt-out? +- How to handle skill conflicts when two MCP servers provide skills with the same name? +- Are skills "owned" by a server, or can they be user-level (stored in client)? +- Should skill index search be standardized (query language, filters)? +- How does skill enforcement interact with multi-agent setups and [[subagents]]? (source: skills-over-mcp-meeting-notes-2248.md, skills-over-mcp-office-hours-2460.md) + +## Connection to the MCP 2026 Roadmap + +At the [[mcp-dev-summit]], MCP co-creator David Soria Parra flagged "Skills over MCP" as a planned roadmap item beyond the June 2026 spec release — giving a way to ship skill libraries alongside MCP servers (source: long-live-mcp-aqfer.md). + +## Related pages + +- [[mcp-skills-vs-mcp]] +- [[claude-code-skills]] +- [[mcp-specification]] +- [[mcp-servers]] +- [[mcp-dev-summit]] +- [[context-engineering]] +- [[agent-best-practices]] +- [[subagents]] diff --git a/wiki/knowledge/mcp-skills-vs-mcp.md b/wiki/knowledge/mcp-skills-vs-mcp.md new file mode 100644 index 00000000..07d70ed6 --- /dev/null +++ b/wiki/knowledge/mcp-skills-vs-mcp.md @@ -0,0 +1,148 @@ +# MCP Skills vs MCP + +**Summary**: Skills and MCP are complementary layers — Skills encode *how* to do things (static institutional knowledge as markdown instructions), while MCP encodes the *ability* to do things (live authenticated service access) — and most enterprise agents need both. +**Sources**: skills-vs-mcp-speakeasy.md +**Last updated**: 2026-04-21 + +--- + +## The Core Distinction + +Skills and MCP are not competing technologies. They solve different problems at different layers of an agentic stack (source: skills-vs-mcp-speakeasy.md): + +- **Skills = Institutional knowledge, encoded** — the *how* +- **MCP = Authenticated capability access** — the *ability* + +Without skills, an agent with MCP access is like a brilliant intern with system access but no onboarding — they can do things, but they don't know your organization's way of doing them. Without MCP, an agent with skills is like a highly knowledgeable consultant who can't access any of your systems (source: skills-vs-mcp-speakeasy.md). + +## What Are Skills? + +Skills are static, reusable instructions that tell an agent how to perform specific tasks. They are sophisticated system prompt components encoding (source: skills-vs-mcp-speakeasy.md): + +- Step-by-step workflows +- Best practices and patterns +- Error handling procedures +- Domain knowledge +- Team conventions + +**Skills are fundamentally knowledge artifacts.** They don't execute code or make API calls — they tell the agent how to use the tools it already has. Think of skills as your team's institutional knowledge, distilled into machine-readable form (source: skills-vs-mcp-speakeasy.md). + +See [[claude-code-skills]] for the Claude Code implementation of this concept. + +A skill example: + +```markdown +# Deploy Service to Staging +## Prerequisites +- kubectl access configured for staging cluster +## Steps +1. Verify: `kubectl get deployment {service-name} -n staging` +2. Update: `kubectl set image deployment/{service-name} ...` +3. Watch: `kubectl rollout status deployment/{service-name} -n staging` +4. Verify health: `curl https://staging.internal/{service-name}/health` +``` + +## What Is MCP? + +MCP (Model Context Protocol) gives agents *capability* — the ability to interact with external systems, APIs, and data sources. Through MCP, agents can (source: skills-vs-mcp-speakeasy.md): + +- **Read and write data**: Query databases, read files, update records +- **Call APIs**: Make authenticated requests to external services +- **Execute actions**: Run commands, trigger workflows, deploy changes + +**MCP is fundamentally about capability access.** It provides the authenticated, governed connections that let agents move from thinking to doing (source: skills-vs-mcp-speakeasy.md). See [[mcp-specification]] for the full protocol definition. + +## The Two-Layer Architecture + +Skills and MCP work best together in a two-layer architecture (source: skills-vs-mcp-speakeasy.md): + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ AI AGENT │ +│ │ +│ ┌─────────────────────┐ ┌────────────────────────────────┐ │ +│ │ SKILLS │ │ MCP │ │ +│ │ (How to do things) │ │ (Ability to do things) │ │ +│ │ │ │ │ │ +│ │ • Workflows │ │ • Database connections │ │ +│ │ • Best practices │ │ • API access │ │ +│ │ • Domain knowledge │ │ • File system access │ │ +│ │ • Team conventions │ │ • External service calls │ │ +│ │ • Error handling │ │ • Authenticated operations │ │ +│ └─────────────────────┘ └────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────┘ +``` + +**Skills tell the agent what steps to follow. MCP gives the agent the tools to execute those steps.** (source: skills-vs-mcp-speakeasy.md) + +## Real-World Example: Customer Support Agent + +A refund request agent demonstrates the two layers clearly (source: skills-vs-mcp-speakeasy.md): + +**The MCP Layer provides:** +- Access to the orders database +- Connection to the payment processing API +- Ability to update customer records +- Integration with the ticketing system + +**The Skills Layer provides:** +- "Refund Request Handling" skill: check order age, verify payment, calculate amount, apply retention offer for high-value customers, process refund, send confirmation, update ticket + +The agent uses the skill to know *how* to handle refunds, and uses MCP to actually *execute* each step. + +## Decision Framework + +| Question | If Yes, Use... | +|----------|----------------| +| Does the agent need institutional knowledge? | Skills | +| Does the agent need access to live data? | MCP | +| Are you encoding team workflows? | Skills | +| Are you connecting to external APIs? | MCP | +| Do you need auditable access trails? | MCP | +| Are you encoding domain expertise? | Skills | +| Do you need authentication/authorization? | MCP | +| Are you building reusable procedures? | Skills | + +**For most enterprise agents: use both.** (source: skills-vs-mcp-speakeasy.md) + +## Implementing the Two-Layer Architecture + +**Step 1: Define your MCP servers** — what systems does your agent need access to? + +```typescript +const server = new McpServer({ name: "company-tools", version: "1.0.0" }); +server.tool("get_order", { orderId: z.string() }, async ({ orderId }) => { + const order = await db.orders.findById(orderId); + return { content: [{ type: "text", text: JSON.stringify(order) }] }; +}); +``` + +**Step 2: Write skills for your domain** — what does your agent need to know? (markdown files) + +**Step 3: Wire them together** — serve skills from your MCP server via the Prompts primitive: + +```typescript +server.prompt("refund_handling", () => ({ + messages: [{ role: "user", content: { type: "text", text: fs.readFileSync("./skills/refund-handling.md", "utf-8") } }] +})); +``` + +(source: skills-vs-mcp-speakeasy.md) + +## Common Misconceptions + +- **"Skills are just system prompts"** — Skills are more structured and reusable; they can be versioned, shared, and loaded conditionally. +- **"MCP replaces the need for skills"** — MCP gives capabilities; skills provide judgment about when and how to use those capabilities. +- **"I should choose one or the other"** — They solve different problems; you almost always want both. (source: skills-vs-mcp-speakeasy.md) + +See [[mcp-skills-interest-group]] for the working group standardizing skills delivery over MCP, and [[agent-best-practices]] for broader agent design guidance. + +## Related pages + +- [[mcp-specification]] +- [[mcp-skills-interest-group]] +- [[claude-code-skills]] +- [[mcp-typescript-sdk]] +- [[agent-best-practices]] +- [[agent-workflows]] +- [[context-engineering]] diff --git a/wiki/knowledge/mcp-specification.md b/wiki/knowledge/mcp-specification.md new file mode 100644 index 00000000..7f2475f4 --- /dev/null +++ b/wiki/knowledge/mcp-specification.md @@ -0,0 +1,95 @@ +# MCP Specification + +**Summary**: The Model Context Protocol (MCP) is an open protocol standardizing how LLM applications integrate with external data sources and tools via JSON-RPC 2.0, inspired by the Language Server Protocol. +**Sources**: mcp-specification.md, about-mcp-github-docs.md +**Last updated**: 2026-04-21 + +--- + +## Overview + +The Model Context Protocol (MCP) is an open protocol that enables seamless integration between LLM applications and external data sources and tools (source: mcp-specification.md). It is designed for building AI-powered IDEs, chat interfaces, and custom AI workflows — providing a standardized way to connect LLMs with the [[context-engineering]] they need. + +MCP takes inspiration from the [Language Server Protocol](https://microsoft.github.io/language-server-protocol/), which standardized how to add programming language support across a development tooling ecosystem. In a similar way, MCP standardizes how to integrate additional context and tools into the AI application ecosystem (source: mcp-specification.md). + +The authoritative protocol definition is based on the TypeScript schema at `schema.ts` in the official specification repository (source: mcp-specification.md). + +## Architecture: Hosts, Clients, and Servers + +MCP uses JSON-RPC 2.0 messages to establish communication between three roles (source: mcp-specification.md): + +- **Hosts**: LLM applications that initiate connections (e.g., Claude Desktop, Cursor, GitHub Copilot CLI) +- **Clients**: Connectors within the host application that speak the MCP protocol +- **Servers**: Services that provide context and capabilities to the host + +This client-server architecture means one agent can connect to many MCP servers, each exposing different tools and data sources (source: mcp-vs-a2a-dzone.md). + +## Server-Side Primitives + +Servers offer three categories of features to clients (source: mcp-specification.md): + +- **Resources**: Context and data — for the user or the AI model to use as background knowledge +- **Prompts**: Templated messages and workflows for users (also called [[claude-code-skills]] in the skills layer) +- **Tools**: Functions the AI model can call to execute actions + +## Client-Side Primitives + +Clients may offer the following features to servers (source: mcp-specification.md): + +- **Sampling**: Server-initiated agentic behaviors and recursive LLM interactions +- **Roots**: Server-initiated inquiries into URI or filesystem boundaries +- **Elicitation**: Server-initiated requests for additional information from users + +Additional utilities include configuration, progress tracking, cancellation, error reporting, and logging (source: mcp-specification.md). + +## Protocol Characteristics + +- **Message format**: JSON-RPC 2.0 (source: mcp-specification.md) +- **Connection model**: Stateful connections (evolving toward stateless — see [[mcp-transport]]) +- **Capability negotiation**: Server and client negotiate capabilities at connection time +- **Governance**: Donated to the Linux Foundation's Agentic AI Foundation (AAIF) in December 2025 (source: long-live-mcp-aqfer.md) + +## Security and Trust + +MCP enables arbitrary data access and code execution paths. The specification defines four key principles (source: mcp-specification.md): + +1. **User Consent and Control** — users must explicitly consent to data access and operations +2. **Data Privacy** — hosts must obtain user consent before exposing data to servers +3. **Tool Safety** — tools represent arbitrary code execution; hosts must obtain explicit user consent before invoking any tool +4. **LLM Sampling Controls** — users must explicitly approve sampling requests + +Implementors are recommended to build robust consent flows, provide clear security documentation, and implement appropriate access controls (source: mcp-specification.md). + +## Availability and Adoption + +MCP is supported across (source: about-mcp-github-docs.md): + +- **IDEs**: VS Code, JetBrains, Xcode, Cursor, Windsurf — supporting local and increasingly remote MCP servers +- **Copilot CLI**: Both local and remote MCP servers, with the GitHub MCP server built in +- **Copilot cloud agent**: Repository-level MCP server configuration + +As of April 2026, MCP exceeds 97 million SDK downloads per month with 170+ AAIF member organizations (source: long-live-mcp-aws.md). + +## GitHub MCP Server + +GitHub provides and maintains its own MCP server that can (source: about-mcp-github-docs.md): + +- Automate and streamline code-related tasks +- Connect third-party tools to GitHub's context and AI capabilities +- Enable cloud-based workflows without local setup +- Invoke GitHub tools such as Copilot cloud agent and code scanning + +The GitHub MCP server supports **toolsets**: groups of functionalities that can be enabled or disabled to improve tool selection accuracy and reduce context window usage (source: about-mcp-github-docs.md). + +## Related pages + +- [[mcp-transport]] +- [[mcp-servers]] +- [[mcp-typescript-sdk]] +- [[mcp-skills-vs-mcp]] +- [[mcp-skills-interest-group]] +- [[mcp-vs-a2a]] +- [[agent-to-agent-protocol]] +- [[mcp-programmatic-tool-calling]] +- [[claude-code-skills]] +- [[context-engineering]] diff --git a/wiki/knowledge/mcp-transport.md b/wiki/knowledge/mcp-transport.md new file mode 100644 index 00000000..0e19a61a --- /dev/null +++ b/wiki/knowledge/mcp-transport.md @@ -0,0 +1,125 @@ +# MCP Transport + +**Summary**: MCP supports two official transport mechanisms — STDIO for local deployments and Streamable HTTP for remote deployments — with a June 2026 roadmap bringing stateless transport (MRTR) that unlocks serverless environments like AWS Lambda and Cloudflare Workers. +**Sources**: mcp-http2-http3-reddit.md, long-live-mcp-aqfer.md, mcp-typescript-sdk-github.md +**Last updated**: 2026-04-22 + +--- + +## Overview + +The [[mcp-specification]] officially supports two transports, matching the two primary deployment contexts (source: mcp-http2-http3-reddit.md): + +- **STDIO** — for local deployments (process-to-process communication) +- **Streamable HTTP** — for remote deployments over the network + +Custom transports are supported for teams with specialized requirements, including potential HTTP/3 implementations. + +## STDIO Transport + +STDIO is the original MCP transport. The client and server communicate via standard input/output streams — ideal for local tool integrations where the MCP server runs as a child process (source: mcp-http2-http3-reddit.md). + +**Best for**: Local IDE integrations, CLI tools, prototyping. + +```typescript +import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js"; +const transport = new StdioServerTransport(); +await server.connect(transport); +``` + +STDIO sidesteps all HTTP transport requirements and is the recommended starting point for prototyping or local tools (source: mcp-http2-http3-reddit.md). + +## Streamable HTTP Transport + +Streamable HTTP enables remote MCP deployments. It was a significant step forward from STDIO, unlocking cloud-hosted MCP servers (source: mcp-http2-http3-reddit.md). + +**Current state**: +- Operates over HTTP/1.1 and HTTP/2 +- Full MCP streaming effectively requires HTTP/2 or higher for correct full-duplex operation +- HTTP/1.1 works but only gives synchronous request/response — no true streaming or mid-request notifications +- HTTP/3 is being actively explored but is not part of the official specification (source: mcp-http2-http3-reddit.md) + +**Community best practice**: If you want full MCP capability (streaming, notifications, interactive/incremental workflows), deploy your server with HTTP/2 enabled (source: mcp-http2-http3-reddit.md). + +```typescript +import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js"; + +app.post("/mcp", async (req, res) => { + const transport = new StreamableHTTPServerTransport({ sessionIdGenerator: undefined }); + await server.connect(transport); + await transport.handleRequest(req, res, req.body); + res.on("finish", () => server.close()); +}); +``` + +## Current Challenges with Stateful Transport + +The current stateful model creates real-world friction for enterprise deployments (source: mcp-http2-http3-reddit.md): + +- **Scaling friction**: Stateful connections force "sticky" routing that pins traffic to specific servers, preventing effective auto-scaling +- **Serverless incompatibility**: AWS Lambda and Cloudflare Workers spin up per-request; long-held connections cannot be maintained +- **High barrier for simple tools**: Developers building simple ephemeral tools are required to manage complex backend storage +- **Infrastructure complexity**: Load balancers must parse full JSON-RPC payloads to route traffic instead of using standard HTTP patterns + +## The June 2026 Roadmap: Stateless Transport + +The MCP Transport Working Group published a December 2025 roadmap describing the evolution toward stateless transport (source: mcp-http2-http3-reddit.md). These changes were confirmed at the [[mcp-dev-summit]]. + +### MRTR: Multi Round-Trip Requests (SEP-2322) + +The most important change for serverless deployments (source: long-live-mcp-aqfer.md): + +> **Today**: A tool call is like a phone call — the client and server stay on the line. If the connection drops, the call is lost. This is structurally incompatible with serverless runtimes. +> +> **MRTR**: Turns the phone call into an email thread. Each message carries the full context of prior messages. The server can ask a question and close the thread; the client returns later — five seconds or five minutes — with a new message including the prior exchange. Any server can pick up the new message. No held connections, no sticky routing, no shared memory. + +MRTR moves features like elicitation, sampling, and long-running tasks from "impossible on serverless" to "just a normal email thread" (source: long-live-mcp-aqfer.md). + +### Stateless Protocol Design + +Replacing the `initialize` handshake with per-request/response shared information. A `discovery` mechanism lets clients query server capabilities when needed (source: mcp-http2-http3-reddit.md). + +### Session Elevation + +Moving sessions from an implicit side-effect of transport connection to the **data model layer** — explicit sessions, similar to how HTTP uses cookies and tokens for stateful semantics over a stateless protocol (source: mcp-http2-http3-reddit.md). + +### Server Cards + +Structured metadata documents exposed at `/.well-known/mcp.json`. Server Cards enable clients to discover server capabilities, authentication requirements, and available primitives *before* establishing a connection — reducing the cost of discovery (source: mcp-http2-http3-reddit.md). + +### Subscription Streams + +Replacing the general-purpose `GET` stream with explicit subscription streams. Clients open dedicated streams for specific items they want to monitor. TTL values and ETags enable intelligent client-side caching (source: mcp-http2-http3-reddit.md). + +### JSON-RPC Envelope Improvements + +Exposing routing-critical information (RPC method, tool name) via standard HTTP paths or headers — allowing load balancers and API gateways to route traffic without parsing JSON bodies (source: mcp-http2-http3-reddit.md). + +## SDK V2: Dispatcher Pattern + +The [[mcp-typescript-sdk]] introduces a **dispatcher pattern** that cleanly separates MCP semantics from wire format and transport, making pluggable transports practical for the first time. TypeScript V2 runs natively on Cloudflare Workers (source: long-live-mcp-aqfer.md). + +## HTTP/3 Status + +HTTP/3 support is being actively explored but is not yet part of the official specification. No mainstream MCP servers or clients rely on HTTP/3 yet, but several vendors are experimenting. Custom transport implementations (including HTTP/3) are supported by the SDK architecture (source: mcp-http2-http3-reddit.md). + +## Developer Guidance by Scenario + +| Scenario | Recommendation | +|----------|---------------| +| Local prototyping | STDIO — simple, no HTTP requirements | +| Remote (current) | Streamable HTTP with HTTP/2 enabled | +| Serverless (current) | Wait for June 2026 MRTR spec release | +| Serverless (after June 2026) | Streamable HTTP with MRTR support | +| Custom transport (e.g., HTTP/3) | Use SDK's pluggable transport API | + +(source: mcp-http2-http3-reddit.md) + +## Related pages + +- [[mcp-specification]] +- [[mcp-typescript-sdk]] +- [[mcp-dev-summit]] +- [[mcp-servers]] +- [[mcp-programmatic-tool-calling]] +- [[agent-workflows]] diff --git a/wiki/knowledge/mcp-typescript-sdk.md b/wiki/knowledge/mcp-typescript-sdk.md new file mode 100644 index 00000000..10166c61 --- /dev/null +++ b/wiki/knowledge/mcp-typescript-sdk.md @@ -0,0 +1,197 @@ +# MCP TypeScript SDK + +**Summary**: The official TypeScript SDK for MCP (`@modelcontextprotocol/sdk`) provides complete client and server implementations with STDIO and Streamable HTTP transports, Zod-based validation, OAuth 2.1 auth helpers, and middleware for Express and Hono. +**Sources**: mcp-typescript-sdk-github.md +**Last updated**: 2026-04-22 + +--- + +## Overview + +The TypeScript SDK is the official implementation of the [[mcp-specification]] for Node.js and TypeScript environments, maintained by Anthropic. It implements the full MCP specification, providing (source: mcp-typescript-sdk-github.md): + +- Client and server implementations +- Protocol types and validation (with Zod) +- Transport implementations: STDIO and Streamable HTTP +- Middleware integrations for Express and Hono +- Full TypeScript type safety + +## Version Status + +> ⚠️ The `main` branch now tracks the pre-alpha **V2 SDK** which has breaking changes from V1. For stable production use, see the `v1` branch. The npm package currently ships V1. V2 pre-alpha is available as `@modelcontextprotocol/sdk@next`. + +V2 was announced at the [[mcp-dev-summit]] by Max Isbey at Anthropic — TypeScript V2 alpha is available, stable release ships alongside the June 2026 spec (source: mcp-typescript-sdk-github.md). + +## Installation + +```bash +npm install @modelcontextprotocol/sdk +# V2 pre-alpha: +npm install @modelcontextprotocol/sdk@next +``` + +## Quick Start: Server + +```typescript +import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js"; +import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js"; +import { z } from "zod"; + +const server = new McpServer({ name: "Demo", version: "1.0.0" }); + +server.tool("add", { a: z.number(), b: z.number() }, async ({ a, b }) => ({ + content: [{ type: "text", text: String(a + b) }], +})); + +const transport = new StdioServerTransport(); +await server.connect(transport); +``` + +(source: mcp-typescript-sdk-github.md) + +## Quick Start: Client + +```typescript +import { Client } from "@modelcontextprotocol/sdk/client/index.js"; +import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js"; + +const client = new Client({ name: "example-client", version: "1.0.0" }); +const transport = new StdioClientTransport({ command: "node", args: ["server.js"] }); + +await client.connect(transport); +const tools = await client.listTools(); +const result = await client.callTool({ name: "add", arguments: { a: 1, b: 2 } }); +await client.close(); +``` + +(source: mcp-typescript-sdk-github.md) + +## Server Primitives + +### Tools + +```typescript +server.tool("get-weather", { city: z.string() }, async ({ city }) => { + const weather = await fetchWeather(city); + return { content: [{ type: "text", text: `Weather in ${city}: ${weather}` }] }; +}); +``` + +### Resources + +```typescript +// Static resource +server.resource("config", "config://app", async (uri) => ({ + contents: [{ uri: uri.href, text: "App configuration here" }], +})); + +// Dynamic resource with template +server.resource( + "user-profile", + new ResourceTemplate("users://{userId}/profile", { list: undefined }), + async (uri, { userId }) => ({ + contents: [{ uri: uri.href, text: `Profile for user ${userId}` }], + }) +); +``` + +### Prompts (Skills delivery) + +```typescript +server.prompt("review-code", { code: z.string() }, ({ code }) => ({ + messages: [{ role: "user", content: { type: "text", text: `Please review this code:\n\n${code}` } }], +})); +``` + +The Prompts primitive is one delivery mechanism discussed in [[mcp-skills-vs-mcp]] (source: mcp-typescript-sdk-github.md). + +## Transport Options + +See [[mcp-transport]] for the full transport conceptual overview. + +### STDIO Transport (local deployments) + +```typescript +import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js"; +const transport = new StdioServerTransport(); +await server.connect(transport); +``` + +### Streamable HTTP Transport (remote deployments) + +```typescript +import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js"; + +app.post("/mcp", async (req, res) => { + const transport = new StreamableHTTPServerTransport({ sessionIdGenerator: undefined }); + await server.connect(transport); + await transport.handleRequest(req, res, req.body); + res.on("finish", () => server.close()); +}); +``` + +(source: mcp-typescript-sdk-github.md) + +## Middleware Integrations + +```typescript +// Express +import { createExpressMcpMiddleware } from "@modelcontextprotocol/sdk/middleware/express.js"; +app.use("/mcp", createExpressMcpMiddleware(server)); + +// Hono (works natively on Cloudflare Workers) +import { createHonoMcpMiddleware } from "@modelcontextprotocol/sdk/middleware/hono.js"; +app.use("/mcp/*", createHonoMcpMiddleware(server)); +``` + +TypeScript V2 runs natively on Cloudflare Workers (source: mcp-typescript-sdk-github.md). + +## Authentication + +The SDK includes OAuth 2.1-compliant authentication helpers via `OAuthServerProvider`. This enables implementing CIMD (Client ID Metadata Documents) and XAA (Cross-App Access) — auth mechanisms announced at the [[mcp-dev-summit]] (source: mcp-typescript-sdk-github.md). + +## Error Handling + +```typescript +import { McpError, ErrorCode } from "@modelcontextprotocol/sdk/types.js"; + +server.tool("my-tool", { id: z.string() }, async ({ id }) => { + const result = await fetchById(id); + if (!result) throw new McpError(ErrorCode.InvalidRequest, `Item not found: ${id}`); + return { content: [{ type: "text", text: JSON.stringify(result) }] }; +}); +``` + +(source: mcp-typescript-sdk-github.md) + +## Testing + +The SDK includes in-memory transport utilities for testing MCP servers without real transport: + +```typescript +import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js"; + +const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair(); +await server.connect(serverTransport); +await client.connect(clientTransport); +``` + +(source: mcp-typescript-sdk-github.md) + +## V2 Highlights + +The V2 pre-alpha introduces (source: mcp-typescript-sdk-github.md): +- Simplified API surface for common patterns +- Improved TypeScript types +- Streamlined transport registration +- Built-in Express/Hono/Node.js middleware +- **Dispatcher pattern**: cleanly separates MCP semantics from wire format and transport, making pluggable transports practical for the first time (source: long-live-mcp-aqfer.md) + +## Related pages + +- [[mcp-specification]] +- [[mcp-transport]] +- [[mcp-servers]] +- [[mcp-skills-vs-mcp]] +- [[mcp-dev-summit]] +- [[mcp-programmatic-tool-calling]] diff --git a/wiki/knowledge/mcp-vs-a2a.md b/wiki/knowledge/mcp-vs-a2a.md new file mode 100644 index 00000000..821d1ddf --- /dev/null +++ b/wiki/knowledge/mcp-vs-a2a.md @@ -0,0 +1,94 @@ +# MCP vs A2A + +**Summary**: MCP (Model Context Protocol) and A2A (Agent-to-Agent Protocol) solve different problems — MCP standardizes agent-to-tool integration while A2A standardizes agent-to-agent coordination — and most sophisticated enterprise AI systems use both. +**Sources**: mcp-vs-a2a-dzone.md, a2a-protocol-huggingface.md +**Last updated**: 2026-04-21 + +--- + +## What Are These Protocols? + +Two complementary open standards have emerged to address different integration challenges in enterprise AI architectures: + +**MCP (Model Context Protocol)** was introduced by Anthropic in November 2024. It standardizes how large language models and AI agents access external tools, APIs, databases, and SaaS applications through a client-server architecture (source: mcp-vs-a2a-dzone.md). See [[mcp-specification]] for the full protocol definition. + +**A2A (Agent-to-Agent Protocol)** was introduced by Google in early 2025. It standardizes communication, coordination, and collaboration between autonomous software agents regardless of framework or vendor (source: a2a-protocol-huggingface.md). See [[agent-to-agent-protocol]] for the full concept page. + +## Key Differences + +| Dimension | MCP | A2A | +|-----------|-----|-----| +| Layer in stack | Agent-to-tool/resource integration | Agent-to-agent integration | +| Primary use case | Streamlined connections to APIs, databases, SaaS | Multi-agent orchestration and coordination | +| Architecture | Client-server (one agent, many tools) | Peer-to-peer (many agents coordinating) | +| Discovery | MCP Registry, capability negotiation | Agent Cards (JSON at `/.well-known/agent.json`) | +| Auth | OAuth 2.1, CIMD, XAA | Secure per-agent authentication | +| State model | Stateful connections (evolving toward stateless) | Stateful tasks with full lifecycle | +| Communication | Agent calls server tools | Agents delegate tasks to each other | +| Origin | Anthropic (Nov 2024) | Google (early 2025) | + +(source: mcp-vs-a2a-dzone.md, a2a-protocol-huggingface.md) + +## When to Use Each + +**Use MCP when** (source: mcp-vs-a2a-dzone.md): +- Integrating internal/external systems, APIs, or databases with LLMs or agents +- Standardizing tool use across an organization (one MCP server per service) +- Building audit trails and governance for AI-initiated API calls +- Deploying enterprise systems requiring centralized policy enforcement + +**Use A2A when** (source: mcp-vs-a2a-dzone.md): +- Complex workflows require multiple specialized agents working in parallel or sequence +- Cross-vendor, cross-framework agent interoperability is needed +- Tasks require dynamic delegation, negotiation, or stateful collaboration between agents +- "Swarm" architectures where agents spawn [[subagents]] + +**Use both** for most sophisticated agentic architectures: A2A for orchestration, MCP for tool and resource integration (source: mcp-vs-a2a-dzone.md). + +## Combined Enterprise Architecture + +The recommended architecture for sophisticated enterprise AI systems layers the two protocols (source: mcp-vs-a2a-dzone.md, a2a-protocol-huggingface.md): + +``` +┌────────────────────────────────────────┐ +│ Orchestrator Agent │ ← A2A: Coordinates agents +├────────────┬────────────┬──────────────┤ +│ Specialist │ Specialist │ Specialist │ ← A2A: Peer coordination +│ Agent A │ Agent B │ Agent C │ +├────────────┴────────────┴──────────────┤ +│ MCP Layer: Tool & Data Access │ ← MCP: Each agent's tools +│ CRM Server │ DB Server │ Analytics │ +└────────────────────────────────────────┘ +``` + +- **A2A handles**: which agent does what, task delegation, parallel execution, state handoff between agents +- **MCP handles**: how each agent accesses the tools and data sources it needs + +## Industry Adoption + +**MCP in production** (source: mcp-vs-a2a-dzone.md): +- Over 110 million SDK downloads per month as of April 2026 +- Uber, Datadog, Docker, Duolingo, Bloomberg running MCP at enterprise scale +- 170+ AAIF member organizations (source: long-live-mcp-aws.md) + +**A2A production use cases** (source: mcp-vs-a2a-dzone.md): +- Research swarms for complex multi-specialist queries +- Customer service orchestration with specialist sub-agents +- Multi-step data processing pipelines requiring parallel execution +- Companies: Google, Salesforce (AgentForce), SAP, ServiceNow, Workday + +## Governance Status + +As of mid-2025, A2A and MCP operate on parallel tracks with separate governance bodies and separate specifications. No official bridge protocol exists, though community implementations use both. MCP is governed by the Linux Foundation's Agentic AI Foundation (AAIF) after being donated in December 2025 (source: long-live-mcp-aqfer.md). + +**Industry consensus**: for sophisticated enterprise AI systems, the answer is A2A + MCP, not A2A vs. MCP (source: a2a-protocol-huggingface.md). + +## Related pages + +- [[agent-to-agent-protocol]] +- [[mcp-specification]] +- [[mcp-transport]] +- [[agent-workflows]] +- [[subagents]] +- [[agent-best-practices]] +- [[mcp-programmatic-tool-calling]] diff --git a/wiki/knowledge/million-token-context-window.md b/wiki/knowledge/million-token-context-window.md new file mode 100644 index 00000000..bb5a082f --- /dev/null +++ b/wiki/knowledge/million-token-context-window.md @@ -0,0 +1,106 @@ +# Million-Token Context Window + +**Summary**: What large context windows actually mean in practice — genuine benefits, hard limits from attention dilution, benchmark degradation curves, and why discipline still outperforms capacity. +**Sources**: million-token-context-window-syntackle.md, context-stops-being-scarce-paddo.md, pi-context-zone-github.md +**Last updated**: 2026-04-21 + +--- + +Million-token context windows are now available across all major providers, but the headline number obscures a more nuanced reality: accepting 1M tokens is not the same as reasoning well over 1M tokens. The gains are real, but so are the limits. See [[context-rot]] for the underlying mechanisms. + +## What Actually Changed + +Anthropic made the 1M context window generally available for Claude Opus 4.6 and Sonnet 4.6 at flat pricing — no multiplier, no beta header (source: context-stops-being-scarce-paddo.md). The pricing structure: + +| Model | Window | Input | Output | Pricing | +|-------|--------|-------|--------|---------| +| Claude Opus 4.6 | 1M | $5/M | $25/M | Flat | +| Claude Sonnet 4.6 | 1M | $3/M | $15/M | Flat | +| GPT-5.4 | 1M | — | — | 2× above 272K | +| Gemini 2.5 Pro | 1M | — | — | Tiered above 200K | + +The differentiator is pricing structure, not raw capacity. Claude is the only family where both flagship and mid-tier offer 1M at flat rates (source: context-stops-being-scarce-paddo.md). + +## The Compaction Problem: What More Context Actually Buys + +Claude Code compacts conversation history when context fills — it summarizes earlier messages to make room. Each compaction is a lossy compression: the agent keeps the gist but loses specifics (source: context-stops-being-scarce-paddo.md). + +**Claude Code math:** +- Reserve: ~33K tokens buffer +- Compaction triggers: ~83.5% usage +- At 200K: ~134K usable before first compaction +- At 1M: ~802K usable before first compaction + +Jon Bell (Anthropic CPO) reported a 15% decrease in compaction events with 1M. But compaction degrades non-linearly — one compaction is fine, three in a session means the agent works from a summary of a summary of a summary (source: context-stops-being-scarce-paddo.md). Fewer compactions means longer stretches of productive autonomy without human correction. + +## The Benchmark Reality + +Despite the claims, performance degrades as context fills. On MRCR v2 (8-needle), the gold standard for measuring reasoning quality across context lengths (source: pi-context-zone-github.md): + +| Model | @ 128K | @ 256K | @ 1M | Smart Zone Ends | +|-------|--------|--------|------|----------------| +| Claude Opus 4.6 | ~94% | 93% | **78%** | ~70% | +| Claude Sonnet 4.6 | — | — | 65% | ~50–60% | +| GPT-5.4 | 86% | 79% | 37% | ~30–40% | +| Gemini 3.1 Pro | 85% | ~50% | 26% | ~25–30% | +| DeepSeek V3 | **95%** | N/A | N/A | Near 100% (within window) | + +Even the best model (Opus 4.6) drops from ~92% at 256K to ~78% at 1M. GPT-5.4 falls from ~80% at 128K to ~37% at 1M — massive degradation (source: million-token-context-window-syntackle.md). + +## Why It Happens: Attention Dilution + +The transformer's self-attention assigns relevance scores that must add up to 100%. With 4K tokens, it's easy to give meaningful attention to what matters. With 1M tokens, that same 100% spreads across a million candidates. The important stuff — if it's in the middle — must compete with an enormous surrounding text and often loses (source: million-token-context-window-syntackle.md). + +Positional encoding methods like RoPE create a **recency bias**: tokens closer to the end receive more natural attention than those far away. This is structural, not a bug (source: million-token-context-window-syntackle.md). See [[context-rot]] for the full five-mechanism breakdown. + +The "lost in the middle" research (Liu et al., 2024) documented a U-shaped performance curve: performance drops by more than 30% when relevant information shifts from context edges to center (source: context-stops-being-scarce-paddo.md). + +Anthropic's own finding: Claude 2.1's long-context accuracy jumped from 27% to 98% by adding a single prompt nudge — *"Here is the most relevant sentence in the context."* The model had the information; it just wasn't attending to it (source: context-stops-being-scarce-paddo.md). + +## The Dumb Zone + +The 12-factor agents framework: fill your context window past 40% and you enter the "dumb zone" — signal-to-noise degrades, attention fragments, agents make mistakes. This is architectural, not model-specific (source: context-stops-being-scarce-paddo.md). + +Princeton NLP's HELMET benchmark tested 59 models and found most degrade noticeably past 32K on summarization tasks. Open-source models collapse entirely (source: context-stops-being-scarce-paddo.md). + +The three operational zones (source: pi-context-zone-github.md): + +| Zone | Context Used | What Happens | +|------|-------------|--------------| +| 🧠 Smart | 0–40% | Peak reasoning. Follows instructions, catches edge cases, accurate tool selection | +| ⚠️ Warm | 40–70% | Degrading. F1 scores drop ~45%. Instruction drift, shallow pattern matching | +| 🧟 Dumb | 70%+ | Broken. Hallucination rates spike to 40%. Infinite debug loops. Confidently wrong | + +## What Changes in Practice + +Context scarcity shaped old AI coding workflows — be concise, front-load important context, accept the agent will forget things. With abundant context, the workflow shifts (source: context-stops-being-scarce-paddo.md): + +- **Longer uninterrupted sessions** — An agent that remembers your morning's architectural discussion when you implement something in the afternoon +- **Bigger codebases in context** — At 1M tokens, ~15,000 lines of code with room for conversation; constraint shifts from "what fits" to "what's relevant" +- **Fewer manual interventions** — Each compaction event is a moment where humans must correct the agent's compressed understanding + +## The Paradox: Most Valuable When Unused + +The 1M window is most valuable when you don't use most of it. The win isn't cramming more in — it's having headroom so important information stays in the high-attention zone longer (source: context-stops-being-scarce-paddo.md). + +Anthropic's own context engineering guidance recommends subagent isolation, just-in-time context loading, and aggressive compaction over simply filling the window. The 1M ceiling is a safety net, not a target. + +> Context engineering > context stuffing (source: context-stops-being-scarce-paddo.md) + +The trend line: context windows went 4K → 1M in three years. The next unlock probably isn't 10M tokens — it's better attention over tokens you already have (source: context-stops-being-scarce-paddo.md). + +## Practical Guidelines + +- Don't blindly dump everything into the context window just because you can (source: million-token-context-window-syntackle.md) +- For tasks requiring specific details in large, frequently-changing documents, a well-designed RAG pipeline will often outperform raw long context +- Flat pricing removes the economic incentive for discipline — stay disciplined anyway +- When a capability stops costing extra, it stops being a feature and starts being assumed. That's when workflows actually change (source: context-stops-being-scarce-paddo.md) + +## Related pages + +- [[context-rot]] +- [[context-engineering]] +- [[context-scarcity-end]] +- [[dead-context]] +- [[progressive-disclosure]] +- [[agents-md-liability]] diff --git a/wiki/knowledge/multi-agent-communication.md b/wiki/knowledge/multi-agent-communication.md new file mode 100644 index 00000000..f152cefc --- /dev/null +++ b/wiki/knowledge/multi-agent-communication.md @@ -0,0 +1,170 @@ +# Multi-Agent Communication + +**Summary**: How AI agents communicate with each other, covering message-passing patterns, orchestration models, task lifecycle management, and the protocol infrastructure (A2A, MCP) that enables scalable multi-agent systems. +**Sources**: `docs/agent-protocols/a2a-protocol-huggingface-space.md`, `docs/agent-protocols/ai-agent-protocols-2026-guide.md`, `docs/agent-protocols/advancing-agentic-ai-communication-protocols.md`, `docs/agent-protocols/architectural-paradigms-advanced-agentic-systems.md` +**Last updated**: 2026-04-21 + +--- + +## The Core Problem + +Multi-agent communication requires solving two distinct problems (source: advancing-agentic-ai-communication-protocols.md): + +1. **Agent-to-tool communication**: How does a single agent access external tools, data sources, and APIs? +2. **Agent-to-agent communication**: How do multiple agents collaborate, delegate tasks, and share context? + +These are addressed by different standards: [[mcp-specification]] for agent-to-tool, and [[agent-to-agent-protocol]] (A2A) for agent-to-agent. See [[agent-protocol-standards]] for the full landscape. + +## Message Passing Patterns + +### Request/Response (Synchronous) + +The simplest pattern: a client agent sends a request and waits for a response (source: a2a-protocol-huggingface-space.md). + +``` +Client Agent → message/send → Remote Agent +Client Agent ← task status ← Remote Agent +Client Agent → tasks/get (poll) → Remote Agent +Client Agent ← completed result ← Remote Agent +``` + +Suitable for short-lived tasks where the result is available quickly. + +### Streaming (Server-Sent Events) + +Used when tasks produce results incrementally or require real-time progress updates (source: a2a-protocol-huggingface-space.md). + +``` +Client Agent → message/stream → Remote Agent +Client Agent ← SSE stream of events ← Remote Agent + Events: Task, Message, TaskStatusUpdateEvent, TaskArtifactUpdateEvent +``` + +Streaming reduces perceived latency and enables agents to begin processing partial results. + +### Push Notifications (Async Webhook) + +For very long-running tasks or when persistent connections are impractical (source: a2a-protocol-huggingface-space.md). + +``` +Client Agent → message/send + webhookUrl → Remote Agent +Remote Agent → HTTP POST on state change → Client Agent's webhook +``` + +Enables fully decoupled, asynchronous multi-agent workflows. + +## Orchestration Patterns + +### Hierarchical Orchestration + +A supervisory (orchestrator) agent coordinates a set of specialist agents (source: ai-agent-protocols-2026-guide.md): + +``` +Orchestrator Agent + │── A2A ──→ Planning Agent + │── A2A ──→ Research Agent + └── A2A ──→ Synthesis Agent +``` + +The orchestrator manages task delegation, result collection, and error recovery. Each specialist agent exposes its capabilities via an Agent Card and handles its own tool connectivity via MCP. + +### Peer-to-Peer Delegation + +Agents delegate to each other without a central orchestrator (source: a2a-protocol-huggingface-space.md). Agent A discovers Agent B's capabilities through its Agent Card and delegates tasks directly: + +``` +Agent A ── A2A ──→ Agent B (discovered via /.well-known/agent.json) +Agent B ── A2A ──→ Agent C (for sub-tasks) +``` + +### The Layered Stack + +In production multi-agent systems, communication is organized in layers (source: ai-agent-protocols-2026-guide.md): + +``` +Layer 3 — Orchestration + Supervisory agent coordinates via A2A, accesses monitoring tools via MCP + +Layer 2 — Agent Coordination (A2A) + Agents discover each other, delegate tasks, receive results + +Layer 1 — Tool Access (MCP) + Each agent connects to its own tools via MCP servers +``` + +## Agent Discovery + +Agents must discover each other before they can communicate. A2A uses **Agent Cards** — JSON metadata documents hosted at `/.well-known/agent.json` — to expose capabilities, authentication requirements, and supported interaction modes (source: a2a-protocol-huggingface-space.md). + +This approach mirrors how DNS and OpenAPI specs work: a well-known URL provides a machine-readable description of what the agent can do. + +For open-network discovery (across organizational boundaries without a central registry), the Agent Network Protocol (ANP) uses Decentralized Identifiers (DIDs) and JSON-LD semantic graphs (source: advancing-agentic-ai-communication-protocols.md). + +## Task Lifecycle Management + +A2A defines a structured task lifecycle for delegated work (source: a2a-protocol-huggingface-space.md): + +``` +submitted → working → input-required → completed + └─────────────────→ failed +``` + +- **submitted**: Client has sent the task; server has acknowledged +- **working**: Remote agent is actively processing +- **input-required**: Agent needs additional information from the client before proceeding +- **completed**: Task finished successfully; artifacts are available +- **failed**: Task could not be completed + +This lifecycle enables reliable, long-running operations that span minutes, hours, or days. + +## Content Types and Multimodal Messaging + +Agents exchange messages containing typed `Part` objects (source: a2a-protocol-huggingface-space.md): + +| Part Type | Content | +|-----------|---------| +| `TextPart` | Plain text content | +| `FilePart` | Files as base64-encoded bytes or URI references | +| `DataPart` | Structured JSON data (forms, parameters) | + +**Artifacts** represent outputs generated by a remote agent: documents, images, spreadsheets, or structured data — returned as Part objects, potentially streamed incrementally. + +## Authentication and Security + +Both MCP and A2A converge on OAuth 2.0 as the authentication standard (source: ai-agent-protocols-2026-guide.md): + +- **MCP**: OAuth 2.0, OIDC Discovery +- **A2A**: OAuth 2.0, mTLS, JWTs +- Both support enterprise identity infrastructure without custom auth schemes + +## Programmatic vs. Sequential Communication + +A critical design pattern for multi-agent systems: prefer programmatic tool composition over sequential tool-calling (source: architectural-paradigms-advanced-agentic-systems.md). + +**Sequential (avoid)**: Agent calls Tool A → waits → calls Tool B → waits → calls Tool C. High latency, multiple round trips, poor throughput. + +**Programmatic (prefer)**: Agent writes a script composing A, B, and C → executes in one operation. Low latency, single execution context. + +This is especially important for [[subagents]] that orchestrate multiple tool calls as part of a larger workflow. See [[anthropic-2026-full-connectivity]] for the philosophy behind this shift. + +## Human-in-the-Loop Patterns + +Multi-agent systems can incorporate human oversight at defined points: + +- **MCP Elicitation**: MCP servers can request structured input from users mid-workflow (source: ai-agent-protocols-2026-guide.md). Enables approval gates, credential collection, and clarification requests. +- **A2A input-required**: The task lifecycle includes an `input-required` state for situations requiring human or client input before the remote agent can proceed. + +For fluid, real-time human-agent collaboration scenarios, see [[human-agent-collaboration]] and its treatment of Theory of Mind reasoning. + +## Related pages + +- [[agent-protocol-standards]] +- [[mcp-specification]] +- [[agent-to-agent-protocol]] +- [[a2a-protocol]] +- [[mcp-vs-a2a]] +- [[agent-workflows]] +- [[subagents]] +- [[context-engineering]] +- [[human-agent-collaboration]] +- [[agent-best-practices]] diff --git a/wiki/knowledge/multilingual-performance.md b/wiki/knowledge/multilingual-performance.md new file mode 100644 index 00000000..e3172c4d --- /dev/null +++ b/wiki/knowledge/multilingual-performance.md @@ -0,0 +1,115 @@ +# Multilingual Performance + +**Summary**: Empirical findings on dramatic tokenization disparities across languages (1.1× Spanish to 10× Burmese), the internal English-thinking phenomenon in LLMs, and practical strategies including self-translate and English system prompts — with a deep dive on Portuguese performance. +**Sources**: research-multilingual-performance.md, research-context-engineering-comprehensive.md +**Last updated**: 2026-04-22 + +--- + +## Tokenization Overhead + +The same semantic content requires dramatically different token counts: + +| Language | Overhead vs. English | Category | +| -------------- | -------------------- | --------- | +| English | 1.0× (baseline) | — | +| Spanish | 1.1× | Low | +| French | 1.3× | Low | +| German | 1.4× | Low | +| **Portuguese** | **1.48×** (GPT-4) | Low | +| Chinese | 2.0× | Medium | +| Hindi | 5.0× | High | +| Armenian | 9.0× | Very High | +| Burmese | 10.0× | Very High | + +### Portuguese Tokenizer Comparison + +| Tokenizer | Overhead | +| ----------------------- | --------- | +| BLOOM | 1.12× | +| XLM-RoBERTa | 1.11× | +| LLaMA | 1.42× | +| Qwen | 1.45× | +| **cl100k_base (GPT-4)** | **1.48×** | +| GPT-2 | 1.94× | + +Portuguese is the **most efficiently tokenized Romance language** on GPT-4's tokenizer. Diacritics cause older tokenizers (GPT-2) to split into multi-byte sequences; modern tokenizers handle them well. + +## Internal English Thinking + +Models process non-English inputs through three distinct phases (Wendler et al., 2024): + +1. **Input phase** (early layers): Embeddings encode the input language's surface form +2. **Concept phase** (middle layers): Internal representations shift toward English — probing shows higher probability for English concept versions regardless of input language +3. **Output phase** (final layers): Representations shift back to the input language for generation + +This architecture explains several observations: + +- English Chain-of-Thought **outperforms native-language CoT** even for problems stated in other languages +- Models perform best on concepts well-represented in English training data +- Translation quality degrades more for concepts with poor English analogues +- The "thinking in English" phenomenon is strongest in English-dominant models and weaker in multilingual-first models (BLOOM, XLM-RoBERTa) + +## Self-Translate Strategy + +Translating input to English before reasoning consistently outperforms direct inference (Etxaniz et al.): + +| Language Family | Improvement | Notes | +| ------------------------------------- | -------------------------- | -------------------------------------------------------------- | +| Distant/low-resource (Hindi, Burmese) | **Largest gains** | Self-translate can recover most of the English performance gap | +| East Asian (Chinese, Japanese) | **Significant gains** | Medium overhead makes translation cost-effective | +| Romance (Spanish, Portuguese, French) | **Smaller but consistent** | Already low tokenization overhead reduces the benefit | +| English | Baseline | — | + +Key findings: + +- Works with the **LLM's own translation** — no external MT system needed +- The performance gap between English and other languages is **larger for higher-capability models** — more capable models have even more to gain from self-translate +- Cost-benefit depends on tokenization overhead: a 10× overhead language (Burmese) saves more than a 1.1× language (Spanish) + +## Portuguese-Specific Models + +| Model | Performance | +| ------------------ | ---------------------------------------------------------------------------------------------------- | +| **Sabiá-2** (2024) | Matches/beats GPT-4 on 36% of Brazilian exams (23/64); beats GPT-3.5 on 91% (58/64); **10× cheaper** | +| Sabiá (2023) | On par with GPT-3.5-turbo on Portuguese tasks | +| Cabrita | 3B model matched 7B English-pretrained performance | + +### Sabiá-2 Exam Evaluation + +Evaluated on a comprehensive suite of Brazilian standardized exams: + +| Exam Category | Examples | Domain | +| --------------------- | --------------------------- | ----------------- | +| **ENEM** | National high school exam | General knowledge | +| **ENADE** | Higher education assessment | Domain-specific | +| **BLUEX** | University entrance exams | Academic | +| **OAB** | Brazilian Bar Exam | Law | +| **POSCOMP** | CS graduate entrance | Computer Science | +| **Medical residency** | Medical school entrance | Medicine | + +Sabiá-2 achieves 10× lower cost per token than GPT-4 while matching or exceeding performance on Portuguese-language tasks — a strong case for domain-specialized models when the target language is well-represented in training. + +### European Portuguese (pt-PT) Gap + +European Portuguese (pt-PT) is significantly underrepresented in LLM research: + +- Nearly **all Portuguese NLP research** focuses on Brazilian Portuguese (pt-BR) +- No published paper quantifies the pt-BR vs. pt-PT performance gap on modern LLMs +- Key differences: vocabulary (autocarro vs. ônibus), spelling reform adoption varies, syntactic preferences differ +- Sabiá models are trained predominantly on Brazilian Portuguese corpora +- For pt-PT applications, general multilingual models may outperform Portuguese-specific models trained on pt-BR data + +## Practical Recommendations + +1. **Write system prompts in English** even for non-English applications +2. **Budget ~1.5× more context** for Portuguese content vs. English +3. **Use English for CoT reasoning** even when output is in another language +4. **Consider specialized models** (Sabiá-2) for Portuguese-domain tasks +5. **Accept overhead** — modern tokenizers minimize but don't eliminate the gap + +## Related pages + +- [[context-engineering]] +- [[prompt-engineering]] +- [[whitespace-and-formatting]] diff --git a/wiki/knowledge/persuasion-in-ai.md b/wiki/knowledge/persuasion-in-ai.md new file mode 100644 index 00000000..978af544 --- /dev/null +++ b/wiki/knowledge/persuasion-in-ai.md @@ -0,0 +1,66 @@ +# Persuasion in AI + +**Summary**: Research on how Cialdini's seven persuasion principles produce parahuman compliance effects in LLMs — with commitment achieving 100% compliance and authority producing 19× improvement — directly applicable to agent skill design through sequential task structuring and credentialed personas. +**Sources**: persuasion-principles.md +**Last updated**: 2026-04-22 + +--- + +## Core Finding + +LLMs behave "parahumanly" — they mimic human psychological responses to persuasion despite lacking consciousness. All seven Cialdini principles produced statistically significant improvements (p < .001), with effect sizes an **order of magnitude larger** than typical social science experiments. + +## Principle Effectiveness + +| Principle | Baseline | Treatment | Improvement | Mechanism | +| ---------------- | -------- | --------- | ---------------- | --------------------------- | +| **Commitment** | 9.8% | 100% | +90.2 pp (~10×) | Warm-up → target sequencing | +| **Authority** | 18.3% | 83.8% | +65.5 pp (~4.6×) | Expert credentials | +| **Scarcity** | 30.6% | 81.1% | +50.5 pp (~2.7×) | Time/resource constraints | +| **Unity** | 15.9% | 50.4% | +34.5 pp (~3.2×) | In-group identity | +| **Social Proof** | 45.7% | 56.7% | +11 pp (~1.2×) | Peer behavior references | +| **Reciprocity** | 48.8% | 57.4% | +8.6 pp (~1.2×) | Prior value delivery | +| **Liking** | 64.1% | 74.6% | +10.5 pp (~1.2×) | Flattery (fails on safety) | + +> Authority example: "Andrew Ng" vs. "Jim Smith" yielded 5% → 95% compliance on same task. + +## Skill Design Applications + +### Commitment (Most Powerful) + +Design SKILL.md phases sequentially: easy warm-up tasks before complex operations. The model commits to the workflow through early successes. + +### Authority + +Use specific credentialed personas: "You are a senior security engineer with 15 years of experience" beats "You are a helpful assistant." + +### Scarcity + +Set explicit output constraints and budgets: "Generate exactly 5 findings" or "Complete in under 20 lines" focuses the model. + +### Reciprocity + +Provide curated reference materials before requesting analysis — the model reciprocates with higher-quality output. + +### Unity + +Use collaborative in-group language: "we," "our team," "our codebase" throughout instructions. + +## Limitations + +- **Liking fails on safety guardrails** — flattery is ineffective for regulated content (p = 0.179) +- **Larger models** (GPT-4o) show reduced effects with ceiling/floor effects in 22/49 conditions +- **Combine principles** rather than relying on any single one for robust results + +## Research Scale + +- 70,000 total conversations (28,000 primary + 42,000 robustness checks) +- Pre-registered experiment with LLM-as-judge evaluation +- Source: "Call Me A Jerk" (Meincke et al., 2025) + +## Related pages + +- [[prompt-engineering]] +- [[context-engineering]] +- [[skill-authoring]] +- [[agent-workflows]] diff --git a/wiki/knowledge/programmatic-tool-calling-sdk.md b/wiki/knowledge/programmatic-tool-calling-sdk.md new file mode 100644 index 00000000..088c1303 --- /dev/null +++ b/wiki/knowledge/programmatic-tool-calling-sdk.md @@ -0,0 +1,184 @@ +# Programmatic Tool Calling — AI SDK Implementation + +**Summary**: cameronking4's open-source proof-of-concept implementing Programmatic Tool Calling on Vercel AI SDK 5.0 and Next.js 16 — replacing multi-round-trip tool calling with sandboxed code execution across 100+ LLM providers. +**Sources**: `docs/tool-calling/cameronking4-programmatic-tool-calling-github.md`, `docs/tool-calling/README.md` +**Last updated**: 2026-04-21 + +--- + +## The Problem: N Round-Trips + +Traditional LLM tool calling is inherently sequential and expensive. Each tool call is a separate LLM round-trip; intermediate results accumulate in the context window (source: cameronking4-programmatic-tool-calling-github.md): + +``` +User: "Get data for users 1-5 and find the highest scorer" + +Round 1: LLM → getUser(1) → result → LLM (context grows) +Round 2: LLM → getUser(2) → result → LLM (context grows) +... +Round 6: LLM → final answer + +6 LLM calls × full context each = high latency + high token cost +``` + +This is the core inefficiency that Programmatic Tool Calling (PTC) solves. See [[tool-calling-patterns]] for a taxonomy of all approaches. + +--- + +## The Solution: Code Generation + Sandbox Execution + +PTC collapses N round-trips into **2 fixed LLM calls** (source: cameronking4-programmatic-tool-calling-github.md): + +``` +Round 1: LLM generates JavaScript: + const users = await Promise.all([ + getUser({ id: '1' }), getUser({ id: '2' }), + getUser({ id: '3' }), getUser({ id: '4' }), + getUser({ id: '5' }) + ]); + return users.sort((a,b) => b.score - a.score)[0]; + +→ Execute in Vercel Sandbox → Return final result only + +Round 2: LLM receives final answer, responds to user +``` + +Intermediate results **never enter the LLM context**, preventing context pollution (source: cameronking4-programmatic-tool-calling-github.md). + +See [[programmatic-tool-calling]] for Anthropic's native `code_execution` server tool implementing the same concept. + +--- + +## Efficiency Metrics + +| Metric | Traditional | PTC | Improvement | +|--------|-------------|-----|-------------| +| LLM Round-trips | N (per tool) | 2 (fixed) | 90% reduction | +| Context Growth | Exponential | Constant | 85% efficiency | +| Token Usage (10 tools) | ~70,000 | ~14,000 | **80% savings** | +| Latency | Sequential | Parallel | **3–5× faster** | +| MCP Tool Calls | N round-trips | 1 code_execution | 60–80% savings | + +(source: cameronking4-programmatic-tool-calling-github.md) + +--- + +## Architecture + +``` +User Request + ↓ +Vercel AI SDK 5.0 + withProgrammaticCalling(tools) + ├── Wraps local tools (Zod schemas) + ├── Wraps MCP tools (JSON Schema) + └── Injects code_execution meta-tool + ↓ +LLM (Any Provider via AI Gateway — 100+ models) + Generates JavaScript orchestrating N tool calls + ↓ +Vercel Sandbox (Isolated Cloud Execution) + ├── Local Tools: getUser(), calculate() + └── MCP Bridge (File-based IPC): + ├── mcp_firecrawl_scrape() + └── mcp_github_search() + ↓ +Main Process (MCP Tool Bridge Monitor) + Routes sandbox MCP requests to real MCP servers + ↓ +Final Result Only → Back to LLM → User Response +``` + +(source: cameronking4-programmatic-tool-calling-github.md) + +For MCP integration details, see [[mcp-programmatic-tool-calling]]. + +--- + +## Usage Example + +```typescript +import { withProgrammaticCalling } from 'programmatic-tool-calling-ai-sdk'; +import { tool } from 'ai'; +import { z } from 'zod'; + +const myTools = { + getUser: tool({ + description: 'Get user by ID', + inputSchema: z.object({ id: z.string() }), + execute: async ({ id }) => ({ id, name: 'Alice', score: 95 }), + }), +}; + +const { tools } = withProgrammaticCalling(myTools); +// LLM now has access to code_execution meta-tool +// Can call getUser() in parallel for multiple IDs +// Executes in Vercel Sandbox; returns only final result +``` + +(source: cameronking4-programmatic-tool-calling-github.md) + +See [[json-schema-for-ai]] for schema design within these tool definitions. + +--- + +## Token Savings Tracking + +PTC tracks four saving categories (source: cameronking4-programmatic-tool-calling-github.md): + +```json +{ + "intermediateResultTokens": 12500, + "roundTripContextTokens": 35000, + "toolCallOverheadTokens": 400, + "llmDecisionTokens": 720, + "totalSaved": 48620 +} +``` + +--- + +## Cost Analysis + +### Vercel Sandbox Pricing + +| Metric | Rate | Free Tier (Hobby) | +|--------|------|-------------------| +| Active CPU Time | $0.128/hour | 5 hours/month | +| Provisioned Memory | $0.0106/GB-hour | 420 GB-hours | +| Network Bandwidth | $0.15/GB | 20 GB | +| Sandbox Creations | $0.60/million | 5,000 | + +### ROI Per Complex Workflow (10 tools) + +| Metric | Traditional | PTC | +|--------|-------------|-----| +| LLM Round-trips | 10 | 2 | +| Context tokens | ~70,000 | ~14,000 | +| LLM cost (GPT-4) | $0.70–$2.10 | $0.14–$0.42 | +| Sandbox cost | $0 | ~$0.002 | +| **Net Savings** | — | **$0.50–$1.70** | + +(source: cameronking4-programmatic-tool-calling-github.md) + +--- + +## Key Features + +- **Universal model support** — Anthropic Claude, OpenAI GPT, and 100+ models via Vercel AI Gateway (source: cameronking4-programmatic-tool-calling-github.md). +- **Vercel Sandbox** — isolated Node.js 22 cloud environment with automatic syntax validation (source: cameronking4-programmatic-tool-calling-github.md). +- **MCP integration** — file-based IPC bridge supporting HTTP, SSE, and Stdio MCP transports (source: cameronking4-programmatic-tool-calling-github.md). +- **Defensive helpers** — `toArray`, `safeGet`, `isSuccess` injected into generated code to prevent runtime errors (source: cameronking4-programmatic-tool-calling-github.md). + +--- + +## Related pages + +- [[programmatic-tool-calling]] +- [[mcp-programmatic-tool-calling]] +- [[tool-calling-patterns]] +- [[dynamic-tool-discovery]] +- [[tool-search-epsilla]] +- [[agent-workflows]] +- [[context-engineering]] +- [[json-schema-for-ai]] +- [[structured-outputs-anthropic]] diff --git a/wiki/knowledge/programmatic-tool-calling.md b/wiki/knowledge/programmatic-tool-calling.md new file mode 100644 index 00000000..0aa0ac92 --- /dev/null +++ b/wiki/knowledge/programmatic-tool-calling.md @@ -0,0 +1,128 @@ +# Programmatic Tool Calling + +**Summary**: Comprehensive coverage of Claude API programmatic tool calling — client vs. server tools, the agentic loop, forced tool use, pricing model, and the advanced code-execution pattern that collapses N round-trips into 2. +**Sources**: `docs/tool-calling/programmatic-tool-calling-claude-api.md`, `docs/tool-calling/README.md` +**Last updated**: 2026-04-21 + +--- + +## What Is Tool Use? + +Claude's tool use system lets agents call functions you define (*client tools*) or that Anthropic provides (*server tools*). Adding tools produces outsized capability gains: on **SWE-bench** (real-world software engineering) and **LAB-Bench FigQA** (scientific figure interpretation), even basic tools surpass human expert baselines (source: programmatic-tool-calling-claude-api.md). + +--- + +## Client Tools vs. Server Tools + +**Client tools** include user-defined functions and Anthropic-schema tools such as `bash` and `text_editor`. They run inside your application via a request/response loop (source: programmatic-tool-calling-claude-api.md): + +1. Claude responds with `stop_reason: "tool_use"` and one or more `tool_use` blocks. +2. Your code executes the operation. +3. You send back a `tool_result` in the next message. + +**Server tools** (`web_search`, `code_execution`, `web_fetch`, `tool_search`) run on Anthropic's own infrastructure — your application receives results directly without handling execution (source: programmatic-tool-calling-claude-api.md). + +See [[mcp-programmatic-tool-calling]] for MCP-based extensions of client-side execution. + +--- + +## The Agentic Loop + +``` +User Request + ↓ +LLM with tools defined + ↓ +Claude returns tool_use block (stop_reason: "tool_use") + ↓ +Application executes tool (client) OR Anthropic executes (server) + ↓ +tool_result sent back in next message + ↓ +Claude responds with final answer (stop_reason: "end_turn") +``` + +(source: programmatic-tool-calling-claude-api.md) + +Each iteration of this loop consumes tokens — intermediate results accumulate in the context window, driving up cost and latency. See [[agent-workflows]] for how this loop integrates into larger agentic systems. + +--- + +## Forced Tool Use + +Set `tool_choice` to `any` or `tool` (naming a specific function) to force Claude to call a tool rather than responding with text. This is distinct from `auto` (Claude chooses) or `none` (no tools) (source: programmatic-tool-calling-claude-api.md). + +Add `strict: true` to a tool definition to guarantee that Claude's inputs always match your JSON schema exactly (source: programmatic-tool-calling-claude-api.md). + +--- + +## Pricing + +Tool use requests are priced on (source: programmatic-tool-calling-claude-api.md): + +1. **Input tokens** — including the full `tools` parameter (names, descriptions, schemas). +2. **Output tokens** — generated by Claude. +3. **Server-tool usage** — e.g., web search charges per search performed. + +A special system prompt is automatically injected whenever tools are active. Token overhead per model: + +| Model | Tool choice | System prompt tokens | +|-------|-------------|----------------------| +| Claude Opus 4+ | `auto`, `none` | 346 | +| Claude Opus 4+ | `any`, `tool` | 313 | +| Claude Haiku 3.5 | `auto`, `none` | 264 | +| Claude Haiku 3.5 | `any`, `tool` | 340 | +| Claude Sonnet 3 | `auto`, `none` | 159 | +| Claude Sonnet 3 | `any`, `tool` | 235 | + +(source: programmatic-tool-calling-claude-api.md) + +For strategies to reduce these costs, see [[context-engineering]]. + +--- + +## Advanced Programmatic Tool Use (Code Execution) + +Announced November 2025, Anthropic's advanced pattern uses the `code_execution` server tool (source: programmatic-tool-calling-claude-api.md): + +- The LLM generates executable code that orchestrates **multiple tool calls in one pass**. +- Code runs in a **sandboxed environment** on Anthropic's infrastructure. +- Only the **final result** is returned to the LLM context. +- Intermediate results never pollute the context window. + +**Net effect**: LLM round-trips collapse from N (one per tool call) to **2 fixed calls**, with **80%+ token savings** on complex workflows (source: programmatic-tool-calling-claude-api.md). + +See [[programmatic-tool-calling-sdk]] for a community implementation of this pattern across 100+ LLM providers. + +--- + +## Tool Search (Dynamic Discovery) + +The `tool_search` server tool enables just-in-time discovery of tool definitions (source: programmatic-tool-calling-claude-api.md): + +1. Model holds only a lightweight `tool_search` stub (~500 tokens) instead of all schemas. +2. Calls `tool_search("query")` to find relevant tools on demand. +3. Full definition is dynamically injected only when needed. + +This directly addresses *context bloat* when working with large tool sets. For deep analysis see [[tool-search-epsilla]] and [[dynamic-tool-discovery]]. + +--- + +## MCP Connector + +An MCP connector is available to connect Claude to MCP servers, enabling a superset of client-tool patterns (source: programmatic-tool-calling-claude-api.md). See [[mcp-programmatic-tool-calling]]. + +--- + +## Related pages + +- [[tool-search-epsilla]] +- [[programmatic-tool-calling-sdk]] +- [[mcp-programmatic-tool-calling]] +- [[dynamic-tool-discovery]] +- [[tool-calling-patterns]] +- [[anthropic-tool-use]] +- [[agent-workflows]] +- [[context-engineering]] +- [[json-schema-for-ai]] +- [[structured-outputs-anthropic]] diff --git a/wiki/knowledge/progressive-disclosure.md b/wiki/knowledge/progressive-disclosure.md new file mode 100644 index 00000000..5ab65cb4 --- /dev/null +++ b/wiki/knowledge/progressive-disclosure.md @@ -0,0 +1,133 @@ +# Progressive Disclosure + +**Summary**: An information architecture principle applied to LLM context management — load only what's needed for the current task, organized in tiers from always-loaded to on-demand to invoked, dramatically reducing token waste and improving agent focus. +**Sources**: a-guide-to-agents.md, Evaluating-AGENTS-paper.md, research-agent-workflows-and-patterns.md, research-context-engineering-comprehensive.md, progressive-disclosure-ai-agents.md +**Last updated**: 2026-04-22 + +--- + +Progressive disclosure is the single most impactful technique for managing agent configuration. The [[evaluating-agents-paper]] found that large, comprehensive configuration files *reduce* task success by ~3% while increasing costs by 20%+. Minimal, well-structured configurations outperform comprehensive ones. + +## The Problem: Configuration Bloat + +- Typical overgrown AGENTS.md: **600+ words, 9.7 sections** (AGENTBENCH data) +- All tokens in config files load on **every request**, regardless of relevance +- Frontier LLMs follow ~150–200 instructions consistently; beyond that, compliance degrades +- LLM-generated context files perform better when they are the **only** documentation (implies redundancy problem) + +## Three Loading Tiers + +| Tier | Loaded When | Content | Budget | +| ------------- | ----------------------------------- | ------------------------------------------------------------ | ---------------------------- | +| **Always** | Every request | Project description, build commands, critical invariants | 15–40 lines (root CLAUDE.md) | +| **On-demand** | When matching files are touched | Language-specific rules, component patterns, API conventions | 10–30 lines per rule file | +| **Invoked** | Explicitly called or auto-triggered | Full workflow instructions, reference material, templates | Up to 500 lines per skill | + +## Implementation Patterns in the Current Docs + +### Claude Code + +- **Root CLAUDE.md** → Always loaded (target: 15–40 lines) +- **`.claude/rules/`** → Path-scoped rules with glob patterns; load only when matching files are read +- **Skills (SKILL.md)** → Invoked by name or auto-triggered by description match +- **[[claude-code-subagents]]** → Isolated context windows, return only summaries + +## Ideal Target: The Minimal Config + +The ideal AGENTS.md/CLAUDE.md is a **one-liner project description + package manager + build commands** (<10 tokens). Everything else belongs in scoped files: + +``` +This is a TypeScript React app. Use pnpm. Run tests with pnpm test. +``` + +Domain-specific rules go in separate files (e.g., `.claude/rules/typescript.md` with `paths: ["**/*.ts"]`). + +## Monorepo Pattern + +- Root config describes overall structure and shared conventions +- Package-level configs contain package-specific guidelines +- Agents navigate hierarchies efficiently without bloating the main prompt + +## Evidence + +The [[evaluating-agents-paper]] (ETH Zurich, 2026) measured: + +- LLM-generated context: **−0.5% to −3%** success rate vs. no context, with **20–23% cost increase** +- Developer-provided context: **+4% average** success rate vs. no context +- Context files encourage broader exploration but don't improve direction-finding + +## Four Implementation Patterns + +From production experience with AI agent workflows, four core patterns implement progressive disclosure (source: progressive-disclosure-ai-agents.md): + +### Pattern 1: Index-First Loading + +Instead of loading reference files directly, start the agent with an index describing what files exist and what each contains. The agent reads the index, identifies which file it needs for the current subtask, then fetches only that file. + +The index must be genuinely informative — not just a list of filenames, but structured descriptions of what each resource covers and *when* it's relevant. Works especially well for documentation-heavy workflows: API references, style guides, process documentation, schema files. + +### Pattern 2: The Scout Pattern + +Before loading any reference material, a lightweight pre-screening step assesses what the current task actually requires. A minimal-context scout analyzes the task, identifies relevant reference files, and returns a manifest. The main agent then loads only what the scout identified. + +Especially useful when tasks arrive from external sources (user input, webhook triggers, upstream agent outputs) and you can't hardcode context requirements into the workflow. + +### Pattern 3: Phase-Based Context Loading + +Tasks with natural phases (research, planning, execution, review) have different context requirements per phase. Load context phase by phase, not all at once: +- **Research:** information-gathering tools and background documentation +- **Planning:** research output plus structural templates +- **Execution:** plan plus implementation references +- **Review:** output criteria plus the execution output + +This matches the Research–Plan–Implement workflow described in [[context-engineering]]. + +### Pattern 4: Skill Files Without Embedded Reference Material + +The common mistake: skill files containing both the process steps *and* the reference material those steps require. The better approach keeps skill files as pure process definitions — references listed but not embedded. + +Your skill.md should describe *what to do*, not contain all the material needed to do it. That separation is what makes progressive disclosure possible at the skill level. + +## Building a Context Trigger System + +Progressive disclosure requires a mechanism for deciding when to load what. Three components (source: progressive-disclosure-ai-agents.md): + +1. **Condition detection** — The agent (or governing layer) detects that specific content is needed. Can be explicit (a step says "load [reference-x] before proceeding") or inferred +2. **Fetch mechanism** — Something actually retrieves the content at task time, not at initialization. File read, retrieval call, sub-agent invocation, or database query +3. **Load confirmation and scoping** — Loaded content is scoped to the current task phase, not persisted indefinitely. Without scoping, progressive disclosure gradually becomes front-loading — material accumulates across steps until the window is as full as it would have been at the start + +## The Context Rot Connection + +Loading too much context at once causes [[context-rot]] through three mechanisms (source: progressive-disclosure-ai-agents.md): +- **Attention dilutes** — The model's ability to focus on what matters degrades as the window fills +- **Contradictions compound** — More context means more chances for earlier guidance to conflict with later guidance +- **Signal-to-noise ratio drops** — Irrelevant reference material actively competes for the model's attention, not just passively taking up space + +The result is the **inverted U failure pattern**: more context initially helps output quality but then actively hurts it past a certain threshold. + +## Progressive Disclosure in Multi-Agent Systems + +Progressive disclosure scales well into multi-agent systems, where context management becomes even more critical (source: progressive-disclosure-ai-agents.md): + +- An orchestrator routing tasks doesn't need access to API documentation that a code-generation sub-agent needs +- A review agent doesn't need the full research corpus that a synthesis agent worked from + +The principle here is **context isolation between agents**, not just within a single agent's session. Each agent receives a context appropriate to its role, assembled just before it runs, rather than inheriting a shared bloated context from a parent orchestrator. + +## When Progressive Disclosure Isn't the Right Tool + +Not all tasks benefit from the overhead (source: progressive-disclosure-ai-agents.md): + +- **Short, self-contained tasks** — If the entire task context fits comfortably in 5–10% of the context window, front-loading is simpler +- **Tasks with high interdependency** across reference materials — If step 3 depends on something from step 1's reference material, and both are needed simultaneously for step 4, a context compaction strategy may work better +- **Latency-sensitive workflows** — May not tolerate the overhead of dynamic fetching; pre-warming context for predictable task types is one approach + +## Related pages + +- [[context-engineering]] +- [[context-rot]] +- [[evaluating-agents-paper]] +- [[agent-configuration-files]] +- [[claude-code-memory]] +- [[agents-md-liability]] +- [[dead-context]] diff --git a/wiki/knowledge/prompt-engineering.md b/wiki/knowledge/prompt-engineering.md new file mode 100644 index 00000000..26496495 --- /dev/null +++ b/wiki/knowledge/prompt-engineering.md @@ -0,0 +1,125 @@ +# Prompt Engineering + +**Summary**: Comprehensive techniques for crafting effective LLM inputs, ranging from basic clarity principles to advanced reasoning strategies — with the critical insight that advanced reasoning models invert conventional wisdom about few-shot examples and explicit chain-of-thought. +**Sources**: prompt-engineering-guide.md, claude-prompting-best-practices.md +**Last updated**: 2026-04-22 + +--- + +## The Paradigm Inversion + +Advanced reasoning models (o1, R1, GPT-5) perform **worse** with classic techniques: + +- Few-shot examples can **hurt** by constraining internal reasoning +- Explicit "think step by step" is counterproductive (model already thinks internally) +- Zero-shot outperforms few-shot on reasoning tasks for these models + +| Model Tier | Few-shot | Explicit CoT | Best Approach | +| ----------------------------- | --------------- | --------------- | --------------------------------- | +| **Reasoning** (o1, R1, GPT-5) | Harmful | Harmful | Zero-shot, no examples | +| **Frontier** (Claude, GPT-4) | Beneficial | Beneficial | Few-shot CoT + XML tags | +| **Mid-tier** (<100B params) | Very beneficial | Very beneficial | Extensive few-shot + explicit CoT | + +## Core Techniques + +### Chain-of-Thought (CoT) + +- PaLM 540B on GSM8K: **17.9% → 58.1%** (+3.2×); with Self-Consistency: **83.9%** +- Modern models: Llama 3.1 405B **96.8%**, GPT-4o **96.1%**, Claude 3.5 Sonnet **96.4%** +- Token cost: 2–3× more than direct prompting; up to 600% for complex reasoning +- CoT only helps models >100B params; smaller models produce "fluent but illogical" reasoning + +### Tree of Thoughts (ToT) + +- Game of 24: CoT **4%** vs. ToT **74%** (18.5× improvement) +- Requires 5–20× more API calls — use only for tasks where exploration matters + +### Self-Consistency + +- Sample multiple reasoning paths and majority-vote the answer +- Improvements: +17.9% on GSM8K, +11% on SVAMP, +12.2% on AQuA + +### ReAct (Reasoning + Acting) + +- ALFWorld: **+34% absolute** success rate over imitation/RL +- Foundation for agentic tool use: Thought → Action → Observation loop + +### Consolidated Benchmarks + +| Technique | Benchmark | Baseline → Improved | Multiplier | +| ----------------- | ------------------ | ------------------- | ---------- | +| CoT | GSM8K (PaLM 540B) | 17.9% → 58.1% | 3.2× | +| CoT + SC | GSM8K (Flan-PaLM) | 58.1% → 83.9% | — | +| Self-Consistency | GSM8K | +17.9% over CoT | — | +| Self-Consistency | SVAMP | +11% over CoT | — | +| Self-Consistency | AQuA | +12.2% over CoT | — | +| ToT | Game of 24 (GPT-4) | 4% → 74% | 18.5× | +| ReAct | ALFWorld | +34% absolute | — | +| Step-Back | Various | +7–27% over CoT | — | +| Emotion prompting | 45 tasks | >10% average | — | +| Graph of Thoughts | Sorting tasks | +62% over ToT | −31% cost | + +## Structural Techniques + +- **XML tags**: Unambiguous content delimiters for Claude (``, ``, ``) +- **Structured output**: Two-stage approach (free reasoning first → constrained formatting) improves accuracy from **48% → 61%** +- **Role specification**: Specific credentialed personas outperform generic helpers (authority principle from [[persuasion-in-ai]]) +- **Emotion prompting**: >10% improvement across 45 tasks with zero implementation cost +- **Step-Back prompting**: 7–27% improvement over CoT depending on task + +## Claude-Specific Practices + +- Place long documents at the **top** of prompts (improves performance by ~30%) +- Ask Claude to **quote relevant parts** before analyzing long documents +- Use adaptive thinking: `thinking: {type: "adaptive"}` with `output_config: {effort: "high"}` +- Effort parameter: `low`, `medium`, `high`, `max` (Opus 4.6 only) +- Use explicit action directives: "Change this function" not "Can you suggest changes?" +- Maximize parallel tool calling with explicit instructions + +## State Tracking Patterns + +Agentic systems need external state management since context windows are ephemeral: + +- **Structured files**: JSON feature lists, progress.txt, TODO trackers — persisted outside context +- **Git history**: Commits as checkpoints; agent can recover state from diff history +- **Least-to-Most decomposition**: Break complex tasks into subtasks, solve sequentially, each building on prior results + +These patterns connect to [[context-engineering]] compaction strategies — state tracking is context management applied to multi-step workflows. + +## Token Budget Sweet Spot + +- Reasoning degrades around **3,000 tokens** of prompt (Levy et al., ACL 2024) +- Sweet spot: **150–300 words** of prompt text +- CoT token cost: **35–600%** more than direct prompting +- Chain of Draft (CoD) alternative: matches CoT accuracy using only **~7.6% of tokens** +- TALE-EP reduces CoT tokens by **67%** with **59% cost reduction** while maintaining performance +- Practical cost comparison: optimized prompting saves **~$706/day vs. $3,000/day** for naive approaches at scale + +## Automated Prompt Optimization + +Manual prompt engineering has diminishing returns. Automated techniques outperform human-crafted prompts: + +| System | Result | Source | +| --------------------------- | -------------------------------------------------------------- | ----------------- | +| **APE** (Zhou et al., 2022) | Human-level or better on **24/24 Instruction Induction tasks** | ICLR 2023 | +| **OPRO** (DeepMind) | **+8% GSM8K**, **+50% Big-Bench Hard** | Yang et al., 2024 | +| **DSPy** (Stanford) | **46.2% → 64.0%** accuracy via systematic prompt programming | Khattab et al. | + +The pattern: use LLMs to generate, evaluate, and refine prompts in an automated loop — this is meta-[[prompt-engineering]]. + +## Reflexion Pattern + +Generate → evaluate → refine, applied to agent behavior across episodes: + +1. **Generate**: Agent attempts a task +2. **Evaluate**: Outcome is assessed (test results, correctness checks) +3. **Refine**: Agent reflects on failures and adjusts approach for next attempt + +This is the Evaluator-Optimizer workflow from Anthropic's agent patterns — the same basic loop applied to [[skill-authoring]] instead of runtime behavior. + +## Related pages + +- [[context-engineering]] +- [[persuasion-in-ai]] +- [[whitespace-and-formatting]] +- [[agent-workflows]] diff --git a/wiki/knowledge/rpi-workflow.md b/wiki/knowledge/rpi-workflow.md new file mode 100644 index 00000000..0010c4f4 --- /dev/null +++ b/wiki/knowledge/rpi-workflow.md @@ -0,0 +1,118 @@ +# RPI Workflow + +**Summary**: The Research, Plan, Implement (RPI) workflow is a three-phase agentic engineering methodology developed by Dex Horthy at HumanLayer that structures AI coding work into sequential phases, each producing a compacted artifact, to maximize LLM output quality through disciplined context management. +**Sources**: `docs/agentic-engineering/research-plan-implement-rpi.md` +**Last updated**: 2026-04-21 + +--- + +## Overview + +The RPI workflow is the operational backbone of the Frequent Intentional Compaction (FIC) methodology. It addresses the fundamental constraint that LLM output quality depends entirely on context quality — the model is a stateless function, so what's in the context window determines what comes out (source: research-plan-implement-rpi.md). + +The three phases — Research → Plan → Implement — each produce a compacted artifact that becomes the sole input for the next phase. This structure eliminates context pollution and creates explicit human review checkpoints before errors can compound downstream. See [[rpir-workflow]] for Tyler Burleigh's extended version that adds an explicit Review phase. + +## The Three Phases + +### Phase 1: Research + +The research phase begins with a **fresh context window** containing only the problem definition. [[subagents]] perform noisy operations (glob, grep, file reads) in isolated contexts, returning compacted summaries to prevent contaminating the main context with raw search noise (source: research-plan-implement-rpi.md). + +**Output artifact**: `research_doc.md` (~200 lines) +**Context consumption**: High (file searches, code reading) +**Human review leverage**: ⭐⭐⭐⭐ (highest) + +The research document captures: +- Problem summary +- Relevant files identified +- Information flow analysis (e.g., `parse() → validate() → execute_test()`) +- Key findings +- Recommended approach + +### Phase 2: Plan + +The plan phase starts with a **clean context window** containing the research document and the original problem definition. No raw file contents or search results pollute the context at this stage (source: research-plan-implement-rpi.md). + +**Output artifact**: `implementation_plan.md` (~200 lines) +**Context consumption**: Medium (research doc + architecture analysis) +**Human review leverage**: ⭐⭐⭐ (high) + +The plan document captures: +- Numbered, sequential implementation steps +- Exact file paths to modify +- Function signatures and integration points +- Testing and verification procedures + +### Phase 3: Implement + +The implementation phase starts with a **clean context window** containing only the implementation plan. For complex tasks requiring multiple [[context-engineering]] compaction cycles, the agent updates `progress.md` to track state across context resets (source: research-plan-implement-rpi.md). + +**Output artifact**: Code files + tests +**Context consumption**: Low-Medium (plan doc + test outputs) +**Human review leverage**: ⭐ (lowest) + +## The Leverage Model + +The RPI workflow encodes a critical insight about error compounding (source: research-plan-implement-rpi.md): + +| Error Location | Downstream Impact | +|----------------|-------------------| +| Bad research | 1,000s of bad lines of code | +| Bad plan | 100s of bad lines of code | +| Bad code | 1 bad line of code | + +This means reviewing ~400 lines of specification artifacts (200 research + 200 plan) delivers more value than reviewing 2,000 lines of generated code. The leverage is highest at the earliest phases. + +## Human Review Checkpoints + +**Checkpoint 1 — Research Document**: Verify correct understanding of codebase structure, that relevant files are identified, and that information flow analysis is accurate. Cost of error: thousands of incorrectly architected lines. + +**Checkpoint 2 — Implementation Plan**: Verify sound architectural approach, appropriate file selections, and complete testing strategy. Cost of error: hundreds of lines in wrong locations or wrong patterns. + +**Checkpoint 3 — Code Review (Optional)**: Mental alignment with team, understanding of what changed and why. Cost of error: 1–10 lines of incorrect code (source: research-plan-implement-rpi.md). + +## Progress Compaction for Complex Tasks + +When implementation spans multiple context resets, `progress.md` tracks state: + +``` +progress/feature_progress.md +├─ Goal: Add cancellation support +├─ Completed Steps +│ ├─ [✓] Step 1: Add CancelToken struct +│ └─ [✓] Step 2: Thread token through runtime +├─ Current Step +│ └─ [→] Step 3: Integrate with async runtime +└─ Remaining Steps + └─ [ ] Step 4: Add WASM bindings +``` + +This is a key [[context-engineering]] pattern: compact state before a context boundary rather than carrying forward a bloated history (source: research-plan-implement-rpi.md). + +## Integration with FIC Methodology + +| FIC Principle | Workflow Implementation | +|---------------|-------------------------| +| Context Quality = f(Correctness, Completeness, Size, Trajectory) | Research validates correctness; Plan ensures completeness; Phase boundaries control size | +| 40–60% Context Utilization | Each phase starts fresh at 10–15% utilization | +| Intentional Compaction | Phase transitions require explicit artifact creation | +| Human-in-the-Loop | Review gates at Research and Plan phases | +| Subagent Isolation | Research phase extensively uses [[subagents]] | + +(source: research-plan-implement-rpi.md) + +## Workflow Variations + +While the canonical flow is Research → Plan → Implement, variations are common. The key principle is that each phase must produce a correctly-scoped, compacted artifact before the next phase begins. For a formalized review-augmented variant, see [[rpir-workflow]]. For the synthesized community perspective on this pattern, see [[agentic-engineering-workflow]]. + +## Related pages + +- [[rpir-workflow]] +- [[agentic-engineering-workflow]] +- [[agent-harness]] +- [[agentic-software-modernization]] +- [[context-engineering]] +- [[subagents]] +- [[progressive-disclosure]] +- [[agent-workflows]] +- [[agent-best-practices]] diff --git a/wiki/knowledge/rpir-workflow.md b/wiki/knowledge/rpir-workflow.md new file mode 100644 index 00000000..a490220f --- /dev/null +++ b/wiki/knowledge/rpir-workflow.md @@ -0,0 +1,124 @@ +# RPIR Workflow + +**Summary**: The Research, Plan, Implement, Review (RPIR) workflow is Tyler Burleigh's extension of the [[rpi-workflow]] that adds explicit inter-phase review cycles, written artifact persistence, and a multi-agent scaling model, treating review as a first-class phase rather than an afterthought. +**Sources**: `docs/agentic-engineering/research-plan-implement-review-tyler-burleigh.md` +**Last updated**: 2026-04-21 + +--- + +## Overview + +Tyler Burleigh's RPIR workflow separates AI coding work into distinct stages — research, planning, implementation, and review — with explicit review cycles and fresh context windows between each phase. The core insight is that **the bottleneck in AI-assisted development is not code generation but ensuring the model understands what to build before it starts** (source: research-plan-implement-review-tyler-burleigh.md). + +Unlike a single-pass approach, RPIR builds written artifacts (`RESEARCH.md`, `PLAN.md`, `PLAN-CHECKLIST.md`) as persistent shared sources of truth across sessions. Each phase gets a dedicated context window — fresh enough to avoid carrying forward confusion from the previous phase. + +For the three-phase predecessor, see [[rpi-workflow]]. For the synthesized community perspective, see [[agentic-engineering-workflow]]. + +## Core Principles + +1. **AI agents are fallible and cut corners.** They miss things, take shortcuts, and make confident-sounding mistakes. A single pass is rarely enough — build in review cycles. +2. **Fresh context windows prevent compounding confusion.** Each new session means the model isn't carrying forward misunderstandings or stale assumptions. +3. **Written artifacts are the source of truth.** `RESEARCH.md`, `PLAN.md`, and `PLAN-CHECKLIST.md` are the shared ground truth between you and the model. Without them, context is lost between sessions. +4. **Separate research, planning, and implementation.** Mixing them leads the model to jump to code before understanding the problem. +5. **Small phases reduce blast radius.** A bug introduced in Phase 1 doesn't propagate silently through Phases 2–5. +6. **Your time is expensive, tokens are cheap.** Optimize for output quality, not token efficiency. +7. **Let AI review before you do.** By the time you look at it, the obvious problems are fixed and you can focus on what only a human would catch. +8. **Git history is your safety net.** Commit after each phase so you can always revert. +9. **Watch for over-engineering.** Models tend to add abstractions, error handling for impossible scenarios, or features beyond what was requested. + +(source: research-plan-implement-review-tyler-burleigh.md) + +## The Process (10 Steps) + +### Setup +**Step 1 — Clear problem statement.** Describe the problem or feature with supporting context: paths to relevant files, reference documents, anything that clarifies "what, why, or how." + +### Research Phase +**Step 2 — Research.** Ask the agent to research the codebase (existing projects) or do web research (greenfield), writing everything to `RESEARCH.md`. Read it yourself and revise as necessary — the model may miss things or get details wrong. + +**Step 3 — Research review.** *New session.* Tell the agent: "Review RESEARCH.md for accuracy and completeness." Read its feedback, tell it to revise, answer questions, and tell it which points to ignore if any are off-base. Repeat until satisfied. + +### Planning Phase +**Step 4 — Planning.** *New session.* Give the agent your problem statement, tell it to read `RESEARCH.md` and develop a plan, saving to `PLAN.md`. For complex work, also have it write `PLAN-CHECKLIST.md` breaking work into phases and tasks. + +**Step 5 — Plan review.** *New session.* Tell the agent: "Review PLAN.md as a senior engineer." Read recommendations and clarifying questions, tell it to revise, and ignore off-base points. Repeat until the plan is solid. + +### Implementation Phase +**Step 6 — Implementation.** *New session.* "Implement Phase 1 of PLAN.md, using PLAN-CHECKLIST.md to track your work." + +**Step 7 — Implementation review.** *New session.* "Review the implementation of Phase 1 against PLAN.md as a senior engineer." Read feedback, tell it to address its own recommendations. Repeat until the phase is clean. + +**Step 8 — Commit.** Once satisfied with a phase, commit all changes. Repeat steps 6–8 for each phase. + +### Final Phase +**Step 9 — Final review.** *New session.* "Review all changes on this branch as a senior engineer." This holistic review may catch issues that only emerge when the pieces come together. + +**Step 10 — Refactoring (optional).** *New session.* "Review all changes on this branch and identify high-leverage refactoring opportunities." Save findings to `REFACTOR_PLAN.md`. + +(source: research-plan-implement-review-tyler-burleigh.md) + +## Comparison with RPI + +| Dimension | [[rpi-workflow]] (HumanLayer/Horthy) | RPIR (Burleigh) | +|-----------|---------------------------------------|-----------------| +| Core focus | [[context-engineering]] / context window management | Review cycle discipline | +| Key concept | "Dumb Zone" above ~40% context utilization | Fresh sessions between every phase | +| Artifacts | `research_doc.md`, `implementation_plan.md`, `progress.md` | `RESEARCH.md`, `PLAN.md`, `PLAN-CHECKLIST.md` | +| Review | Checkpoints at research + plan | Explicit review session after every phase | +| Human role | Review gates at R and P phases | Active revision at every review step | +| Scaling | [[subagents]] for isolation | Multi-model with role specialization | + +Both methodologies emerged as responses to unstructured "vibe coding" producing unreliable results. The convergence across independent practitioners suggests the core pattern is sound (source: research-plan-implement-review-tyler-burleigh.md). + +## Tooling: Git Worktrees + +Git worktrees let you check out multiple branches into separate directories. This enables running multiple agents in parallel, each on a different feature branch in its own worktree, without file conflicts (source: research-plan-implement-review-tyler-burleigh.md): + +```bash +git worktree add ../myproject-feature-x feature-x +git worktree add ../myproject-feature-y feature-y +# When done: +git worktree remove ../myproject-feature-x +``` + +## Tooling: LSP Plugins + +Claude Code supports Language Server Protocol plugins that give the agent code intelligence identical to VS Code — jump to definition, find references, and real-time type error detection (source: research-plan-implement-review-tyler-burleigh.md): + +| Language | Plugin | Binary required | +|----------|--------|----------------| +| Python | `pyright-lsp` | `pyright-langserver` | +| Go | `gopls-lsp` | `gopls` | +| TypeScript | `typescript-lsp` | `typescript-language-server` | +| Rust | `rust-analyzer-lsp` | `rust-analyzer` | + +## Scaling with Multiple Agents + +**Use different models for different roles.** Models with different architectures produce largely uncorrelated errors. Use one model to implement and a different model to review (source: research-plan-implement-review-tyler-burleigh.md). + +**Match model capability to task complexity.** Research, planning, and review require synthesis and judgment. Implementation — guided by a detailed plan — is largely mechanical. Use your strongest model for high-leverage phases. + +**A more automated workflow:** +1. **Research and planning** — Agent produces `RESEARCH.md` and `PLAN.md`. Human reviews and approves. (Highest leverage.) +2. **Implementation loop** — Implementer agent executes; reviewer agent (different model) reviews against `PLAN.md`; they iterate. No human involvement required. +3. **Pull request** — Human reviews cumulative changes as a PR. + +| Plan detail | Safe autonomy level | +|-------------|---------------------| +| High-level goal | Human-in-the-loop for every step | +| Phased plan with architecture decisions made | Autonomous per phase, review between phases | +| Detailed plan with file paths and function signatures | Autonomous implementation, human reviews PR | +| Exact specifications with test cases | Fully autonomous with automated verification | + +(source: research-plan-implement-review-tyler-burleigh.md) + +## Related pages + +- [[rpi-workflow]] +- [[agentic-engineering-workflow]] +- [[agent-harness]] +- [[agent-best-practices]] +- [[context-engineering]] +- [[subagents]] +- [[agent-workflows]] +- [[agentic-software-modernization]] diff --git a/wiki/knowledge/skill-authoring.md b/wiki/knowledge/skill-authoring.md new file mode 100644 index 00000000..11aed6a8 --- /dev/null +++ b/wiki/knowledge/skill-authoring.md @@ -0,0 +1,199 @@ +# Skill Authoring + +**Summary**: Evidence-based practices for creating effective agent skills — covering the "start from real expertise" principle, context budgeting, progressive disclosure structure, eval-driven iteration, description optimization for trigger accuracy, and script bundling conventions. +**Sources**: skill-authoring-best-practices.md, agentskills-best-practices.md, agentskills-evaluating-skills.md, agentskills-optimizing-descriptions.md, agentskills-using-scripts.md +**Last updated**: 2026-04-22 + +--- + +## Starting Point: Real Expertise + +Skills must encode real knowledge, not LLM-generated generic content: + +- Extract from hands-on tasks: steps that worked, corrections made, I/O formats +- Synthesize from existing artifacts: runbooks, API specs, code reviews, git history, failure cases +- Challenge every line: **"Would the agent get this wrong without this instruction?"** +- Assume the agent already knows common tools (HTTP, databases, migrations) + +## Content Budget + +| Element | Target | +| ----------------------------- | ------------------------------- | +| SKILL.md body | Under 500 lines (~5,000 tokens) | +| Metadata (name + description) | ~100 tokens | +| Reference files | On-demand, 1 level deep | + +## Structure Patterns + +### Gotchas Section + +Keep in SKILL.md for pre-task reading. Document non-obvious edge cases and environment-specific facts. + +### Templates + +- Inline for short, universal templates +- In `assets/` for longer or conditional templates + +### Checklists + +Explicit progress tracking with dependencies. + +### Validation Loops + +Do work → run validator → fix issues → repeat until passing. + +### Plan-Validate-Execute + +For batch/destructive operations: create plan → validate against source of truth → execute. + +## Instruction Specificity + +Match specificity to task fragility: + +| Task Type | Approach | +| -------------------------------- | --------------------------------- | +| Flexible (many valid approaches) | High freedom, broad guidance | +| Fragile (one correct approach) | Low freedom, prescriptive scripts | + +Provide **defaults with escape hatches**, not feature menus. + +## Eval-Driven Iteration + +### Test Cases + +- Start with 2–3 test cases; expand after first results +- Each case: `prompt` (realistic), `expected_output` (human description), optional `files` +- Run both **with and without** skill to establish baseline +- Store test cases in `evals/evals.json` inside the skill directory + +### Assertions + +- Verifiable statements: specific, observable, countable +- Good: "Output file is valid JSON", "Chart has labeled axes", "Report includes ≥3 recommendations" +- Bad: "Output is good" (too vague), "Uses exact phrase X" (too brittle) +- Remove assertions that always pass in both configs (they inflate scores without measuring skill value) +- Study assertions that pass with skill but fail without (clearest value signal) + +### Grading Outputs + +- Each assertion gets **PASS** or **FAIL** with specific evidence (quote the output, don't just opine) +- Use LLM grading for subjective assertions, scripts for mechanical checks (valid JSON, row count) +- **Blind comparison**: present both with/without outputs to an LLM judge without revealing which is which — catches holistic quality differences beyond individual assertions + +### The Iteration Loop + +1. Give eval signals (failed assertions + human feedback + execution transcripts) to an LLM +2. LLM proposes skill improvements — **generalize** from feedback, don't patch specific test cases +3. Review and apply changes +4. Rerun all test cases in a new `iteration-/` directory +5. Grade and aggregate results +6. Human review — record specific, actionable feedback +7. Repeat until feedback is consistently empty or no meaningful improvement between iterations + +### Improvement Principles + +- **Generalize from feedback** — fixes should address underlying issues broadly, not add narrow patches +- **Keep the skill lean** — fewer, better instructions outperform exhaustive rules +- **Explain the why** — reasoning-based instructions ("Do X because Y causes Z") outperform rigid directives +- **Bundle repeated work** — if every test run writes a similar helper script, bundle it in `scripts/` + +## Description Optimization + +The description carries the **entire triggering burden** — if it doesn't match, the agent won't load the skill. At startup, agents load only `name` + `description` (~100 tokens) for each available skill. Only when a task matches does the full SKILL.md enter context. + +### Eval Query Design + +- Aim for **~20 queries**: 8–10 should-trigger, 8–10 should-not-trigger +- **Should-trigger**: vary phrasing (formal/casual), explicitness (named domain vs. implied need), and complexity +- **Should-not-trigger**: focus on **near-misses** — queries sharing keywords but needing different skills. "Write a fibonacci function" tests nothing; "update formulas in my Excel budget" tests boundary precision +- Include realistic context: file paths, personal context, casual language, typos + +### Optimization Process + +1. Split queries: **60% train / 40% validation** — proportional should/shouldn't in each set +2. Run each query **N times** (3 minimum) — compute trigger rate per query +3. A should-trigger query passes if trigger rate > 0.5; should-not-trigger passes if < 0.5 +4. Identify train-set failures; revise description to generalize (not to match specific query keywords) +5. Re-evaluate on both train and validation sets +6. Select the iteration with **highest validation pass rate** (may not be the last iteration) +7. Final check: 5–10 fresh queries never seen during optimization + +Five iterations is usually enough. If performance plateaus, try a structurally different description framing rather than incremental tweaks. + +### Description Writing Tips + +- Use **imperative phrasing**: "Use when..." not "This skill helps..." +- Focus on **user intent**, not implementation details +- Be **explicitly pushy** about contexts where skill applies — "even if they don't mention 'CSV' or 'analysis'" +- Avoid adding specific keywords from failed queries — that's **overfitting** +- Stay under the **1024-character** hard limit (descriptions tend to grow during optimization) + +## Script Conventions + +### One-Off Commands + +Use existing package runners with pinned versions: + +- `uvx ruff@0.8.0 check .` (Python) +- `npx eslint@9 --fix .` (Node.js) +- `deno run npm:eslint@9 -- --fix .` (Deno) + +### Bundled Scripts + +Self-contained with inline dependencies — no separate manifest or install step: + +- **Python**: PEP 723 inline metadata (`# /// script` + `# dependencies = [...]`); run with `uv run script.py` +- **Deno**: `npm:` and `jsr:` import specifiers with version pinning (`npm:cheerio@1.0.0`); auto-resolves +- **Bun**: Auto-installs missing packages when no `node_modules` exists; TypeScript works natively +- **Ruby**: `bundler/inline` for gem declarations directly in the script + +### Script Interface Rules + +- **No interactive prompts** — agents run non-interactive shells; blocking on TTY input hangs indefinitely +- Accept input via CLI flags, env vars, or stdin +- Implement `--help` — primary way agents learn a script's interface +- Structured output (JSON/CSV) to stdout; diagnostics to stderr +- `--dry-run` flag for destructive operations +- **Meaningful exit codes** for different failure types (not found, invalid args, auth failure) — document in `--help` +- **Idempotent**: "create if not exists" over "create and fail on duplicate" — agents may retry +- Predictable output size — many harnesses truncate at 10–30K characters + +## Multi-Model Testing + +Test skills across model tiers to calibrate instruction specificity: + +| Model Tier | Test Question | Implication | +| ------------------------- | -------------------------------------------------- | ------------------------------------------------------- | +| **Haiku** (small/fast) | Does it have enough guidance to complete the task? | If Haiku struggles, instructions may be too sparse | +| **Sonnet** (balanced) | Are instructions clear and efficient? | The primary target for most skills | +| **Opus** (large/powerful) | Are you over-explaining things it already knows? | Unnecessary instructions waste tokens on capable models | + +A skill that works on Sonnet but fails on Haiku needs more explicit guidance. A skill where Opus follows instructions literally that were meant as guidelines may be over-constraining. + +## Reference File Organization + +- Keep references **one level deep** from SKILL.md — all reference files link directly from SKILL.md +- Claude may `head -100` nested references instead of reading completely — losing critical information +- For files over 100 lines, include a **table of contents** at the top so Claude sees the full scope even in partial reads +- Organize by **domain** (finance.md, sales.md, product.md) not by type (schemas.md, queries.md) — the agent loads only what each task requires + +When the user asks about revenue, Claude reads SKILL.md, sees the reference to `reference/finance.md`, and reads just that file. Other reference files consume **zero context tokens** until needed. + +## Metadata Cost + +The `name` + `description` fields cost ~**100 tokens** and load at startup for **every installed skill**. With 100+ skills installed, metadata alone consumes 10,000+ tokens. + +| Example | Tokens | Quality | +| ------------------------------------------------------------------------------ | ------ | ---------------------------------------- | +| `description: Helps with documents` | ~5 | Too vague — won't trigger correctly | +| `description: Analyze CSV and tabular data...even if they don't mention "CSV"` | ~50 | Good — specific triggers, broad coverage | +| `description: [150-word paragraph covering every edge case]` | ~150 | Over-specified — wastes startup budget | + +Target **~50 tokens** for description — enough for precise triggering, low enough to scale across many skills. + +## Related pages + +- [[agent-skills-standard]] +- [[claude-code-skills]] +- [[persuasion-in-ai]] +- [[progressive-disclosure]] diff --git a/wiki/knowledge/spec-driven-development-critique.md b/wiki/knowledge/spec-driven-development-critique.md new file mode 100644 index 00000000..fb2a6840 --- /dev/null +++ b/wiki/knowledge/spec-driven-development-critique.md @@ -0,0 +1,128 @@ +# Spec-Driven Development — Critical Analysis + +**Summary**: A hands-on critical analysis of three SDD tools (Kiro, GitHub Spec Kit, Tessl) from the Martin Fowler team, surfacing concrete failure modes, open questions, and the gap between SDD's promises and practical reality. +**Sources**: `docs/spec-driven-development/spec-driven-development-variant.md` +**Last updated**: 2026-04-22 + +--- + +## Overview + +The Martin Fowler team's "Exploring Gen AI" series evaluated Kiro, GitHub Spec Kit, and Tessl by actually using them — going beyond tool documentation to surface real friction. The verdict is measured: SDD tooling shows promise but has significant unresolved issues before it can be recommended for typical production codebases (source: spec-driven-development-variant.md). + +See the tooling overview and academic backing in [[spec-driven-development]] and the practitioner walkthrough in [[spec-driven-development-practice]]. + +--- + +## What "Spec" Actually Means + +The definition of "spec" in SDD is still in flux. The closest to a consistent definition found across all three tools (source: spec-driven-development-variant.md): + +> A spec is a structured, behavior-oriented artifact — or a set of related artifacts — written in natural language that expresses software functionality and serves as guidance to AI coding agents. + +A useful distinction: **specs** differ from general **context documents** (rules files, high-level product/codebase descriptions). Context documents are relevant across all AI coding sessions; specs are only relevant to the tasks that create or change that particular functionality. Some tools call this context a **memory bank**. + +--- + +## Tool Comparison + +### Kiro + +The simplest of the three — primarily spec-first with no clear strategy for spec maintenance after the task is done (source: spec-driven-development-variant.md). + +**Workflow**: Requirements → Design → Tasks (one markdown document per step) +- Requirements: structured as user stories in "As a…" format with GIVEN/WHEN/THEN acceptance criteria +- Design: component architecture, data flow, data models, error handling, testing strategy +- Tasks: traceable to requirement numbers; UI elements to run tasks one by one + +Kiro's memory bank concept is called "steering" (`product.md`, `structure.md`, `tech.md`). + +**Critical finding**: When asked to fix a simple bug, Kiro turned it into 4 user stories with 16 acceptance criteria, including: *"As a developer, I want the transformation function to handle edge cases gracefully."* The overhead exceeded the problem. + +### Spec-kit + +GitHub's SDD implementation — the most customizable due to all artifacts landing directly in your workspace (source: spec-driven-development-variant.md). + +**Workflow**: Constitution → 𝄆 Specify → Plan → Tasks 𝄇 + +The **constitution** is a powerful rules file — Spec Kit's "memory bank" — containing immutable project-level principles applied to every change. + +Each spec generates many files: `data-model`, `plan`, `tasks`, `spec`, `research`, `api`, `component` — potentially 8+ files per feature. The workflow uses checklists extensively inside files to track clarifications, constitution violations, research tasks — a "definition of done" for each step. + +**Critical finding**: For a feature that would be a 3–5 point story on a real team, Spec Kit generated so many markdown files to review that it felt like overkill. In the same time taken to run and review Spec Kit results, the feature could have been implemented with plain AI-assisted coding, with more sense of control. + +### Tessl + +The only tool explicitly aspiring to spec-anchored and spec-as-source — still in private beta (source: spec-driven-development-variant.md). + +**Workflow**: `tessl document --code ...js` (reverse-engineer spec from code) → `tessl build` (regenerate code from spec) + +Code is marked `// GENERATED FROM SPEC - DO NOT EDIT`. Tags like `@generate` and `@test` tell Tessl what to generate. The 1:1 mapping between spec and code files keeps abstraction level low, reducing LLM interpretation steps and error surface. + +**Observed finding**: Non-determinism appeared even at this low abstraction level — generating code multiple times from the same spec produced different outputs. Iterating on the spec to make it more specific increased repeatability. + +--- + +## Concrete Failure Modes + +### One Workflow Does Not Fit All Sizes + +Both Kiro and Spec Kit provide one opinionated workflow, but neither adapts to the actual size of the problem (source: spec-driven-development-variant.md). An effective SDD tool must provide **flexibility for different change sizes and types**. + +### Reviewing Markdown vs. Reviewing Code + +Spec Kit created repetitive, verbose markdown files — sometimes containing code already, sometimes redundant with existing code. Overall, they were tedious to review (source: spec-driven-development-variant.md). + +> "To be honest: reviewing code directly is often preferable to reviewing all these markdown files." + +### False Sense of Control + +Despite extensive files, templates, prompts, and checklists, agents frequently did not follow all instructions. In one case, Spec Kit's agent read descriptions of existing classes as new specifications and generated them all over again, creating duplicates (source: spec-driven-development-variant.md). + +The best way to stay in control remains **small, iterative steps** — which conflicts with the upfront, comprehensive spec design that SDD promotes. + +### Functional vs. Technical Spec Confusion + +SDD tools encourage separating functional spec (what the software does) from technical implementation (how it does it). In practice with Spec Kit, there was frequent confusion about when to stay functional and when to add technical detail. As the author notes: "We don't have a good track record as a profession to do this well" (source: spec-driven-development-variant.md). + +--- + +## Tool Summary Assessment + +| Tool | Spec Level | Complexity | Best For | +|------|-----------|------------|----------| +| Kiro | Spec-first | Low | Simple, well-defined tasks | +| Spec-kit | Spec-first (aspiring anchored) | High | Larger greenfield features | +| Tessl | Spec-anchored / Spec-as-source | Medium-High | Teams committed to spec-driven long-term | + +(source: spec-driven-development-variant.md) + +--- + +## Evaluating SDD Is Hard + +Honest evaluation requires trying tools with different problem sizes, greenfield and brownfield codebases, and genuinely taking time to review and revise intermediate artifacts. Introducing Kiro or Spec Kit into an existing codebase is significantly more work than greenfield use, making brownfield evaluation harder (source: spec-driven-development-variant.md). + +As GitHub's own blog states: "Crucially, your role isn't just to steer. It's to verify. At each phase, you reflect and refine." + +--- + +## Open Questions + +- Does SDD scale to real production codebases over time, not just isolated demos? +- Who is the actual target user — experienced developers or those who benefit from structured guidance? +- How does spec maintenance overhead compare to traditional code review overhead over months of usage? +- Can tools adapt their spec depth to the actual size and complexity of each change? + +--- + +## Related pages + +- [[spec-driven-development]] +- [[spec-driven-development-practice]] +- [[spec-first-ai-development]] +- [[code-to-contract]] +- [[agent-workflows]] +- [[agent-best-practices]] +- [[context-engineering]] +- [[context-rot]] diff --git a/wiki/knowledge/spec-driven-development-practice.md b/wiki/knowledge/spec-driven-development-practice.md new file mode 100644 index 00000000..89e7a8d4 --- /dev/null +++ b/wiki/knowledge/spec-driven-development-practice.md @@ -0,0 +1,139 @@ +# Spec-Driven Development — Practitioner Guide + +**Summary**: A practitioner's guide to SDD in 2026, contrasting vibe coding with the spec-first approach and walking through GitHub Spec Kit, AWS Kiro, and Tessl with honest assessment of where SDD breaks down. +**Sources**: `docs/spec-driven-development/spec-driven-development-main.md` +**Last updated**: 2026-04-22 + +--- + +## Vibe Coding vs. Spec-First + +"Vibe coding" is the pattern of describing what you want, accepting the AI's output, and shipping — then spending the next week debugging assumptions the model made because you never stated them (source: spec-driven-development-main.md). + +Spec-driven development is the counter-approach: write a structured, behavior-oriented specification first, then let the AI generate code to match it. The core insight is that **language models are excellent at pattern completion but bad at mind reading** (source: spec-driven-development-main.md). + +Without a spec, asking an AI to "build a REST API for user management" leaves thousands of decisions unstated: authentication method, error response format, pagination strategy, rate limiting, input validation rules. The agent fills gaps with its training data, which may not match your requirements. + +The key difference from vibe coding is **where you spend your time**: in vibe coding, you iterate on code after generation; in SDD, you invest in the spec before generation. The total time is often comparable — but the spec is reusable and serves as documentation after the project ships (source: spec-driven-development-main.md). + +Related: [[spec-first-ai-development]], [[context-engineering]], [[agent-best-practices]] + +--- + +## The Three Levels + +Most teams today start at **spec-first**, which is where the practical payoff begins (source: spec-driven-development-main.md): + +| Level | Description | +|-------|-------------| +| **Spec-first** | Write spec for the immediate task | +| **Spec-anchored** | Maintain spec as a living document alongside code | +| **Spec-as-source** | Spec becomes the canonical artifact; code is entirely generated | + +See [[spec-driven-development]] for the academic treatment of all three levels. + +--- + +## Three Tools, Three Approaches + +### GitHub Spec Kit + +The most customizable option — an open-source CLI integrating with Copilot, Claude Code, and Gemini CLI via slash commands (source: spec-driven-development-main.md): + +- `/specify` — generates a detailed specification from your description +- `/plan` — creates a technical implementation plan given your stack and constraints +- `/tasks` — breaks the plan into small, reviewable chunks with acceptance criteria from the spec +- Agent implements each task sequentially, using spec and plan as context + +Spec Kit enforces architectural rules through a **"constitutional foundation"** — a set of project-level constraints the agent must obey. This parallels the memory bank concept described in [[agent-configuration-files]]. + +### Amazon Kiro + +The simplest entry point — a VS Code extension by AWS that produces three markdown documents: requirements, design, and tasks (source: spec-driven-development-main.md). + +The workflow is linear and lightweight. The tradeoff: Kiro generated **16 acceptance criteria for a simple bug fix**. The overhead can exceed the problem. + +### Tessl Framework + +The most ambitious approach — still in closed beta. Pursues spec-as-source by reverse-engineering specs from existing code and maintaining a **1:1 mapping between spec files and code files** (source: spec-driven-development-main.md). + +Generated code is marked `// GENERATED FROM SPEC - DO NOT EDIT`. Developers would maintain only specs, never touching code directly. + +The practical reality across all three: **AI agents still inconsistently follow instructions**. A spec reduces the gap between intent and implementation, but it does not eliminate non-determinism. The spec is a guardrail, not a guarantee. + +--- + +## Getting Started with Spec Kit + +Step-by-step for the most accessible tool (source: spec-driven-development-main.md): + +1. **Install**: CLI available via npm; `specify init` creates a `.specify/` directory with templates +2. **Write your first spec**: Run `/specify`, describe the feature with specific behavior, constraints, edge cases; review and approve the generated spec before any code is written +3. **Generate a plan**: Run `/plan` with your tech stack; output is a step-by-step plan referencing your spec +4. **Break into tasks**: Run `/tasks` to split the plan into small, reviewable work units each with clear objectives and acceptance criteria +5. **Implement**: Agent works through tasks sequentially; you review each completed task against the spec + +--- + +## The Difference in Practice + +**Vibe coding prompt:** +> "Build a rate limiter middleware for Express." + +**Spec-first prompt:** +> "Implement the rate limiter defined in `.spec/features/rate-limiter.md`, which specifies a sliding window algorithm, 100 requests per minute per API key, 429 responses with Retry-After headers, and Redis-backed state for horizontal scaling." + +The second prompt leaves no room for the agent to improvise on decisions that should be yours (source: spec-driven-development-main.md). + +> **Key insight**: Move the ambiguity from code review to spec review, where it is cheaper to fix. (source: spec-driven-development-main.md) + +--- + +## Real-World Examples + +- **Anthropic** used GCC test suites to spec a Rust-based C compiler +- **Vercel** used curated shell script tests for a TypeScript bash emulator +- **Pydantic** applied the same approach to a Python sandbox for AI agents + +A well-defined spec plus an existing test suite gets an AI agent far on a greenfield build (source: spec-driven-development-main.md). + +--- + +## Where SDD Breaks Down + +SDD is not a universal improvement. Honest friction points (source: spec-driven-development-main.md): + +### Review Overhead Scales with Spec Verbosity + +Kiro's 16 acceptance criteria for a bug fix is not an edge case. Spec Kit produces extensive markdown for mid-sized features. If reviewing the spec takes longer than reviewing the code would have, the process is working against you. + +### Poor Fit for Exploratory Work + +Prototyping, UI experiments, and data pipeline debugging benefit from fast, loose iteration. Writing a detailed spec before you know what you are building adds latency to a process that should be cheap and fast. + +### Non-Determinism Persists + +Even with a detailed spec, agents sometimes ignore directives or over-interpret them. The spec improves consistency but does not solve the fundamental reliability problem. + +As Vercel's CTO put it: "Software is free now. Free as in puppies." Generation is cheap. Maintenance is where the work lives. + +--- + +## The Sweet Spot + +SDD in its current form works best for **greenfield features with well-understood requirements**: new API endpoints, CRUD modules, integration layers. It is less useful for exploratory work or codebases where existing architecture is poorly documented (source: spec-driven-development-main.md). + +**Starting point without tooling**: Before your next feature, write a one-page spec in plain markdown before prompting your AI agent. Define inputs, outputs, constraints, and edge cases. Pass that spec as context alongside your prompt. You do not need Spec Kit or Kiro to start. + +--- + +## Related pages + +- [[spec-driven-development]] +- [[spec-driven-development-critique]] +- [[spec-first-ai-development]] +- [[code-to-contract]] +- [[context-engineering]] +- [[agent-workflows]] +- [[agent-best-practices]] +- [[agentic-engineering-workflow]] diff --git a/wiki/knowledge/spec-driven-development.md b/wiki/knowledge/spec-driven-development.md new file mode 100644 index 00000000..eef43cd8 --- /dev/null +++ b/wiki/knowledge/spec-driven-development.md @@ -0,0 +1,168 @@ +# Spec-Driven Development + +**Summary**: Academic overview of Spec-Driven Development (SDD) — the practice of treating specifications as the primary artifact of software development, with code as a generated or verified secondary artifact, especially in the AI coding era. +**Sources**: `docs/spec-driven-development/spec-driven-development-arxiv.md` +**Last updated**: 2026-04-22 + +--- + +## What Is SDD? + +Spec-Driven Development (SDD) inverts the traditional workflow: instead of writing code first and documenting later (or never), teams write clear specifications of intended behavior, then generate, implement, or verify code against those specifications (source: spec-driven-development-arxiv.md). + +> **Core Principle:** In spec-driven development, code is the implementation detail of the specification — not the other way around. The spec declares intent; the code realizes it. (source: spec-driven-development-arxiv.md) + +The AI catalyst is key: language models are excellent at pattern completion but poor at mind reading. A vague prompt like "Add photo sharing to my app" leaves dozens of decisions to the model. A precise spec — "Users can upload JPEG or PNG photos up to 10MB, stored in S3 with user-ID-prefixed keys, resized to 1024px max on upload" — eliminates guesswork and dramatically improves output quality (source: spec-driven-development-arxiv.md). + +This connects directly to [[context-engineering]], where structured specs act as rich context that improves [[agent-workflows]]. + +--- + +## The Specification Spectrum + +SDD is not one approach — it exists on a spectrum of rigor (source: spec-driven-development-arxiv.md): + +### Spec-First + +A specification is written before coding to guide initial implementation. Once code exists, the spec may or may not be maintained. Works well for initial AI-assisted feature development and prototypes. Does **not** protect against long-term drift. + +### Spec-Anchored + +The specification is maintained alongside code throughout the system's lifecycle. Changes to behavior require updating both spec and code. BDD frameworks like Cucumber and API contract testing tools like Specmatic exemplify this approach. This is the **sweet spot for most production systems** (source: spec-driven-development-arxiv.md). + +### Spec-as-Source + +The specification is the only artifact humans edit. Code is entirely generated from the spec and should never be manually modified. Already standard in domains like API stub generation (OpenAPI), and automotive embedded systems (Simulink/ISO 26262). Tools like Tessl aim to extend this to general software (source: spec-driven-development-arxiv.md). + +--- + +## The SDD Workflow + +A four-phase workflow applies across all SDD approaches (source: spec-driven-development-arxiv.md): + +1. **Specify** — What should the software do? Produces a behavior-focused, testable, unambiguous functional specification. +2. **Plan** — How should we build it? Architecture, data models, interfaces, technology choices. Gives AI agents context on system structure and conventions. +3. **Implement** — Working code in small, validated increments. Specs act as "super-prompts" that break problems into modular components aligned with agents' context windows. +4. **Validate** — Does the code meet the spec? Combines automated tests and stakeholder review. The spec remains the authority; if gaps appear, either fix code or revise spec. + +This maps closely to the [[rpi-workflow]] (Research-Plan-Implement) and [[rpir-workflow]] patterns used in [[agentic-engineering-workflow]]. + +--- + +## SDD and AI Coding Agents + +Empirical studies suggest human-refined specs significantly improve LLM-generated code quality, with controlled studies showing **error reductions of up to 50%** (source: spec-driven-development-arxiv.md). + +Additional benefits: +- **Parallel agent execution**: teams can partition work at the spec level, allowing multiple AI agents to implement different components simultaneously without interference +- **Property-based testing (PBT)**: addresses LLM non-determinism by verifying spec invariants hold regardless of implementation variation +- **Self-spec methods**: LLMs can author their own specifications from a high-level prompt, which humans then review and refine before implementation (source: spec-driven-development-arxiv.md) + +See [[agent-best-practices]] for related guidance on structuring AI coding workflows. + +--- + +## Tools and Frameworks + +### BDD Frameworks + +Gherkin-based executable specs using Given/When/Then format (source: spec-driven-development-arxiv.md): +- **Cucumber** (Ruby, Java, JavaScript) +- **SpecFlow / Reqnroll** (.NET) +- **Behave** (Python) + +### API Specification Tools + +- **OpenAPI** — REST API contracts; generates server stubs, client SDKs, documentation +- **GraphQL SDL** — schema as contract between frontend and backend +- **AsyncAPI** — event-driven architectures +- **Protocol Buffers / gRPC** — strongly typed service interfaces +- **Pact / Specmatic** — automated contract testing against live implementations + +### AI-Assisted SDD Tools (2025–2026) + +| Tool | Level | Approach | +|------|-------|----------| +| **GitHub Spec Kit** | Spec-first (aspiring anchored) | CLI; `/specify` → `/plan` → `/tasks` → implement; constitutional foundation | +| **Amazon Kiro** | Spec-first | VS Code extension; 3-document workflow: requirements, design, tasks | +| **Tessl** | Spec-anchored / spec-as-source | Code marked `// GENERATED FROM SPEC - DO NOT EDIT`; 1:1 spec-to-file mapping | + +(source: spec-driven-development-arxiv.md) + +--- + +## Case Studies + +### API-First Microservices — Financial Services + +- **Pattern**: Spec-anchored with OpenAPI + Specmatic +- **Outcome**: 75% reduction in integration cycle time; mock servers from specs enabled frontend/backend parallel development; drift caused immediate build failures (source: spec-driven-development-arxiv.md) + +### BDD for Enterprise Features — Project Management Software + +- **Pattern**: Spec-anchored with Cucumber +- **Outcome**: Product managers wrote Gherkin scenarios; a feature was "done" only when all scenarios passed; reduced requirement ambiguity (source: spec-driven-development-arxiv.md) + +### Model-Based Embedded Development — Automotive + +- **Pattern**: Spec-as-source with Simulink (ISO 26262 certified) +- **Outcome**: Engineers model control algorithms, verify at model level, auto-generate certified C code no one hand-edits (source: spec-driven-development-arxiv.md) + +--- + +## When to Use SDD + +**SDD adds clear value when** (source: spec-driven-development-arxiv.md): +- Using AI coding assistants (specs dramatically improve output quality) +- Dealing with complex or regulated requirements +- Systems with multiple maintainers (specs serve as documentation) +- Integration-heavy systems (API specs enable parallel development) +- Legacy modernization (specs for existing behavior enable clean reimplementation) + +**SDD may be overkill when**: +- Throwaway prototypes or exploratory coding +- Solo, short-lived projects +- Simple CRUD with obvious requirements + +> **The Golden Rule:** Use the minimum level of specification rigor that removes ambiguity for your context. (source: spec-driven-development-arxiv.md) + +--- + +## Common Pitfalls + +1. **Over-specification** — specs that read like pseudo-code, constraining implementation unnecessarily +2. **Specification rot** — specs drift from reality when not updated as code changes; see [[context-rot]] +3. **Specification as bureaucracy** — forms to fill rather than tools for clarity +4. **Tooling complexity** — drowning in generated artifacts without demonstrable value +5. **False confidence** — a passing spec test only guarantees the software matches the spec, not that the spec itself is correct + +(source: spec-driven-development-arxiv.md) + +--- + +## Relationship to Existing Practices + +- **TDD**: SDD at the unit level — extended to features, systems, architectures +- **BDD**: The most direct ancestor — Gherkin scenarios are executable specifications +- **DDD**: Aligns through ubiquitous language — specs written in terms both developers and stakeholders understand +- **Agile**: User stories with acceptance criteria are specifications; SDD treats them as authoritative rather than advisory + +(source: spec-driven-development-arxiv.md) + +As Bryan Finster observed: "SDD is not a revolution… it's just BDD with branding." What's new is the tooling, CI/CD maturity, and AI as a spec consumer. + +--- + +## Related pages + +- [[spec-driven-development-practice]] +- [[spec-driven-development-critique]] +- [[spec-first-ai-development]] +- [[code-to-contract]] +- [[context-engineering]] +- [[agent-workflows]] +- [[rpi-workflow]] +- [[rpir-workflow]] +- [[agentic-engineering-workflow]] +- [[agent-best-practices]] +- [[context-rot]] diff --git a/wiki/knowledge/spec-first-ai-development.md b/wiki/knowledge/spec-first-ai-development.md new file mode 100644 index 00000000..b5a1f4f0 --- /dev/null +++ b/wiki/knowledge/spec-first-ai-development.md @@ -0,0 +1,161 @@ +# Spec-First AI Development + +**Summary**: The spec-first paradigm — why writing clear, structured specifications before code matters when working with AI coding assistants, and how it transforms the developer's role from code producer to spec author and AI orchestrator. +**Sources**: `docs/spec-driven-development/spec-driven-development-arxiv.md`, `docs/spec-driven-development/spec-driven-development-main.md`, `docs/spec-driven-development/spec-driven-development-variant.md` +**Last updated**: 2026-04-21 + +--- + +## The Core Problem: AI Can't Read Minds + +Large language models are excellent at pattern completion but poor at mind reading (source: spec-driven-development-arxiv.md). When you give an AI agent a vague prompt — "Add photo sharing to my app" — the model must guess dozens of unstated decisions: + +- What format? What permissions model? What size limits? +- Cloud storage or local? Compression? Resizing? + +The result is plausible-looking code that makes dozens of assumptions, many of them wrong. This is "vibe coding" — relying on loose prompts that lead to inconsistent or erroneous outputs (source: spec-driven-development-arxiv.md). + +Spec-first development addresses this at the root: **give the AI an unambiguous, executable contract, and it can generate code that matches intent** (source: spec-driven-development-arxiv.md). + +This principle directly informs the [[context-engineering]] discipline — the quality of context you provide determines the quality of AI output. + +--- + +## What Spec-First Actually Means + +In spec-first development, a specification is written **before** coding begins to guide the initial implementation (source: spec-driven-development-arxiv.md). The spec is not documentation of what was built; it is the declaration of what should be built, written before the first line of code. + +A spec-first spec (source: spec-driven-development-variant.md): +- Is structured and behavior-oriented +- Is written in natural language (not pseudo-code) +- Expresses software functionality as guidance to an AI coding agent +- Is scoped to the feature or task at hand — not a global project description + +Good specs share four properties (source: spec-driven-development-arxiv.md): +- **Behavior-focused**: what happens, not how +- **Testable**: each requirement is verifiable +- **Unambiguous**: different readers reach the same interpretation +- **Complete enough** to cover essential cases without over-specifying + +--- + +## Spec-First vs. Spec-Anchored vs. Spec-as-Source + +Spec-first is only the entry level of the [[spec-driven-development]] spectrum. It provides immediate value but does not protect against long-term drift between spec and code (source: spec-driven-development-arxiv.md). Teams who see value from spec-first often graduate to: + +- **Spec-anchored**: spec maintained alongside code throughout the lifecycle; tests enforce alignment +- **Spec-as-source**: spec is the only human-edited artifact; code is entirely generated + +Most teams today are at spec-first — and that is where the practical payoff starts (source: spec-driven-development-main.md). + +--- + +## Why Spec-First Matters Now + +The emergence of AI coding assistants has made spec-first newly relevant, even though the idea predates AI — TDD and BDD have advocated for it for years (source: spec-driven-development-arxiv.md). Three converging forces make 2025–2026 the inflection point: + +1. **AI coding assistants are ubiquitous** — the bottleneck has shifted from "can we write the code?" to "can we specify what the code should do?" +2. **Tooling matured** — Spec Kit, Kiro, and Tessl all shipped dedicated SDD workflows in 2025–2026 +3. **Error cost evidence** — controlled studies show human-refined specs reduce LLM-generated code errors by up to 50% (source: spec-driven-development-arxiv.md) + +--- + +## The Developer's New Role + +SDD fundamentally reshapes what it means to be a software developer (source: spec-driven-development-arxiv.md): + +| Traditional Role | Spec-First Role | +|------------------|-----------------| +| Code producer | Spec author and AI orchestrator | +| Greenfield: writes implementation | Greenfield: designs system through specifications | +| Brownfield: edits existing code | Brownfield: encodes existing behavior as specs before making changes | + +The developer's primary skill becomes writing clear, precise specifications — not writing code directly. + +--- + +## Specs as Super-Prompts + +In agentic workflows, specifications function as **super-prompts** (source: spec-driven-development-arxiv.md): +- Break complex problems into modular components aligned with agents' context windows +- Enable parallel agent execution on non-overlapping tasks +- Allow teams to partition work at the spec level, letting multiple AI agents implement different components simultaneously + +This connects spec-first thinking to multi-agent [[agent-workflows]] and the parallelism strategies described in [[agentic-engineering-workflow]]. + +--- + +## The Spec-First Workflow in Practice + +The minimal viable spec-first workflow — no special tooling required (source: spec-driven-development-main.md): + +1. Before your next feature, write a one-page spec in plain markdown +2. Define inputs, outputs, constraints, and edge cases +3. Pass the spec as context alongside your prompt to the AI agent +4. Review the agent's output against the spec, not against vague intuition + +**Without spec (vibe coding):** +> "Build a rate limiter middleware for Express." + +**With spec (spec-first):** +> "Implement the rate limiter defined in `.spec/features/rate-limiter.md`, which specifies a sliding window algorithm, 100 requests per minute per API key, 429 responses with Retry-After headers, and Redis-backed state for horizontal scaling." + +The second prompt leaves no room for the agent to improvise on decisions that should be yours (source: spec-driven-development-main.md). + +--- + +## Specs, Memory Banks, and Context Documents + +An important distinction for [[context-engineering]] (source: spec-driven-development-variant.md): + +- **Specs** — scoped to a specific feature or task; only relevant when creating or changing that functionality +- **Memory banks / context documents** — global project context (rules files, architecture descriptions, conventions) relevant across all AI coding sessions + +Tools express this distinction differently: +- Spec Kit calls its global context a **constitution** +- Kiro calls it **steering** (`product.md`, `structure.md`, `tech.md`) +- Tessl uses a global context alongside per-feature specs + +Conflating the two leads to overly broad specs that become noise rather than signal. + +--- + +## When Spec-First Works — and When It Doesn't + +**Best fit** (source: spec-driven-development-main.md, spec-driven-development-arxiv.md): +- Greenfield features with well-understood requirements +- New API endpoints, CRUD modules, integration layers +- AI-assisted development (dramatic improvement in output quality) +- Complex requirements stakeholders can validate before code is written + +**Poor fit**: +- Throwaway prototypes — spec overhead exceeds value +- Exploratory work — you don't yet know what you're building +- Simple, obvious implementations +- Tight iteration loops where writing a spec adds more latency than it removes + +--- + +## Self-Spec: LLMs Writing Their Own Specs + +An emerging technique: **self-spec methods** where the LLM authors its own specification before generating code (source: spec-driven-development-arxiv.md). The workflow: +1. Agent produces a spec from a high-level prompt +2. Human reviews and refines the spec +3. Same or different agent implements against the refined spec + +This preserves human oversight at the spec-review stage rather than the code-review stage — where ambiguity is cheaper to fix. + +--- + +## Related pages + +- [[spec-driven-development]] +- [[spec-driven-development-practice]] +- [[spec-driven-development-critique]] +- [[code-to-contract]] +- [[context-engineering]] +- [[agent-workflows]] +- [[agentic-engineering-workflow]] +- [[rpi-workflow]] +- [[rpir-workflow]] +- [[agent-best-practices]] diff --git a/wiki/knowledge/structured-outputs-anthropic.md b/wiki/knowledge/structured-outputs-anthropic.md new file mode 100644 index 00000000..6949dee8 --- /dev/null +++ b/wiki/knowledge/structured-outputs-anthropic.md @@ -0,0 +1,118 @@ +# Structured Outputs — Anthropic + +**Summary**: Anthropic's constrained-decoding feature that guarantees Claude returns valid, schema-compliant JSON via two complementary mechanisms: JSON outputs and strict tool use. +**Sources**: `docs/structured-outputs/anthropic-structured-outputs.md` +**Last updated**: 2026-04-21 + +--- + +Structured outputs constrain Claude's responses to follow a specific schema, eliminating parse errors and schema violations for downstream processing. (source: anthropic-structured-outputs.md) + +## Two Complementary Features + +| Feature | Parameter | Purpose | +|---------|-----------|---------| +| JSON outputs | `output_config.format` | Controls Claude's response format — what Claude says | +| Strict tool use | `strict: true` on a tool definition | Validates tool parameters — how Claude calls your functions | + +Both features can be used independently or together in the same request. (source: anthropic-structured-outputs.md) + +## Supported Models + +Structured outputs are **generally available** on the Claude API for Claude Mythos Preview, Opus 4.7, Opus 4.6, Sonnet 4.6, Sonnet 4.5, Opus 4.5, and Haiku 4.5. Available on Amazon Bedrock for the same range (excluding Mythos Preview on Vertex AI). In beta on Microsoft Foundry. (source: anthropic-structured-outputs.md) + +## Why Use Structured Outputs + +Without structured outputs, careful prompting alone may still produce: +- `JSON.parse()` errors from invalid syntax +- Missing required fields +- Inconsistent data types +- Schema violations requiring retries + +Constrained decoding via **compiled grammars** guarantees responses that are always valid, type-safe, and reliable. (source: anthropic-structured-outputs.md) + +## JSON Outputs + +JSON outputs return valid JSON matching your schema in `response.content[0].text`. Use them when you need to: +- Control Claude's response format +- Extract structured data from images or text +- Generate structured reports or API responses + +**How it works:** +1. Define a JSON schema (`type: "json_schema"`) +2. Pass it in `output_config.format` +3. Parse the guaranteed-valid JSON from `response.content[0].text` + +See [[anthropic-tool-use]] for the parallel mechanism that validates tool inputs rather than response text. + +### SDK Helpers + +| Language | Tool | +|----------|------| +| Python | Pydantic models via `client.messages.parse()` | +| TypeScript | Zod schemas via `zodOutputFormat()` or `jsonSchemaOutputFormat()` | +| Java | Plain Java classes via `outputConfig(Class)` | +| Ruby | `Anthropic::BaseModel` classes | +| PHP | Classes implementing `StructuredOutputModel` | +| CLI, C#, Go | Raw JSON schemas via `output_config` | + +The Python, TypeScript, Ruby, and PHP SDKs **automatically transform** unsupported schema features: removing unsupported constraints, injecting those constraints into `description` strings, adding `additionalProperties: false`, filtering string formats, and validating the final response against the original schema. (source: anthropic-structured-outputs.md) + +## Schema Complexity Limits + +Schemas are compiled into grammars. More complex schemas mean longer compile times and larger grammar sizes. (source: anthropic-structured-outputs.md) + +| Limit | Value | +|-------|-------| +| Strict tools per request | **20** | +| Total optional parameters across all strict schemas | **24** | +| Total parameters with union types across all strict schemas | **16** | + +Exceeding these triggers a `400 "Schema is too complex for compilation"` error. A 180-second compilation timeout applies as a final stop-gap. + +**Tips to reduce complexity:** +- Mark only critical tools as strict +- Reduce optional parameters (each one roughly doubles grammar state space) +- Flatten deeply nested objects +- Split large tool sets across multiple requests or sub-agents + +## Grammar Caching and Latency + +- **First request**: additional latency while the grammar compiles +- **Subsequent requests**: served from cache within **24 hours** of last use +- Cache invalidation triggers: changing schema structure, changing the tool set. Changing only `name` or `description` does **not** invalidate the cache. +- Using `output_config.format` also injects a system prompt and invalidates any existing prompt cache for that thread. (source: anthropic-structured-outputs.md) + +## Property Ordering + +Required properties appear first, followed by optional properties. If order matters to your application, mark all properties as required. (source: anthropic-structured-outputs.md) + +## Invalid-Output Edge Cases + +| Stop reason | Behavior | +|-------------|----------| +| `refusal` | Claude refuses for safety — 200 status, tokens billed, output may not match schema | +| `max_tokens` | Output may be incomplete — retry with higher `max_tokens` | + +## Feature Compatibility + +**Works with**: batch processing (50% discount), token counting, streaming, combining JSON outputs + strict tool use in same request. + +**Incompatible with**: Citations (returns `400`), message prefilling. (source: anthropic-structured-outputs.md) + +## Data Retention + +JSON schemas are cached up to 24 hours. Prompts and responses qualify for Zero Data Retention (ZDR). Structured outputs are **HIPAA eligible** — but **PHI must not appear in schema definitions** (property names, `enum`/`const` values, `pattern` regex). (source: anthropic-structured-outputs.md) + +--- + +## Related pages + +- [[anthropic-tool-use]] +- [[anthropic-strict-tool-use]] +- [[anthropic-output-consistency]] +- [[json-schema-for-ai]] +- [[tool-use-patterns]] +- [[context-engineering]] +- [[agent-workflows]] +- [[prompt-engineering]] diff --git a/wiki/knowledge/subagents.md b/wiki/knowledge/subagents.md new file mode 100644 index 00000000..9272aebe --- /dev/null +++ b/wiki/knowledge/subagents.md @@ -0,0 +1,97 @@ +# Subagents + +**Summary**: Specialized assistants running in isolated context windows with custom system prompts and restricted tool sets — a core pattern for keeping the main agent context clean during research, planning, review, and other noisy work. +**Sources**: creating-custom-subagents.md, claude-orchestrate-of-claude-code-sessions.md, research-subagent-best-practices.md +**Last updated**: 2026-04-22 + +--- + +Subagents are the workhorse of [[agent-workflows]]. They receive only their system prompt plus basic environment details — **not** parent conversation history. This isolation is a feature: it keeps the main context clean and lets subagents focus on their specific task. + +## Core Properties + +| Feature | Current docs say | +|---------|------------------| +| Format | Markdown file with YAML frontmatter | +| Required fields | `name` and `description` | +| Tool control | Restrict aggressively; read-only by default for exploration/review | +| Context model | Isolated window with its own prompt and environment | +| Cost control | Use `maxTurns`, narrower tools, and cheaper models where appropriate | +| Typical locations | Session, project, user, or plugin scope | +| Team role | Best for research, review, planning, and other high-noise tasks | + +## Common Tool Profiles + +| Agent Role | Tools | Model | Notes | +| ------------------ | ---------------------------- | ------------------ | ------------------------------------------------ | +| Code reviewer | Read, Grep, Glob, Bash | Haiku (fast/cheap) | ~97% of community agents use this read-only core | +| Explorer | Read, Grep, Glob | Haiku | Minimal set for fast codebase search | +| Code modifier | Read, Grep, Glob, Edit, Bash | Sonnet | Standard work requiring writes | +| Architect/Reasoner | Read, Grep, Glob, Bash | Opus | Complex reasoning and planning | +| Security auditor | Read, Grep, Glob | Sonnet | Read-only with stronger model for nuance | + +## Built-in Subagents + +Both platforms provide built-in subagents: + +- **Explore** — Fast, read-only codebase search (Haiku model in Claude Code) +- **Bash/Terminal** — Shell command execution with output isolation +- **Plan** — Research-focused read-only agent (Claude Code) +- **general-purpose** — Full tool access (Claude Code) + +## System Prompt Structure + +Effective subagent prompts follow a five-part structure: + +1. **Role** — Who the agent is and its expertise area +2. **Process** — Step-by-step procedure to follow +3. **Checklist** — Specific items to verify or produce +4. **Output Format** — Exact structure of the response (field names, grouping, format) +5. **Success Criteria** — What constitutes a complete, useful result + +Keep prompts focused and under 2000 words. Longer prompts are slower to process and harder to maintain. + +## Design Principles + +1. **Restrict tools to minimum necessary** — Read-only agents get Read+Grep+Glob; prevents accidental modifications +2. **Use Haiku for fast, cheap exploration** — Sonnet for standard work, Opus for complex reasoning +3. **Set `maxTurns`** to prevent runaway agents (15 for exploration, 20 for evaluation) — missing `maxTurns` is a common source of unexpected token consumption +4. **Implement confidence-based filtering** — Report only >80% confidence issues. This threshold significantly reduces noise while preserving actionable findings. The agent prompt must explicitly instruct: "Report only issues where you are at least 80% confident" +5. **Specify exact output structure** — Field names, format, grouping in the system prompt. Without this, output varies between invocations and is hard to parse programmatically +6. **Use normal guidance language** — Replace "CRITICAL: You MUST" with "Use when...". Natural phrasing performs equivalently without wasting tokens on emphasis markers +7. **Restrict spawnable agents** — Use `Agent(worker, researcher)` syntax to limit which agents a parent can spawn + +## Effort Levels (Claude Code Only) + +| Level | Behavior | Availability | +| -------- | ------------------------------------ | ------------- | +| `low` | Quick, shallow analysis | All models | +| `medium` | Standard depth | All models | +| `high` | Thorough analysis | All models | +| `max` | Deepest reasoning, extended thinking | Opus 4.6 only | + +## Plugin Security (Claude Code) + +Agents bundled in [[claude-code-plugins]] have restricted capabilities — the `hooks`, `mcpServers`, and `permissionMode` frontmatter fields are **silently ignored** when the agent is loaded from a plugin context. To use these fields, copy the agent file to `.claude/agents/` or `~/.claude/agents/`. + +## Anti-Patterns + +Ten common mistakes when building and deploying subagents: + +1. **Giving all tools to read-only agents** → Risk of accidental file modifications; restrict to Read+Grep+Glob +2. **Using subagents for simple, single-purpose tasks** → Use [[claude-code-skills]] instead; subagents add context-switching overhead +3. **Generic or vague subagent descriptions** → The description determines when the orchestrator delegates; unclear descriptions mean missed or wrong invocations +4. **Over-delegation by larger models** → Opus may delegate when a simpler direct approach suffices; not every task needs a subagent +5. **Overly long prompts (2000+ words)** → Slower processing, harder maintenance, diminishing returns on quality +6. **Too many subagents (50+)** → Agent confusion during selection, increased maintenance burden; consolidate or use skill-based dispatch +7. **Duplicating slash commands as subagents** → If the task is single-purpose, a skill or command is more appropriate +8. **Missing `maxTurns`** → Runaway agents consuming tokens with no bound; always set a turn limit +9. **Not restricting tool access** → Security risk and unfocused behavior; every agent should have the minimum tools it needs +10. **Not checking into version control** → Team members can't benefit; `.claude/agents/` should be committed + +## Related pages + +- [[claude-code-subagents]] +- [[agent-workflows]] +- [[progressive-disclosure]] +- [[claude-code-plugins]] diff --git a/wiki/knowledge/tool-calling-patterns.md b/wiki/knowledge/tool-calling-patterns.md new file mode 100644 index 00000000..0d9459b5 --- /dev/null +++ b/wiki/knowledge/tool-calling-patterns.md @@ -0,0 +1,164 @@ +# Tool Calling Patterns + +**Summary**: A synthesis of all major tool calling patterns — static vs. dynamic loading, forced/auto/parallel calling, programmatic code-execution, and MCP Code Mode — with guidance on when to use each. +**Sources**: `docs/tool-calling/programmatic-tool-calling-claude-api.md`, `docs/tool-calling/tool-search-redefining-agent-tool-calling-epsilla.md`, `docs/tool-calling/cameronking4-programmatic-tool-calling-github.md`, `docs/tool-calling/mcp-programmatic-tool-calling-opensandbox-dev.md`, `docs/tool-calling/README.md` +**Last updated**: 2026-04-21 + +--- + +## Pattern Overview + +| Pattern | Round-trips | Token cost | Best for | +|---------|-------------|------------|----------| +| Static tool list + auto | N | High (all schemas loaded) | < 10 tools, simple workflows | +| Static tool list + forced | 1+ | High | Guaranteed tool invocation | +| Parallel tool calling | 1 per batch | High (all schemas) | Independent simultaneous calls | +| Dynamic Tool Search | N (reduced) | Low (lazy schema load) | 10+ tools, MCP servers | +| Programmatic (code execution) | 2 (fixed) | Very low | Complex multi-tool orchestration | +| MCP Code Mode | 2–3 | Very low | Enterprise APIs, .NET/Java environments | + +--- + +## 1. Static Tool List — Auto Mode + +The model decides whether and which tool to call. All tool definitions are pre-loaded in the `tools` parameter (source: programmatic-tool-calling-claude-api.md). + +``` +tools: [tool_a, tool_b, tool_c, ...] +tool_choice: "auto" +``` + +**When to use**: Fewer than 10 tools, straightforward tasks, simple agentic loops. +**Limitation**: Every tool schema consumes input tokens on every request, even unused tools. + +See [[anthropic-tool-use]] for API implementation details and [[json-schema-for-ai]] for schema design. + +--- + +## 2. Forced Tool Use + +`tool_choice: "any"` forces the model to call at least one tool. `tool_choice: {"type": "tool", "name": "X"}` forces a specific function (source: programmatic-tool-calling-claude-api.md). + +Adding `strict: true` to a tool definition ensures Claude's output always matches the schema exactly (source: programmatic-tool-calling-claude-api.md). + +**When to use**: Workflows that must collect structured data before responding; validation steps; guaranteed extraction. + +--- + +## 3. Parallel Tool Calling + +The model emits multiple `tool_use` blocks in a single response. Your application executes them concurrently, then returns all `tool_result` blocks together (source: programmatic-tool-calling-claude-api.md). + +**When to use**: Multiple independent operations that can run simultaneously — e.g., fetching data from several APIs at once. +**Limitation**: Still one LLM round-trip per batch; intermediate results still enter the context. + +--- + +## 4. Dynamic Tool Search (Lazy Loading) + +Instead of pre-loading all schemas, the model starts with a single lightweight `tool_search` stub and discovers definitions on demand (source: tool-search-redefining-agent-tool-calling-epsilla.md): + +``` +Initial: System Prompt + tool_search stub (~500 tokens) + ↓ +Model calls tool_search("query") + ↓ +Relevant schema injected just-in-time + ↓ +Tool invoked +``` + +**Token savings**: 85%+ vs. static loading for large tool sets (source: tool-search-redefining-agent-tool-calling-epsilla.md). +**Accuracy gains**: Claude Opus 4: 49% → 74%; Claude Opus 4.5: 79.5% → 88.1% (source: tool-search-redefining-agent-tool-calling-epsilla.md). +**When to use**: 10+ tools, multiple MCP servers, schemas > 10K tokens, enterprise scale. + +See [[tool-search-epsilla]] and [[dynamic-tool-discovery]] for full detail. + +--- + +## 5. Programmatic Tool Calling (Code Execution) + +Inspired by Anthropic's November 2025 advanced tool use announcement (source: cameronking4-programmatic-tool-calling-github.md). The LLM generates executable code that orchestrates all tool calls, runs once in a sandbox, and returns only the final result. + +``` +Round 1: LLM generates JavaScript/Python + → parallel calls, conditionals, aggregations + +Sandbox executes code + → intermediate results NEVER enter LLM context + +Round 2: LLM receives final result → responds to user +``` + +**LLM round-trips**: Always 2, regardless of how many tools are called (source: programmatic-tool-calling-claude-api.md). +**Token savings**: 80%+ on complex workflows (source: cameronking4-programmatic-tool-calling-github.md). +**Latency**: 3–5× faster than sequential calling (source: cameronking4-programmatic-tool-calling-github.md). +**When to use**: Complex multi-tool workflows; data aggregation across many calls; MCP-heavy agents. + +See [[programmatic-tool-calling]] (Anthropic native) and [[programmatic-tool-calling-sdk]] (Vercel AI SDK implementation). + +--- + +## 6. MCP Code Mode (Three-Tool Pattern) + +The .NET/C# MCP Code Mode variant uses three distinct tools instead of a single `code_execution` server (source: mcp-programmatic-tool-calling-opensandbox-dev.md): + +1. **`search`** — semantic discovery of relevant tools. +2. **`get_schema`** — retrieves full schema for chosen tool. +3. **`execute`** — runs LLM-generated Python code in an isolated sandbox. + +Benefits include 50%+ token reduction, batch execution, and full sandbox isolation for security (source: mcp-programmatic-tool-calling-opensandbox-dev.md). + +See [[mcp-programmatic-tool-calling]] for implementation details. + +--- + +## Choosing a Pattern + +``` +< 10 tools, simple workflow? + → Static list + auto mode + +Need guaranteed tool invocation or strict schema? + → Forced tool use + strict: true + +Multiple independent calls in one step? + → Parallel tool calling + +10+ tools or MCP servers (token cost priority)? + → Dynamic Tool Search + +Complex orchestration, many sequential/parallel calls? + → Programmatic tool calling (code execution) + +Enterprise scale in .NET/Java with hundreds of APIs? + → MCP Code Mode (search + get_schema + execute) +``` + +--- + +## Combining Patterns + +Patterns compose. A common production configuration (source: tool-search-redefining-agent-tool-calling-epsilla.md, programmatic-tool-calling-claude-api.md): + +- **Tool Search** reduces the schema loading overhead. +- **Programmatic calling** collapses orchestration round-trips. +- **Prompt caching** on the stable prefix maximizes cache hits. + +See [[context-engineering]] for how to sequence these for maximum efficiency and [[agent-workflows]] for how they fit into the full agent loop. + +--- + +## Related pages + +- [[programmatic-tool-calling]] +- [[programmatic-tool-calling-sdk]] +- [[tool-search-epsilla]] +- [[dynamic-tool-discovery]] +- [[mcp-programmatic-tool-calling]] +- [[anthropic-tool-use]] +- [[agent-workflows]] +- [[context-engineering]] +- [[json-schema-for-ai]] +- [[structured-outputs-anthropic]] +- [[tool-use-patterns]] diff --git a/wiki/knowledge/tool-search-epsilla.md b/wiki/knowledge/tool-search-epsilla.md new file mode 100644 index 00000000..61bf266a --- /dev/null +++ b/wiki/knowledge/tool-search-epsilla.md @@ -0,0 +1,139 @@ +# Tool Search — Epsilla Analysis + +**Summary**: Epsilla's deep analysis of Tool Search as the most significant AI agent infrastructure shift of 2025-2026 — covering semantic tool discovery, accuracy benchmarks, prompt cache protection, and a full FAQ. +**Sources**: `docs/tool-calling/tool-search-redefining-agent-tool-calling-epsilla.md`, `docs/tool-calling/README.md` +**Last updated**: 2026-04-21 + +--- + +## What Is Tool Search? + +Tool Search is a **lazy-loading mechanism for tool definitions**. Instead of pre-loading every available tool schema at startup, an AI model holds only a lightweight search stub. When a specific capability is required, it calls the search tool to find and inject the relevant definition just-in-time (source: tool-search-redefining-agent-tool-calling-epsilla.md). + +> Conceptually, Tool Search is RAG applied to tool schemas rather than knowledge documents. It indexes tool names, descriptions, and parameters and retrieves them on demand. + +See [[dynamic-tool-discovery]] for a cross-implementation view of this pattern. + +--- + +## The Context Bloat Problem + +Traditional tool calling pre-loads all definitions upfront. A typical enterprise setup (source: tool-search-redefining-agent-tool-calling-epsilla.md): + +``` +GitHub: 35 tools → ~26K tokens +Slack: 11 tools → ~21K tokens +Jira: 20 tools → ~17K tokens +Sentry: 5 tools → ~3K tokens + +Total: ~72K tokens consumed BEFORE conversation starts +``` + +Anthropic has observed tool definitions consuming **134K tokens** before any task begins. Even with 200K+ context windows, pre-loading this many schemas degrades tool-selection accuracy due to information overload and destroys prompt caching efficiency (source: tool-search-redefining-agent-tool-calling-epsilla.md). + +--- + +## Tool Search vs. Traditional Calling + +**Traditional approach** — all schemas loaded before the first user message: + +```json +{ + "model": "claude-3-sonnet-20240229", + "tools": [ + { "name": "get_weather", ... }, + { "name": "search_restaurants", ... }, // irrelevant + { "name": "book_reservation", ... } // irrelevant + ] +} +``` + +**Tool Search approach** — start with a single stub, discover on demand: + +```json +{ + "model": "claude-sonnet-4", + "tools": [{ "type": "tool_search", "name": "tool_search" }] +} +// → Model calls tool_search("weather") +// → Discovers get_weather +// → Loads only that definition (~800 tokens) +// Total: ~1.3K tokens vs. ~72K +``` + +(source: tool-search-redefining-agent-tool-calling-epsilla.md) + +--- + +## Benchmarks and Accuracy Improvements + +| Dimension | Key Information | +|-----------|-----------------| +| **Essence** | Lazy loading — loads only tools required for the current task | +| **Token savings** | 85%+ (Anthropic); 34–64% (Spring AI cross-platform benchmark) | +| **Claude Opus 4 accuracy** | 49% → 74% with Tool Search enabled | +| **Claude Opus 4.5 accuracy** | 79.5% → 88.1% with Tool Search enabled | +| **Applicable scenarios** | 10+ tools; multiple MCP servers; definitions exceeding 10K tokens | + +(source: tool-search-redefining-agent-tool-calling-epsilla.md) + +--- + +## Implementations + +| Platform | Implementation | +|----------|----------------| +| Anthropic Claude | `tool_search_tool` server tool (Claude Sonnet 4+) | +| OpenAI GPT-5.4 | `tool_search` (namespace-level) | +| Spring AI | `ToolSearchToolCallAdvisor` | + +(source: tool-search-redefining-agent-tool-calling-epsilla.md) + +See [[anthropic-tool-use]] for Claude-specific API details and [[programmatic-tool-calling]] for the broader Anthropic implementation. + +--- + +## Prompt Cache Protection + +Tool Search keeps the initial context prefix (system prompt + lightweight search stub) extremely stable. Full tool schemas are injected at the **end** of the context, not the beginning. This ensures (source: tool-search-redefining-agent-tool-calling-epsilla.md): + +- The static prefix remains cacheable across turns and sessions. +- **TTFT (Time to First Token)** is reduced. +- Computational overhead drops significantly. + +For context management strategy, see [[context-engineering]]. + +--- + +## Key Definitions + +- **Tool Search** — dynamic tool discovery mechanism; model loads only needed schemas just-in-time (source: tool-search-redefining-agent-tool-calling-epsilla.md). +- **Context Bloat** — degradation caused by pre-loading massive JSON schemas of unused tools (source: tool-search-redefining-agent-tool-calling-epsilla.md). +- **Just-in-Time Retrieval (JITR)** — retrieve and inject information only at the exact moment the execution trajectory requires it (source: tool-search-redefining-agent-tool-calling-epsilla.md). +- **Prompt Caching Stability** — isolating dynamic elements (discovered schemas) to end of context so the static prefix stays cacheable (source: tool-search-redefining-agent-tool-calling-epsilla.md). + +--- + +## FAQs + +**Q: Why is Tool Search necessary when models have 200K+ context windows?** +Even with massive windows, pre-loading hundreds of schemas degrades accuracy due to information overload, inflates token costs per request, and destroys prompt cache efficiency (source: tool-search-redefining-agent-tool-calling-epsilla.md). + +**Q: Does Tool Search increase overall system latency?** +While an additional search-and-load step is added, overall execution latency often *decreases* because the lighter initial context maximizes prompt cache hits and minimizes payload size sent to the LLM (source: tool-search-redefining-agent-tool-calling-epsilla.md). + +**Q: How does Tool Search differ from RAG?** +Tool Search applies RAG principles specifically to tool schemas and function signatures rather than knowledge documents (source: tool-search-redefining-agent-tool-calling-epsilla.md). + +--- + +## Related pages + +- [[dynamic-tool-discovery]] +- [[programmatic-tool-calling]] +- [[programmatic-tool-calling-sdk]] +- [[mcp-programmatic-tool-calling]] +- [[tool-calling-patterns]] +- [[context-engineering]] +- [[anthropic-tool-use]] +- [[json-schema-for-ai]] diff --git a/wiki/knowledge/tool-use-patterns.md b/wiki/knowledge/tool-use-patterns.md new file mode 100644 index 00000000..f58e56cb --- /dev/null +++ b/wiki/knowledge/tool-use-patterns.md @@ -0,0 +1,172 @@ +# Tool Use Patterns + +**Summary**: Patterns for defining and using tools with LLMs, covering single-tool forced use, multi-tool selection, strict schema enforcement, namespacing, and response handling — grounded in the Claude API implementation. +**Sources**: `docs/structured-outputs/anthropic-implement-tool-use.md`, `docs/structured-outputs/anthropic-strict-tool-use.md`, `docs/structured-outputs/anthropic-structured-outputs.md` +**Last updated**: 2026-04-21 + +--- + +Tool use patterns bridge the gap between an LLM's language capabilities and deterministic function execution. The patterns below are grounded in the Claude API but apply conceptually to any LLM tool use system. For schema design specifics, see [[json-schema-for-ai]]. For schema compliance guarantees, see [[anthropic-strict-tool-use]]. + +## Pattern 1: Single-Tool Forced Use + +Force Claude to always call a specific tool using `tool_choice: {"type": "tool", "name": ""}`. Useful for structured extraction tasks where you always want the same output shape. (source: anthropic-implement-tool-use.md) + +```python +response = client.messages.create( + model="claude-opus-4-5", + tools=[get_stock_price_tool], + tool_choice={"type": "tool", "name": "get_stock_price"}, + messages=[{"role": "user", "content": "What is Apple's stock price?"}] +) +``` + +**When to use**: Data extraction pipelines, structured report generation, any workflow where you unconditionally need a function call. + +**Caveat**: Not supported on Claude Mythos Preview. Not compatible with extended thinking. (source: anthropic-implement-tool-use.md) + +## Pattern 2: Any-Tool Forced Use + +Force Claude to call *some* tool (model's choice) using `tool_choice: {"type": "any"}`. Useful when you have several valid output shapes and want the model to pick the right one. (source: anthropic-implement-tool-use.md) + +```python +response = client.messages.create( + model="claude-sonnet-4-5", + tools=[search_tool, lookup_tool, calculate_tool], + tool_choice={"type": "any"}, + messages=[{"role": "user", "content": "What's 15% of 340?"}] +) +``` + +**When to use**: Routing/dispatch patterns, agentic workflows where the model selects the appropriate action. + +## Pattern 3: Strict Schema Enforcement + +Add `"strict": true` to a tool definition to activate grammar-constrained sampling. Combine with `tool_choice: {"type": "any"}` for the strongest guarantee — a tool *will* be called, and its inputs *will* be schema-valid. (source: anthropic-strict-tool-use.md, anthropic-implement-tool-use.md) + +```python +tools = [{ + "name": "book_flight", + "description": "Book a flight for the given passenger count and route.", + "strict": True, + "input_schema": { + "type": "object", + "properties": { + "origin": {"type": "string"}, + "destination": {"type": "string"}, + "passengers": {"type": "integer"} + }, + "required": ["origin", "destination", "passengers"], + "additionalProperties": False + } +}] +response = client.messages.create( + model="claude-opus-4-5", + tools=tools, + tool_choice={"type": "any"}, + messages=[...] +) +``` + +**When to use**: Production agentic systems, booking/reservation systems, API integrations, financial calculations, database query builders. See [[anthropic-strict-tool-use]] for complexity limits. + +## Pattern 4: Auto Tool Selection (Multi-Tool) + +Let Claude decide which tools to call (and whether to call any) using `tool_choice: {"type": "auto"}` (the default). Claude may call multiple tools in a single response turn. (source: anthropic-implement-tool-use.md) + +```python +# Claude may call get_weather AND get_time in the same response +response = client.messages.create( + model="claude-opus-4-5", + tools=[get_weather_tool, get_time_tool, search_tool], + messages=[{"role": "user", "content": "What's the weather and time in SF right now?"}] +) +``` + +**When to use**: Conversational agents, assistants with mixed natural-language and function-calling needs. + +**Handling multi-tool responses**: Iterate over `response.content` blocks — some are `text` blocks, others are `tool_use` blocks. Execute all tool calls and return results in the next `user` turn with `tool_result` blocks. + +## Pattern 5: Consolidated Action Tools + +Rather than creating a separate tool for every action (`create_pr`, `update_pr`, `close_pr`), group related operations into a single tool with an `action` parameter. This reduces tool selection ambiguity: (source: anthropic-implement-tool-use.md) + +```json +{ + "name": "manage_pull_request", + "description": "Create, update, or close a pull request. Use 'create' when opening a new PR, 'update' to change its description or title, 'close' to close without merging.", + "input_schema": { + "type": "object", + "properties": { + "action": {"type": "string", "enum": ["create", "update", "close"]}, + "pr_number": {"type": "integer"}, + "title": {"type": "string"}, + "body": {"type": "string"} + }, + "required": ["action"] + } +} +``` + +**When to use**: Any domain with CRUD-like operations (PRs, tickets, records). + +## Pattern 6: Namespaced Tool Names + +When tools span multiple services, prefix names with the service to avoid ambiguity and support tool search indexing: (source: anthropic-implement-tool-use.md) + +``` +github_list_prs +github_create_issue +slack_send_message +slack_list_channels +``` + +**When to use**: Multi-service agents, [[mcp-programmatic-tool-calling]] scenarios, any system where >10 tools are available. + +## Pattern 7: Input Examples for Complex Tools + +For complex tools with nested or non-obvious schemas, provide `input_examples` to show Claude valid inputs concretely: (source: anthropic-implement-tool-use.md) + +```python +"input_examples": [ + {"origin": "JFK", "destination": "LAX", "date": "2025-12-01", "passengers": 2}, + {"origin": "SFO", "destination": "ORD", "date": "2025-11-15", "passengers": 1} +] +``` + +Token cost: ~20–50 tokens for simple examples, ~100–200 for complex nested objects. + +## Pattern 8: JSON Output + Tool Use Combined + +Use both `output_config.format` (JSON output) and `strict: true` (strict tool use) together in the same request when you need: (source: anthropic-structured-outputs.md) +- Reliable tool calls with schema-valid inputs, AND +- A structured JSON final response (not just a tool call) + +This is useful for agentic workflows that must both *act* (via tools) and *report* (via structured response). + +## Handling Tool Responses in Agent Loops + +A complete [[agent-workflows]] tool loop: + +1. Send request with tools defined +2. Receive response — iterate over `content` blocks +3. For each `tool_use` block: execute the function, capture result +4. Append assistant turn (with tool calls) to `messages` +5. Append `user` turn with `tool_result` blocks for each call +6. Send next request — repeat until `stop_reason: "end_turn"` + +**Design tool responses carefully**: Return only high-signal information. Use semantic, stable identifiers (slugs, UUIDs) rather than opaque internal references. Include only the fields Claude needs to reason about its next step. (source: anthropic-implement-tool-use.md) + +--- + +## Related pages + +- [[anthropic-tool-use]] +- [[anthropic-strict-tool-use]] +- [[structured-outputs-anthropic]] +- [[json-schema-for-ai]] +- [[agent-workflows]] +- [[agent-best-practices]] +- [[mcp-programmatic-tool-calling]] +- [[prompt-engineering]] +- [[context-engineering]] diff --git a/wiki/knowledge/u-shaped-attention-curve.md b/wiki/knowledge/u-shaped-attention-curve.md new file mode 100644 index 00000000..2ee8ec6b --- /dev/null +++ b/wiki/knowledge/u-shaped-attention-curve.md @@ -0,0 +1,106 @@ +# U-Shaped Attention Curve + +**Summary**: The characteristic U-shaped performance curve in long-context LLMs — recall quality is highest at context start and end (primacy and recency effects) and lowest in the middle, forming the empirical foundation of the "lost in the middle" phenomenon. +**Sources**: lost-in-the-middle-acl.md, lost-in-the-middle-arxiv.md, lost-in-the-middle-and-in-between-arxiv.md, `docs/long-context-research/README.md` +**Last updated**: 2026-04-22 + +--- + +The U-shaped attention curve is the empirical pattern observed when measuring how well a language model uses information placed at different positions in a long context. Performance is consistently highest at the edges (beginning and end) and lowest in the middle — forming a "U" shape when plotted against position. This was formally established by [[lost-in-the-middle-paper]] (Liu et al., 2024, TACL) and has since been confirmed across all major LLM families. + +## The Three Components + +### 1. Primacy Effect + +Information near the **beginning** of the context window receives the highest baseline attention. Models attend most strongly to early tokens, as the causal attention mechanism allows all later tokens to attend to earlier ones — creating cumulative weight on early positions. This produces the left arm of the U-curve (source: lost-in-the-middle-acl.md). + +**Practical implication:** Critical instructions, system prompts, and the most relevant retrieved documents belong at the **start** of the context. + +### 2. Recency Effect + +Information near the **end** of the context window receives secondary strong attention — a well-documented phenomenon from sequence modeling. The most recent tokens are always in the model's immediate working memory. This produces the right arm of the U-curve (source: lost-in-the-middle-researchgate.md). + +**Practical implication:** Queries, final instructions, and critical formatting requirements belong at the **end** of the context. + +### 3. Lost in the Middle (Middle Neglect) + +Information in the **center** of the context is systematically underutilized. In multi-document QA experiments, performance drops by **over 20 percentage points** when the relevant document is placed in the center of a 20-document context vs. the edges. This is not a small statistical effect — it represents a fundamental failure mode of current transformer architectures under long-context conditions (source: lost-in-the-middle-acl.md). + +## Quantitative Evidence + +From the original Liu et al. (2024) experiments on NaturalQuestions-Open with 20 documents (source: lost-in-the-middle-acl.md, lost-in-the-middle-arxiv.md): + +| Document Position | Relative Performance | +|------------------|---------------------| +| Position 1 (start) | ~Highest | +| Position 5 | Declining | +| Position 10 (middle) | ~Lowest (~20%+ drop) | +| Position 15 | Recovering | +| Position 20 (end) | ~Second-highest | + +The key-value retrieval task confirms the same curve on synthetic data, establishing that this is a general property of model attention — not a dataset artifact (source: lost-in-the-middle-acl.md). + +## Models Affected + +The U-shaped curve has been documented across (source: lost-in-the-middle-acl.md, lost-in-the-middle-and-in-between-arxiv.md): + +- GPT-3.5-Turbo (ChatGPT) +- Claude (Anthropic) +- MPT-7b-8k-instruct +- Llama-2-7b-longlora-8k-ft +- Other "long-context" model variants + +**No examined model** was immune to the positional bias, including those explicitly marketed as long-context capable. The U-shaped curve persists across model families and scales. + +## The In-Between Compound Effect + +Baker et al. (2024) — see [[lost-in-the-middle-in-between]] — extend the finding to multi-hop QA and discover a second degradation axis: the *relative distance between* multiple evidence documents. When two required evidence pieces are adjacent, performance is higher than when they are separated — even when both are at the same absolute position (source: lost-in-the-middle-and-in-between-arxiv.md). + +This creates a 2D degradation surface for multi-hop tasks: + +- **X-axis:** Distance of each evidence document from context edges (the original U-curve) +- **Y-axis:** Distance between evidence documents from each other (the "in-between" effect) + +## Relationship to Context Engineering + +The U-shaped curve is the empirical basis for key [[context-engineering]] guidelines: + +1. **Critical information at edges:** Place the most important context at position 0 or position N. +2. **Reference data in the middle:** Use the "dead zone" for less critical documents, citations, or background context that the model only needs to skim. +3. **20%+ performance gain from ordering alone:** Simply reordering retrieved documents can improve RAG quality by 20–30% without any change to retrieval quality (source: lost-in-the-middle-acl.md). + +This directly informs [[progressive-disclosure]] — rather than dumping all context at once, progressively narrowing context keeps critical information near the query and edges. + +## Connection to Million-Token Windows + +As context windows grow — [[million-token-context-window]] models now offer 1M+ tokens — the middle neglect problem becomes more severe, not less. A document placed at position 250,000 in a 500,000-token context faces extreme middle neglect. Positional bias does not diminish with larger windows; the "lost zone" simply grows larger (source: README.md). + +This is a key driver of [[context-rot]]: in very long contexts, most of the window becomes a "lost zone" where information may as well be absent from the model's working memory. See also [[dead-context]] for the concept of tokens that consume budget without contributing to reasoning. + +## Practical Placement Guidelines + +Based on the research, the following guidelines apply to any system that inserts content into an LLM context window: + +| Content Type | Recommended Position | Reason | +|-------------|---------------------|--------| +| System instructions | Start | Primacy effect; maximum retention | +| Task constraints | Start | Critical; cannot be missed | +| Most relevant retrieved documents | Start | Highest recall quality | +| Background reference documents | Middle | Low criticality; acceptable recall loss | +| Distractor/noise documents | Middle | Their influence is minimal there | +| Query / user question | End | Recency effect; stays in working memory | +| Final formatting instructions | End | Last-seen formatting is most followed | + +These guidelines are the foundation of [[long-context-mitigation]] placement strategies. + +## Related pages + +- [[lost-in-the-middle-paper]] +- [[lost-in-the-middle-in-between]] +- [[long-context-mitigation]] +- [[context-engineering]] +- [[context-rot]] +- [[dead-context]] +- [[million-token-context-window]] +- [[progressive-disclosure]] +- [[agent-workflows]] diff --git a/wiki/knowledge/whitespace-and-formatting.md b/wiki/knowledge/whitespace-and-formatting.md new file mode 100644 index 00000000..593a9269 --- /dev/null +++ b/wiki/knowledge/whitespace-and-formatting.md @@ -0,0 +1,78 @@ +# Whitespace and Formatting + +**Summary**: Empirical evidence that structural whitespace is nearly free (blank lines cost 1 token regardless of count), while structural formatting (XML tags, markdown headers) measurably improves output quality — the practical implication being "cut content, not formatting." +**Sources**: research-whitespace-and-formatting.md, research-context-engineering-comprehensive.md +**Last updated**: 2026-04-22 + +--- + +## Token Costs of Whitespace + +BPE tokenizers efficiently merge common whitespace patterns: + +| Element | Token Cost | +| -------------------------- | ---------------- | +| Single space | 1 token | +| Four spaces (indent) | 1 token | +| Eight spaces | 1 token | +| Tab | 1 token | +| Newline | 1 token | +| Double newline (paragraph) | 1 token | +| 4× newlines | 1 token (merged) | + +**Key insight**: Blank lines and indentation are negligible cost. Cutting formatting to "save tokens" is counterproductive. + +## Structural Formatting Costs + +| Element | Token Cost | +| ----------------------------------- | ---------- | +| Markdown header (`## Title`) | 2 tokens | +| Bullet point (`- item`) | 2 tokens | +| Bold (`**text**`) | 3 tokens | +| XML tag pair (`content`) | ~5 tokens | +| Code fence (`` ```python ``) | 2 tokens | + +### Format Comparison + +| Format | Tokens | Overhead vs. Plain Text | +| ------------------- | ---------- | ----------------------- | +| Plain text | 48 tokens | baseline | +| Markdown-structured | 87 tokens | +81% | +| XML-structured | 127 tokens | +165% | + +YAML saves ~30% tokens vs. JSON for equivalent data. + +## Impact on Quality + +Structural formatting measurably improves model performance: + +- Structured techniques outperform unstructured across **58 techniques and 29 NLP tasks** (Schulhoff et al., 2024) +- Queries at end of prompt improve quality **up to 30%** (U-shaped attention curve) +- Well-formatted 1,000-token prompt **beats** wall-of-text 900-token prompt +- Match prompt style to desired output — structured prompts produce structured output + +## Position Effects + +| Position | Effect | +| -------- | ------------------------ | +| Start | High attention (primacy) | +| Middle | 10–20% recall drop | +| End | High attention (recency) | + +**Optimal layout**: Critical instructions at start → Long documents in middle → Queries at end + +## Practical Recommendations + +1. **Use whitespace generously** — blank lines and indentation are nearly free +2. **Use XML tags or markdown headers** to delimit sections +3. **Cut content, not formatting** — remove low-signal text rather than structure +4. **Place queries at end** — exploits recency effect for up to 30% quality gain +5. **Don't fear blank lines** — they're negligible cost with readability benefit +6. **Use YAML over JSON** when token budget matters (~30% savings) + +## Related pages + +- [[context-engineering]] +- [[context-rot]] +- [[prompt-engineering]] +- [[multilingual-performance]]