feat(kb): AI-native representation Phase 2 — consume the block — 0.24.2#15
Merged
Conversation
…(Phase 2) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ns it as the snippet The result description (which session-load + persona-context inject) is now the block's shared-intermediate when present, so consumers get the structured proposition instead of a prose snippet. Block tokens are a proposition-level BM25 field; the body field excludes the block so terms aren't double-counted. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ediate directly Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Consumption: search BM25-weights the block (1.5, body de-duped) + returns it as the result snippet (→ session-load/persona injection); knowledge_fetch 'block' tier reads the shared intermediate. Additive/back-compat. Full suite green (66 shell + vitest). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- MEDIUM: cap the block-snippet result description at SNIPPET_CHARS (200) — restores the 2afcfe3 context-budget bound so a big block can't blow session-load's enrichment budget + truncate the slug list. - LOW: avgDL now measures stripAiBlock(doc.body) (consistent with the block-excluding body field); stripAiBlock strips ALL blocks (g-flag, defensive). - validate-the-real-capability: added an ai-block to the eval fixture (oauth-bare-flag) so test-knowledge-eval actually exercises + bounds the new description path (recall 1.000, 309 tokens). Guards: description-cap test, multi-block strip test. Full suite green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Phase 2 of the AI-native KB representation: the per-page <!-- ai:begin … ai:end --> “shared intermediate” is now consumed by retrieval flows, so search can index and return the structured block as the snippet, and knowledge_fetch can return the block directly via a new tier.
Changes:
- Update
knowledge_searchBM25 scoring to index ai-block field values (weight 1.5) while excluding the block from prose-body tokenization, and return a budget-capped ai-block snippet when present. - Add
knowledge_fetchtier"block"to fetch the compact ai-block representation (with fallback behavior) and add targeted tests/fixtures. - Introduce
aiBlockSnippet()plus defensivestripAiBlock()behavior (strip all blocks), and bump plugin/MCP versions to 0.24.2 / 2.6.0.
Reviewed changes
Copilot reviewed 11 out of 38 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/fixtures/eval-wiki/wiki/learnings/oauth-bare-flag.md | Adds an ai-block to ensure eval fixtures exercise the new consumption path. |
| skills/upgrade/SKILL.md | Documents the 0.24.2 migration marker and Phase 2 behavior changes. |
| mcp/src/tools/knowledge-search.ts | BM25 now indexes ai-block values and returns ai-block snippets (budget-capped) when available. |
| mcp/src/tools/knowledge-search.test.ts | Adds tests covering ai-block indexing and snippet length capping. |
| mcp/src/tools/knowledge-fetch.ts | Adds tier: "block" to return the shared intermediate with fallback behavior. |
| mcp/src/tools/knowledge-fetch.test.ts | New tests for the knowledge_fetch "block" tier and fallback. |
| mcp/src/tools/ai-block.ts | stripAiBlock() now strips all blocks; adds aiBlockSnippet() helper. |
| mcp/src/tools/ai-block.test.ts | Adds coverage for multi-block stripping and aiBlockSnippet() output. |
| mcp/src/server.ts | Bumps MCP server version to 2.6.0 and exposes knowledge_fetch "block" tier in the tool schema/description. |
| mcp/dist/tools/knowledge-validate.bundle.js | Regenerated bundle reflecting updated stripAiBlock() behavior. |
| mcp/dist/tools/knowledge-search.test.js.map | Regenerated sourcemap for updated tests. |
| mcp/dist/tools/knowledge-search.test.js | Regenerated compiled test output. |
| mcp/dist/tools/knowledge-search.js.map | Regenerated sourcemap for updated search logic. |
| mcp/dist/tools/knowledge-search.js | Regenerated compiled search output. |
| mcp/dist/tools/knowledge-search.d.ts.map | Regenerated type sourcemap reflecting source changes. |
| mcp/dist/tools/knowledge-search-cli.bundle.js | Regenerated CLI bundle reflecting ai-block search changes. |
| mcp/dist/tools/knowledge-reindex.bundle.js | Regenerated bundle reflecting updated stripAiBlock() behavior. |
| mcp/dist/tools/knowledge-fetch.test.js.map | Regenerated sourcemap for new fetch tests. |
| mcp/dist/tools/knowledge-fetch.test.js | Regenerated compiled output for new fetch tests. |
| mcp/dist/tools/knowledge-fetch.test.d.ts.map | Regenerated type sourcemap for new fetch tests. |
| mcp/dist/tools/knowledge-fetch.test.d.ts | Generated type stub for new fetch tests. |
| mcp/dist/tools/knowledge-fetch.js.map | Regenerated sourcemap for updated fetch logic. |
| mcp/dist/tools/knowledge-fetch.js | Regenerated compiled fetch output (includes "block" tier). |
| mcp/dist/tools/knowledge-fetch.d.ts.map | Regenerated type sourcemap reflecting "block" tier addition. |
| mcp/dist/tools/knowledge-fetch.d.ts | Updates Tier union to include "block". |
| mcp/dist/tools/ai-block.test.js.map | Regenerated sourcemap for updated ai-block tests. |
| mcp/dist/tools/ai-block.test.js | Regenerated compiled output for updated ai-block tests. |
| mcp/dist/tools/ai-block.js.map | Regenerated sourcemap for updated ai-block module. |
| mcp/dist/tools/ai-block.js | Regenerated compiled output (includes aiBlockSnippet, global strip). |
| mcp/dist/tools/ai-block.d.ts.map | Regenerated type sourcemap reflecting aiBlockSnippet. |
| mcp/dist/tools/ai-block.d.ts | Exposes aiBlockSnippet and updates strip docstring. |
| mcp/dist/tools/ai-block-render-cli.bundle.js | Regenerated CLI bundle reflecting updated ai-block helpers. |
| mcp/dist/server.js.map | Regenerated sourcemap reflecting server/tool schema changes. |
| mcp/dist/server.js | Regenerated compiled server output (2.6.0, "block" tier). |
| mcp/dist/server.bundle.js | Regenerated server bundle reflecting all updates. |
| mcp/dist/cli/sb-entry.bundle.js | Regenerated CLI entry bundle reflecting all updates. |
| .claude-plugin/plugin.json | Bumps plugin version to 0.24.2. |
| .claude-plugin/marketplace.json | Bumps marketplace version to 0.24.2. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+117
to
+121
| description: (doc.aiBlock && Object.keys(doc.aiBlock).length) | ||
| ? aiBlockSnippet(doc.type, doc.aiBlock).slice(0, SNIPPET_CHARS) // shared intermediate, budget-capped (Phase 2) | ||
| : (source === 'local-doc' | ||
| ? doc.description | ||
| : (doc.description || rawContent.slice(0, SNIPPET_CHARS).replace(/\s+/g, ' ').trim())), |
Comment on lines
+106
to
+113
| } else if (tier === 'block') { | ||
| // The machine-first shared intermediate. Falls back gracefully when the page has no block. | ||
| if (doc.aiBlock && Object.keys(doc.aiBlock).length) { | ||
| text = aiBlockSnippet(doc.type, doc.aiBlock); | ||
| } else { | ||
| const summary = summarySection(doc.body); | ||
| text = summary || [gist, '', ...headings(doc.body), '', '(no ai-block — use tier=summary/full)'].join('\n').trim(); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What — Phase 2 (the block gets used)
The machine-first shared intermediate now drives retrieval + injection:
knowledge_searchindexes the block as a proposition-level BM25 field (weight 1.5; the prose-body field excludes the block so terms aren't double-counted) and returns the block as the result snippet — sosession-load+persona-context, which inject the search snippet, now hand the reader the structured proposition instead of a prose fragment. The snippet is budget-capped (200 chars).knowledge_fetchblocktier reads just the shared intermediate (graceful fallback to summary when no block).aiBlockSnippet(compact schema-ordered one-line form).MCP server → 2.6.0 (new fetch tier). Additive + back-compat: a page with no block behaves exactly as before. Phase 2b (the block's own embedding/vector) needs embeddings → deferred (offline-first stays BM25). Phase 3 = dream/maintainer refresh + lint staleness + backfill.
Release-gate review — 4 findings, 0 false positives, all fixed + guarded
2afcfe3) → could blowsession-load's enrichment budget + truncate the slug list. Now capped; regression test added.avgDLnow consistent with the block-excluding body field;stripAiBlockstrips all blocks (defensive).oauth-bare-flag→ the gate now exercises + bounds it (recall 1.000, 309 tokens).History review otherwise clean (blockless pages byte-identical; tier set purely widened).
Verification
Full suite green (66 shell + vitest, 0 fail); lockstep 0.24.2 / MCP 2.6.0; validate + migration-row + eval gates; 4 TDD tasks each RED→GREEN + gate-fix.
🤖 Generated with Claude Code