Skip to content

rebase: give module agents the curated knowledge base - #167

Merged
tzhouam merged 2 commits into
mainfrom
feat/rebase-knowledge-base
Sep 18, 2026
Merged

tzhouam merged 2 commits into
mainfrom
feat/rebase-knowledge-base

Conversation

@tzhouam

@tzhouam tzhouam commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator

Problem

The rebase module agents could reach past failures (search_debug_memory) and learned skills (search_skills), but not the curated knowledge base — the human-written pages under knowledge/repos/vllm-omni/ covering components, models, CI and git lore. So an agent rediscovered by shell what a page already stated.

Measured on the v0.30.0 campaign, 526 tool calls in one run:

tool calls
run_shell 169
read_file 114
edit_file 76
grep 26
search_debug_memory 17
search_skills 10
doc_search 2
doc_read 0

doc_search was already reachable on the harness path via the MCP tool bridge, so the gap was instruction, not capability: neither module prompt template mentioned the knowledge base at all, while the DEBUG_MEMORY tokens actively push agents toward search_debug_memory.

Change

  • doc_search / doc_read added to the adapter tool schemas, appended after the parent dispatcher's 20 so that order stays an intact prefix.
  • RebaseBackends gains both handlers, fail-closed by default like the rest, implemented in _build_backends over the same KnowledgeDocs view the review flows use (repo-scoped through the manifest's knowledge.repo_subdir).
  • Guidance rides the builder-supplied ADAPTIVE_GUIDANCE slot, not a template edit — so the shipped templates stay byte-identical to the parent's copies and test_templates_are_parent_verbatim still passes. (I edited the templates first; that test caught it and I backed it out.)

Guidance tells agents to search the knowledge base before shell exploration, cite the page path when it informed a decision, and that where a page and the code disagree the CODE is authoritative for a rebase — say so in the decision file rather than editing to match the page.

Reviewer attention: goldens

Three goldens are deliberately regeneratedmodule_prompt_model_config.txt, module_prompt_worker_runner.txt, request_shape_turn1.json. The render and the tool surface both change by design, so no mechanism preserves them.

The prompt goldens no longer record the PARENT builder's output, and I rewrote that test's docstring to say so rather than let it keep claiming a parity it no longer has. The parent agent had no knowledge-base access, so this divergence is the point — but if retiring that parity needs an explicit decision, this is the line to object to.

Verification

  • Full copilot suite green.
  • doc_search returns real pages for the vllm-omni slice: components/scheduler/architecture.md, models/qwen-omni/_index.md, debug/guides/....
  • Live-path render (live=True) carries the guidance, 24,567 chars, no unsubstituted tokens, and preserves supplied adaptive rules.

Honest limitation

Wired correctly, but on deepseek-flash the agents still did not call it — 0 doc_search in the first 99 tool calls of a run with this change live. Every layer verified (prompt render, schema, handler), so this is instruction-following, not wiring. Making it effective likely needs either a stronger tier or an enforcement point in the plan gate; that is follow-up, not this PR.

🤖 Generated with Claude Code

The rebase module agents could reach past failures (search_debug_memory) and
learned skills (search_skills), but not the human-written knowledge pages --
component maps, model notes, CI and git lore. So an agent rediscovered by
shell what a page already stated.

Evidence from the v0.30.0 campaign: of 526 tool calls, 169 were run_shell and
26 grep, against 2 doc_search and 0 doc_read. doc_search was already reachable
on the harness path via the MCP tool bridge, so the gap was instruction, not
capability -- neither module prompt template mentioned the knowledge base at
all, while DEBUG_MEMORY tokens actively push agents toward search_debug_memory
(17 calls) and search_skills (10).

This wires it on both backends:

- doc_search/doc_read added to the adapter tool schemas, APPENDED after the
  parent dispatcher's 20 so that order stays an intact prefix.
- RebaseBackends gains the two handlers, fail-closed by default like the rest,
  implemented in _build_backends over the same KnowledgeDocs view the review
  flows use (repo-scoped through the adapter manifest's knowledge.repo_subdir).
- The instruction rides the builder-supplied ADAPTIVE_GUIDANCE slot rather
  than a template edit, so the shipped templates stay byte-identical to the
  parent's copies (test_templates_are_parent_verbatim still passes).

Guidance tells agents to search the knowledge base before shell exploration,
to cite the page path when it informed a decision, and that where a page and
the code disagree the CODE is authoritative for a rebase.

Goldens: module_prompt_*.txt and request_shape_turn1.json are deliberately
regenerated -- the render and the tool surface both change by design. The
prompt goldens no longer record the PARENT builder's output, and the test
docstring now says so instead of claiming a parity it no longer has. The
parent agent had no knowledge-base access, so this divergence is the point.

Verified: full suite green; doc_search returns real pages for the vllm-omni
slice (components/scheduler/architecture.md, models/qwen-omni/_index.md); a
rendered module prompt carries the guidance with no unsubstituted tokens.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

check_spec_freshness --strict flagged engine/steps/rebase_v3 and rebase_engine
as STALE: their source changed after the specs were last verified (2026-09-06).

Records what actually changed rather than only bumping the date:
- rebase_engine: rebase_tools now carries the parent's 20 tools PLUS
  doc_search/doc_read, appended after the parent order.
- rebase_v3: _build_backends wires those two over the same KnowledgeDocs view
  the review flows use, repo-scoped by the manifest's knowledge.repo_subdir.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@tzhouam
tzhouam merged commit 762cbb5 into main Sep 18, 2026
2 checks passed
@tzhouam
tzhouam deleted the feat/rebase-knowledge-base branch September 18, 2026 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant