app: FastMCP server instructions + tool annotations (#121) - #125
Merged
Conversation
Agents did not reach for the code-search tools when asked about a repo that is not checked out locally -- they ran local Glob/Grep, reported not-found, and never consulted the index. A server's `instructions` string is returned in the `initialize` result and reaches the client unconditionally, whereas tool schemas may be deferred behind a tool-search step when many servers are connected. The server passed no `instructions`, so it contributed zero always-on context: only the bare tool names survived, and `search_code` next to a built-in grep reads like a worse grep. Extract `build_mcp() -> FastMCP` from `create_app()`. The Starlette app does not expose the FastMCP object it was built from, so tool metadata was unreachable without standing up a full HTTP session -- which would have put the tests in the e2e file that never executes. Each call still yields a fresh instance, preserving the single-use session-manager constraint. Add `title=` and read-only/open-world annotations to all six tools on semantic-accuracy grounds only; no routing or approval-friction credit is claimed. `_READ_ONLY` is a module constant because inlining pushes all six registration lines past line-length 100. Tool names, query semantics, and payload shapes are unchanged. The six tool docstrings are untouched -- that is #122, deferred behind #123. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tannerwendland-db
approved these changes
Jul 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #121.
Summary
Agents did not reach for the code-search MCP tools when a prompt named a repo, file, or project that is not checked out in the working directory. They ran local
Glob/Grep, reported not-found, and never consulted the index.The cause is a channel asymmetry. A server's
instructionsstring is returned in theinitializeresult and reaches the client unconditionally; tool schemas are not — when many MCP servers are connected, clients may defer tool schemas behind a tool-search step and load only tool names up front.create_app()passed noinstructions, so this server contributed zero always-on context: only the bare names survived, andsearch_codesitting next to a built-in grep reads like a worse grep.Scope
build_mcp() -> FastMCPextracted fromcreate_app(). The Starlette app returned bystreamable_http_app()does not expose theFastMCPobject it was built from, soinstructionsandlist_tools()were unreachable without standing up a full HTTP session — which would have forced these tests intotests/integration/test_mcp_server.py, which is@pytest.mark.e2eand never executes (make testruns-m "unit or observability";ci-lakebase.ymlgates on an unsetCI_LAKEBASE_ENABLED). Each call still yields a fresh instance, preserving the single-useStreamableHTTPSessionManagerconstraint.SERVER_INSTRUCTIONS(573 chars). A routing signpost, not documentation — it costs system-prompt tokens in every session of every connected client. It routes discovery throughlist_reposrather than asserting a corpus exists, so it stays true against a freshly forked template with an empty index.find_references/list_importsare deliberately excluded: this answers "should I engage this server," not "here is the full catalog."title=andannotations=on all six tools,readOnlyHint/openWorldHint, on semantic-accuracy grounds only. All six are genuinely read-only against an external open-world index. No routing or approval-friction credit is claimed._READ_ONLYis a module constant rather than inline because inlining puts all six registration lines at 108–127 chars againstline-length = 100.Unchanged: tool names (zoekt parity), query semantics, payload shapes. The six tool docstrings are untouched — that is #122.
Tests
New
tests/unit/test_mcp_descriptions.py, markedunitso it actually runs. No database required.instructionsis non-empty, containslist_repos, matches/working directory/iand/cannot find locally/i, and equalsSERVER_INSTRUCTIONS. The trigger assertions matter: without them the baseline isinstructions=None, so any non-empty string passes —'See list_repos.'would satisfy everything else./not checked out/iis deliberately not asserted; it isFalseagainst the real text, which wraps betweenNOTandchecked.app/query/parser.py:_SUPPORTEDrather than being hand-copied, per the convention atapp/query/semantic_filters.py:26-27("never a second hand-written table") — a future field is denylisted automatically.list_tools(): exact six-name roster, non-empty description and title, both annotation hintsTrue.All three fail on pre-change code.
What is verified, and what is not
The mechanism is verified.
instructionsround-trips throughinitialize(mcp/server/lowlevel/server.py:152,188), and it survives the documenteduvx uc-mcp-proxyregistration path, which is a transparent server→client copy.The behavioral effect is not measured, and is not measurable at this scale. Whether agents actually consult the index more often is unproven. A 10-prompt before/after check was considered and rejected as theater: unblinded, author-run, prompts chosen after the text was written, and a baseline no reviewer can reproduce. Please do not read the mechanism verification as outcome evidence. If the original failure recurs after this ships, that is the signal to build a real eval harness.
One further limit, from review: the protocol guarantees
instructionsis delivered to the client. Whether a given client injects it into model context is client behavior. That is confirmed for Claude Code; it is not a protocol guarantee for every client.Sequencing
Consensus planning (Planner → Architect → Critic, 4 revisions) resequenced this work to #121 → #123 → #122.
#122 was originally planned to ship in this PR and was cut. Its premise — that tool-search ranks candidates on name + description — is not verifiable from this repo, and has been demoted to a hypothesis in its issue body. #123 (a skill matching on request text) is the stronger trigger for the client where the failure was actually observed, so #122 now lands last, where its marginal value can be judged against a shipped skill rather than guessed at.
🤖 Generated with Claude Code