Skip to content

app: FastMCP server instructions + tool annotations (#121) - #125

Merged
tannerwendland-db merged 1 commit into
masterfrom
feat/mcp-server-instructions
Jul 27, 2026
Merged

app: FastMCP server instructions + tool annotations (#121)#125
tannerwendland-db merged 1 commit into
masterfrom
feat/mcp-server-instructions

Conversation

@IceRhymers

Copy link
Copy Markdown
Owner

Closes #121.

Summary

Agents did not reach for the code-search MCP tools when a prompt named a repo, file, or project that is not checked out in the working directory. They ran local Glob/Grep, reported not-found, and never consulted the index.

The cause is a channel asymmetry. A server's instructions string is returned in the initialize result and reaches the client unconditionally; tool schemas are not — when many MCP servers are connected, clients may defer tool schemas behind a tool-search step and load only tool names up front. create_app() passed no instructions, so this server contributed zero always-on context: only the bare names survived, and search_code sitting next to a built-in grep reads like a worse grep.

Scope

build_mcp() -> FastMCP extracted from create_app(). The Starlette app returned by streamable_http_app() does not expose the FastMCP object it was built from, so instructions and list_tools() were unreachable without standing up a full HTTP session — which would have forced these tests into tests/integration/test_mcp_server.py, which is @pytest.mark.e2e and never executes (make test runs -m "unit or observability"; ci-lakebase.yml gates on an unset CI_LAKEBASE_ENABLED). Each call still yields a fresh instance, preserving the single-use StreamableHTTPSessionManager constraint.

SERVER_INSTRUCTIONS (573 chars). A routing signpost, not documentation — it costs system-prompt tokens in every session of every connected client. It routes discovery through list_repos rather than asserting a corpus exists, so it stays true against a freshly forked template with an empty index. find_references/list_imports are deliberately excluded: this answers "should I engage this server," not "here is the full catalog."

title= and annotations= on all six tools, readOnlyHint/openWorldHint, on semantic-accuracy grounds only. All six are genuinely read-only against an external open-world index. No routing or approval-friction credit is claimed. _READ_ONLY is a module constant rather than inline because inlining puts all six registration lines at 108–127 chars against line-length = 100.

Unchanged: tool names (zoekt parity), query semantics, payload shapes. The six tool docstrings are untouched — that is #122.

Tests

New tests/unit/test_mcp_descriptions.py, marked unit so it actually runs. No database required.

  1. instructions is non-empty, contains list_repos, matches /working directory/i and /cannot find locally/i, and equals SERVER_INSTRUCTIONS. The trigger assertions matter: without them the baseline is instructions=None, so any non-empty string passes — 'See list_repos.' would satisfy everything else. /not checked out/i is deliberately not asserted; it is False against the real text, which wraps between NOT and checked.
  2. Boundary: query-grammar denylist primary, 650-char cap secondary. An earlier draft carried the entire grammar at 731 chars under a 900-char cap while hitting zero denylist tokens, so a length check alone does not catch grammar creep. The field half derives from app/query/parser.py:_SUPPORTED rather than being hand-copied, per the convention at app/query/semantic_filters.py:26-27 ("never a second hand-written table") — a future field is denylisted automatically.
  3. Tool metadata over list_tools(): exact six-name roster, non-empty description and title, both annotation hints True.

All three fail on pre-change code.

make lint    All checks passed! / 155 files already formatted / mypy: no issues in 39 source files
make test    1322 passed, 274 deselected

What is verified, and what is not

The mechanism is verified. instructions round-trips through initialize (mcp/server/lowlevel/server.py:152,188), and it survives the documented uvx uc-mcp-proxy registration path, which is a transparent server→client copy.

The behavioral effect is not measured, and is not measurable at this scale. Whether agents actually consult the index more often is unproven. A 10-prompt before/after check was considered and rejected as theater: unblinded, author-run, prompts chosen after the text was written, and a baseline no reviewer can reproduce. Please do not read the mechanism verification as outcome evidence. If the original failure recurs after this ships, that is the signal to build a real eval harness.

One further limit, from review: the protocol guarantees instructions is delivered to the client. Whether a given client injects it into model context is client behavior. That is confirmed for Claude Code; it is not a protocol guarantee for every client.

Sequencing

Consensus planning (Planner → Architect → Critic, 4 revisions) resequenced this work to #121#123#122.

#122 was originally planned to ship in this PR and was cut. Its premise — that tool-search ranks candidates on name + description — is not verifiable from this repo, and has been demoted to a hypothesis in its issue body. #123 (a skill matching on request text) is the stronger trigger for the client where the failure was actually observed, so #122 now lands last, where its marginal value can be judged against a shipped skill rather than guessed at.

🤖 Generated with Claude Code

Agents did not reach for the code-search tools when asked about a repo
that is not checked out locally -- they ran local Glob/Grep, reported
not-found, and never consulted the index.

A server's `instructions` string is returned in the `initialize` result
and reaches the client unconditionally, whereas tool schemas may be
deferred behind a tool-search step when many servers are connected. The
server passed no `instructions`, so it contributed zero always-on
context: only the bare tool names survived, and `search_code` next to a
built-in grep reads like a worse grep.

Extract `build_mcp() -> FastMCP` from `create_app()`. The Starlette app
does not expose the FastMCP object it was built from, so tool metadata
was unreachable without standing up a full HTTP session -- which would
have put the tests in the e2e file that never executes. Each call still
yields a fresh instance, preserving the single-use session-manager
constraint.

Add `title=` and read-only/open-world annotations to all six tools on
semantic-accuracy grounds only; no routing or approval-friction credit
is claimed. `_READ_ONLY` is a module constant because inlining pushes
all six registration lines past line-length 100.

Tool names, query semantics, and payload shapes are unchanged. The six
tool docstrings are untouched -- that is #122, deferred behind #123.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tannerwendland-db
tannerwendland-db merged commit d2190c2 into master Jul 27, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

app: FastMCP server instructions so agents route not-found-locally requests to the index

2 participants