Cartographer

A codebase-knowledge agent that tells you why the code is the way it is, and refuses to make it up.

Ask "why is this built this way?" about a codebase. Cartographer answers with an exact, clickable file:line citation and the deciding pull request or issue thread. When it cannot prove a claim from the source, it refuses instead of fabricating. And it ships a ranked bus-factor gap report: the modules whose rationale lives only in someone's head.

Built for the Kaggle × Google "5-Day AI Agents: Intensive Vibe Coding" capstone (track: Agents for Business). Stack: Google ADK and native Gemini, developed in Claude Code.

Watch the five-minute demo: https://youtu.be/2yPX0n4MAU0

Why it's different

Most "chat with your codebase" tools answer anything you ask, including things that aren't in the code. Cartographer inverts that. Provenance is the product. Every answer is either grounded (teal, with a source line you can click) or refused (red, logged as a bus-factor gap). The refusal is a feature, not a failure.

	Typical code-chat	Cartographer
Unprovable claim	Answers anyway (hallucinates)	Refuses, deterministically
Evidence	"Trust me"	Exact file:line + the deciding PR/issue
Code right, reason unwritten	Invents a plausible reason	Amber partial: "the code checks out, but the why was never written down"
Output of not knowing	Silence or a guess	A ranked bus-factor gap report

Answering is three-way: a grounded ANSWER (teal, cited to a PR), an honest PARTIAL (amber, code-grounded but rationale-undocumented), or a REFUSAL (red, logged as a gap). The combination is the moat: a deterministic refuse-to-fabricate gate, plus the bus-factor report, plus the undocumented-rationale class. See docs/WRITEUP.md for prior art and an honest list of limitations.

The grounding gate (real and tested)

The heart of the product is a deterministic gate. A claim is grounded only when every piece of its evidence appears as a whole identifier token that real code in the cited span actually references, with comments and docstrings excluded. It is a high-precision fabrication trap (necessary, not sufficient); the semantic "does this span actually answer the question" layer is an advisory Gemini LLM-judge on top.

bash scripts/verify.sh          # lint + the grounding-gate tests
python grounding/test_gate.py   # or run the gate tests directly (no pytest needed)

The tests include adversarial bypasses that a naive substring check would wrongly accept and this gate rejects: "S3" hiding inside the identifier bucket_S3_DISABLED, a library named only in a comment, or a behavior described only in a docstring. See grounding/.

Status

Built, deployed, and demoable. The full pipeline works end to end, offline and deterministic; the core beats were observed end to end via Playwright (web/observe_demo.py).

• Grounding gate (grounding/): deterministic, 13/13 (6 grounded controls + 7 adversarial bypasses); emits per-token receipts.
• ADK + Gemini agent (agent/): classify, retrieve, draft, verify, route to answer / partial / gap. The verifier is a @node.
• Custom MCP server (mcp_server/): get_source_span, git_blame_why, verify_claim.
• Bus-factor gap report and memory (agent/gaps.py, agent/memory.py).
• Eval gate (eval/): 0 fabrications (no LLM), refusal pass^k, gap precision/recall, running in CI.
• Provenance map: all 52 src/werkzeug modules across 7 package communities; 9 proven (teal), 43 inferred (amber).
• Routed web UI (api/, web/): same-origin FastAPI. / overview, /ask cockpit (MAP / ASK / EVIDENCE, with ?q= deep links), /gaps, /module/<path>, /tour. Responsive, WCAG-AA.
• Deployed demo: authenticated Cloud Run behind IAP, cached mode. Real live AI runs locally via run-local.bat (below).

The full security and cost posture is in docs/SECURITY.md.

Architecture

Cartographer is an ADK + Gemini agent, not a deterministic lookup. The gate is its guardrail.

flowchart TD
    Q["Question: why is this built this way?"] --> C["Classify · LlmAgent<br/>gemini-2.5-flash-lite"]
    C --> R["Retrieve · custom MCP tools<br/>get_source_span · git_blame_why · list_symbols"]
    R --> SRC[("real code + PR / issue thread")]
    SRC --> D["Draft candidate answer · LlmAgent<br/>gemini-2.5-pro"]
    D --> G{"DETERMINISTIC GATE (no LLM)<br/>grounding/gate.py · AST-token, whole-word"}
    G -->|"ACCEPT + deciding PR"| P["PROVEN<br/>file:line + the deciding PR"]
    G -->|"ACCEPT, no thread"| PA["PARTIAL<br/>code checks out, why undocumented"]
    G -->|"REJECT"| RF["REFUSAL<br/>logged as a bus-factor gap"]
    P --> J["Advisory LLM-judge<br/>downgrade or reject, never upgrade"]
    PA --> J
    RF --> GAP[["Bus-factor gap report<br/>+ compounding memory"]]
    classDef proven fill:#0b3b34,stroke:#2dd4bf,color:#e9f9f5,stroke-width:2px
    classDef partial fill:#3a2c08,stroke:#f59e0b,color:#fdf3e0,stroke-width:2px
    classDef refuse fill:#3a1414,stroke:#ef4444,color:#fde8e8,stroke-width:2px
    classDef gate fill:#0d1117,stroke:#2dd4bf,color:#e6edf3,stroke-width:3px
    class P proven
    class PA partial
    class RF refuse
    class G gate

Loading

The agent runs two LlmAgents: classify on gemini-2.5-flash-lite and draft on gemini-2.5-pro (verified live on Vertex). An ADK Workflow / @node graph routes classify, retrieve, draft, verify, then answer / partial / gap. A custom MCP toolset (mcp_server/) fetches the real source and history, untrusted input is fenced (agent/pipeline.py:_fence), and compounding memory (agent/memory.py) persists refusals across sessions. The grounding verifier is a @node, and the verdict comes from the deterministic gate, never the LLM. The model proposes; the gate disposes.

Two engines stay separate. A stdlib-ast indexer is the source of truth for citations (deterministic line ranges); the import graph is display-only and never a citation source. The why-layer recovers rationale from the PR or issue thread (via CHANGES.rst links), not commit messages, and a thread only counts if it isn't a chore and substantively states a reason.

For serving, FastAPI mounts the built SPA same-origin (no CORS) as a routed shell. The deployed path serves cached answers with no live LLM (a denial-of-wallet control) on Cloud Run scale-to-zero. The live lane runs locally via run-local.bat in hybrid mode (agent/llm.py:HybridProvider: cache-first plus budgeted live Gemini through your own gcloud ADC), and every live answer goes through the same deterministic gate.

Course concepts

ADK Workflow/@node orchestration (D1), a custom MCP toolset (D2), compounding memory (D3), the refuse-to-fabricate verifier plus eval-as-CI-gate plus spotlighting of untrusted input (D4), and spec-driven development (9 executable specs/*.feature) on Cloud Run (D5).

Quickstart (local, no cloud, no creds, cached hot path)

python -m pip install -r requirements-dev.txt
npm --prefix web ci && npm --prefix web run build
uvicorn api.main:app --port 8080        # open http://127.0.0.1:8080/

Run with real live AI (local, hybrid)

Ask about any of the 52 modules in real time. Cached answers are instant; anything new is answered live by Gemini through your own gcloud ADC (no IAM, no deploy). The first run builds the venv and UI.

gcloud auth application-default login    # one-time
run-local.bat                            # Windows: double-click or run; serves http://localhost:8080 (hybrid)

Live answers go through the same deterministic gate and surface a LIVE pill. The deployed Cloud Run service stays cached by design (denial-of-wallet), so the five-minute video is the canonical demo.

Verify and re-run the eval (the honest numbers)

bash scripts/verify.sh                  # ruff + tests + eval gate + tsc + web build + e2e (the one-command gate)
python eval/run_eval.py --gate          # 0 fabrications + refusal pass^k + coverage + gap P/R, with denominators
python web/observe_demo.py              # drive the demo beats headlessly (Playwright)

Deploy (authenticated Cloud Run)

scripts/deploy.sh builds the Dockerfile via Cloud Build and deploys to authenticated Cloud Run (--no-allow-unauthenticated, scale-to-zero, max 3 instances, cached mode). It is written to reuse an existing project without creating any IAM, service account, or VPC: it takes an existing builder SA and subnet, runs on the default compute SA, and uses all-traffic VPC egress where the platform requires it. Every value is env-overridable for your own project.

gcloud auth login                # one-time, interactive
export GOOGLE_CLOUD_PROJECT=your-gcp-project
bash scripts/deploy.sh

Reach the authenticated endpoint via gcloud run services proxy cartographer --region <region> or IAP. The deployed endpoint serves cached answers only; live AI is the local run-local.bat lane above.

Submission

The capstone cover is COVER-B-v3-FINAL (user-supplied, AI-made, disclosed). The five-minute demo video (https://youtu.be/2yPX0n4MAU0) follows the /tour walkthrough beat for beat and is the canonical demo, since the deployed service is authenticated and not public.

License

MIT.