Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,10 @@ jobs:
with:
python-version: ${{ matrix.python-version }}

# The selection + persistence logic is deliberately free of the heavy
# app stack (ChromaDB, LangChain, Streamlit), so CI runs it with just
# pytest. test_cli.py self-skips without the full deps; the complete
# 45-test suite runs locally via `uv run pytest`.
- run: pip install pytest
# The selection, persistence, and MCP-tool logic are deliberately free
# of the heavy app stack (ChromaDB, LangChain, Streamlit), so CI runs
# them with just pytest + the (lightweight) mcp SDK. test_cli.py
# self-skips without the full deps; the complete suite runs locally
# via `uv run pytest`.
- run: pip install pytest "mcp>=1.6,<2"
- run: pytest tests/ -v
60 changes: 53 additions & 7 deletions DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ Architecture and data model for Java Interview Coach. See [README.md](README.md)

## Architecture overview

Two pipelines feed into two interchangeable UIs (Streamlit's `app.py`, or `cli.py`'s stdin
loop) that each drive the exact same LangGraph-style workflow — one button click, or one
`input()` call, at a time.
Two pipelines feed into three interchangeable front-ends (Streamlit's `app.py`, `cli.py`'s
stdin loop, or the `mcp_server/` MCP tools) that each drive the exact same LangGraph-style
workflow — one button click, one `input()` call, or one tool call, at a time.

```mermaid
flowchart TD
Expand Down Expand Up @@ -178,6 +178,42 @@ cumulative stats carry over between the two.
and only the `evaluate` node's actual network call does. See Known limitations below for
exactly how far this was verified to run in this environment.

### MCP server (round 3)

`mcp_server/` exposes the coach over the [Model Context Protocol](https://modelcontextprotocol.io)
as a third front-end alongside Streamlit and the CLI, so any MCP client (Claude Desktop,
Claude Code, Cursor, VS Code) can run mock interviews using the same retrieval, grading,
persistence, and spaced-repetition logic.

- **Layering.** `mcp_server/tools.py` is the tool logic as plain functions with heavy
dependencies passed in (`collection`, `llm`, `db_path`). It imports only `memory.store`
and `topics` at module level — deliberately *not* `graph.workflow` (which pulls the whole
LangChain/LangGraph chain) or `chromadb` or `mcp`. The 3-line RAG `query` call and the
evaluate/hint prompt text are inlined / moved to `prompts.py` so the module stays light.
Result: all nine tools are unit-tested with just `pytest` + a temp SQLite file + a fake
collection / fake LLM (`tests/test_mcp_tools.py`).
- **Wire layer.** `mcp_server/server.py` is the only file that imports `mcp`. It wraps each
`tools.py` function with `@mcp.tool()`, adds three resources
(`interview://topics`, `interview://progress`, `interview://question-bank/{topic}`) and a
`mock_interview` prompt, and owns the lazily-built ChromaDB collection and `ChatGroq`
client — so `list_tools` and the store-backed tools respond instantly and a missing
`GROQ_API_KEY` only surfaces (as a clear `RuntimeError`) if a grading tool is actually
called.
- **Transports.** `python -m mcp_server` runs stdio (local clients); `--http` runs
FastMCP's streamable-HTTP on `MCP_HOST`/`MCP_PORT` (default `127.0.0.1:8000/mcp`) for a
hosted deployment. Both were smoke-tested with a real MCP client handshake.
- **Sessions.** MCP tool calls are stateless, so `evaluate_answer` only records an attempt
when the caller threads through a `session_id` from `start_session` (plus a `topic`).
Without them it still grades, just doesn't persist — the `mock_interview` prompt tells the
client to always pass them.
- **`mcp` version.** Pinned to `>=1.6,<2`: v2.x renamed `FastMCP` to `MCPServer` and
changed the API, and the v1 `FastMCP` surface is what current client docs and examples
assume.

`topics.py` and `prompts.py` were extracted in this round so the CLI, the workflow graph,
and the MCP server share one topic list and one set of prompt strings instead of three
copies drifting apart.

## Notebooks vs. real modules

| Still notebook-only | Promoted to a real module |
Expand All @@ -188,9 +224,12 @@ exactly how far this was verified to run in this environment.

## Tests

`tests/test_store.py` and `tests/test_selection.py` are real `pytest` tests (40 total, all
passing as of this writing — run `uv run pytest tests/ -v` to reproduce), added in round 2.
They replace round 1's manual-script verification for these two modules:
`uv run pytest tests/ -v` runs 61 tests, all passing as of this writing. `test_store.py`
and `test_selection.py` (40 tests) were added in round 2 to replace round 1's manual-script
verification; `test_mcp_tools.py` and `test_mcp_server.py` (16 tests) were added in round 3
with the MCP server. All of them except `test_cli.py` run with just `pytest` (+ the
lightweight `mcp` SDK for the two MCP files) — no ChromaDB, Groq key, network, or
`questions_db.json`.

- `test_store.py` exercises `memory/store.py` end to end against a fresh temp SQLite file
per test (`tmp_path`, via the `db_path` parameter every `store` function already accepts)
Expand All @@ -216,7 +255,14 @@ They replace round 1's manual-script verification for these two modules:
instead of looping, EOF-mid-answer still submitting what was typed) by monkeypatching
`builtins.input`. It's the only I/O-free logic in `cli.py` — everything else in that file
does real ChromaDB/Groq I/O, so it's verified by hand instead (see Known limitations).
- None of the three test files require `GROQ_API_KEY`, network access, or
- `test_mcp_tools.py` calls every MCP tool through `mcp_server/tools.py` with a fake
collection (returns a fixed candidate list) and a fake LLM (returns fixed content), plus a
temp SQLite file — covering topic accuracy roll-up, `get_interview_question` selection and
its empty-bank branch, `evaluate_answer` CORRECT/INCORRECT parsing and the persist-only-
when-scoped rule, and the spaced-repetition tools. `test_mcp_server.py` checks the FastMCP
registration (all nine tools, three resources, the prompt) and that a grading tool raises
a clear `GROQ_API_KEY` error when unset; it self-skips if the `mcp` SDK isn't installed.
- None of the committed test files require `GROQ_API_KEY`, network access, or
`questions_db.json`.

## Known limitations
Expand Down
73 changes: 65 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,12 @@ pipeline**: semantic search over 1,715 real interview questions finds a
relevant pool, then a difficulty- and history-aware ranker decides which one
to actually ask. An LLM grades each answer against an ideal answer. Every
attempt is persisted, so weak-topic weighting and a spaced-repetition
schedule accumulate across sessions. A Streamlit UI and a headless CLI share
the exact same core.
schedule accumulate across sessions.

Three front-ends drive the identical core: a **Streamlit UI**, a **headless
CLI**, and an **MCP server** ([Model Context Protocol](https://modelcontextprotocol.io))
that exposes the whole coach as tools any MCP client — Claude Desktop, Claude
Code, Cursor, VS Code — can run mock interviews against.

## The pipeline

Expand All @@ -30,7 +34,7 @@ topic ─▶ ChromaDB semantic search ─▶ candidate pool ─▶ selection.py
| **Selection layered on top of RAG, not replacing it** | Vector search is good at "relevant to this topic," bad at "the right difficulty for this user right now." `graph/selection.py` takes the retrieved pool and re-ranks it against persisted per-topic accuracy, so retrieval stays simple and the adaptivity is testable in isolation (no vector store needed). |
| **Difficulty is an explicit heuristic, labelled as a proxy** | The 1,715-question bank has no difficulty labels. `_estimate_difficulty` approximates it from question length and phrasing ("why"/"how does X work"/internals vs. "what is"/"define"). Called out in code and docs so it isn't mistaken for ground truth. |
| **Persistence is stdlib `sqlite3`, no ORM** | `st.session_state` dies with the process. `memory/store.py` writes every attempt (topic, question, correctness, timestamp, session id) so stats survive restarts. One small module, one file, fully unit-tested against temp DBs. |
| **UI and CLI call the identical core** | `app.py` (Streamlit) and `cli.py` both drive the same `graph/` and `memory/` modules — no business logic behind a Streamlit import. The CLI is both a scripting entry point and proof the split is real. |
| **One core, three front-ends** | `app.py` (Streamlit), `cli.py`, and `mcp_server/` all drive the same `graph/` + `memory/` modules — no business logic behind a Streamlit import. `mcp_server/tools.py` holds the tool logic as dependency-injected plain functions (kept off the LangChain/ChromaDB import chain), so it's unit-tested with just `pytest`; `mcp_server/server.py` is the only file that touches the MCP wire protocol. |
| **Spaced repetition on the same file** | A post-answer confidence rating (Again/Hard/Good/Easy) schedules that exact question 1/3/7/14 days out; "Due for Review" mode resurfaces what's due instead of pulling from RAG. |

See [DESIGN.md](DESIGN.md) for the full architecture and data model.
Expand Down Expand Up @@ -64,22 +68,69 @@ echo "my answer\n\nquit" | uv run python cli.py --questions 5 # scripted
Same prerequisites and same persistence as the Streamlit app — CLI and UI
sessions share stats.

## MCP server

Exposes the coach over the [Model Context Protocol](https://modelcontextprotocol.io)
so any MCP client can use it as a tool.

```bash
uv run python -m mcp_server # stdio (for Claude Desktop / Cursor / VS Code)
uv run python -m mcp_server --http # streamable-HTTP on 127.0.0.1:8000/mcp
```

**Tools:** `list_topics`, `start_session`, `get_interview_question` (RAG +
adaptive selection), `evaluate_answer`, `get_hint`, `record_attempt`,
`rate_question`, `get_due_reviews`, `get_progress`.
**Resources:** `interview://topics`, `interview://progress`,
`interview://question-bank/{topic}`.
**Prompt:** `mock_interview(topic, num_questions)` — a template that drives a
full session through the tools.

The ChromaDB collection and the Groq client are built lazily, so `list_tools`
and the store-backed tools respond instantly. `evaluate_answer` / `get_hint`
need `GROQ_API_KEY`; without it they return a clear message and everything
else still works.

<details>
<summary>Claude Desktop / Cursor config</summary>

`claude_desktop_config.json` (or `.cursor/mcp.json`):

```json
{
"mcpServers": {
"java-interview-coach": {
"command": "uv",
"args": ["--directory", "/ABS/PATH/TO/java-interview-coach", "run", "python", "-m", "mcp_server"],
"env": { "GROQ_API_KEY": "your_key_here" }
}
}
}
```

Claude Code: `claude mcp add java-interview-coach -- uv --directory /ABS/PATH run python -m mcp_server`
</details>

## Tests

```bash
uv run pytest tests/ # 45 tests
uv run pytest tests/ # 61 tests
```

- `test_store.py` — session/attempt CRUD, cumulative stats, recent-question
filtering, spaced-repetition interval math, upsert-on-rerate, due-question
ordering (all against temp SQLite files).
- `test_selection.py` — difficulty heuristic, difficulty re-ranking,
recently-asked filtering, auto-topic weighting.
- `test_mcp_tools.py` — every MCP tool via a fake collection / fake LLM and a
temp DB.
- `test_mcp_server.py` — the FastMCP wiring: all tools/resources/prompt
registered, graceful error without `GROQ_API_KEY`.
- `test_cli.py` — `cli.py`'s stdin-parsing helper.

`test_store.py` and `test_selection.py` need only `pytest` — no ChromaDB,
Groq key, or network — which is what CI runs on 3.11 and 3.12. `test_cli.py`
self-skips unless the full app stack is installed.
Everything except `test_cli.py` runs with just `pytest` + the (lightweight)
`mcp` SDK — no ChromaDB, Groq key, or network — which is what CI runs on 3.11
and 3.12. `test_cli.py` self-skips unless the full app stack is installed.

## Tech stack

Expand All @@ -89,14 +140,16 @@ self-skips unless the full app stack is installed.
| Agent framework | LangChain + LangGraph |
| Vector store | ChromaDB |
| Persistence | SQLite (stdlib `sqlite3`) |
| UI | Streamlit |
| Front-ends | Streamlit · CLI · MCP server (`mcp` SDK, FastMCP) |
| Env / packaging | Python 3.11, `uv` |

## Project structure

```
app.py Streamlit UI, wires graph nodes to session state
cli.py headless practice loop: ask -> answer -> evaluate -> score
topics.py the fixed topic list, shared by every front-end
prompts.py evaluate / hint prompt text, shared by workflow.py and mcp_server
corpus.py shared ChromaDB collection loader
report.py exportable Markdown session report
rag.ipynb fetches + parses the question corpus into questions_db.json
Expand All @@ -106,6 +159,10 @@ graph/
selection.py difficulty-adaptive re-ranking on top of RAG retrieval
memory/
store.py SQLite persistence for sessions / attempts / reviews + stats
mcp_server/
tools.py tool logic as dependency-injected plain functions
server.py FastMCP wiring (tools / resources / prompt) + lazy deps
__main__.py `python -m mcp_server [--http]`
tests/ pytest suite (see above)
```

Expand Down
8 changes: 1 addition & 7 deletions cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,7 @@
from graph.selection import pick_topic_for_auto_mode
from graph.workflow import build_nodes
from memory import store

AUTO = "auto"
TOPICS = [
"OOP", "Java Core", "Java Collections", "Spring",
"JVM", "Multithreading", "Databases", "Java 8",
"Patterns", "Testing",
]
from topics import AUTO, TOPICS


def _prompt_answer() -> str | None:
Expand Down
22 changes: 3 additions & 19 deletions graph/workflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,29 +29,13 @@
from graph.selection import select_question
from graph.state import InterviewState
from memory import store
from prompts import EVAL_PROMPT, HINT_PROMPT

# RAG candidates fetched per question before the adaptive layer ranks them.
QUESTION_POOL_SIZE = 12

eval_prompt = ChatPromptTemplate.from_template("""
You are a Java technical interviewer evaluating an answer.

Question: {question}
Candidate's Answer: {answer}

Respond with:
1. CORRECT or INCORRECT
2. Brief feedback (2-3 sentences)
3. Ideal answer in simple terms

Start your response with either CORRECT or INCORRECT on the first line.
""")

hint_prompt = ChatPromptTemplate.from_template("""
You are a helpful Java tutor.
Give a short hint (2-3 sentences) for this question without giving away the answer.
Question: {question}
""")
eval_prompt = ChatPromptTemplate.from_template(EVAL_PROMPT)
hint_prompt = ChatPromptTemplate.from_template(HINT_PROMPT)


def retrieve_candidates(collection, topic: str, n_results: int = QUESTION_POOL_SIZE) -> list[str]:
Expand Down
11 changes: 11 additions & 0 deletions mcp_server/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
"""MCP server for Java Interview Coach.

Exposes the same retrieval + adaptive-selection + grading + spaced-repetition
core that the Streamlit app and the CLI drive, as Model Context Protocol
tools/resources/prompts — so any MCP client (Claude Desktop, Claude Code,
Cursor, VS Code) can run mock interviews against it.

``mcp_server.tools`` holds the logic as plain functions (dependency-injected,
unit-testable). ``mcp_server.server`` wires them to FastMCP and owns the
lazily-built ChromaDB collection and Groq client.
"""
3 changes: 3 additions & 0 deletions mcp_server/__main__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
from mcp_server.server import main

main()
Loading
Loading