feat: many indexes on one host — usable explorer, URL-path routing, proxy-safe MCP

[dimittal]

Four commits, each reviewable on its own. Everything here came out of running the tool for real rather than from reading the code.

1. Many brains from one process

serve --brains <root> mounts every brain under a directory at its own path, so /<name>/mcp is that brain's endpoint. One process, one port, one container.

A container per brain was the obvious approach and does not scale — each carries its own interpreter and model cache. Measured: 200 brains in one process is ~373MB (~1.8MB marginal each) against roughly 53GB as containers. SQLite matters here too; one OpenSearch index per brain makes each brain a shard, which is the thing that stops scaling first.

Starlette does not run a mounted app's lifespan, so each child's is entered explicitly via AsyncExitStack. Without it the MCP session manager never starts and every request hangs.

Per-brain tokens via OPEN_INDEX_TOKEN_<NAME>, so one open endpoint doesn't open the rest. The container entrypoint no longer assumes exactly one brain at /brain.

2. A usable explorer

The UI was only useful for search — it showed no structure and no way to see the graph. Rebuilt around what a first-time reader needs, in order:

Tab
How to use?	default tab — what a doc_type and an entity are, that relationships are optional, the MCP endpoint, the tool list
Schema	every doc_type and its fields, so the shape is visible at all
Explore	search
Map	the whole index, not one entity's neighbourhood

The map was unreadable. It drew every entity's name beside its dot; names are long and arbitrary, so they overlapped each other and their own edges. The canvas now carries shape and colour only, with full detail on hover. It also opened blank until you picked an anchor — the wrong default for someone who has never seen the index. It now shows the whole index, capped at the 250 most-connected nodes, and says so when the cap bites rather than presenting a subset as everything.

Two bugs that only appear once one process serves many brains:

• Every path rendered the same brain. Streamlit's router resolves programmatic pages by an internal identifier, so /a and /b both served whichever sorted first. A curl status check cannot catch this — Streamlit returns its shell for any path — so this was confirmed with a WebSocket probe carrying a per-brain marker. The index now comes from st.context.url, and nothing else selects it.
• The help tab showed no MCP endpoint. A shared explorer cannot be handed one correct URL as configuration; the answer depends on which index you're looking at. It's derived from the page URL, which also keeps it right behind any proxy or hostname. OPEN_INDEX_PUBLIC_URL stays as the fallback for single-brain deployments.

Presentation logic lives in ui/view.py with no Streamlit import, so it's testable directly; app.py is widgets over it.

3. OpenSearch — three bugs, all found with real data

1. Search failed outright on any brain with a numeric, boolean or timestamp field. OpenSearch rejects the whole multi_match if a non-text field is listed, so one numeric field broke all search.
2. A string field holding date-shaped values was mapped as a date by dynamic detection, producing the same rejection for a correctly-declared field. date_detection is off, so the schema decides the type rather than the first document indexed.
3. If the index was dropped after the brain was opened, the next write auto-created it with a dynamic mapping — doc_type as text, breaking every aggregation long after the write that caused it.

4. Serving behind a proxy

serve could not sit behind any reverse proxy: the MCP SDK enables DNS-rebinding protection with a localhost-only allow-list, so a proxied request arrives with a foreign Host and is rejected 421. Passing a non-localhost bind address to the SDK would have silently disabled the protection, so the allow-list is built explicitly from --public-url plus --allowed-host / OPEN_INDEX_ALLOWED_HOSTS. * opts out for a trusted proxy.

Tests

535 pass, ui/view.py at 100%. New coverage for multi-brain mounting and lifespans, URL-path selection and its fallbacks, endpoint derivation, the overview graph, label-free rendering, the schema tables, the host allow-list (including the with/without-port case that caused the 421), and field-type filtering.

Verified end to end on a real deployment over HTTPS, several indexes behind one host.

🤖 Generated with Claude Code