feat(api): delete, external ids, and a JSON HTTP API

[dimittal]

Stacked on #21 (which is stacked on #20) — based on feat/retrieval-audit so this diff shows only the CRUD and API work.

Completes CRUD and puts it on a second surface. The explorer stays read-only — writes keep going through a validated path, as decided.

delete_entity

End to end through both backends, Brain, the CLI, MCP and the API. Three things it has to get right:

The file. For a storage: file doc_type the JSON on disk is the source of truth, so removing only the index row is a pause, not a delete — the entity returns on the next reindex. There is a test that deletes, reindexes, and asserts it stays gone.

The edges, in both directions. An edge outliving its target still renders on the map and in neighbour lists; it reads as corruption rather than as a deletion. On OpenSearch that needs an update_by_query as well as the delete, because edges live on the source document.

The order. The file goes first. If the index delete then fails, a reconcile cleans up; the other order leaves a file that silently restores a deleted entity. A delete that cannot remove the file deletes nothing and says so.

external_id

The id the source system already knows a thing by — a ticket key, a CRM record, a UUID.

id keeps its <doc_type>:<slug> shape, because that is what lets an agent read a document's type straight off its id. The free-form identifier goes beside it, with lookup_by_external_id on MCP, the API and the CLI. Also get_entities([ids]), which agents were emulating with N round trips.

OpenSearch maps fields explicitly — the code already warns that anything unnamed is dropped on write — so external_id needed adding to the mapping and both conversion directions. Omitted, the lookup would have silently never matched.

A JSON HTTP API

At /<index>/api/v1, mounted beside the explorer, so one process and one port serve the UI, the API and the MCP endpoint for every index on the host.

It mirrors the MCP tools rather than inventing a second vocabulary, and calls the same Brain methods, so the two cannot disagree about what a search means or what a write validates.

GET    /api/v1/search?q=…&mode=keyword&filter.tenant_id=acme
GET    /api/v1/entities/{id}
GET    /api/v1/entities?id=a&id=b
GET    /api/v1/entities/by-external-id/{external_id}
GET    /api/v1/schema
GET    /api/v1/traces/{trace_id}
PUT    /api/v1/entities/{id}
DELETE /api/v1/entities/{id}

• The URL is authoritative on PUT. A body id that disagrees is a 400, so PUT /entities/a can never write entity b.
• A caller's mistake is a 4xx with a message that says how to fix it. An unknown mode and a filter on an undeclared field are 400, not 500 — the search contract's fail-closed behaviour surfaces as a fixable error rather than an opaque one.
• /api/... is registered before the /{name} page routes, so it is never read as an index named "api".

Auth

Off unless a token is configured, and it gates writes only — reads stay open, matching how the MCP endpoint already behaves. A deployment holding real data sets OPEN_INDEX_TOKEN (or a per-index OPEN_INDEX_TOKEN_<NAME>); one serving public demo data does not, and nothing changes for it.

401 with a WWW-Authenticate challenge rather than 403: the caller can fix it by presenting a token, and should be told how.

A lost test, restored

The assertion tying the help tab's documented tool list to the tools the server actually registers went missing when the Streamlit test suite was replaced. It is back — and it immediately did its job, catching that the page had gone stale against three new tools. A tool list that lies about what an agent can do is worse than no list at all.

629 tests pass, api.py at 100%. New docs page for the API, and the navigation guide now covers identifiers and when not to delete.

🤖 Generated with Claude Code