[FEATURE] Implement really small and simple LLM

[WimvandenHeijkant]

Problem & value — what problem, for whom, and why now?

Some of the features of Identity Atlas require a bit to much knowledge. Already creating a new matrix requires a 4 step wizard. People might need to create a context. And then make the matrix from there.

I would like to have an interface where a human can ask in human can express what he or she wants and then get the result. A chat bot kind of interface where you can say something like. I want to create a new matrix view that shows all assignements to DHMR groups over the full organization.. this should then create a matrix view for them.

What should it let someone do?

Find what you need in Identity Atlas by asking for it in human language.

✅ I'm delighted when…

• When it is really easy to find the data you are looking for in identity atlas without needing to know how to make a matrix or a context or any of the other complexity
• If this could be done with a local LLM that run as part of the Identity Atlas solution

❌ I'm disappointed when…

• When I still need to know a lot of complexity

How general should this be?

A general capability (this case is only the first example)

How complete? (optional)

No response

What it deliberately does NOT do (optional)

No response

Where should it appear? (optional)

No response

Related issues / dependencies (optional)

No response

Screenshots / examples (optional)

No response

What's driving this now? (optional)

None

[github-actions]

Hi @WimvandenHeijkant — DoR review of #935 ("really small and simple LLM" / natural-language interface). I ran the build-readiness probe against the live code first, so the questions below are only the ones the code can't answer. The intent is clear and I think this is very buildable — but the scope-ambition choices below are yours to make, and the decomposition depends on them.

What the probe found in the codebase (good news)

Most of the substrate for this feature already exists:

1. LLM plumbing is in place. app/api/src/llm/providers.js + service.js provide a provider abstraction (Anthropic / OpenAI / Azure OpenAI) behind one chat() signature, with API keys in the encrypted secrets vault and an Admin → LLM Settings page. A local LLM was explicitly deferred in v1 (docs/architecture/llm-and-risk-scoring.md) but the abstraction was designed so a local provider is an additive adapter, not a rewrite.
2. A chat-style LLM UX already ships — the Risk Profile wizard's generate/refine chat (RiskProfileWizard.jsx), where drafts stay client-side and nothing touches the database until the user saves. That's the proven interaction pattern to reuse.
3. A matrix is just data. The 3-step wizard produces a structured JSON filter object stored in SavedMatrixFilters (org-wide, app/api/src/routes/matrix/savedFilters.js). So "create a matrix from a sentence" = LLM translates the sentence into that existing, schema-validatable JSON — it never needs to touch SQL or invent a new storage model. Your example ("all assignments to DHMR groups over the full organization") maps to: subject = everyone, resource condition = groups matching "DHMR", orientation/rollup defaults.

Proposed decisions — please ratify (or object) in one reply

These I can defend from the code/architecture; I'll treat a 👍 as ratified:

• P1 — Local LLM = a connectable local provider, not bundled weights. Add an OpenAI-compatible/Ollama adapter as a fourth provider in the existing abstraction, plus an optional compose service — rather than baking multi-GB model weights into the Identity Atlas image (image size, model licensing, and hardware variance make bundling the wrong default). "Runs as part of the solution" is then a compose-profile toggle.
• P2 — The LLM never writes SQL and never touches the database. It only emits structured intents (a matrix filter JSON, a navigation target, a search query) that are schema-validated and executed by the existing APIs. For an authorization product this is the security line: a model must not free-form query who-can-access-what.
• P3 — Grounded, preview-first flow. The assistant resolves names ("DHMR groups") against the real data via the existing search endpoints, then shows you the interpreted matrix through the existing wizard/Apply path — visible and editable before anything is saved. A misunderstanding degrades to a pre-filled wizard, never a silently wrong result. Drafts stay client-side until you save (same as the risk wizard).

Questions only you can answer (numbered — one batched reply covers everything)

1. Operation set (how complete is v1?). "Find what you need by asking" spans several distinct capabilities. Which are in the first cut, which are phase-2?

• (a) Create a matrix view from a sentence (your example)
• (b) Find / navigate — "show me user X", "which groups match Y" → jump to the right page
• (c) Answer questions in the chat — "who has access to Z?" answered as text
• (d) Create contexts or other artifacts
  You chose "general capability", so all of these are presumably the destination — the question is the order and what the first shippable slice must contain.

2. Local LLM: hard requirement or strong preference? This is the single biggest fork:

• Is it acceptable to ship v1 using the already-configured cloud provider (Anthropic/OpenAI/Azure key the operator sets today) with the local provider following as its own slice (P1)? Or is no-cloud a hard constraint for the very first release?
• Related and important either way: what data may leave the boundary? A cloud-LLM assistant would send group/user names (not secrets, but real authz metadata) in prompts. If that's unacceptable for your customers, that's a strong argument to sequence the local provider first — say so and I'll spec it that way.
• What hardware should the local path assume — a CPU-only Docker host (the typical Identity Atlas deployment), or can a GPU be assumed?

3. "Really small" vs. "actually works" — where's your floor? Honest counsel from the probe: translating fuzzy human asks into correct structured filters, with real entity-name grounding, is a hard task for tiny models. Sub-3B "really small" models will misinterpret often; the v1 architecture doc already benchmarked ~Qwen-2.5-14B / Mistral-Small-22B as the realistic CPU floor for structured-JSON quality (several GB RAM, seconds-per-answer on CPU). If forced to choose, which wins: smaller footprint or higher answer quality? (P3's preview-first design is the safety net either way.)

4. Accept/reject criteria — give me 3–5 concrete utterances that MUST work. Your delighted/disappointed criteria are currently abstract ("still need to know a lot of complexity"). The DHMR sentence is utterance #1. Please add a few more you'd personally test with (ideally including one vague one, so we pin the graceful-clarification behaviour: the assistant should ask a follow-up or open the pre-filled wizard, never guess silently). These become the fixture acceptance criteria.

5. Language. English-only for v1, or must Dutch utterances work too? (Backlog #723 — multi-language support — is open but not started; LLMs handle Dutch input largely for free, but testing it is scope.)

6. Placement (form — you skipped this field). Proposal to ratify or hand to the designer: a global assistant entry point (button/panel available on every page), with matrix-creation results opening in the Matrix tab. Fine, or did you picture it living only inside the matrix area?

Backlog context

• [FEATURE] matrix to see which access packages a certain group of users have #931 (matrix: which access packages a group of users has) — a perfect candidate utterance for Q4; the assistant should eventually handle exactly that ask.
• [UX] Per-page contextual help/doc links + in-app glossary #772 (per-page contextual help + in-app glossary) — attacks the same "too much required knowledge" problem from the deterministic side; complementary, not duplicate.
• [FEATURE] Reports Tab #934 (Reports Tab) — adjacent: "ask for data in human language" and "canned reports" overlap in audience; worth keeping the operation-set answer (Q1) consistent with it.
• [Feature] Multi-language support (starting with Dutch) — site, product & docs i18n #723 (multi-language) — see Q5.

What happens next

This is too large to build as one feature — once you've answered, I expect to split it (likely: S1 local/OpenAI-compatible provider in LLM Settings → S2 NL→matrix assistant with entity grounding → S3 find/Q&A → S4 broader artifact creation + global surface), with the order driven by your Q1/Q2 answers. One consolidated reply here un-blocks all of it.

Route: state:awaiting-requestor — the open items are scope-ambition choices that belong to the requestor.