feat(kg): support local/OpenAI-compatible LLM endpoints

[bferanmi806-sketch]

Fixes #598. @keysersoft — implemented per your approved contract, plus the review follow-ups below.

Contract compliance:

• Uses KG_LLM_BASE_URL (not AI_BASE_URL); optional KG_LLM_API_KEY (not AI_*); keeps using KG_LLM_MODEL.
• Base URL set → OpenAI-compatible request goes there; empty key valid (Ollama/LM Studio need none).
• Authorization: Bearer … only when a key exists (custom path); hosted paths byte-identical.
• No response_format on the custom endpoint — JSON instruction + existing parsing instead.
• Unusable local JSON → explicit { json: null, skipped: true }; enrich()/generateForConnectors()/generateForServer() return early with no hash write and no suggestion replacement, so the next run retries normally. consolidate() unchanged and safe (json?.skills → early return).
• Endpoint logging uses new URL(base).origin only (safe fallback when unparseable).
• OpenAI/OpenRouter/Anthropic behaviour preserved when no base URL is set; batch stays Anthropic-only.

Tests: llm-client.spec.ts (9 tests), new kg-llm.service.spec.ts (hash untouched on skip + retry stores normally), kg-skill.service.spec.ts additions (pending suggestions untouched on skip, still replaced on usable reply, consolidate untouched on skip). Focused suites 17/17; broader src/knowledge-graph + src/ee/cloud 132/132; eslint clean; zero tsc errors in touched files (one pre-existing cookie-parser types error repo-wide).

Docs/Docker: docs/knowledge-graph.md block now comments out the localhost URL, explains Docker localhost, documents host.docker.internal:11434 + Linux extra_hosts + compose service-name option. docker-compose.yml passes all five KG_LLM_* vars with empty defaults (docker compose config validates). docker-compose.quickstart.yml intentionally untouched (minimal eval stack, carries no KG vars at all).

Real local run (honest harness note: the Ollama binary download is blocked in this sandbox, so this ran Qwen2.5-0.5B-Instruct via llama.cpp behind a minimal /v1/chat/completions front — the identical request/response contract Ollama serves):

• resolved: {provider:custom, model:qwen2.5, apiKey:'', baseUrl:http://localhost:11435/v1}
• server saw: response_format=null auth=absent
• parsed: {relationships:[{from:e0, to:e1, kind:same_identity, confidence:0.9, reason:e0 is a person and e1 is a customer}]}

Happy to sign the CLA when the bot asks.

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[github-actions]

👋 Welcome, @bferanmi806-sketch, and thanks for opening your first PR on AnythingMCP!

A few quick pointers:

• Make sure CI is green before requesting review (Backend, Frontend, Playwright, CodeQL, Trivy).
• If this is a new adapter, the parametrised catalog.spec.ts test will validate it automatically.
• Sign off your commits if you can — it's not blocking, just nice to have.

Someone from the core team will look at this within ~48h. If you don't hear back, please ping us in Discussions / Q&A.

⭐ While you wait — if you find AnythingMCP useful, a star helps others discover it.

[bferanmi806-sketch]

recheck

[bferanmi806-sketch]

I have read the CLA Document and I hereby sign the CLA

[keysersoft]

Thanks @bferanmi806-sketch, this is close. The hosted paths are byte-identical, I ran the KG and cloud cron suites locally (128 passing, tsc clean), and since the base URL only comes from the instance env there's no new SSRF surface.

One thing has to change before I merge. Returning { json: {} } on a bad reply doesn't skip the pass, because the callers treat it as a valid empty answer:

1. KgLlmService.enrich() hands it to applyEnrichResult(), which stores kg_llm_hash. The next run sees an unchanged graph and returns skipped, so one garbled reply blocks enrichment until someone forces a rerun.
2. applyConnectorResult() and generateForServer() delete the pending skill suggestions before inserting the new ones, so an empty result wipes the pending list.

consolidate() already ignores an empty result, so that one is fine.

Could you make the skip explicit? For example return { json: null, skipped: true } (or throw a small LlmUnusableReplyError) on the custom path, and have enrich(), generateForConnectors() and generateForServer() return early without touching the hash or the pending suggestions. A test for the enrich case and one for the skills case would be great. While you're there, please log new URL(base).origin instead of the full base, in case someone puts credentials in the URL.

Two smaller things on the docs:

• The block in docs/knowledge-graph.md is meant to be copied, and KG_LLM_BASE_URL=http://localhost:11434/v1 is uncommented there. Someone pasting it with an OpenAI key would end up on localhost without noticing. Please comment it out like you did in .env.example.
• Most self-hosters run us in Docker, where localhost is the container itself. Please add a line about http://host.docker.internal:11434/v1 (on Linux that needs extra_hosts: ["host.docker.internal:host-gateway"]), or the service name when Ollama runs in the same compose project. Also, docker-compose.yml doesn't pass any KG_LLM_* variable to the app container today. If you're up for it, add KG_LLM_ENABLED, KG_LLM_PROVIDER, KG_LLM_MODEL, KG_LLM_BASE_URL and KG_LLM_API_KEY there with empty defaults. If not, I'll do it in a follow-up.

Thanks for being upfront about the llama.cpp harness, that's fine with me.

[bferanmi806-sketch]

@keysersoft addressed in 3125f79, same branch:

• Explicit skip: custom path now resolves { json: null, skipped: true } instead of { json: {} }. enrich() returns early before applyEnrichResult (no kg_llm_hash write, next run retries normally); generateForConnectors()/generateForServer() return early before deleteMany (pending suggestions untouched). consolidate() untouched - json?.skills on null takes its existing empty-result early return.
• Tests: enrich skip leaves the hash unstored + retry stores normally; skills skip leaves pending suggestions untouched (with a usable-reply control proving replacement still works); consolidate untouched on skip. Focused 17/17, broader KG + cloud cron suites 132/132, eslint clean, no new tsc errors.
• Logging: warn line now uses new URL(base).origin with an unparseable-endpoint fallback, so no credentials/path/query can leak.
• Docs/Docker: docs block comments out the localhost URL and covers host.docker.internal, Linux extra_hosts, and the compose service-name option. docker-compose.yml passes all five KG_LLM_* vars with empty defaults (docker compose config validates). Left docker-compose.quickstart.yml alone - it carries no KG vars at all and is a deliberately minimal eval stack.

No replacement PR opened, nothing merged.

[bferanmi806-sketch]

recheck

[keysersoft]

Thanks @bferanmi806-sketch, that was quick and it's exactly what I asked for. I ran the KG and cloud cron suites on your branch (132 passing, tsc clean) and the skip tests read well.

I pushed one small commit on top so we don't need another round: the Docker note had split the env block in docs/knowledge-graph.md, so everything after it rendered as headings. It's back in one code fence with the note below it. I also added OPENAI_API_KEY, OPENROUTER_API_KEY and ANTHROPIC_API_KEY to docker-compose.yml, otherwise the hosted path still couldn't get its key into the container.

CI is approved and running. I'll merge once it's green.

[bferanmi806-sketch]

Thanks @keysersoft, really appreciate the review and the follow-up fixes. Glad the skip behavior landed the way you wanted.

I enjoyed working through this one. If there are other issues around the KG, local model support, or anything else you think would be a good fit, feel free to tag or assign me happy to keep contributing.

[keysersoft]

Merged, thanks again @bferanmi806-sketch. It'll be in the next release.