Feature request: Empower Gortex with a pluggable LLM layer#25
Merged
Conversation
~/.config/gortex/config.yaml gains an `llm:` block. MergeLLMInto fills zero fields of the repo-local llm config from the global one (local non-zero values win, including an explicit per-repo override of an inherited model path), with leading `~/` in model paths expanded against $HOME. The daemon and `gortex mcp` startup paths both load and merge the global config before SetupLLM; env vars still override last.
search_symbols gains an `assist` arg (auto/on/off/deep). `auto` uses a cheap NL heuristic to skip identifier lookups; active modes run grammar- constrained query expansion + name/signature rerank, and `deep` adds a body-grounded verification pass that reads candidate bodies and callers and honestly drops irrelevant matches — an empty result is preserved as the load-bearing negative signal. The svc layer adds a pre-warmed assist context with its own mutex and KV cache so short assist calls don't head-of-line block a long `ask`, plus LRU caches for expand/rerank/verify. The stub build returns errServiceUnavailable for all three new methods.
Introduce an llm.Provider interface so the `ask` agent and the search_symbols assist passes run on any of four backends, selected by the `llm.provider` config key: - local — in-process llama.cpp, the only `-tags llama` package; ships a non-llama stub so the provider factory always compiles - anthropic / openai / ollama — pure-Go HTTP clients, available in every build, via forced tool-use / json_schema / `format` schema respectively Provider is a single-method surface (Complete) over a provider-neutral []Message conversation; the agent tool-loop and the three assist passes are all built on it. Tool calls travel as plain text (emulated protocol) so one Message shape works across every provider. llm.Config gains a `provider` key plus per-provider sub-blocks, replacing the flat llama-only shape; MergedWith / MergeLLMInto merge per sub-block. Assist prompts are tiered by capability (small / frontier) and keyed off the active provider's name rather than carried per-provider. Knock-on: agent and svc are now pure Go — the build-tag split is contained entirely in provider/local. The `ask` tool and assist modes work with any provider, not just llama builds; service_stub.go and tools_llm_stub.go are deleted and registerLLMTools is unconditional. SetupLLM logs provider-construction errors instead of silently disabling.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an opt-in LLM capability to Gortex and generalizes it into a multi-provider backend. Three commits, building on each other.
1. Global LLM config
~/.config/gortex/config.yamlgains anllm:block. A repo-local.gortex.yamlllm:block layers over it per non-zero field (local wins), with~/model paths expanded against$HOME. Lets users keep model/tuning config in one placeacross repos.
2. LLM-assisted search ranking
search_symbolsgains anassistarg —auto/on/off/deep:autouses a cheap NL heuristic to skip identifier lookupsdeepadds a body-grounded verification pass that reads candidate bodies + callers and honestly drops irrelevant matches (an empty result is the load-bearing negative signal)Backed by a pre-warmed assist context with LRU caches for expand/rerank/verify.
3. Pluggable multi-provider backend
Generalizes the LLM layer behind an
llm.Providerinterface, selected by thellm.providerconfig key:-tags llamapackage, ships a non-llama stubformatschema)Provideris a single-method surface (Complete) over a provider-neutral[]Messageconversation; theaskagent loop and the three assist passes are all built on it. Tool calls travel as plain text so one message shape works across everyprovider. Assist prompts are tiered by capability (small / frontier) keyed off the active provider.
Knock-on:
agentandsvcare now pure Go — the build-tag split is contained entirely inprovider/local. Theasktool and assist modes work with any provider, not just llama builds; the stub files are deleted andregisterLLMToolsis unconditional. CGO is now opt-in (only
localneeds it).Test plan
go build ./...andgo build -tags llama ./...go vet— clean both tag configsgo test -race ./...— passesgo test -tags llama -race ./...— passes (3664 tests, 113 packages)-tags llamae2e assist test exercises the local provider against a real GGUF modelllm.provider: anthropic(+ANTHROPIC_API_KEY) and confirmask/search_symbols assist:deepwork without a-tags llamabuild