Skip to content

Dedup Ollama provider model-refresh fan-out by daemon base URL - #4263

Merged
atomantic merged 2 commits into
mainfrom
claim/issue-4154
Aug 15, 2026
Merged

Dedup Ollama provider model-refresh fan-out by daemon base URL#4263
atomantic merged 2 commits into
mainfrom
claim/issue-4154

Conversation

@atomantic

Copy link
Copy Markdown
Owner

Summary

refreshOllamaBackedProviders (post-install / post-delete on the Local LLMs tab) fanned out with no dedup: one refreshProviderModels(id) per Ollama-backed provider. Several providers commonly resolve to the same daemon — the built-in ollama provider plus the four shipped CLI/TUI providers (claude-ollama, claude-ollama-tui, opencode-ollama, opencode-ollama-tui), all defaulting to the same local endpoint — so each independently re-fetched /api/tags and re-ran the entire per-model /api/show tool-capability probe for an answer that cannot differ.

Providers are now bucketed by a new ollamaRefreshGroupKey(provider) before the fan-out:

  • Compute split from persist. fetchProviderModels(id) is the probe half of refreshProviderModels(id) with the write removed; refreshProviderModels is now a thin composition of it plus updateProvider(id, { models }), so the two halves cannot drift. No new persist helper was needed — the existing updateProvider spread already has exactly the right semantics.
  • A bucket with 2+ members is probed once through its first member, then applied to every member (that one included) via updateProvider. Writes are sequential because saveProviders is a whole-file read-modify-write.
  • The two probe shapes stay in separate key namespaces. An api-type provider short-circuits to ${endpoint}/api/tags and persists the unfiltered tag list; a cli/tui provider routes to _fetchOllamaToolCapableModels and persists the tool-use-capable subset. Collapsing them would persist a tool-filtered list onto the plain ollama provider — or an unfiltered one onto a Claude harness that then silently fails to edit files.
  • An api provider carrying an apiKey is deliberately ungroupable: when the /api/tags short-circuit misses, _refreshAPIProviderModels falls through to a generic /models fetch that sends the key, so two providers on one endpoint with different keys can legitimately see different catalogs.
  • A null key means "refresh me individually", never "shared bucket" — and a bucket with a single member also keeps the plain one-call refresh, so the common single-provider install behaves byte-for-byte as it did before.
  • A failed shared probe skips its whole group with one log line rather than N copies of the same unreachable-daemon error. [] (a legitimately empty catalog after the user deleted their last model) is still persisted; only null is the skip signal.

The group key lives in internal/modelFetchers.js and reuses resolveModelFetcher — the same resolver the real dispatch uses — so the grouping rule can't drift from the probe it is predicting. ollamaBaseFromProvider moved to internal/ollamaBacked.js (beside isOllamaBackedProvider) to avoid a module cycle, and stays internal: the group key is the public contract, not the normalizer.

The toolkit stays self-contained — no imports out to other PortOS modules.

Scope note: this is the dedup half only. The sibling issue about batching provider-refresh writes (one providers.json save per provider) shares the same compute/persist split as a prerequisite but is intentionally left alone here so the two changes don't collide.

Test plan

  • server/lib/aiToolkit/internal/modelFetchers.test.js — new ollamaRefreshGroupKey block: all four shipped CLI/TUI Ollama providers collapse onto one key (driven off data.reference/providers.json, so a future seed addition on the same daemon is caught); the api-type ollama provider stays out of the tool-filtered bucket; different daemons stay apart; trailing-slash and /v1 spellings normalize together; non-Ollama and keyed-api shapes return null rather than a shared bucket.
  • server/lib/aiToolkit/providers.test.js — new fetchProviderModels block: returns the probed list without persisting; agrees with what refreshProviderModels persists; applying a fetched list to a sibling provider matches refreshing that sibling directly (and preserves its other fields); returns null only for a missing provider; throws 502 on a failed probe and leaves the stored list untouched.
  • server/services/localLlm.test.js — new fan-out cases: four providers sharing a daemon produce exactly one fetchProviderModels call and four updateProvider writes; providers on different daemons / different probe shapes stay on the individual path; a failed shared probe writes nothing and logs once; an empty catalog is still persisted across the group. The suite's provider mocks now use the real isOllamaBackedProvider / ollamaRefreshGroupKey instead of hand-mirrored copies, so the grouping assertions aren't vacuous.
  • Fixed a latent leak in that suite while there: vi.clearAllMocks() clears recorded calls but not queued …Once values, and the fire-and-forget fan-out meant a test where it correctly never ran left its getAllProviders answer queued for the next test to consume.
  • Full server suite: cd server && NODE_ENV=test npm test → 1395 files passed, 29243 tests passed, 26 files / 252 tests skipped (DB-backed suites, correctly gated off the non-test database).

Closes #4154

…4154)

Several providers commonly resolve to one Ollama daemon (the built-in `ollama`
provider plus the four shipped Claude/OpenCode-over-Ollama CLI/TUI providers, all
defaulting to the same local endpoint). The post-install/delete fan-out in
`refreshOllamaBackedProviders` called `refreshProviderModels` once per provider,
so each one independently re-fetched `/api/tags` and re-ran the whole per-model
`/api/show` tool-capability probe against an identical daemon and model set.

Providers are now bucketed by `ollamaRefreshGroupKey` — the daemon base URL plus
the probe shape — and a bucket with more than one member is probed once via the
new compute-only `fetchProviderModels`, then applied to every member with
`updateProvider`. The two probe shapes stay in separate key namespaces: an
`api`-type provider persists the unfiltered tag list while a CLI/TUI one
persists the tool-use-capable subset, so collapsing them would cross-contaminate
the lists. A provider with no group key, or the only member of its bucket, keeps
the plain one-call refresh.
…the vanished-lead skip (#4154)

Three findings from the review pass:

- Every refresh ends in `saveProviders`, a whole-file read-modify-write of
  providers.json, so the fan-out no longer runs under `Promise.all` — concurrent
  members interleaved and clobbered each other's model arrays. The chain is
  fire-and-forget background work, so nothing waits on the wall clock.
- A group whose lead provider was deleted between the listing and the probe got
  `null` back from `fetchProviderModels` and was dropped in silence. Probe
  failure now returns a distinct sentinel so the vanished-lead case logs.
- Corrected the `ollamaBaseFromProvider` docstring, which still claimed a
  re-export that was deliberately dropped.

A fourth finding — normalize `/v1` away for api-type providers too — is declined
and pinned with a test: that arm probes `${endpoint}/api/tags` and
`${endpoint}/models` verbatim, so `…:11434` and `…:11434/v1` are different
requests and only the `/v1` spelling answers `/models`.
@atomantic
atomantic merged commit 8d10c48 into main Aug 15, 2026
7 checks passed
@atomantic
atomantic deleted the claim/issue-4154 branch August 15, 2026 04:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dedup Ollama provider model-refresh fan-out by daemon base URL

1 participant