perf: batch provider model refreshes into one providers.json write - #4269
Merged
Conversation
…4155) A post-install Ollama fan-out refreshed each matching provider through refreshProviderModels, and every one of those ends in saveProviders — a cache invalidate plus a full rewrite of providers.json. N providers meant N whole-file writes, each immediately superseded by the next, and N cache invalidate/repopulate cycles for any concurrent reader. Adds refreshProviderModelsBatch(ids) to the toolkit's provider service: it groups by ollamaRefreshGroupKey (one probe per daemon + probe shape), probes each group's lead without persisting, then applies every result and calls saveProviders exactly once. Per-group statuses (updated / failed / missing) replace the host-side sentinel juggling, so localLlm's refreshOllamaBackedProviders is now just the ollama-backed filter plus one log line per skipped group.
…rray, not against null (#4155)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A post-install/delete Ollama fan-out refreshed each matching provider through
refreshProviderModels, and every one of those ends insaveProviders— a cache invalidate plus a full rewrite ofproviders.json. N providers meant N whole-file writes, each immediately superseded by the next, plus N cache invalidate/repopulate cycles for any concurrentgetAllProviders()/getProviderById()reader in that window. Commit cba168e already deduped the probes by daemon base URL and 8d10c48 serialized them; the batch write entry point this issue asks for was the piece still missing.Adds
refreshProviderModelsBatch(ids)to the toolkit's provider service (server/lib/aiToolkit/providers.js). Three phases:ollamaRefreshGroupKey, so providers sharing a daemon and a probe shape are probed once. Anullkey stays the "not a shared Ollama probe" sentinel — those providers each become a group of one, never a bucket.saveProvidersruns exactly once — or not at all when nothing was probed successfully.It never throws for a per-provider failure: each group carries its own status —
updated(probed;[]is a real empty catalog and IS persisted),failed(probe threw,errorattached, stored lists untouched),missing(no such provider, or the lead was deleted mid-probe). That lets the host log one line per group instead of one per member.refreshOllamaBackedProvidersinserver/services/localLlm.jscollapses to the ollama-backed filter plus that logging — the grouping, sentinel juggling and per-provider write loop all move into the toolkit next to the group-key helper they belong to.server/services/providers.jsre-exports the new entry point, andserver/lib/aiToolkit/CLAUDE.mdrecords the contract ("refreshing more than one provider goes through the batch form, never a loop overrefreshProviderModels").Test plan
server/lib/aiToolkit/providers.batch.test.js(9 tests) runs the real service against a tempproviders.jsonwith a counting delegating spy onatomicWrite— the write count is the whole point, so the spy still performs the real write rather than stubbing it:atomicWritecalled exactly once, one/api/tagsprobe per daemon, every member ends up with its group's models[]) is persisted, still in one writemissingwithout probing or blocking the restapi-type Ollama provider keeps its own (unfiltered) probe rather than joining the tool-filtered bucket for the same daemonserver/services/localLlm.test.jsupdated to assert the fan-out issues ONE batch call carrying exactly the ollama-backed ids, skips the call entirely when nothing matches or the pull failed, logs one line perfailed/missinggroup, stays silent forupdatedgroups, and survives a rejected batch.cd server && NODE_ENV=test npm test— 1396 files / 29263 tests passing, 26 DB-backed suites skipped as designed (no DB-backed suites run).Closes #4155