[pull] master from mudler:master - #1476
Merged
Merged
Conversation
LiquidAI now publishes official DSpark draft sidecars for its 2.6B target. Pair Q4 and Q8 targets with matching draft choices so LocalAI can use speculative decoding across different memory budgets. Assisted-by: Codex:gpt-5 [systematic-debugging] Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add the 2B and 4B distilled models alongside the 9B model. Their compact GGUF builds make the Qwen3.8 reasoning distill practical on smaller hosts. Assisted-by: Codex:gpt-5 Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add Q4_K_M and Q8_0 builds for Tencent's Qwen3.6-based computer-use model. Assisted-by: Codex:gpt-5.6 [Codex] Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add the publisher\047s Q4_K_M build for the recent Muse Glimmer agent fine-tune. Assisted-by: Codex:gpt-5 Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
"stale model config revision" reported only that two hashes differed. It named neither, so an operator could not tell an edited configuration from a revision that is not reproducible for one unchanged file, and the failing value appears in no table. The error now carries the revision the request brought and the one the controller holds. It still wraps ErrStaleModelConfigRevision, so callers that classify the error keep working. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
A request that carries a file stages it to the worker, which writes it under its staging directory. Nothing removed it afterwards. The frontend expires ephemeral keys from object storage, but that sweep never covered a worker's local disk, so every image, audio clip and video a worker ever served stayed on it. One worker had accumulated 175 request directories over three months. The volume reached 100 percent, and from that point every backend start failed because the process manager could not create a state directory. The worker now sweeps its ephemeral staging directory on a timer and once at startup, so files left by a crash are reclaimed too. Staged model files live beside that directory and are not touched. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
process.New applies its options and discards the error they return. When WithTemporaryStateDir could not create a directory, StateDir stayed empty and every later option went unapplied, so the failure surfaced from Run as "mkdir : no such file or directory" naming no path. That message cost a full day of diagnosis on a worker whose volume was full: the real errno was ENOSPC and nothing reported it. The loader now creates the directory itself and returns the underlying error with the path attached. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
Picks up "surface option errors instead of discarding them". New records
the error from applying its options and Run returns it, so a process
whose state directory cannot be created reports the real cause instead
of failing later inside os.MkdirAll("") with an empty path.
LocalAI already resolves that directory itself, so this covers the other
process.New call sites rather than changing behaviour on the backend
start path.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5
A node's status comes from its HTTP heartbeat. Backend installs travel over NATS. The two are independent, so a worker that dies stops answering on the bus at once but stays healthy in the database until its heartbeat ages out. Inside that window the scheduler picked a node it could not reach, and the request failed with "no responders available" rather than moving to a node that was up. The scheduler now probes the node it selected and, when nothing answers, marks it unhealthy and selects again. The demotion is what makes the retry terminate: the next selection reads only healthy nodes. It also tells the other frontends what this one learned, so the cluster does not rediscover a dead worker one failed request at a time. Only nats.ErrNoResponders counts as absent. A worker that answers slowly stays eligible, because dropping it would cost capacity that is really there. The probe reuses the models.running subject: a new subject would go unanswered by workers that have not been upgraded, and every one of them would then look dead. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
A replica row in staging or loading holds its slot, because slot allocation counts every state except unloading. Nothing ever reclaimed such a row: every reconciler pass and the router's eviction query filter state = "loaded", and the per-model probe skips rows without an address, which is exactly what a row that never finished loading has. So a worker that dropped out mid-transfer left a row that pinned the only replica slot for that model on that node. Scheduling then found no free slot and eviction found nothing it was allowed to evict, and the request failed with "no replica slot on <node> and eviction failed: all models busy". The state persisted until an operator intervened. The reconciler now reclaims a row stuck before serving when no load job is driving it. Ownership is decided by the job's LastProgress heartbeat, not by elapsed time: staging a large checkpoint legitimately runs for a long while without touching the replica row, so a deadline would either be a model-size cliff or reclaim a healthy transfer. That heartbeat is the same signal job takeover already trusts. Any error reading the job leaves the slot held, because holding one for another pass costs a scheduling opportunity while a wrong reclaim restarts a multi-gigabyte transfer. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
⬆️ Checksum updates in gallery/index.yaml Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
…89da` (#11685) ⬆️ Update antirez/ds4 Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
The controller pins a model's replicas to a stored revision and rejects any request carrying a different one. Nothing ever re-derived that value from the configuration on disk: it moved only on an edit, a gallery install, or a peer's change broadcast. An inference request may only establish a revision, never replace one. So any other way for the two to diverge left the model permanently unroutable. A configuration edited while a frontend was down lands there, and so does a change in what the revision is computed over: an upgrade that alters the hashed form leaves every stored revision describing a configuration that no longer exists. The only recovery was deleting the row by hand, which is not something a cluster should need. Each frontend now reconciles the stored revisions against the loaded configurations at startup and republishes the ones that disagree. Only those: republishing quarantines every replica loaded under the old revision, so doing it for a model that did not drift would unload a healthy replica for nothing. A model with no stored revision has never been served and is left for its first request to establish. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )