Skip to content

feat(llamacpp): allow verified estimator overruns - #79

Merged
sjug merged 1 commit into
quadsproject:mainfrom
sjug:feat-llamacpp-estimator-overrun
Aug 6, 2026
Merged

feat(llamacpp): allow verified estimator overruns#79
sjug merged 1 commit into
quadsproject:mainfrom
sjug:feat-llamacpp-estimator-overrun

Conversation

@sjug

@sjug sjug commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Allow a custom llama.cpp relaunch policy to attempt its exact configuration when a valid estimator result predicts less free VRAM than the requested target.
  • Persist the permission separately from whether the bypass was actually used, preserving auditability across retries and transactional rollback.
  • Add the custom-only runtime-editor checkbox, expose verified estimator status, and align the KV cache selector with the neighboring unit-backed controls.

Safety

  • Automatic sizing remains estimator-strict, and malformed output or estimator execution failures cannot be bypassed.
  • Static context limits, full GPU offload, and measured post-load free VRAM remain mandatory.
  • Failed launch or verification attempts use the existing transactional rollback to restore the previous persisted policy.

Testing

  • Python 3.12: 1,495 passed
  • Python 3.13: 1,495 passed
  • Coverage: 92.59%
  • mypy: 143 files, no errors
  • ruff check, ruff format, shellcheck, and git diff --check: clean

Let custom relaunch policies attempt an exact configuration when a valid
llama.cpp estimate misses the requested free-VRAM target. Keep
automatic sizing and estimator failures strict, then require full
offload and measured post-load headroom before accepting the runtime.

Persist the permission and whether it was used so retries, rollback,
logs, and the node-detail view remain auditable. Add the custom-only
control and align the KV cache selector with neighboring unit-backed
inputs.
@sjug
sjug requested review from grafuls and sadsfae August 6, 2026 13:09
@sjug
sjug merged commit e41b242 into quadsproject:main Aug 6, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant