Modelfile

# OpenClaw — the one shared local-LLM model alias for all of my apps.
#
# Apps never name a raw Ollama model (qwen3.5:4b, etc.) directly. They all ask
# for "openclaw", and this file decides what that actually is. Swap the FROM
# line, re-run ./install.sh, and every app moves together — with one copy of the
# weights resident in RAM instead of each app loading its own.
#
# Current pick: qwen3:4b-instruct-2507 (Q4_K_M). This is the *instruct*
# (non-reasoning) variant on purpose — it never emits <think> blocks, so
# callers do NOT need to send "think": false and there's no reasoning latency.
# (The base qwen3:4b is a hybrid whose template hard-codes a <think> opener
# and can't be turned off via Ollama's chatml template — avoid it here.)
#
# WHY num_gpu=36 / full offload (2026-07-04): all 36 layers fit on the GPU
# (~3.1 GiB VRAM incl. KV) ONLY because the desktop GUI is disabled
# (multi-user.target). With GNOME/Xorg running, their GPU context stole enough
# memory that the ~2.3 GiB weight buffer couldn't allocate and every 4B model
# OOM'd — it looked like a hard ~1.9 GiB single-alloc ceiling but was really
# the desktop competing for GPU memory. Headless, full offload works and runs
# ~15-18 tok/s (vs ~8 tok/s when 12 layers were forced onto the CPU at 24/36).
#
# If the GUI is ever re-enabled (graphical.target), full offload will OOM
# again — drop num_gpu to ~24 as a fallback, or keep it headless.
#
# NOTE: failed loads leak GPU memory that only a *reboot* clears (a service
# restart is not enough). If loads start OOMing after a bad attempt, reboot.
FROM qwen3:4b-instruct-2507-q4_K_M

# Qwen3-Instruct recommended sampling params.
PARAMETER temperature 0.7
PARAMETER top_p 0.8
PARAMETER top_k 20
PARAMETER min_p 0

# Full GPU offload (all 36 layers). Works because the GUI is disabled; see WHY.
PARAMETER num_gpu 36

# Match the server runtime (-c 4096). Keep callers' prompt theirs.
PARAMETER num_ctx 4096