entrypoint.sh: container wedges instead of failing when model load exceeds 120s

[harshaneel]

Affects the published gemma3-4b and gemma3-12b images. Present on main, not introduced by #12.

Observed

Under host load, a container started from the default image wedges permanently. llama-server is healthy on 127.0.0.1:8080 inside the container, but nothing ever listens on 8090, so every client request fails to connect and docker logs stops after localaik: loading model....

Reproduced once during verification for #12 while the machine was busy: the first docker run sat for roughly 8.5 minutes with no health response. A second run on an idle machine went healthy in 30 seconds.

Mechanism

Two problems compound, both in entrypoint.sh.

The model-load timeout is hardcoded at 120 seconds.

until curl -sf http://127.0.0.1:8080/health >/dev/null 2>&1; do
  tries=$((tries + 1))
  if [ "${tries}" -ge 120 ]; then
    echo "localaik: model failed to load after 120s" >&2
    cleanup
    exit 1
  fi
  sleep 1
done

A 12B model on a loaded machine, or any machine with slow disk, can exceed that. There is no way to raise it: every other tunable is an LK_* variable, this one is not.

cleanup then blocks forever instead of exiting.

cleanup() {
  jobs -p | xargs -r kill 2>/dev/null || true
  wait || true
}

jobs -p in a non-interactive POSIX shell does not reliably list background jobs, so nothing is killed. wait then waits on a llama-server that is still running and never returns. The script never reaches exit 1, so the container neither starts nor dies: it sits with a healthy inference server and no proxy in front of it.

The same cleanup is on trap ... INT TERM, so docker stop on a container in this state also hangs until the timeout and gets SIGKILLed.

The net effect is worse than a plain timeout would be. A container that exited non-zero would be visible in docker ps -a and restartable. This one looks alive.

Suggested fix

• Make the wait configurable, for example LK_MODEL_LOAD_TIMEOUT, defaulting to something well above 120s. A 7.89 GB image implies slow first loads.
• Record llama-server's PID at launch and kill "$PID" directly rather than going through jobs -p.
• Bound the wait, or drop it: the script is about to exit anyway, and PID 1 exiting tears the container down regardless.

Worth deciding at the same time whether the proxy should start before the model is ready and report 503 from /health while loading, which is what the :no-model variant will need anyway. That would remove this startup gate entirely rather than tuning it.