Follow-up to #12, which added the :proxy image.
Problem
LK_UPSTREAM defaults to http://127.0.0.1:8080/v1 (cmd/localaik/main.go:23). That default is correct for the model-bundled images, where entrypoint.sh starts llama-server on 127.0.0.1:8080 in the same container, and it is correct for running the binary on a dev machine against a llama.cpp you started yourself.
It cannot ever be correct in :proxy. That image contains no inference engine, so its own loopback has nothing on port 8080 by construction.
An operator who forgets to set LK_UPSTREAM gets a container that starts cleanly, then answers 503 from /health and fails every request, while pointing at an address that could never have worked. The resolved upstream is printed at startup, so it is diagnosable, but the failure looks like "my model server is unreachable" rather than "I did not configure one."
Suggested fix
Options, roughly in order of preference:
- Have the
:proxy image require LK_UPSTREAM and exit non-zero with a clear message when it is unset, rather than inheriting a default that is meaningless there. Needs a way for the binary to know which variant it is in, for example an ENV set only in the proxy stage.
- Keep the default but detect the specific case at startup: no credential configured, upstream is loopback, and no inference engine present. Warn loudly.
- Leave the code alone and document it harder in the README's
:proxy section.
Option 1 is a real behaviour change for a published image, so it should land before :proxy has users rather than after.
Note that the flag and environment precedence must stay as it is: flag over environment over default. entrypoint.sh passes --upstream explicitly, so any change here must not disturb the bundled images.
Follow-up to #12, which added the
:proxyimage.Problem
LK_UPSTREAMdefaults tohttp://127.0.0.1:8080/v1(cmd/localaik/main.go:23). That default is correct for the model-bundled images, whereentrypoint.shstartsllama-serveron127.0.0.1:8080in the same container, and it is correct for running the binary on a dev machine against a llama.cpp you started yourself.It cannot ever be correct in
:proxy. That image contains no inference engine, so its own loopback has nothing on port 8080 by construction.An operator who forgets to set
LK_UPSTREAMgets a container that starts cleanly, then answers 503 from/healthand fails every request, while pointing at an address that could never have worked. The resolved upstream is printed at startup, so it is diagnosable, but the failure looks like "my model server is unreachable" rather than "I did not configure one."Suggested fix
Options, roughly in order of preference:
:proxyimage requireLK_UPSTREAMand exit non-zero with a clear message when it is unset, rather than inheriting a default that is meaningless there. Needs a way for the binary to know which variant it is in, for example anENVset only in theproxystage.:proxysection.Option 1 is a real behaviour change for a published image, so it should land before
:proxyhas users rather than after.Note that the flag and environment precedence must stay as it is: flag over environment over default.
entrypoint.shpasses--upstreamexplicitly, so any change here must not disturb the bundled images.