Skip to content

LK_UPSTREAM default is unreachable by construction in the :proxy image #14

Description

@harshaneel

Follow-up to #12, which added the :proxy image.

Problem

LK_UPSTREAM defaults to http://127.0.0.1:8080/v1 (cmd/localaik/main.go:23). That default is correct for the model-bundled images, where entrypoint.sh starts llama-server on 127.0.0.1:8080 in the same container, and it is correct for running the binary on a dev machine against a llama.cpp you started yourself.

It cannot ever be correct in :proxy. That image contains no inference engine, so its own loopback has nothing on port 8080 by construction.

An operator who forgets to set LK_UPSTREAM gets a container that starts cleanly, then answers 503 from /health and fails every request, while pointing at an address that could never have worked. The resolved upstream is printed at startup, so it is diagnosable, but the failure looks like "my model server is unreachable" rather than "I did not configure one."

Suggested fix

Options, roughly in order of preference:

  1. Have the :proxy image require LK_UPSTREAM and exit non-zero with a clear message when it is unset, rather than inheriting a default that is meaningless there. Needs a way for the binary to know which variant it is in, for example an ENV set only in the proxy stage.
  2. Keep the default but detect the specific case at startup: no credential configured, upstream is loopback, and no inference engine present. Warn loudly.
  3. Leave the code alone and document it harder in the README's :proxy section.

Option 1 is a real behaviour change for a published image, so it should land before :proxy has users rather than after.

Note that the flag and environment precedence must stay as it is: flag over environment over default. entrypoint.sh passes --upstream explicitly, so any change here must not disturb the bundled images.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions