pi-local-servers

Start the local inference server behind the model you selected, and stop it when you leave.

---

Running models locally means running servers: llama-server on one port, vLLM on another, a custom binary that eats most of your RAM on a third. Switching models in pi then means remembering which process to start, and remembering to kill it afterwards. This extension makes that part disappear: pick a model, the server for its provider comes up before the first request goes out; quit pi, and what pi started is shut down again.

// ~/.pi/agent/local-servers.json — that is the whole setup
{
  "servers": {
    "llamacpp": {
      "probeUrl": "http://127.0.0.1:8080/v1/models",
      "start": { "command": "llama-server", "args": ["--models-dir", "~/models", "--port", "8080"] }
    }
  }
}

/model  →  llamacpp/Qwen3.6-27B      server starts, request waits for it, answer comes back
/exit                                server stops, memory is free again

Table of Contents

• Requirements
• Quick Start
• What Happens, and When
• Configuration
  • Where the config lives
  • Server fields
  • More examples
• Commands
• Logs
• Turning It Off
• Troubleshooting
• Design Notes
• Compatibility

Requirements

• pi 0.84 or newer.
• A local model server you already run by hand, such as llama.cpp, ollama, or vLLM.
• Nothing else: the extension has no runtime dependencies.

This extension spawns the commands you put in its config file, with your user's permissions. It never downloads or installs anything.

Quick Start

1. Install it.

pi install npm:pi-local-servers

Other sources work too — a git repo, or a folder on disk:

pi install git:github.com/matrixfede/pi-local-servers
tar xzf pi-local-servers-1.0.0.tgz && pi install ./package
pi -e ./package        # try it for one run, without installing

2. Find your provider id. A provider is how pi groups models — the keys under providers in ~/.pi/agent/models.json, and the first column of:

pi --list-models

provider      model                       context  max-out  thinking  images
llamacpp      Qwen3.6-27B-…-Q6_K          65.5K    32.8K    yes       no
ollama        qwen3.8:27b                 262.1K   32.8K    yes       yes

Here the provider ids are llamacpp and ollama. Those are the names the config uses.

3. Write the config. The quickest way is to let pi draft it for you — inside pi, run:

/servers init

It reads your models.json, writes one entry per provider that points at this machine, and leaves a REPLACE-ME where the start command goes. Remote providers are skipped, and an existing config is never overwritten.

To write ~/.pi/agent/local-servers.json by hand instead, the smallest useful version is two lines of real content — the address that tells you the server is alive, and the command that starts it:

{
  "servers": {
    "llamacpp": {
      "probeUrl": "http://127.0.0.1:8080/v1/models",
      "start": { "command": "llama-server", "args": ["--models-dir", "~/models", "--port", "8080"] }
    }
  }
}

4. Use pi as usual. There is no command to run. Select a model from that provider — /model, Ctrl+P, or --model at startup — and the server is up before the first request leaves. Quit, and it is stopped.

5. Check it worked. Inside pi:

/servers

● llamacpp     llamacpp — started by pi
● ollama       ollama (:11434)

A filled dot means the server answers. If a server refuses to start, the reason is in ~/.pi/agent/logs/<provider>.log.

What Happens, and When

When a model from a configured provider becomes active:

Situation	What the extension does
The server already answers	Nothing. Your long-running server is left alone.
The server is down, start is configured	Spawns it, then waits for probeUrl before letting the request through.
The server is down, no start configured	Warns you, so a failing request is not a mystery.

That check runs when the session starts, every time you switch model, and once more right before each request — so a server that dies mid-session comes back on the next message.

When the session ends:

1. providers with an unloadCommand run it once per model used, freeing memory without stopping a server you do not own (this is how ollama is handled);
2. servers this extension started get SIGTERM;
3. anything else is left exactly as it was found.

The rule worth remembering: a server that was already running before pi started is never stopped. Attaching to your always-on llama.cpp instance is safe.

Configuration

Out of the box, with no config file at all, the extension knows one thing: ollama runs as a service, so on exit it runs ollama stop for each model you used. Everything else you declare yourself.

Where the config lives

The first file found wins:

1. $PI_LOCAL_SERVERS_CONFIG, if that variable is set
2. .pi/local-servers.json in the project directory — for a per-project setup
3. ~/.pi/agent/local-servers.json — your global setup

The built-in ollama entry is always present unless your config defines ollama itself, in which case yours replaces it.

Server fields

Only probeUrl is required. Everything else has a working default.

Field	Type	Default	Meaning
probeUrl	string	required	URL that returns 2xx only when the server is ready. /v1/models is the usual choice.
label	string	provider id	Name shown in notifications and /servers.
start.command	string	—	Executable to spawn. ~/ is expanded. Omit for servers you do not manage (systemd, docker, a remote host).
start.args	string[]	[]	Arguments. ~/ is expanded here too, so "~/models" reaches the server as an absolute path.
start.cwd	string	inherited	Working directory. ~/ is expanded.
start.env	object	inherited	Extra environment variables, merged over the current environment.
readyTimeoutMs	number	60000	How long to wait for probeUrl after spawning. On timeout the spawned process is killed rather than left half-started.
stopOnExit	boolean	true	Set to false to leave the server running after pi exits — useful to keep a model warm between sessions.
exclusive	boolean	true when start is set	Marks servers that compete for the same GPU or RAM. Before starting one, others that pi started are stopped; servers pi did not start only produce a warning.
unloadCommand	object	—	Runs on exit instead of killing a server pi does not own. {model} inside args expands to the selected model id.

More examples

A server you do not start yourself — ollama runs under systemd, so there is nothing to spawn; on exit its memory is freed without touching the service:

{
  "servers": {
    "ollama": {
      "label": "ollama (:11434)",
      "probeUrl": "http://127.0.0.1:11434/v1/models",
      "unloadCommand": { "command": "ollama", "args": ["stop", "{model}"] }
    }
  }
}

A slow server that should survive the session — vLLM takes minutes to load, so give it a long timeout and keep it running afterwards:

{
  "servers": {
    "vllm": {
      "label": "vLLM (:8001)",
      "probeUrl": "http://127.0.0.1:8001/v1/models",
      "start": { "command": "vllm", "args": ["serve", "Qwen/Qwen3.8-27B", "--port", "8001"] },
      "readyTimeoutMs": 300000,
      "stopOnExit": false
    }
  }
}

Two servers that cannot both be loaded — on a machine where each model fills most of the memory, exclusive (already the default here) makes pi shut one down before starting the other:

{
  "servers": {
    "llamacpp": {
      "probeUrl": "http://127.0.0.1:8080/v1/models",
      "start": { "command": "~/bin/llama-router" }
    },
    "big-model": {
      "probeUrl": "http://127.0.0.1:8000/v1/models",
      "start": { "command": "~/bin/big-server", "args": ["--ctx", "100000"], "cwd": "~/big" },
      "readyTimeoutMs": 120000
    }
  }
}

A ready-to-edit config ships in examples/local-servers.json.

Commands

/servers

Lists every configured server, whether it answers right now, and which ones this session started. It is also the quickest way to spot a typo: a provider that does not appear is a provider the extension has no config for.

/servers init

Writes a starting config from the providers in your models.json: every provider whose baseUrl points at this machine becomes an entry with its probeUrl already filled in, and a REPLACE-ME where its start command goes. A provider on port 11434 is assumed to be ollama and gets an unloadCommand instead. Remote providers are listed as skipped, and an existing config file is left untouched — delete it first if you want a fresh draft.

Logs

File	Contents
~/.pi/agent/logs/<provider>.log	stdout and stderr of the spawned server — the first place to look when a server refuses to come up
~/.pi/agent/logs/local-servers.log	one line per action taken by the extension

Turning It Off

pi -ne                                 # disable all extensions for one run
pi remove npm:pi-local-servers         # uninstall (use the source you installed with)

Removing it never leaves a server behind: shutdown handling runs while it is still loaded.

Troubleshooting

Nothing happens when I select a model. The key in local-servers.json must match the provider id exactly as pi --list-models prints it. Run /servers: if the provider is missing from that list, the config is not reaching it.

The server never becomes ready. Read ~/.pi/agent/logs/<provider>.log. After readyTimeoutMs the extension kills what it spawned rather than leaving a half-started process behind, so that log is the only trace left. A server that legitimately needs longer just needs a bigger readyTimeoutMs.

The first request fails even though the server is up. The server is up but the model is not loaded, and your server rejects requests for unloaded models instead of loading them on demand. Enable on-demand loading server-side, or pre-load the model in your start command — see Design Notes.

A server I started by hand got stopped. It should not have been: only servers this extension spawned are killed. If the port was free when pi looked, pi started its own instance and owns it. Use stopOnExit: false if you would rather keep whatever is running at the end of a session.

Memory pressure when switching between heavy providers. Leave the competing providers exclusive (the default when start is set). Servers pi started are shut down before a new one comes up; servers pi does not own are only reported, never touched.

Design Notes

Model loading belongs to the server, not here. The extension waits for the server to answer, not for a specific model to be resident. Model loading should be handled server-side: llama.cpp's router in --models-dir mode, for example, loads a model on the first request and holds that request until it is ready. Configure your server so a cold request blocks rather than fails, and correct behaviour comes for free.

This is not a stylistic preference. A server busy loading a multi-gigabyte model typically stops answering its own control endpoints, so any "poll until the model is loaded" logic here would time out against a perfectly healthy server.

Startup is serialized. Two quick model switches cannot start two servers at once; the second waits for the first.

Handlers block. pi awaits extension event handlers, so waiting inside session_start, model_select, and before_provider_request is what guarantees the first request never hits a cold port.

Compatibility

• pi 0.84 or newer (uses session_start, model_select, before_provider_request, session_shutdown, and pi.exec).
• Linux and macOS. Servers are stopped through their process group, so a wrapper script goes down together with the server it exec'd. Windows is untested.
• No runtime dependencies.

Contributing

Issues and pull requests are welcome. If you have a server that does not fit the config model — one that needs a health check other than an HTTP probe, or a different shutdown ritual — open an issue describing how you run it; that is the useful input.

License

MIT © matrixfede