Hello,
v2.0.127, Ollama backend. The first request in a session uses the configured contextWindow (32768). Subsequent requests in the same session send 200000 and unload the model between calls (keep_alive: 0), causing a full reload and a CPU overflow. This can be reproduced with ollama ps between each message.
Thanks.
Hello,
v2.0.127, Ollama backend. The first request in a session uses the configured contextWindow (32768). Subsequent requests in the same session send 200000 and unload the model between calls (keep_alive: 0), causing a full reload and a CPU overflow. This can be reproduced with
ollama psbetween each message.Thanks.