Unfork llama.cpp

[ericcurtin]

We should be able to use the code, cmake files, etc. from llama.cpp as is:

#471

[doringeman]

We have to work around the patch we have which is for checking the readiness:

model-runner/pkg/inference/scheduling/runner.go

Line 213 in ac90dd3

readyRequest, err := http.NewRequestWithContext(ctx, http.MethodGet, "http://localhost/v1/models", http.NoBody)

[ericcurtin]

@doringeman you seem like you are in the know here, I can do this to vanilla llama-server (which we should be using) no problem, what is difference about the fork that we cannot do similar?

$ curl -k http://127.0.0.1:8080/v1/models
{"models":[{"name":"unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M","model":"unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M","modified_at":"","size":"","digest":"","type":"model","description":"","tags":[""],"capabilities":["completion"],"parameters":"","details":{"parent_model":"","format":"gguf","family":"","families":[""],"parameter_size":"","quantization_level":""}}],"object":"list","data":[{"id":"unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M","object":"model","created":1764770422,"owned_by":"llamacpp","meta":{"vocab_type":2,"n_vocab":151936,"n_ctx_train":262144,"n_embd":2048,"n_params":30532122624,"size":18550716416}}]}

[copilot-swe-agent]

The agent encountered an error and was unable to start working on this issue: The token used to assign the agent doesn't have the necessary permissions. Please make sure your token has read and write access to actions, contents, issues, and pull requests, and has permission to assign issues to the target repository. Learn more: https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/create-a-pr