Skip to content

Ollama API support - #9

Open
bgreene2 wants to merge 2 commits into
mateusz:masterfrom
bgreene2:ollama_api_support
Open

bgreene2 wants to merge 2 commits into
mateusz:masterfrom
bgreene2:ollama_api_support

Conversation

@bgreene2

@bgreene2 bgreene2 commented Mar 3, 2024

Copy link
Copy Markdown

This adds the ability to point at a model hosted on an Ollama installation instead of running the model within the app.

  1. Added two commandline arguments (ollama-host, ollama-model)
  2. Added Ollama modelfile to create the psyfighter model
  3. Updated readme with instructions
  4. Added requirements.txt to make installing dependencies with pip easier

Tested on my system, which is a PC running Windows 11 and WSL. The Ollama installation is in Windows, and the app was run in WSL.

@mateusz

mateusz commented Mar 3, 2024

Copy link
Copy Markdown
Owner

Thanks - this works well! However I've been playing downstream with using grammars (GBNF), and Ollama is not there yet for guided generation.

I might try looking at Outlines instead of GBNF, they integrate with llama.cpp, but unsure about ollama.

[thinking in progress]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants