Agentic AI for the Masses!
If you have the Nix package manager, you can simply clone this repo, cd into the project directory and run nix develop to get an development environment.
This project uses llama.cpp, specifically llama-server. Also fastAPI. See the requirements.txt for all the (python) dependencies.
You can install this project via the flake.nix file if you have Nix installed. The flake also has a docker build. nix build .#docker