MultiAgent Chat-Bot

Hardware

My project is hosted on 4 different hosts:

1. PC0 API: My x86 server.
2. PC1 Worker1: R9 7945HX 64GB 5200MT/s + RTX 4060 (mobile) 8GB GDDR6
3. PC2 Worker2: Nvidia Jetson AGX Developer Kit 16GB LPDDR4x with MAXN mode enabled
4. PC3 Judge: i7-14700KF 64GB 6400MT/s + RTX 4070 Ti 12GB GDDR6x

Back-end

Starting the cluster

In order to start the cluster use this command in the API host:

./start.sh pc0

in the first PC:

./start.sh pc1

and in the second PC:

./start.sh pc2

and finally the third (the most powerful):

./start.sh pc3

Make sure to edit the IP adresses in the .env file.

Testing

Start by loading the models:

curl -X POST "http://127.0.0.1:8000/system/init_mode" \
     -H "Content-Type: application/json" \
     -d '{
           "mode": "coding"
         }'

Then test the connection:

curl -X POST "http://127.0.0.1:8000/chat" \
     -H "Content-Type: application/json" \
     -d '{
           "message": "Write a Python function to check if a number is prime and optimize its time complexity.",
           "mode": "coding"
         }'