incremental model not consuming?

[maartenjv]

Hi there,

thanks for the excellent tutorial, very useful!
I've got most of the components running, however the adding data and retraining part does not seem to work.
I understand that you need to actually run a kafka broker server? I've used some other tutorials (from here: https://github.com/wurstmeister/kafka-docker) to try and setup a Kafka server and after some strugling everything seems to run fine, expect that the script never gets into the "for message in consumer:'' loop in incremental_model.py. I'm not at all familiar with this kind of stuff, so it would be great if you can share how you did this.

thx

-M

[mstale007]

Hi @maartenjv !
I am glad you found this repo useful.
For this project, I installed Kafka and zookeeper both on my machine directly instead of using docker
And just after installation is done first try to check if it's working correctly by starting a Kafka server and zookeeper server.
Then start Kafka CLI producer and consumer on different terminals and try sending a message from the producer terminal (Explained well in this video)

If your messages are not getting sent or received correctly on the consumer side, something is wrong with your Kafka setup.

If this is working correctly you can pip install the python libraries of kafka that I have used and start running the scripts and then see if it's throwing any errors.
(PS: You can close Kafka CLI producer and consumer terminals once you tested it, you will only require zookeeper and Kafka server to be running when running the repo)

Also if you still want to work with kafka-docker method, you might want to look into Port forwarding since your front end and model will be on your local machine and your producer/consumer will be on docker VM

Hope that helps!