Employee Status (Select one)
No
Summary
The asn1_codec application currently runs a single Kafka consumer/producer loop in a single process. This does not make efficient use of CPU resources. It would be possible to make more efficient use of CPU and memory resources for better scalability to be able to run multiple Kafka consumers within each container. A robust and simple way to implement that is to use supervisord to run multiple instances of the application within each container. This approach is in use and has been tested in production by an IOO deployment, as detailed in the accompanying PR.
Reasoning for new functionality
The asn1_codec application currently runs a single Kafka consumer/producer loop in a single process which is single threaded. This does not make efficient use of CPU resources. It would be advantageous to be able to run multiple Kafka consumers within one container or application instance to make more efficient use of resources, and to avoid the overhead of having to run multiple instances of the codec in multiple containers.
One approach would be to refactor the code to make it thread safe, and run multiple threads within the application. However, the asn1_codec Kafka code currently is not thread safe. It is much simpler to add the ability to run multiple processes with the existing code to avoid the complexity of refactoring the existing code to add multithreading and ensuring thread safety.
Employee Status (Select one)
No
Summary
The asn1_codec application currently runs a single Kafka consumer/producer loop in a single process. This does not make efficient use of CPU resources. It would be possible to make more efficient use of CPU and memory resources for better scalability to be able to run multiple Kafka consumers within each container. A robust and simple way to implement that is to use supervisord to run multiple instances of the application within each container. This approach is in use and has been tested in production by an IOO deployment, as detailed in the accompanying PR.
Reasoning for new functionality
The asn1_codec application currently runs a single Kafka consumer/producer loop in a single process which is single threaded. This does not make efficient use of CPU resources. It would be advantageous to be able to run multiple Kafka consumers within one container or application instance to make more efficient use of resources, and to avoid the overhead of having to run multiple instances of the codec in multiple containers.
One approach would be to refactor the code to make it thread safe, and run multiple threads within the application. However, the asn1_codec Kafka code currently is not thread safe. It is much simpler to add the ability to run multiple processes with the existing code to avoid the complexity of refactoring the existing code to add multithreading and ensuring thread safety.