instead of putting it to module.config.php you can move all closures into Module.php methods getServiceConfig or getControllerConfig or seperate Factory class which implements FactoryInterface.
in that case application with your module will be able to cache configuration and factories with closures will be loaded in your Module.php