[FR] Run save_images in parallel to speed-up preprocessing

[vfdev-5]

@samedii thanks for making this template example running with python 3.6 ! That's great as I can run it now inside Google Colab.

I'd like to ask another FR to improve UX with the template code, namely to run image preprocessing part in multi-processing way to speed up the data preparation:
https://github.com/Aiwizo/template-nvae/blob/d8a7470dcb1dec59dad6ca5299e25e82ace2f9ac/prepare.py#L17-L19

For example, naively it can be done using joblib which could create a pool of workers to process the dataset by chunks.
What do you think ?

[samedii]

It would be very easy to do. I don't try to optimize things that happen once but it might be more reasonable if it's being run more often in a showcase

[vfdev-5]

I don't try to optimize things that happen once

Yes, I agree, this also make sense. Feel free to close the issue as answered.

[samedii]

I added a draft PR for this since I was unable to test it (celeba has reached the download limit). It's quite ugly since I believe you can't have "inner" functions when using joblib.

I'm not sure how long it takes to prepare the data so I don't have an opinion yet. I'll come back to this later