This supersedes #4 and #25.
We need a "smart" converter mode which splits the work by channel / tile like the slow mode, but also processes the work units in parallel (up to a user-specified memory limit). This would replace the current "slow" mode.
This mode should also resolve some outstanding memory allocation issues in the slow mode:
- Don't allocate the full tile size if the dataset is smaller than a tile
- Don't allocate the full depth at once if the dataset is too deep to fit in memory
- Don't allocate the full channel at once if the channel is too large to fit in memory
In principle this mode should allow the converter to convert an image of any shape and size (eventually). We could also explore dealing with an unexpected axis order generically, although that may be complex enough to require a separate issue.
This supersedes #4 and #25.
We need a "smart" converter mode which splits the work by channel / tile like the slow mode, but also processes the work units in parallel (up to a user-specified memory limit). This would replace the current "slow" mode.
This mode should also resolve some outstanding memory allocation issues in the slow mode:
In principle this mode should allow the converter to convert an image of any shape and size (eventually). We could also explore dealing with an unexpected axis order generically, although that may be complex enough to require a separate issue.