The strong scaling tests showed that loading the baselines data is the bottleneck once the number of ranks starts to increase.
It might be that rather than each rank loading its own data and creating a parallel reading bottleneck, a subset of the ranks act as middle managers and load a subset of the data which they then scatter to a number of workers. So e.g. if you had 100 ranks, maybe 10 of those ranks could load data, and scatter to a different 10 ranks each.
The strong scaling tests showed that loading the baselines data is the bottleneck once the number of ranks starts to increase.
It might be that rather than each rank loading its own data and creating a parallel reading bottleneck, a subset of the ranks act as middle managers and load a subset of the data which they then scatter to a number of workers. So e.g. if you had 100 ranks, maybe 10 of those ranks could load data, and scatter to a different 10 ranks each.