It would be a nice feature if the compute server could cache datasets downloaded from recent user jobs.
- How would the cache prioritize which datasets to save?
- How would the server tell if a dataset already exists in the cache? How would it know it has the same version (like with a hash) without downloading the entire dataset again?
Medium priority, possibly worth implementing before a v1 release depending how repetitive our downloads seem.
Est. 1 - 3 days to evaluate options, more to implement.
It would be a nice feature if the compute server could cache datasets downloaded from recent user jobs.
Medium priority, possibly worth implementing before a v1 release depending how repetitive our downloads seem.
Est. 1 - 3 days to evaluate options, more to implement.