For correctness and reproducibility, it makes sense to isolate target execution as much as possible, meaning:
Stage 1
- Always run with a minimal set of environment variables (
PATH, HTTP_PROXY, HTTPS_PROXY, SLURM_*, USER, ...)
- Create a temporary folder (possible on local disk, but it should be possible to overwrite this per-target).
- This folder becomes the working directory of the target when it executes.
- All input files are symlinked into the temporary folder.
- Once the target has executed:
- If the temporary is on the same filesystem as the main storage, move output files to the main storage.
- If not, copy output files to the main storage.
- Wipe the temporary folder (and maybe count how much data was there, not including input files, and report this in the log).
Stage 2
For security and further isolation, we may use landlock to further lock down the process executing the target, e.g. blocking Internet access on a per-target basis.
For correctness and reproducibility, it makes sense to isolate target execution as much as possible, meaning:
Stage 1
PATH,HTTP_PROXY,HTTPS_PROXY,SLURM_*,USER, ...)Stage 2
For security and further isolation, we may use landlock to further lock down the process executing the target, e.g. blocking Internet access on a per-target basis.