Troubles with SLURM

[sarmientoj24]

I am having all sorts of troubles with SLURM for following the section on Localized Channels.

Is it possible to just run it directly without using SLURM?

[betterze]

It takes around 40 mins to calculate the gradient map per images. To get 1k gradient maps, I use 250 GPUs, each gpu will be assigned with 4 images.

If you dont use slurm, then pls run this line manually. You could increase number of images per gpu by changing -num_per. But remember to adjust the starting index in -img_sindex.

For example, if you want 10 images per gpu, then do

one=($(seq 0 10 990))
one_index=$((${SLURM_ARRAY_TASK_ID}%${#one[@]}))

python invert_mask.py -model_path './model/ffhq.pkl' -data_path './npy/ffhq' -img_sindex ${one[$one_index]} -num_per 10

[sarmientoj24]

Can I do that simultaneously on an 8-GPU machine wherein I have the commands manually done?

e.g.

python invert_mask.py -model_path './model/ffhq.pkl' -data_path './npy/ffhq' -img_sindex 0 -num_per 10

python invert_mask.py -model_path './model/ffhq.pkl' -data_path './npy/ffhq' -img_sindex 10 -num_per 10

python invert_mask.py -model_path './model/ffhq.pkl' -data_path './npy/ffhq' -img_sindex 20 -num_per 10
...

Each running on a different GPU?

[betterze]

Yep，it should work.