Skip to content

Under-estimated memory prediction #42

@Jordatious

Description

@Jordatious

When running the converter in memory prediction mode (-m), it was suggested I use ~136 GB (see below), which is for a spectral-line cube of shape (1, 1010, 4096, 4096). However, I allocated 150 GB (see below) and the job crashed with an OOM error. I then ran it using 232 GB and the SLURM reported 190.27 GB MaxRSS, ~40% higher than predicted.

jcollier@slurm-login:~$ /carta_share/hdf_convert/run_hdf_converter -m blah.fits
APPROXIMATE MEMORY REQUIREMENTS:
Z stats:	0.536871 GB
XYZ stats:	0.0331285 GB
Rotation:	67.78 GB
XY stats:	0.033128 GB
Main dataset:	67.78 GB
Mipmaps:	67.7138 GB
TOTAL:	136.163GB (Rotated dataset and mipmaps are not allocated at the same time.)
srun --mem=150GB --time=30 --cpus-per-task=30 /carta_share/hdf_convert/run_hdf_converter -p -o /carta_share/current/users/jcollier/blah.hdf5 blah.fits

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions