duplicate fitres (one gzipped) files error in SALT2mu stage

[djbrout]

Hi Rick,
MaKenzie (who is creating her github account now) has encountered a new issue with her SALT2mu jobs and we seem to have isolated it to a snana change that happened in the last week. Here is the error she is seeing.

/scratch/midway2/rkessler/PIPPIN_OUTPUT/KE_LSST_MWEBV/6_BIASCOR/BBC_LSST_ALL/output_ITER1/SCRIPTS_BBCFIT

	 ../OUTPUT_BBCFIT/FITOPT000_MUOPT000.FITRES

 WARNING(colnum_in_file->open_TEXTgz): Found gzip and unzip file for
	 ../OUTPUT_BBCFIT/FITOPT000_MUOPT000.FITRES
	 ... callFun=colnum_in_file ... try again in 5 sec ...


# ================================================
  PRE-ABORT DUMP from function colnum_in_file->open_TEXTgz :
  Found ../OUTPUT_BBCFIT/FITOPT000_MUOPT000.FITRES.gz
  Found ../OUTPUT_BBCFIT/FITOPT000_MUOPT000.FITRES

   `|```````|`
   <| o\ /o |>
    | ' ; ' |
    |  ___  |     ABORT program on Fatal Error.
    | |' '| |
    | `---' |
    \_______/

 FATAL ERROR ABORT called by colnum_in_file->open_TEXTgz
   Found gzipped and unzipped file.
   Cannot open both files.

but when we run the FAIL_REPEAT command in stage 6 BIASCOR it runs successfully which is odd (and we dont see two files gzipped and non-gzipped) and this made us think that this is a gzip timing issue that eventually resolves itself but not before the crash happens. We went ahead and looked through your git commit history and saw this change snana_close() from last week and we suspect it has some effect on the timing of gzip and is causing the error above.

5fad2dd#diff-391a9d83cd4d4358f141c8bed1eef2f6074d50c9b301e7b69613398531f618ddR10776

[RickKessler]

with such a large number of events (600k), I suspect that gzipping FITOPT000_MUOPT000.FITRES takes a bit longer, and next BBC job starts before gzipping is done and thus finds both gzipped and unzipped files.

I fixed the CPU*CMD logic to not just check that FITOPT000_MUOPT000.FITRES.gz exists, but also ensure that FITOPT000_MUOPT000.FITRES does NOT exist. See updated "while" logic.

[makenzieelliott]

Ahh that makes sense. I will give this another look tomorrow, but it sounds like that should fix the issue. Thank you!

Edit: This worked!