missing .zattrs in zarr

[crawld]

i'm using the old FF API for the attached burn unit, and the fuelgrid zarr that gets created is missing the .zattrs file. this occurred once before with a different burn unit and Anthony was unable to determine the cause.

PLC1.zip

Leticia also gets a zarr missing .zattrs when using the QF GUI.

fwiw, in bp3d we experienced random missing .zattrs and other dot-files when copying zarrs using s3fs. the problem no longer occurs after upgrading s3fs to 2024.2.0.

[amarcozzi]

Thanks @crawld. I'm finishing up some dissertation stuff today so I'll look into this tomorrow.

[iperezx]

If you need more examples, I have a bunch of burn units that generate the same error.

[amarcozzi]

We don't use s3fs, but I vaguely recall us having a discussion about this being an issue with fsspec. I updated the network packages from their 2-3 year old versions to current versions and deployed.

Let me know if that fixes this issue, or if you are still running into this.

[iperezx]

On what repo did you make these changes and where is it deployed to?

[iperezx]

Nvm. You guys made changes to your in-house APIs. So from our end we don't have to change anything and just test it out.

[iperezx]

Just tested it. Still getting the same error.

[amarcozzi]

Okay. Unmarked my answer, ha. Out of curiosity is it deterministic or random? Also, does it depend on a specific unit or site?

Lastly, how high of a priority is this for you all? On one hand, I'm not particularly inclined to dive deep into a potentially challenging to replicate and debug issue for a codebase that we'd like to deprecate–no plans for that yet though and we'll keep you well informed in advance. On the other hand, I recognize and respect that this API is still used and people depend on these things working properly. I think I just need to know how big of an issue this is for you all so I can prioritize accordingly.

[crawld]

@amarcozzi thanks for trying a newer fsspec. It always seems to happen with the burn unit in PLC1.zip.

Priority is a good question. We had a user report this issue using this burn unit they created. It would be interesting to see if the new FF has the same issue with this burn unit.