Restart netCDF file on checkpoint restart (NGWPC-10459)

[idtodd]

[Short description explaining the high-level reason for the pull request]

Additions

• Enum for netCDF file open mode instead of controlling mode by a boolean representing whether the file should be read-only.
• Functions on the NetCDFManager class for opening and validating an existing netCDF file against expectations based on the realization configuration JSON.

Removals

Changes

• Layer::get_catchment_output_data_for_timestep() returns const reference to prevent copying potentially large data.
• NgenSimulation will attempt to open an existing netCDF catchment file instead of creating a new one if checkpointing has been defined in the realization config JSON.

Testing

1. RTE runs tested to be unaffected by changes.
2. Jeff verified cancelling a simulation and restarting it generates the same catchment netCDF output results.

Screenshots

Notes

Todos

Checklist

• PR has an informative and human-readable title
• Changes are limited to a single goal (no scope creep)
• Code can be automatically merged (no conflicts)
• Code follows project standards (link if applicable)
• Passes all existing automated tests
• Any change in functionality is tested
• New functions are documented (with a description, list of inputs, and expected output)
• Placeholder code is flagged / future todos are captured in comments
• Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)
• Reviewers requested with the Reviewers tool ➡️

Testing checklist (automated report can be put here)

Target Environment support

• Linux

[sivasankkar]

All changes are looking good. This can be merged.

[jswade-rtx]

After building with this branch, I ran two short range default forecasts for gage 01123000. For one of the forecasts, I interrupted the run after 3 time steps and restarted from a saved checkpoint. For the other forecast, I ran the full period continuously. I then compared differences across runs for both the catchment_output.nc and troute_output.nc files using the xarray xr.testing.assert_identical(x, y) function, which raises any differences between the passed files. Both the catchment_output.nc and troute_output.nc files were identical between continuous and restarted runs.

This is good to merge on my end.