Programmatically create hourly profiles for github tests

[patrickbrown4]

Summary

We end up downloading the Zenodo files fairly frequently on the github runner. That stresses the Zenodo servers, slows down the runner, and increases the per-run storage requirements for the runner.

Proposed solution

Write some functions to programmatically generate fake CF/load profiles, for use only when a GSw_UseFakeData (or something) flag is turned on. This approach would only be used for the github tests.

• CF profiles
  • It could read the site class and average CF from inputs/supply_curve/supplycurve_{tech}-{siting}.csv and then use a suitable distribution from np.random to create a noisy profile with the same mean for each site, matching the structure of the real site-level files on Zenodo. (Or they could be completely random.)
  • I think it would go here in recf.py as a new if/then:

    ReEDS/input_processing/recf.py

    Lines 132 to 139 in 4bf761b

    	# Get site-level hourly CFs
    	weather_year_site_cf_hourly = reeds.io.get_site_cf_hourly(
    	tech=tech,
    	year=year,
    	case=inputs_case,
    	)
    	# Downselect to relevant sites
    	weather_year_site_cf_hourly = weather_year_site_cf_hourly[df_sc.index]

• Demand
  • Same idea in hourly_load.py:

    ReEDS/input_processing/hourly_load.py

    Lines 618 to 623 in 4bf761b

    	state_load_hourly = reeds.io.get_load_hourly(inputs_case)
    	state_load_hourly = downselect_to_weather_years(
    	state_load_hourly,
    	weather_years
    	)
    	historical_state_load_annual = reeds.io.get_historical_state_load_annual()

• Infrastructure in runbatch.py
  • Add a new test to check_compatibility() to enforce that all parallel cases must use the same GSw_UseFakeData setting for simplicity
  • Bypass reeds.remote.download_remote_files() if GSw_UseFakeData is turned on

Alternative solution

We could instead store a subset of the actual data on Zenodo. But we'd need to update it whenever we update any of the other Zenodo files, or the regions used in the runner tests, which is not ideal. It would also keep us from doing larger-area simplified tests.

(copied from https://github.nrel.gov/ReEDS/ReEDS-2.0/issues/2036)

[patrickbrown4]

Addressed by #26