Flexible start year option & tech classes addition by atpham88 · Pull Request #12 · ReEDS-Model/ReEDS

atpham88 · 2026-04-15T21:50:06Z

Summary

This PR:

Adds an option to specify a flexible start year, instead of 2010
Removes startyear switch and instead establishes startyear based on yearset
Adds algorithm to assign pv, wind, and geothermal tech classes to NEMS units

Technical details

These changes are made in this PR:

process_unitdata.py: Read unitdata.csv from inputs_case and process it to assign sc_point_pids, capacity factors/mean temps to pv, wind, and geothermal units. This file is run immediately after copy_files.py. This allows us to remove all the mapping of NEMS units to sc_point_pids in the ReEDS_input_processing repo.
WriteHintage.py: Allow start year to be flexible instead of 2010.
writecapdat.py: Add mappings to solar, wind, geothermal resource class are based on the resource quality (capacity factors for wind and solar and temperature for geothermal) of the technology as it comes from reV.
c_supplymodel.gms
- pcat set is removed from the model: The equations eq_force_prescription_power and eq_force_prescription_energy, along with the variables EXTRA_PRESCRIP and EXTRA_PRESCRIP_ENERGY, are currently indexed by pcat. Since the prescribed generator classes and vintages are now defined in writecap.py, the pcat index is no longer needed in these equations or variables. Accordingly, the equations are revised so that the investment (INV) of prescribed generators—indexed by (i, newv, r, t)—is equal to the sum of the prescribed build values and the additional prescribed investments (EXTRA_PRESCRIP) for those same (i, newv, r, t) combinations.
- Integrate prescribed retirements when the prescribed facilities retires before their life time: In the NEMS database, the retirement year of some facilities occurs before the end of their expected lifetimes. To accurately capture these early retirements for prescribed facilities, the prescribed retirement values are subtracted from total investments when calculating facility capacities in eq_cap_new_noret and eq_cap_energy_new_noret. Although this implementation follows the same logic as m_capacity_exog, it explicitly captures prescribed generator builds in the INV variables in the current version of the model.
hourlize:
- Remove scripts that generate exog_cap_{tech}.csv files as they are no longer needed.
geothermal_classification.csv is not generated in hourlize but taken from https://pages.github.nrel.gov/ReEDS/ReEDS-2.0/model_documentation.html#technical-resource-potential.
@wesleyjcole Do you want to add some description of what you did to resolve the geothermal prescribed build issue?

Issues resolved

Known incompatibilities

Relevant sources or documentation

Validation, testing, and comparison report(s)

Pending

Checklist for author

Details to double-check

General information to guide review

Zero impact on results of default case
No large data file(s) added/modified
No substantive impact on runtime for full-US reference case
No substantive impact on folder size for full-US reference case
No change to process flow (runbatch.py, d_solve_iterate.py)
No change to code organization
No change to package requirements (environment.yml or Project.toml)

Did you use LLM tools (chatbot or copilot) in the preparation of this PR? If so, describe how

No

Tag points of contact here if you would like additional review of the relevant parts of the model

Co-authored-by: Wesley Cole <49044852+wesleyjcole@users.noreply.github.com>

patrickbrown4

I haven't gone through the whole thing yet and will take a closer look next week, but some initial thoughts in the meantime. Happy to discuss if helpful (including whether to try some of the suggested changes here or in a followup PR). Thanks!

…tdata.py

bsergi

Happy to chat through any of the comments on this one.

bsergi · 2026-05-11T14:31:55Z

+
+# Only used for one-off modification of geothermal supply curves
+# Remove after hourlize is rerun and resource temp is added to geothermal supply curve
+if sys.platform == 'win32':


Does this work for all windows systems?

I remember it worked for windows for a a stdscens processing script, which Pieter already tested. But we can get someone else with a windows laptop to test again here to be sure.

bsergi · 2026-05-11T14:48:25Z

 ```

+### define_tech_classes.py
+Run this scrip after `run_hourlize.py` to generate classification of UPV and wind technologies (`classification_upv.csv`, `classification_wind-ofs.csv`, and `classification_wind-ons.csv`)


It would be nice to not have to add another step that users have to remember. Can we add a if __name__== '__main__': block to tech_classes.py to make it importable and then to call directly from resource.py?

Alternatively, you could also just call this function from the create_exog_rsc from within writecapdat.py and create these on the fly for the relevant access case. That would avoid having to store the extra files in the repo; if they are only used for the assigning the exogenous capacity that might be the better route.

Yes, either way sounds good to me. Let's talk about which one we want to implement today.

define tech class in create_exog_rsc function now.

bsergi · 2026-05-11T14:53:03Z

+    # Aggregate min/max by class and attach access_case
+    if tech == 'wind-ofs':
+        df['subtech'] = 'fixed'
+        df.loc[df['class']>=6,'subtech'] = 'floating'


I'm a little worried about hardcoding this value in here. If this depends on the class_bin_num specification in the config file maybe we can read from that?

I got this from the raw supply curves. It's always that class 1-5 = fixed and above 5 = floating. Originally I processed the classification from the raw windows supply curve which already has fixed and floating categories so I didn't have to do this. But the supply curve in ReEDS repo does not have these categories, so this is more of a one-off modification to add these categories there. I can remove it the next time hourlize is run. Also, it doesn't impact runs since only the class column is read.

bsergi · 2026-05-11T15:36:34Z

-capacity_exog(i,"init-1",r,t)${[yeart(t)-sum{tt$tfirst(tt),yeart(tt) }<maxage(i)]} =
-                                 max(0,capnonrsc(i,r,"value")
-                                       - sum{allt$[allt.val <= t.val],  prescribedretirements(allt,r,i,"value") }
+capacity_exog(i,v,r,t)${[yeart(t)-sum{tt$tfirst(tt),yeart(tt) }<maxage(i)]$sameas(v,'init-1')} =


The comment above notes "existing capacity equals all 2010 capacity less retirements" -- should this be changed to "all pre-start year capacity"? If so I'm guessing there are other 2010 references that might need to be updated.

bsergi · 2026-05-11T15:37:46Z


-*beginning year value is zero (i.e., no elasticity)
-cd_beta(cendiv,t)$[not tfirst(t)] = cd_beta0_allsector(cendiv) ;
+*beginning year value of 2010 is zero (i.e., no elasticity)


Why hardcode this 2010?

bsergi · 2026-05-11T15:39:39Z

-                     $(yeart(tt)<=yeart(t))$(tmodel(tt) or tfix(tt))],
-        INV(i,newv,r,tt) + INV_REFURB(i,newv,r,tt)$[refurbtech(i)$Sw_Refurb]}
+
+        INV(i,newv,r,t) + INV_REFURB(i,newv,r,t)$[refurbtech(i)$Sw_Refurb]


this approach seems a lot cleaner so I like it. Was the goal to help with numerical instability?

bsergi · 2026-05-11T15:42:38Z

-                df_pre['capacity'] =  df_pre['capacity'].round(decimals)
-                df_pre = df_pre.sort_values(['year',profile_id_col])
-                df_pre.to_csv(os.path.join(outpath, 'results', tech + '_prescribed_builds.csv'), index=False)
        #Reduce supply curve based on exogenous (pre-start-year) capacity


I don't understand why you've left this reduction block in. Doesn't the supply curve equation already (eq_rsc_INVlim) already account for exogenous capacity? Wouldn't subtracting here be double couting that capacity?

Maybe it doesn't matter since you need subtract_exog=True for this to be applied and that isn't the default, but I'm wondering whether we should keep it at all since I can't see a time we'd want to use it.

I missed this when I deleted the blocks above. Removed this now.

bsergi · 2026-05-11T15:47:08Z

If we're processing the existing capacity downstream now I think it might be good to remove the remaining pieces in here, otherwise some folks might wander in here and think this is still being used.

…h classes in process_unitdata.csv

patrickbrown4 · 2026-05-11T22:28:19Z

      / (1$[not rsc_capacity_scalar_i(ii)] + rsc_capacity_scalar(ii,r,tt)$rsc_capacity_scalar_i(ii)) } ) / 1000 ;

+
+*** Do we need this part when pcat is removed?


pcat is still used above; is the intent to remove it completely? (Don't go out of your way to remove it if it's not necessary for the PR; I was just curious since it's also mentioned in the PR text.)

ReEDS/b_inputs.gms

Lines 1179 to 1208 in 097ea5f

set prescriptivelink0(pcat,ii) "initial set of prescribed categories and their technologies - used in assigning prescribed builds"

/

$offlisting

$ondelim

$include inputs_case%ds%prescriptivelink0.csv

$offdelim

$onlisting

/ ;

*include non-numeraire CSPs and then exclude numeraire CSPs in ii dimension of

*prescriptivelink0(pcat,ii) set when Sw_WaterMain is ON

prescriptivelink0("csp-ws",ii)$[(csp1(ii) or csp2(ii) or csp3(ii) or csp4(ii))$Sw_WaterMain] = yes ;

prescriptivelink0("csp-ws",ii)$[csp(ii)$i_numeraire(ii)$Sw_WaterMain] = no ;

set prescriptivelink(pcat,i) "final set of prescribed categories and their technologies - used in the model" ;

prescriptivelink(pcat,i)$prescriptivelink0(pcat,i) = yes ;

alias(pcat,ppcat) ;

* active prescriptivelink for all techs not included in the table above

* but restrict out csp techs in this calculation - since they

* are indexed by a separate pcat (csp-ws) and have special considerations

prescriptivelink(pcat,i)$[sameas(pcat,i)$(not sum{ppcat, prescriptivelink(ppcat,i) })$(not csp1(i))] = yes ;

*only geo_hydro techs are considered to meet geothermal prescriptions

prescriptivelink(pcat,i)$[geo_extra(i)] = no ;

*upgrades have no prescriptions

prescriptivelink(pcat,i)$[upgrade(i)] = no ;

Thank you for the catch. I removed the remaining mentions of pcat now.

…during run in writecapdat.py

… instead

…atch since FIPS have changed

…a instead of 'r' column

atpham88 added 6 commits April 15, 2026 15:43

moved to github.com

35b2339

Update ReEDS_generator_database_final_EIA-NEMS.csv

8a17399

put climate folder back

04d2355

update upload-artifact to v4 to pass github tests

b8779fa

Merge branch 'main' into ap_flex-start-year

2c1fec0

fix issue with wind-ons supply curves

64ea661

atpham88 requested a review from wesleyjcole April 16, 2026 22:32

Merge branch 'main' into ap_flex-start-year

bacf02a

wesleyjcole reviewed Apr 16, 2026

View reviewed changes

Update sources.csv

e760e49

Co-authored-by: Wesley Cole <49044852+wesleyjcole@users.noreply.github.com>

atpham88 requested review from bsergi and patrickbrown4 April 17, 2026 00:01

atpham88 added 4 commits April 16, 2026 23:21

remove redundant statements

ad71748

remove exog_cap and exog_prescribe files from hourlize

1d989fc

Update sources.csv

2afc17b

add more description for tech classification files

3dda333

patrickbrown4 reviewed Apr 17, 2026

View reviewed changes

atpham88 added 13 commits April 17, 2026 18:08

Update e_report.gms

2848ff5

Merge branch 'main' into ap_flex-start-year

eb1f1ce

only use max cf for class cut off to avoid gaps

2f78001

keep rounding to 4 decimals

a880c2a

add function to get missing_class_resource

bb0282f

clean up

13079d8

clean up

c7d90f6

remove sc_point_gid and geothermal temp from NEMS

c208046

remove unitdata.csv from runfiles.csv

91cec02

Create generate_unitdata.py

74ae71c

remove reading NEMS data and its processing from copy_files.py

496246c

Add generate_unitdata.py script

10fe8ef

move all NEMS data processing from copy_files and add to generate_uni…

8e160ac

…tdata.py

atpham88 added 3 commits May 9, 2026 18:16

Update inputs.py

1ec5338

Clean up

b4ebb33

Update inputs.py

c3e3945

bsergi reviewed May 11, 2026

View reviewed changes

atpham88 added 4 commits May 11, 2026 11:07

Merge branch 'main' into ap_flex-start-year

cd767f2

still write unitdata.csv to inputs_case in copy_files but process tec…

ed213b5

…h classes in process_unitdata.csv

remove unnecessary process

097ea5f

add __name__ == '__main__' block to define_tech_classes.py

f061367

patrickbrown4 reviewed May 11, 2026

View reviewed changes

atpham88 added 21 commits May 11, 2026 17:09

remove classification_{tech}.csv for pv and wind, create these files …

fe3ff00

…during run in writecapdat.py

clean up

65d2938

clean up

7d3022d

exclude geothermal from missing resource

f03fedf

Update recf.py

752109f

remove missing class resource check

2ad78ae

use supply_curves in inputs_case for sc_point_gids mapping

fedfa98

clean up

c0c7649

separate unitdata_orig.csv and unitdata.csv

918963c

adjust class cutoffs to match with resources

391b89d

Update futurefiles.csv

eae710b

Merge branch 'main' into ap_flex-start-year

6c99ee2

change path for reeds functions

288b46a

resolve conflict

8402b72

remove reassignment of missing class resources and add an error check…

a0deb81

… instead

add exogenous offshore wind cap

46cf43e

remove comment ='#' when reading NEMS data to avoid data loss

15c6644

reassign reeds_ba in unitdata as well to avoid resource/capacity mism…

d2c400f

…atch since FIPS have changed

remove prescriptivelink from b_inputs.gms

1c60b21

use 'ba' column in county2zone to update 'reeds_ba' column in unitdat…

5ec49d2

…a instead of 'r' column

remove remaining pcat and clean up

2663d10

		/ (1$[not rsc_capacity_scalar_i(ii)] + rsc_capacity_scalar(ii,r,tt)$rsc_capacity_scalar_i(ii)) } ) / 1000 ;


		*** Do we need this part when pcat is removed?

	set prescriptivelink0(pcat,ii) "initial set of prescribed categories and their technologies - used in assigning prescribed builds"
	/
	$offlisting
	$ondelim
	$include inputs_case%ds%prescriptivelink0.csv
	$offdelim
	$onlisting
	/ ;

	*include non-numeraire CSPs and then exclude numeraire CSPs in ii dimension of
	*prescriptivelink0(pcat,ii) set when Sw_WaterMain is ON
	prescriptivelink0("csp-ws",ii)$[(csp1(ii) or csp2(ii) or csp3(ii) or csp4(ii))$Sw_WaterMain] = yes ;
	prescriptivelink0("csp-ws",ii)$[csp(ii)$i_numeraire(ii)$Sw_WaterMain] = no ;

	set prescriptivelink(pcat,i) "final set of prescribed categories and their technologies - used in the model" ;

	prescriptivelink(pcat,i)$prescriptivelink0(pcat,i) = yes ;

	alias(pcat,ppcat) ;

	* active prescriptivelink for all techs not included in the table above
	* but restrict out csp techs in this calculation - since they
	* are indexed by a separate pcat (csp-ws) and have special considerations
	prescriptivelink(pcat,i)$[sameas(pcat,i)$(not sum{ppcat, prescriptivelink(ppcat,i) })$(not csp1(i))] = yes ;
	*only geo_hydro techs are considered to meet geothermal prescriptions
	prescriptivelink(pcat,i)$[geo_extra(i)] = no ;


	*upgrades have no prescriptions
	prescriptivelink(pcat,i)$[upgrade(i)] = no ;

Conversation

atpham88 commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Technical details

Issues resolved

Known incompatibilities

Relevant sources or documentation

Validation, testing, and comparison report(s)

Checklist for author

Details to double-check

General information to guide review

Did you use LLM tools (chatbot or copilot) in the preparation of this PR? If so, describe how

Tag points of contact here if you would like additional review of the relevant parts of the model

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

patrickbrown4 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bsergi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

patrickbrown4 May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

atpham88 commented Apr 15, 2026 •

edited

Loading

patrickbrown4 May 11, 2026 •

edited

Loading