Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
35b2339
moved to github.com
atpham88 Apr 15, 2026
8a17399
Update ReEDS_generator_database_final_EIA-NEMS.csv
atpham88 Apr 15, 2026
04d2355
put climate folder back
atpham88 Apr 15, 2026
b8779fa
update upload-artifact to v4 to pass github tests
atpham88 Apr 15, 2026
2c1fec0
Merge branch 'main' into ap_flex-start-year
atpham88 Apr 16, 2026
64ea661
fix issue with wind-ons supply curves
atpham88 Apr 16, 2026
bacf02a
Merge branch 'main' into ap_flex-start-year
atpham88 Apr 16, 2026
e760e49
Update sources.csv
atpham88 Apr 17, 2026
ad71748
remove redundant statements
atpham88 Apr 17, 2026
1d989fc
remove exog_cap and exog_prescribe files from hourlize
atpham88 Apr 17, 2026
2afc17b
Update sources.csv
atpham88 Apr 17, 2026
3dda333
add more description for tech classification files
atpham88 Apr 17, 2026
2848ff5
Update e_report.gms
atpham88 Apr 18, 2026
eb1f1ce
Merge branch 'main' into ap_flex-start-year
atpham88 Apr 20, 2026
2f78001
only use max cf for class cut off to avoid gaps
atpham88 Apr 21, 2026
a880c2a
keep rounding to 4 decimals
atpham88 Apr 21, 2026
bb0282f
add function to get missing_class_resource
atpham88 Apr 21, 2026
13079d8
clean up
atpham88 Apr 21, 2026
c7d90f6
clean up
atpham88 Apr 21, 2026
c208046
remove sc_point_gid and geothermal temp from NEMS
atpham88 Apr 21, 2026
91cec02
remove unitdata.csv from runfiles.csv
atpham88 Apr 22, 2026
74ae71c
Create generate_unitdata.py
atpham88 Apr 22, 2026
496246c
remove reading NEMS data and its processing from copy_files.py
atpham88 Apr 22, 2026
10fe8ef
Add generate_unitdata.py script
atpham88 Apr 22, 2026
8e160ac
move all NEMS data processing from copy_files and add to generate_uni…
atpham88 Apr 22, 2026
4a1ffa6
use current NEMS data for this PR
atpham88 Apr 22, 2026
ab58f8e
add mean resource temp to geothermal supply curves
atpham88 Apr 22, 2026
068e7ea
add one-off script to add mean temp to geothermal resources
atpham88 Apr 22, 2026
5d6ed6d
use egs supply curve for existing geohydro units in NEMS
atpham88 Apr 22, 2026
7730a81
clean up
atpham88 Apr 22, 2026
0917120
Merge branch 'main' into ap_flex-start-year
atpham88 Apr 22, 2026
fab7c02
clean up
atpham88 Apr 22, 2026
91fd4a5
Update generate_unitdata.py
atpham88 Apr 23, 2026
71dccc9
Update generate_unitdata.py
atpham88 Apr 23, 2026
72455b6
clean up
atpham88 Apr 23, 2026
13bea65
Update generate_unitdata.py
atpham88 Apr 23, 2026
6cd9c44
Merge branch 'main' into ap_flex-start-year
atpham88 Apr 23, 2026
57cdef8
update mapping between offshore wind sc_point_gid and NEMS units
atpham88 Apr 23, 2026
7a06fa0
Merge branch 'main' into ap_flex-start-year
atpham88 Apr 23, 2026
fb7ab38
clean up and add more description
atpham88 Apr 23, 2026
d2e5cbd
remove csp from upv_exog_cap
atpham88 Apr 24, 2026
1dfa19d
Update c_supplymodel.gms
atpham88 May 2, 2026
b79c2c7
Update c_supplymodel.gms
atpham88 May 2, 2026
ece954d
reduce decimal in resource temp and change classification file names
atpham88 May 2, 2026
37672cc
Merge branch 'ap_flex-start-year' of https://github.com/ReEDS-Model/R…
atpham88 May 2, 2026
f8bb631
Merge branch 'main' into ap_flex-start-year
atpham88 May 2, 2026
752767b
rename exog_cap_{tech} files
atpham88 May 2, 2026
bb813de
remove duplicate testing lines
atpham88 May 2, 2026
c4ea341
Update input_processing/generate_unitdata.py
atpham88 May 2, 2026
1c28dc6
wrap under 100 characters
atpham88 May 2, 2026
2114afd
clean up datafile names
atpham88 May 2, 2026
75c126b
Update hourlize/define_tech_classes.py
atpham88 May 2, 2026
ff7d420
Update b_inputs.gms
atpham88 May 2, 2026
fe847ea
Update input_processing/recf.py
atpham88 May 2, 2026
ef38eee
Update b_inputs.gms
atpham88 May 2, 2026
ab45787
remove Sw_onlineyearcol
atpham88 May 2, 2026
f82b6bf
Merge branch 'ap_flex-start-year' of https://github.com/ReEDS-Model/R…
atpham88 May 2, 2026
26191fb
move main function after supporting functions
atpham88 May 2, 2026
9807765
clean up coordinates and sc_points mapping function
atpham88 May 2, 2026
85f382d
Update README.md
atpham88 May 2, 2026
f14c35d
Update input_processing/aggregate_regions.py
atpham88 May 2, 2026
7c398a2
wrap under 100 characters
atpham88 May 2, 2026
9e8ed14
reduce number of dfs/gdfs
atpham88 May 2, 2026
df83f58
fix to pass tests
atpham88 May 3, 2026
e859335
clean up generate_unitdata.py
atpham88 May 4, 2026
b364b1a
clean up
atpham88 May 4, 2026
c635b80
Merge branch 'main' into ap_flex-start-year
atpham88 May 5, 2026
9c1780d
assign missing tech classes to resources closest to them
atpham88 May 6, 2026
3f09cfd
Merge branch 'main' into ap_flex-start-year
atpham88 May 6, 2026
db0ff30
Update recf.py
atpham88 May 6, 2026
7002c5f
exclude geothermal from missing resources
atpham88 May 6, 2026
20cd128
Merge branch 'main' into ap_flex-start-year
atpham88 May 7, 2026
2df57c3
Use supply curves in main repo stead of nrelnas to generate tech clas…
atpham88 May 8, 2026
06c948a
Merge branch 'main' into ap_flex-start-year
atpham88 May 8, 2026
b092f29
Update define_tech_classes.py
atpham88 May 9, 2026
3ca7b3b
Merge branch 'main' into ap_flex-start-year
atpham88 May 9, 2026
bda4e11
remove startyear from cases.csv and establish startyear based on yearset
atpham88 May 10, 2026
1ec5338
Update inputs.py
atpham88 May 10, 2026
b4ebb33
Clean up
atpham88 May 10, 2026
c3e3945
Update inputs.py
atpham88 May 10, 2026
cd767f2
Merge branch 'main' into ap_flex-start-year
atpham88 May 11, 2026
ed213b5
still write unitdata.csv to inputs_case in copy_files but process tec…
atpham88 May 11, 2026
097ea5f
remove unnecessary process
atpham88 May 11, 2026
f061367
add __name__ == '__main__' block to define_tech_classes.py
atpham88 May 11, 2026
fe3ff00
remove classification_{tech}.csv for pv and wind, create these files …
atpham88 May 11, 2026
65d2938
clean up
atpham88 May 11, 2026
7d3022d
clean up
atpham88 May 12, 2026
f03fedf
exclude geothermal from missing resource
atpham88 May 12, 2026
752109f
Update recf.py
atpham88 May 12, 2026
2ad78ae
remove missing class resource check
atpham88 May 12, 2026
fedfa98
use supply_curves in inputs_case for sc_point_gids mapping
atpham88 May 12, 2026
c0c7649
clean up
atpham88 May 12, 2026
918963c
separate unitdata_orig.csv and unitdata.csv
atpham88 May 13, 2026
391b89d
adjust class cutoffs to match with resources
atpham88 May 13, 2026
eae710b
Update futurefiles.csv
atpham88 May 13, 2026
6c99ee2
Merge branch 'main' into ap_flex-start-year
atpham88 May 13, 2026
288b46a
change path for reeds functions
atpham88 May 13, 2026
8402b72
resolve conflict
atpham88 May 13, 2026
a0deb81
remove reassignment of missing class resources and add an error check…
atpham88 May 13, 2026
46cf43e
add exogenous offshore wind cap
atpham88 May 13, 2026
15c6644
remove comment ='#' when reading NEMS data to avoid data loss
atpham88 May 13, 2026
d2c400f
reassign reeds_ba in unitdata as well to avoid resource/capacity mism…
atpham88 May 13, 2026
1c60b21
remove prescriptivelink from b_inputs.gms
atpham88 May 13, 2026
5ec49d2
use 'ba' column in county2zone to update 'reeds_ba' column in unitdat…
atpham88 May 13, 2026
2663d10
remove remaining pcat and clean up
atpham88 May 14, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion cases.csv
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ timetype,Sequential or Intertemporal,seq; int; win,seq,
ignore,"0=run this case, 1=ignore this case",0; 1,0,
AWS,Switch for when running on AWS,0; 1,0,
yearset,"_-delimited list of years to model OR bash-formatted year ranges (start..end..step). For example, 2010_2015_2020..2035..3_2040..2050..5 means: 2010, 2015, then 2020 through 2035 in 3-year steps, then 2040 through 2050 in 5-year steps. startyear will be prepended if it is not included. Truncated at endyear.",N/A,2010_2015_2020..2050..3,
startyear,First year to be modeled,N/A,2010,
endyear,Last year to be modeled,N/A,2050,
windows_suffix,Set of windows that are modeled,default; step5; step10; 2100,default,
solver,Solver for GAMS to use,CPLEX; gurobi,CPLEX,
Expand Down
11 changes: 1 addition & 10 deletions docs/sources.csv
Original file line number Diff line number Diff line change
Expand Up @@ -117,14 +117,7 @@ RelativeFilePath,RelativeFolderPath,FileName_new,FileExtension,Description_new,I
/inputs/capacity_exogenous/cappayments.csv,/inputs/capacity_exogenous,cappayments,.csv,,,,,,
/inputs/capacity_exogenous/cappayments_ba.csv,/inputs/capacity_exogenous,cappayments_ba,.csv,,,,,,
/inputs/capacity_exogenous/demonstration_plants.csv,/inputs/capacity_exogenous,demonstration_plants,.csv,Nuclear-smr demonstration plants; active when GSw_NuclearDemo=1,"t,r,i,coolingwatertech,ctt,wst,value",,See 'notes' column in the file and https://www.energy.gov/oced/advanced-reactor-demonstration-projects-0,Prescribed capacity,MW
/inputs/capacity_exogenous/exog_cap_geohydro_allkm_reference.csv,/inputs/capacity_exogenous,exog_cap_geohydro_allkm_reference,.csv,,,,,,
/inputs/capacity_exogenous/exog_cap_geohydro_reference.csv,/inputs/capacity_exogenous,exog_cap_geohydro_reference,.csv,,,,,,
/inputs/capacity_exogenous/exog_cap_upv_limited.csv,/inputs/capacity_exogenous,exog_cap_upv_limited,.csv,,,,,,
/inputs/capacity_exogenous/exog_cap_upv_open.csv,/inputs/capacity_exogenous,exog_cap_upv_open,.csv,,,,,,
/inputs/capacity_exogenous/exog_cap_upv_reference.csv,/inputs/capacity_exogenous,exog_cap_upv_reference,.csv,,,,,,
/inputs/capacity_exogenous/exog_cap_wind-ons_limited.csv,/inputs/capacity_exogenous,exog_cap_wind-ons_limited,.csv,,,,,,
/inputs/capacity_exogenous/exog_cap_wind-ons_open.csv,/inputs/capacity_exogenous,exog_cap_wind-ons_open,.csv,,,,,,
/inputs/capacity_exogenous/exog_cap_wind-ons_reference.csv,/inputs/capacity_exogenous,exog_cap_wind-ons_reference,.csv,,,,,,
/inputs/supply_curve/classification_geothermal.csv,/inputs/capacity_exogenous,classification_geothermal,.csv,Mean tempurature range of geothermal facilities classes defined based on reV resource supply curves and taken from Table 6 of the model documentation https://reeds-model.github.io/ReEDS/model_documentation.html#technical-resource-potential ,"class,access_case,min_reV_mean_temp,max_reV_mean_temp",,,Use to define prescribed capacity class of rsc technology ,celsius degree
/inputs/capacity_exogenous/interconnection_queues.csv,/inputs/capacity_exogenous,interconnection_queues,.csv,,,,,,
/inputs/capacity_exogenous/prescribed_builds_wind-ofs_meshed_limited.csv,/inputs/capacity_exogenous,prescribed_builds_wind-ofs_meshed_limited,.csv,,,,,,
/inputs/capacity_exogenous/prescribed_builds_wind-ofs_meshed_open.csv,/inputs/capacity_exogenous,prescribed_builds_wind-ofs_meshed_open,.csv,,,,,,
Expand Down Expand Up @@ -532,10 +525,8 @@ RelativeFilePath,RelativeFolderPath,FileName_new,FileExtension,Description_new,I
/inputs/sets/orcat.csv,/inputs/sets,orcat,.csv,set of operating reserve categories,,,,GAMS set,
/inputs/sets/ortype.csv,/inputs/sets,ortype,.csv,set of types of operating reserve constraints,,,,GAMS set,
/inputs/sets/p.csv,/inputs/sets,p,.csv,set of products produced,,,,GAMS set,
/inputs/sets/pcat.csv,/inputs/sets,pcat,.csv,set of prescribed technology categories,,,,GAMS set,
/inputs/sets/plantcat.csv,/inputs/sets,plantcat,.csv,set of categories for plant characteristics,,,,GAMS set,
/inputs/sets/prepost.csv,/inputs/sets,prepost,.csv,,,,,GAMS set,
/inputs/sets/prescriptivelink0.csv,/inputs/sets,prescriptivelink0,.csv,initial set of prescribed categories and their technologies - used in assigning prescribed builds,,,,GAMS set,
/inputs/sets/pvb_agg.csv,/inputs/sets,pvb_agg,.csv,crosswalk between hybrid pv+battery configurations and technology options,,,,GAMS set,
/inputs/sets/pvb_config.csv,/inputs/sets,pvb_config,.csv,set of hybrid pv+battery configurations,,,,GAMS set,
/inputs/sets/quarter.csv,/inputs/sets,quarter,.csv,,,,,GAMS set,
Expand Down
10 changes: 0 additions & 10 deletions docs/sources_documentation.md
Original file line number Diff line number Diff line change
Expand Up @@ -2715,11 +2715,6 @@
- **Description:** set of products produced
---

- [pcat.csv](/inputs/sets/pcat.csv)
- **File Type:** GAMS set
- **Description:** set of prescribed technology categories
---

- [plantcat.csv](/inputs/sets/plantcat.csv)
- **File Type:** GAMS set
- **Description:** set of categories for plant characteristics
Expand All @@ -2729,11 +2724,6 @@
- **File Type:** GAMS set
---

- [prescriptivelink0.csv](/inputs/sets/prescriptivelink0.csv)
- **File Type:** GAMS set
- **Description:** initial set of prescribed categories and their technologies - used in assigning prescribed builds
---

- [pvb_agg.csv](/inputs/sets/pvb_agg.csv)
- **File Type:** GAMS set
- **Description:** crosswalk between hybrid pv+battery configurations and technology options
Expand Down
5 changes: 2 additions & 3 deletions hourlize/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,13 +72,13 @@ Follow setup details in the [Running hourlize](#running-hourlize) section.

Hourlize relies on a set of columns being in the reV supply curve. In some cases hourlize can fill in missing columns hourlize in a pre-processing step, but in others these missing columns can cause hourlize to fall. A list of required columns can be found in `hourlize/inputs/resource/rev_sc_columns.csv`


### 4. Synchronize shared directories

By default hourlize will copy the required files in your ReEDS repository (`copy_to_reeds=True`). You can also copy the hourlize outputs back to the shared repository by setting `copy_to_shared=True`.

Note that hourlize copies to only one of the shared locations (either the HPC or nrelnas01, depending on where you are running it), so even with `copy_to_shared=True` you'll want to sync up the two shared folders. When starting from the HPC, be sure to open up permissions to the supply curve outputs you've just created (e.g., `chmod -R 777 UPV/2023_06_06_Update`). Then, from your local computer, use WinSCP or rsync to copy the files from Kestrel to nrelnas01:


```bash
rsync -aPu [username]@kestrel.nlr.gov://projects/shared-projects-reeds/reeds/Supply_Curve_Data/UPV/2023_11_02_LandCover /Volumes/ReEDS/Supply_Curve_Data/UPV
```
Expand Down Expand Up @@ -168,9 +168,8 @@ The main inputs to hourlize are reV outputs for a given reV scenario:
By default, the outputs will be dumped to a subdirectory named `results` within `hourlize/out/[casename]`. In addition, with `copy_to_reeds` set to true (as is default), we'll copy the results to the ReEDS repo containing this hourlize directory, and with`copy_to_shared` set to true (not default), we'll copy to the shared drive (see Shared Drive Locations below).

* `supplycurve_{tech}.csv`: A supply curve with rows for each site and columns for region, class, available capacity, and costs. E.g. see `inputs/supply_curve/wind-ons_supply_curve-reference_ba.csv` (within ReEDS repo)
* `{tech}_exog_cap.csv`: Exogenous (built pre-2010) capacity with columns for region, site and year. This is not capacity builds in each year, but rather cumulative capacity of each existing site over time. E.g. see `inputs/capacity_exogenous/wind-ons_exog_cap_reference_ba.csv` (within ReEDS repo)
* `{tech}_prescribed_builds.csv`: Capacity prescribed builds (2010 - present) with columns for region, year, capacity. This is the installed capacity in each year rather than cumulative capacity over time. E.g. see `inputs/capacity_exogenous/wind-ons_prescribed_builds_reference_ba.csv` (within ReEDS repo)
* `{tech}_.h5`: Hourly capacity factor profiles for each region/class. See `inputs/profiles_cf/cf_wind-ons_reference_ba.h5` within the ReEDS repository as an example. These files include datasets with column names (class|region) and an index with datetime and timezone information.
* `classification_{tech}.csv`: Tech classes based on capacity factors.

### Shared Drive Locations

Expand Down
61 changes: 0 additions & 61 deletions hourlize/resource.py
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're processing the existing capacity downstream now I think it might be good to remove the remaining pieces in here, otherwise some folks might wander in here and think this is still being used.

Original file line number Diff line number Diff line change
Expand Up @@ -771,46 +771,6 @@ def save_sc_outputs(
df_sc.to_csv(os.path.join(outpath, 'results', tech + '_supply_curve_raw.csv'), index=False)
#Round now to prevent infeasibility in model because existing (pre-2010 + prescribed) capacity is slightly higher than supply curve capacity
df_sc[['capacity','existing_capacity']] = df_sc[['capacity','existing_capacity']].round(decimals)
if existing_sites:
# bincol = ['sc_point_gid'] if exog_bin else []
df_exist = df_sc[df_sc['existing_capacity'] > 0].copy()
#Exogenous (pre-start-year) capacity output
df_exog = df_exist[df_exist['online_year'] < start_year].copy()
# Aggregate existing capacity to (i,rs,t)
if not df_exog.empty:
df_exog = df_exog[[profile_id_col, 'class','retire_year','existing_capacity']].copy()
max_exog_ret_year = df_exog['retire_year'].max()
ret_year_ls = list(range(start_year,max_exog_ret_year + 1))
df_exog = df_exog.pivot_table(index=[profile_id_col,'class'], columns='retire_year', values='existing_capacity')
# Make a column for every year until the largest retirement year
df_exog = df_exog.reindex(columns=ret_year_ls).fillna(method='bfill', axis='columns')
df_exog = pd.melt(
df_exog.reset_index(), id_vars= [profile_id_col,'class'],
value_vars=ret_year_ls, var_name='year', value_name='capacity')
df_exog = df_exog[df_exog['capacity'].notnull()].copy()
if(tech == 'egs' or tech == 'geohydro'):
df_exog['tech'] = tech + '_allkm_' + df_exog['class'].astype(str)
else:
df_exog['tech'] = tech + '_' + df_exog['class'].astype(str)
df_exog = df_exog[[profile_id_col,'tech','year','capacity']].copy()
df_exog = df_exog.groupby(['tech','year',profile_id_col], sort=False, as_index=False).sum()
df_exog = df_exog.sort_values(['year',profile_id_col]).round(decimals)
df_exog.to_csv(os.path.join(outpath, 'results', tech + '_exog_cap.csv'), index=False)
#Prescribed capacity output
if tech in ['wind-ons', 'wind-ofs']:
df_pre = df_exist[df_exist['online_year'] >= start_year].copy()
if not df_pre.empty:
df_pre = df_pre[[profile_id_col,'online_year','existing_capacity']].copy()
df_pre = df_pre.rename(columns={'online_year':'year', 'existing_capacity':'capacity'})
df_pre = df_pre.groupby([profile_id_col,'year'], sort=False, as_index =False).sum()
df_pre['capacity'] = df_pre['capacity'].round(decimals)
df_pre = df_pre.sort_values(['year',profile_id_col])
df_pre.to_csv(os.path.join(outpath, 'results', tech + '_prescribed_builds.csv'), index=False)
#Reduce supply curve based on exogenous (pre-start-year) capacity
if subtract_exog:
criteria = (df_sc['online_year'] > 0) & (df_sc['online_year'] < start_year)
df_sc.loc[criteria, 'capacity'] = (
df_sc.loc[criteria, 'capacity'] - df_sc.loc[criteria, 'existing_capacity'])

cfcol = 'capacity_factor_ac' if 'capacity_factor_ac' in df_sc else 'mean_cf'
df_sc_out = (
Expand Down Expand Up @@ -860,27 +820,6 @@ def copy_outputs(
os.path.join(inputspath, 'supply_curve', f'supplycurve_{tech}-{access_case}.csv')
)

#Prescribed builds and exogenous capacity (if they exist)
try:
shutil.copy2(
os.path.join(resultspath, f'{tech}_prescribed_builds.csv'),
os.path.join(
inputspath, 'capacity_exogenous',
f'prescribed_builds_{techlabel}_{access_case}.csv',
)
)
except Exception:
print('WARNING: No prescribed builds')

try:
df = pd.read_csv(os.path.join(resultspath,f'{tech}_exog_cap.csv'))
df.rename(columns={df.columns[0]: '*'+str(df.columns[0])}, inplace=True)
df.to_csv(
os.path.join(inputspath,'capacity_exogenous',f'exog_cap_{tech}_{access_case}.csv'),
index=False
)
except Exception:
print('WARNING: No exogenous capacity')

## Metadata
# rev configs
Expand Down
Loading
Loading