feat: California configuration on CPUC SERVM 2026 demand (weights, demand profile, import caps, CA config, benchmark) - #801
Merged
Conversation
Builds a fractional per-(SERVM region, cluster bus) allocation table so CPUC
SERVM demand for the six California load regions can be disaggregated onto the
simplified network.
The shares are derived by composing the two busmaps (base bus -> substation ->
{simpl} cluster bus) against elec_base_network.nc rather than reading a bus
column of the clustered network: aggregate_to_substations drops
`balancing_area` from the buses, and a cluster can straddle two SERVM regions
anyway (LA County holds both LDWP and CISO-SCE buses). The base network is the
last stage that still carries both `balancing_area` and the population-based
`load_weight`, so the composition recovers where each base bus's weight lands.
A straddling cluster therefore appears once per region it overlaps, each row
carrying only that region's share, and the factors sum to 1.0 within a region.
Balancing areas that are deliberately out of scope (CISO-VEA, Nevada
footprint) keep a row with an empty region in servm_region_map.csv, so an
unknown balancing area introduced by upstream relabeling still hard-fails
while the excluded one is dropped with a logged load share.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…p, PR 3) Rewires the two dead config keys `model_topology.interface_transmission_limits` and `electricity.transmission_interface_limits`: the RESOLVE interface table is now applied as a per-snapshot MW cap on the aggregate flow across each interface. Scope is the electricity import/export Links added by add_extra_components, so the constraint is inert when trade is disabled. Region_2 entries that are inside the network contribute no trade links, so internal AC lines escape the cap; that understatement is documented in the module, not corrected here. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds `electricity.demand.profile: servm`, wiring CPUC SERVM 2026 hourly load
for the six California load regions (IID, LADWP, NCNC, PGE, SCE, SDGE) through
the existing read/write demand strategies.
Retrieve: `retrieve_cpuc_servm_load` pulls one ~118 MB CSV per forecast year
(2026-2045) from files.cpuc.ca.gov, and `retrieve_cpuc_baseline_generators`
pulls the CAISO baseline generator list, both through a thin
`retrieve_cpuc_data.py` downloader patterned on `retrieve_eer_data.py`.
ReadServm parses the three-row header positionally: the seventh index column
("Hour of Day") carries the stray ('Region', 'Unit Type') labels, so the
calendar block cannot be found by testing the upper header levels for
blankness. Every published component is kept on the `subsector` index level so
the zonal artifact stays component-resolved; only `Net Load` reaches the model,
selected by a new `ReadStrategy.default_subsector` hook that `Context` applies
so `main()` stays profile-agnostic. Files are indexed by the forecast year
parsed from each basename rather than by list order. Strips are fixed PST
(UTC-8, no DST -- verified against the BTMPV solar-noon centroid, Dec 12.52 vs
Jul 12.68) and are rolled to UTC with the same `np.roll` convention ReadEer
uses for CST.
Two calendar misalignments are accepted and documented on
`ReadServm._assign_snapshots`:
1. SERVM lays its hours on a synthetic Monday-start calendar, so weekday and
weekend hours do not line up with the real weekdays of the planning horizon.
2. For a leap *weather* year the strip contains February 29 and omits
December 31, while the model snapshots do the opposite, so every hour after
February lands one calendar day earlier than it sat in the source file.
The 8760-hour strip is therefore mapped positionally onto the network's own
per-period snapshots rather than onto a synthesised `date_range`, which would
run a day short of December 31 for the leap planning horizons (2028/2032/2040)
because `get_snapshots` drops February 29 from them.
WriteServm disaggregates by matrix product against the PR-1 weights table
pivoted to (region x bus), rather than the base class's one-zone-per-bus
mapping: a cluster bus can straddle two SERVM regions (LA County holds both
LDWP and CISO-SCE buses), and the product gives such a bus the sum of its share
of every region it overlaps.
`build_electrical_demand` gains a second output, the reader's zonal
component-resolved demand written before disaggregation. It is produced for
every profile (for efs/eer it is simply the single subsector="all" slice), and
is captured off the `Context` rather than re-read.
Config: `electricity.demand.scenario.servm_weather_years` selects the weather
year from the stacked 2000-2024 record. A single-entry list is deterministic;
multiple entries raise NotImplementedError pending stochastic scenarios. A
value differing from the top-level `renewable_weather_years` logs a warning.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ubs (PR 4) Ties the SERVM demand profile (PR 2) and the aggregate interface transmission limits (PR 3) together into a runnable, maintained California entry point. New `workflow/repo_data/config/config.california.yaml`: CA-only Western model on CPUC SERVM 2026 demand, at REeDS-zone resolution (clusters must be 4 — the ReEDS backbone cannot cluster below the four CA zones p8/p9/p10/p11), with planning horizons restricted to SERVM forecast years (2030/2035/2040/2045), the RESOLVE CAISO interface caps switched on, and imports/exports enabled at a 25%-of-demand annual volume limit. A clearly-marked commented alternative carries the county-resolution variant (clusters: 58, simpl: 'county'); the county NARIS flowgate file is selected automatically by `add_extra_components` from `topological_boundaries`. Phase-2 hook `conventional.ambient_derate` is stubbed in both the California and default configs and guarded in `add_electricity`: enabling it raises NotImplementedError rather than silently doing nothing. The comments record that when implemented it REPLACES the EIA-860 seasonal derate and must never stack with it or with a UCAP-derated capacity credit. `retrieve.smk` reserves `retrieve_cpuc_thermal_derate` for the profiles it will need. Tests: `config.test.california.yaml` is a SERVM variant of the CA test harness config. It keeps a full 8760-hour snapshot year — `ReadServm._assign_snapshots` maps its hourly strips positionally onto the network's own per-period snapshots and raises unless the horizon carries exactly 8760, so a truncated window cannot build SERVM demand. Three dry-run cases are added (california -> cluster_network, california -> solve_network, and a county-mode case driven by `--config` overrides), and `tests/integration/test_servm_demand_artifacts.py` asserts weight conservation, demand/network alignment, and the PST->UTC roll against a `--until add_demand` build. Docs: a SERVM section in data-demand.md (source URL pattern, the nine forecast years, the six regions and their balancing-area mapping including the CISO-VEA exclusion, Net-Load-only dispatch with the component-resolved zonal artifact, weather-year semantics, fixed PST, and both calendar caveats); the interface constraint documented in model-constraints.md and data-transmission.md including the p8 gap; the `transmission_interface_limits` config row corrected from "not currently consumed"; a California example in config-spatial.md; and a release-notes entry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Compares installed capacity between the CPUC Baseline Generator List and the model's generator fleet, aggregated by SERVM benchmark region and technology category, per planning horizon. Emits a long-format comparison CSV and a per-horizon deviation heatmap under the run's figures/benchmark/ directory. Gated by the new run.benchmark_cpuc flag (default false), which is independent of the demand profile: the fleet benchmark is useful for any California run. Region attribution: EIA reports every CAISO plant under the single BA code CISO, so powerplants.csv cannot separate PGE from SCE from SDGE. The benchmark therefore runs at the coarsest resolution both sides support — CAISO (= PGE + SCE + SDGE), LADWP (LDWP), IID, NCNC (BANC + TIDC) — with the collapse stated in repo_data/CPUC/servm_benchmark_regions.csv. The model side is restricted to state == CA so EIA's CISO code does not drag in the Nevada CISO-VEA footprint. Technology attribution: repo_data/CPUC/servm_tech_map.csv maps both SERVM tech categories and PyPSA carriers into a shared compare_category. Anything unmapped on either side becomes an explicit UNMAPPED:<name> row rather than being dropped. Vintage/retirement filtering is shared by both sides and mirrors add_electricity.load_powerplants, so the benchmark measures the fleet the model actually builds. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- servm_region_map.csv: PACW/WALC/NEVP added as known-but-excluded rows —
CA slivers served by non-CPUC-region BAs (0.48% of CA load weight);
their load lives in CPUC's forthcoming non-CA regional files.
- conftest: --rerun-triggers mtime so a fresh worktree sharing data/ via
symlink does not re-download (and rewrite) existing retrieve outputs.
- config.test.california.yaml / config.california.yaml: add the solving
block validate.smk reads at parse time, costs.aeo.scenario, and the
godeeep renewable_land_access pin; test solver gurobi for the
distribute_clusters QP (fixture never reaches solve_network).
- test_row_count_matches_snapshots: compare against the demand-attached
network — elec_s{simpl}.nc still carries PyPSA's default ['now'] index.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Tier B (pytest -m integration) had never actually run — CI was silently
skipping it — and its first execution surfaced four latent defects:
- config.test.yaml could not even parse: validate.smk reads
config["solving"] at Snakefile parse time. Added the minimal solving
block (+ costs.aeo.scenario, renewable_land_access pins).
- build_cost_data.py: the AEO scenario fallback was "Reference", but the
PUDL parquet's model_case_eiaaeo vocabulary is lowercase ("reference")
-> empty frame and IndexError whenever a config omitted the key.
- godeeep_wind_height shipped as "_100m", a hub height with no published
Zenodo record (only wind_125m_* exist) -> downloader returns None and
xarray fails with a misleading "no IO backend" error. Config value and
script fallback both set to "_125m".
- test_bus_count_reasonable expected O(50) substations, but on the ReEDS
transport path aggregate_to_substations keeps TAMU granularity
(CA-only Western ~2,000 buses); bound widened with an explanatory note.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Aug 30, 2026
Closed
The meaningful fleet-vs-fleet comparison is today's plants against today's baseline: at 2026 every unit on both sides exists, so there is no forecast ambiguity — and the CPUC list is a baseline (near-static to 2045), so later horizons only measure model expansion against it. - rule benchmark_cpuc_baseline no longer consumes the assembled network: it needs only powerplants.csv and the CPUC workbook, so enabling run.benchmark_cpuc never triggers a model build (2-job DAG). Outputs move to results/<run>/cpuc_benchmark/ without cluster wildcards; the network reconciliation stays available as a library function. - new run.benchmark_cpuc_horizons key (default [2026]) decouples the benchmark year from scenario planning_horizons; empty list falls back. - drop a mid-file `from __future__` import that broke under snakemake's script preamble injection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The CPUC Baseline benchmark exposed the mechanism: pudl_path pinned v2025.5.0, whose most-recent filings freeze the fleet at the 2023/24 EIA-860 vintage — missing exactly the fastest-growing CAISO categories (the 2026 benchmark was short 7.9 GW of battery and 3.9 GW of solar). v2026.8.0 carries filings through report_date 2026-01; the existing most-recent-non-null query picks them up with no schema changes (verified column-complete on S3). data_year (the EIA-923 heat-rate / fuel-cost window) moves 2023 -> 2025, the latest complete year. CAISO 2026 benchmark after the bump: total gap -13.0% -> -8.3% (battery -7.9 -> -5.7 GW, solar -3.9 -> -2.1 GW); thermal deltas are unchanged, consistent with those being classification, not vintage. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…lants EIA leaves nerc_region NULL for plants that first appear in a recent 860 vintage, and add_electricity maps nerc_region -> interconnect, so the plain isin() filter silently deleted new-build (2.2 GW of CA renewables under PUDL v2025.5.0; 204 plants / 7.3 GW nationally under v2026.8.0). Found by unit-level RCA against the CPUC Baseline Generator List. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
load_powerplants pinned every existing/proposed unit to a 2100 retirement, discarding EIA's planned_generator_retirement_date — the CPUC benchmark showed this keeps ~2.9 GW of retired CAISO OTC steam (Ormond Beach, Alamitos, Huntington Beach) in the fleet forever. - add_electricity.load_powerplants: existing/proposed units now retire on their announced date where one exists (config gate electricity.honor_planned_retirements, default true). Applies at the first investment period; post-clustering periods aggregate build_year/lifetime capacity-weighted, so per-period exogenous retirement inside a cluster remains out of scope (documented). - build_powerplants: planned_generator_retirement_date now takes the LATEST filing's value as-is instead of most-recent-non-null — a newer NULL means the announcement was withdrawn (Diablo Canyon post-SB846 carried a stale 2025-08 date from an old filing, which would have retired Unit 2 spuriously). - benchmark_cpuc_baseline.prepare_model_plants mirrors the new rule. - New unit tests for the retirement semantics; benchmark test updated. CAISO 2026 benchmark: Gas CT/ICE/Steam +5,720 -> +4,387 (Alamitos + Huntington Beach out at 2026; Ormond Beach stays until its current EIA date of 2027-01 — a genuine EIA-vs-CPUC source disagreement, no longer a model artifact). Tier A 200 passed, Tier B 32 passed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CPUC's SERVM regions are contractual ledgers: they carry AZ/NV/UT/BC/ Mexico resources (Palo Verde share, Intermountain, Apex, Hoover shares, Powerex dynamic hydro, Mexicali CCs, Energia Sierra Juarez, Desert Star, and ~2 GW of out-of-state solar/battery contracts) under California region labels. A physically-located model can never carry these, so the benchmark scored 10.9 GW of them as model shortfall. - repo_data/CPUC/servm_out_of_state_units.csv: 74 units, 10,873 MW, each verified at unit level (EIA plant ids / CAISO queue / physical county), with evidence strings. - benchmark_cpuc_baseline: the listed units leave the scored regions and surface on an "EXCLUDED: out-of-state contracted" row per category (kept out of the deviation heatmap, like the RECONCILE rows); stale list entries warn loudly instead of silently un-excluding. CAISO 2026 after exclusion: scored CPUC 95.2 GW vs model 93.9 GW (-1.4%, from -9.5%). Nuclear lands at +23 MW (Diablo nameplate delta), LADWP Gas CC at +13 MW. Known residuals now stand exposed instead of being offset: Solar +1,105 (model-only small plants), Gas CC +2,442 (cogen reclassification + nameplate convention), Battery -3,744 (real CODs 2027-29 behind CPUC 2026 dates + ~1.2 GW with no EIA identity). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
conventional.unit_commitment: true for the CA configs (linearized UC —
the solve stays an LP). Making the flag real surfaced three latent
defects, all fixed:
- p_min_pu was computed in the unit_commitment branch of
attach_conventional_generators but never passed to n.add — minimum
stable levels were silently dropped.
- PyPSA's up_time_before=1 default forces every committable unit online
into the first snapshot; with min_up_time > 1 that pins minimum load
onto the January-night load trough and is infeasible without load
shedding (reproduced on a toy). Committable units now start with
up_time_before=0.
- min_up/min_down are in snapshots and ramps are per-snapshot in PyPSA,
but the data is hours / pu-per-hour. prepare_network now rescales them
to the {opts} temporal resolution (no-op when nothing is committable).
Data sanitation (build_powerplants.sanitize_uc_parameters): per-carrier
UC_BOUNDS keep in-band ADS values untouched and clamp/fill only the
physically impossible — absolute-$ start costs imputed onto sub-MW units
(up to $1.03M/MW), 108h min-up on OCGT peakers, p_min_pu up to 31x
nameplate, $0 geothermal cycling, and NaNs that PyPSA reads as
infinitely flexible. 17,749 clamped / 6,087 filled across 17,502
committable rows, logged per carrier; non-committable rows verified
untouched. apply_ucap now scales committable p_min_pu alongside
p_max_pu so the feasibility invariant survives.
Clustering strategies: start_up_cost capacity_weighted_average -> sum
(the aggregate acts as one unit; the average understated cycling cost
~N-fold) and ramp limits max -> capacity_weighted_average.
40 new UC tests; fast tier 241 passed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
cluster_simpl shipped resolve_simpl_mode() and build_county_busmap() but main() never called them — it int()'d the wildcard, so simpl='county' crashed with ValueError at runtime (the county-mode DAG dry-run test could not catch an execution-time failure). The county branch now passes the county-FIPS busmap through clustering_for_n_clusters' custom_busmap path: no k-means, no cluster-count QP. Verified: CA-only western at simpl='county' yields the expected 58 county buses (e.g. p10_p06037 = Los Angeles). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two defects in add_elec_imports_exports, exposed by the first county-
resolution run with trade enabled:
- the exports call omitted zone_col, silently falling back to the
"reeds_zone" default. On county networks every county then fails the
zone test and boundary rows get their inside/outside roles scrambled
(an Arizona county classified as inside).
- the cost cache checked `zone_outside not in costs` but wrote
costs[zone_inside]; once an earlier row's inside zone matched a later
row's outside zone, the write was skipped and costs[zone_inside]
raised KeyError ('p04012', La Paz County AZ).
reeds_zone runs were unaffected (default coincided; key collision not
triggered at that cardinality).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… CA buses CPUC attributes ~10.9 GW of physically out-of-state resources to CA regions. New opt-in option (default off; on for the CA configs) attaches the 65 units with EIA identities as generators/storage at CA buses: - unit <-> EIA matching reused from repo_data/CPUC/servm_out_of_state_units.csv (the benchmark exclusion ledger); techno-economics (heat rates, costs, derates, sanitized UC params) come from the same PUDL powerplants build, captured before the CA footprint filter drops out-of-state plants. - p_nom = min(contracted Capmax, live plant capacity); bus = max-LAF bus of the unit's SERVM region from the servm_load_weights artifact. - batteries attach as StorageUnits (max_hours from EIA MWh, sqrt(eta) convention matching attach_battery_storage); remote VRE copies the attachment bus's same-carrier profile; firm units get the identical seasonal derate treatment applied to their own columns only. - rows with no EIA identity (Mexicali CCs, Powerex BC hydro, ESJ Baja wind, Tahoe/NV entitlements - 1,923 MW) are skipped with a summary warning and remain represented by the import machinery. - non-extendable by construction (contracted shares, not expansion candidates). Attach summary for the CA config: 8,818 MW across solar 3,546 / CCGT 2,117 / battery 1,715 / nuclear 635 (Palo Verde) / hydro 592 (Hoover) / OCGT 108 / geothermal 105. 14 new tests; 99 targeted tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The land-access refactor (#745) left the compressed historical records with a single weather year (2012), stranding the pre-#745 bus-aggregated archives (solar 1980-2022, wind 100m 2001-2022; Zenodo 18293999 / 18331699) that cover every other year. This restores them as a fallback: - historical years with a compressed record (2012) route through the screened NREL land-access path unchanged; other years download the aggregated archive and remap substation-keyed profiles through busmap_s{simpl} with a caps-weighted mean (verified end-to-end on the real 2019 solar file: 1,972 in-footprint substations -> 75 clusters, schema identical to the compressed path). - HARD GATES per issue #803: explicit opt-in (godeeep_allow_unscreened_fallback, default false); hard error when any renewable carrier is extendable (unscreened profiles must never drive siting against screened p_nom_max); hard error when a run mixes screened and unscreened years (checked for every historical run, not just fallback-routed ones - a [2012, 2019] config previously slipped through); provenance attrs (land_access, hub_height, source) stamped on the output .nc; loud three-caveat warning. - NREL caps remain the p_nom_max source (weather-year independent); the screened-caps/unscreened-profile inconsistency is documented and rendered moot by the extendable gate. 52 new tests. Wind archive structure inferred from the old code + solar file; first real wind fallback run is the confirmation. Refs #803. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
for more information, see https://pre-commit.ci
…bility - new docs/source/california-model.md: definitive CA reference — data inventory (SERVM demand, PUDL v2026.8.0 fleet, GODEEEP renewables, NARIS/RESOLVE trade limits, CPUC benchmark + out-of-state ledger, remote contracted resources), demand weather-year options (2000-2024 via servm_weather_years), spatial resolutions, UC defaults, caveats. - renewable weather-year availability stated precisely per path: screened compressed = 2012 only (125m wind); unscreened aggregated fallback (opt-in, #803 gates) = solar 1980-2022 / wind 2001-2022 (100m); climate scenarios = planning-horizon-indexed 2030/2040/2050. Stale "one historical year" claims replaced in data-generators.md and config-configuration.md. - fixed the self-contradictory land-access ordering (empirically limited < reference < open, from the avail_* artifacts). - data-demand.md: excluded-BA list corrected (VEA + PACW + WALC + NEVP). - config.california.yaml: warn that rcp85cooler has no 2035/2045 CF records (documented landmine; resolution deferred). - tests/docs NREL_EXCLUSION key set updated (pre-existing failure). Sphinx builds clean; tests/docs 39 passed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… Mateo
assign_missing_regions overwrote ALL seven region attributes whenever a
bus was missing ANY of them. The Trans Bay Cable's Potrero terminal
(sub 36177) resolves its county correctly to p06075 (San Francisco) but
sits in a reeds_shapes gap, so the whole-row copy replaced its county
with the nearest complete bus's — San Mateo. SF's only HVDC infeed
(375 MW, from the TAMU dcline data, present and correctly rated all
along) delivered one county south of reality; 11 of 4,248 western buses
were misassigned by the same mechanism.
Fix: fill only the attributes that are actually NaN. Six regression
tests, including the Potrero case.
Empirical confirmation (county-58 WY2019 operational run): the cable now
terminates at p06075 and load shedding collapses 805.4 -> 25.4 MWh
(19 -> 2 hours, peak 125.8 -> 17.3 MW); the SF evening-ramp deliverability
bind is resolved. Found by an HVDC-modeling audit of the shedding result.
Related follow-up (logged, not fixed): the ReEDS DC ITL files
(transmission_capacity_init_nonAC_{ba,county}.csv - PDCI, IPP DC) exist
in repo_data and are consumed nowhere; irrelevant for CA-only runs but a
real gap for western/usa transport-model runs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…mark
The EXCLUDED aggregate row becomes per-benchmark-region rows
("EXCLUDED: OOS CAISO" etc.) carrying BOTH the CPUC ledger MW and the
share the remote_contracted_resources option can attach on the model
side (ledger rows with an EIA identity whose plant is live at the
horizon, capped at the contracted Capmax; multi-plant entitlements like
Hoover's "154;8902" are summed). A new third output,
cpuc_capacity_composition.pdf, renders stacked in-state + out-of-state
bars per region for both sides.
CA 2026: ledger 10,873 MW, model-attachable 8,946 MW; the 1,927 MW
residual is exactly the no-EIA-identity rows (Mexicali CCs, Powerex BC
hydro, ESJ Baja wind, Tahoe NV, Fish Lake) that stay with the import
machinery.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Resolves conflicts from #806 (GODEEEP CF registry, no fallback paths). Resolution policy: the registry wins everywhere it overlaps — - build_renewable_profiles.py / zenodo_downloader.py: theirs wholesale; the interim unscreened aggregated fallback (and its opt-in flag, configtable row, docs and 52 tests) is retired. Screened historical years 1980-2022 now come from the registry's SHA256-verified mirror, which resolves #803 properly. - config.common.yaml: registry semantics for godeeep_wind_height (no default); our PUDL v2026.8.0 pin and nrel_caps_reassign block kept. - docs: availability rewritten registry-era in data-generators.md, config-configuration.md and california-model.md (source × years matrix; every SERVM demand year 2000-2022 now pairs with a screened profile of the same year). - config.california.yaml switched to renewable_scenarios: ['historical'] + WY2019: the registry validates configured (dataset, year) pairs at parse time, and the previous rcp85cooler pairing with 2035/2045 horizons (a documented landmine) became a hard parse failure. Verified parsing (35-job data_model DAG). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
for more information, see https://pre-commit.ci
Adapts the California/SERVM feature set to the auto-loaded layered base and schema validation from #794: - schema: admit our config surface in the closed sections — run.benchmark_cpuc / benchmark_cpuc_horizons; electricity.honor_planned_retirements, remote_contracted_resources, demand.profile enum += servm, demand.scenario.servm_weather_years; conventional.ambient_derate. - config.california.yaml and config.test.california.yaml rewritten as SPARSE OVERLAYS (programmatic deep-diff against the merged base); the test overlay drops its now-redundant solving/costs/land-access blocks and its stale pre-fix clustering strategies, inheriting the corrected defaults (start_up_cost: sum, capacity-weighted ramps). - policy_constraints paths follow the tracked repo_data/config/ layout (solve rules' interface_limits input, configtables, conftest, docs). - test_dag_dryrun: their repo_data paths + our California cases (incl. county-mode override) as a union. - configtables: their reference-default land-access row (ordering now consistent); our un-deadified transmission_interface_limits description kept, path-updated. Verified: both CA overlays parse against the schema (35-job / 26-job data_model DAGs); fast tier 350 passed, docs 41 passed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ate ledger CMRRON_2_CMNWD1 (300 MW) is the third Energia Sierra Juarez phase in La Rumorosa, Mexico -- contracted to Silicon Valley Power and delivered over the existing ESJ cross-border 230 kV line. No EIA-860 record exists (Mexican plant), so it is excluded from scoring rather than attachable. CAISO Wind raw benchmark delta moves from -256 MW to +44 MW. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes and collapses the five-PR stack #796 / #797 / #798 / #799 / #800 into a single PR (author's request). Commit history preserves the original PR boundaries.
Changes proposed in this Pull Request
A California-only PyPSA-USA configuration driven by the CPUC 2026 SERVM update, with selectable demand weather year (2000–2024) per run:
load_weight; exact for clusters straddling regions and agnostic tosimpl/topological_boundaries(county resolution works unchanged). CA slivers served by non-CPUC-region BAs (CISO-VEA, PACW, WALC, NEVP; 0.48% of CA load weight) are explicitly excluded and logged.profile: servmdemand — retrieve rules for the nine per-forecast-year CPUC CSVs (2026–2045) + Baseline Generator List;ReadServm(3-row-header parse with format-drift guards, weather-year filter viaelectricity.demand.scenario.servm_weather_years, fixed-PST→UTC shift verified empirically, leap-safe positional snapshot mapping);WriteServmmatrix-product disaggregation; every SERVM load component preserved in a new zonal parquet artifact for later component-level modeling (phase 2/3 feedstock).transmission_interface_limits.csv/model_topology.interface_transmission_limitspair into a per-snapshot cap on import/export link flows (RESOLVE CAISO_Imports 9,728/10,208 MW), alongside the existing per-flowgate NARIS caps and percent-of-load volume limits.config.california.yaml— maintained CA config:western+include: {reeds_state: ['CA']}, reeds_zone/4-cluster default with a documented county-mode alternative,planning_horizons: [2030, 2035, 2040, 2045], imports/exports enabled (wholesale costs, capacity + 25%/yr volume caps), interface limits on. Plusconfig.test.california.yaml+ integration tests, docs across data-demand/model-constraints/config-spatial/configtables, and phase-2 (conventional.ambient_derateguard) / phase-3 (multi-weather-yearNotImplementedError) stubs.run.benchmark_cpuc) comparing model fleet vs CPUC baseline by benchmark region × technology category per horizon (CSV + deviation heatmap); unmapped techs/regions surface explicitly, never dropped.pytest -m integrationactually executable surfaced four pre-existing develop defects, fixed in the final commit:config.test.yamlcould not parse (missingsolvingblock), the AEO fuel-cost scenario fallback used the wrong case ("Reference"vs the parquet'sreference),godeeep_wind_heightshipped as"_100m"(no Zenodo record exists for that height — wind profiles could not build), and an unvalidated substation-count bound.Testing
pytest -m fast: 195 passed, 1 pre-existing skip.pytest -m integration: 32 passed — first real execution of this tier; builds the CA test slice throughcluster_networkand the SERVM chain throughadd_demand(downloads one ~118 MB CPUC CSV).pre-commitclean on all changed files. No equivalence run for this branch (new opt-in profile/config; the only behavior changes to existing paths are the Tier B defect fixes in commita13d7ee, of whichgodeeep_wind_height: _125mchanges renewable profiles for any config that relied on the broken shipped value — which could not have built at all).Checklist
pre-commitandpytest -m fastpass locally.workflow/envs/environment.yaml(no dependency changes).workflow/repo_data/config/config.default.yaml.docs/source/configtables/*.csv.🤖 Generated with Claude Code