Skip to content

feat(validate): CPUC Baseline Generator List capacity benchmark (CPUC stack 5/5) - #800

Closed
ktehranchi wants to merge 1 commit into
feat/servm-demandfrom
feat/cpuc-benchmark
Closed

ktehranchi wants to merge 1 commit into
feat/servm-demandfrom
feat/cpuc-benchmark

Conversation

@ktehranchi

Copy link
Copy Markdown
Collaborator

Stacks on #798 (base set to feat/servm-demand; retarget after the stack lands). Gated by a new run.benchmark_cpuc flag — useful for any California run, independent of the SERVM demand profile.

Changes proposed in this Pull Request

A validate-stage benchmark comparing installed capacity against the CPUC Baseline Generator List (June 2026), per planning horizon:

  • workflow/scripts/benchmark_cpuc_baseline.py + rule benchmark_cpuc_baseline (runs on the assembled unsolved network — no solver needed). Outputs a long-format comparison CSV and a region × technology deviation heatmap PDF, surfaced through rule all when run.benchmark_cpuc: true.
  • Region attribution: EIA publishes all CAISO plants as a single CISO BA (no PGE/SCE/SDGE split), so the benchmark compares CAISO-total (CPUC PGE+SCE+SDGE, which per the CPUC dictionary is the CAISO definition) plus individual non-CAISO regions (LDWP→LADWP, IID→IID, BANC+TIDC→NCNC), declared in repo_data/CPUC/servm_benchmark_regions.csv. Model side filtered to CA (EIA's CISO also spans the NV VEA footprint).
  • Technology mapping: both sides map into common compare categories (repo_data/CPUC/servm_tech_map.csv) — SERVM's 27 tech categories are finer than PyPSA carriers. Unmapped techs on either side surface as explicit UNMAPPED: rows, never dropped; gas Cogen/CHP is kept CPUC-only (no CHP concept in the model — expect the model long in the gas buckets and short ~3.7 GW in Cogen, documented).
  • Vintage filtering mirrors add_electricity.load_powerplants exactly on the model side — which makes visible (by design) that the model pins existing plants to a 2100 retirement and does not honor announced retirements. Note also a genuine boundary case in the CPUC data: both Diablo Canyon units carry RetireDates of 2029-10/2030-10 and therefore exit CPUC's own 2030 column.

Real-workbook sanity (2030, CA): CAISO 103.0 GW / LADWP 10.3 / NCNC 6.7 / IID 2.9; e.g. Battery 25.0 GW, Solar 30.8 GW, Gas CC 23.1 GW totals — table in the script's tests/docstring lineage.

Testing

  • Tier A pytest -m fast: 195 passed, 1 pre-existing skip (combined stack).
  • New tests: test_benchmark_cpuc_baseline.py22 passed (header layout, vintage boundary cases, unmapped-tech reporting, CAISO roll-up; plus a guard asserting the tech map covers every carrier in constants.EIA_TECH_MAP so new carriers can't silently vanish).
  • DAG dry-runs: rule appears with the flag on (29 jobs), absent with it off (27), absent for default config; --until benchmark_cpuc_baseline pulls the workbook retrieve rule.
  • Tier B not run for this PR. No equivalence run for this branch (flag defaults false; no existing outputs change).

Checklist

  • I tested my contribution locally and it seems to work fine.
  • pre-commit and pytest -m fast pass locally.
  • Code and workflow changes are sufficiently documented.
  • Changed dependencies are added to workflow/envs/environment.yaml (no dependency changes).
  • Changes in configuration options are added in workflow/repo_data/config/config.default.yaml.
  • Changes in configuration options are also documented in docs/source/configtables/*.csv.

🤖 Generated with Claude Code

Compares installed capacity between the CPUC Baseline Generator List and
the model's generator fleet, aggregated by SERVM benchmark region and
technology category, per planning horizon. Emits a long-format comparison
CSV and a per-horizon deviation heatmap under the run's figures/benchmark/
directory. Gated by the new run.benchmark_cpuc flag (default false), which
is independent of the demand profile: the fleet benchmark is useful for any
California run.

Region attribution: EIA reports every CAISO plant under the single BA code
CISO, so powerplants.csv cannot separate PGE from SCE from SDGE. The
benchmark therefore runs at the coarsest resolution both sides support —
CAISO (= PGE + SCE + SDGE), LADWP (LDWP), IID, NCNC (BANC + TIDC) — with
the collapse stated in repo_data/CPUC/servm_benchmark_regions.csv. The
model side is restricted to state == CA so EIA's CISO code does not drag in
the Nevada CISO-VEA footprint.

Technology attribution: repo_data/CPUC/servm_tech_map.csv maps both SERVM
tech categories and PyPSA carriers into a shared compare_category. Anything
unmapped on either side becomes an explicit UNMAPPED:<name> row rather than
being dropped.

Vintage/retirement filtering is shared by both sides and mirrors
add_electricity.load_powerplants, so the benchmark measures the fleet the
model actually builds.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ktehranchi

Copy link
Copy Markdown
Collaborator Author

Superseded by #801 — the five-PR stack was collapsed into a single PR at the author's request. All commits (plus Tier B fixes) are on feat/cpuc-servm-california.

@ktehranchi ktehranchi closed this Aug 30, 2026
@ktehranchi
ktehranchi deleted the feat/cpuc-benchmark branch August 30, 2026 01:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant