Skip to content

Mechanically-safe cleanups — 57 items, no behaviour change #782

Description

@ktehranchi

Mechanically-safe cleanups — 57 items, no behaviour change

Split out of the bulk cleanliness review (#780 and #781) because every item here is
provably no behaviour change — mostly deletions of unreachable code, plus a handful of one-line corrections
where the current code cannot execute the branch it appears to offer.

These are good onboarding tasks: each is self-contained, each has a concrete file and line, and none of them can
alter model results. Roughly 2,900 lines are deleted across the dead-code items — about 6% of the Python in
the repo.

57 findings — 19 whole-modeling-domain, 28 power model, 10 sector coupling.

Important

Five of these are critical despite being safe. They are one-line fixes, but three of them are currently
producing wrong output or a hard crash. Please take these before the rest of the list — do not treat them as
ordinary good-first-issue material.

/plot_statistics.py:1 — Repo-root directory named " " holds 3 committed scripts; 2 are stale forks of live code

~100 KB of shadow copies of three of the most-edited scripts in the repo, committed by an unquoted-variable cp/mv in #737. Two are stale forks diverging by hundreds of lines from the live files. Grep, IDE search, and find all surface them alongside the real ones, and the space-prefixed path breaks naive tooling (an xargs awk over git ls-files errors with can't open file /plot_statistics.py). Anyone who opens the wrong copy edits code that is never executed.

Evidence
$ git ls-files --stage | head -3
100644 97b8a5ba3489b5404bd5d75e965f999571c75139 0	 /plot_statistics.py
100644 5aaa6f4a6e39c6743b0c7f0d3c6cd9b5d2060f68 0	 /policy.py
100644 b6a58c891e06e0ec5eaed55c1593dfbaf93a4fc9 0	 /solve_network.py

$ ls -la
drwxr-xr-x@  5 kamrantehranchi  staff     160 Aug 23 13:11  

$ diff -q " /policy.py" workflow/scripts/opts/policy.py   -> IDENTICAL
$ diff " /plot_statistics.py" workflow/scripts/plot_statistics.py | grep -c '^[<>]'   -> 285
$ diff " /solve_network.py" workflow/scripts/solve_network.py | grep -c '^[<>]'      -> 261

$ git log --oneline --diff-filter=A -- " /plot_statistics.py"
363631ee Hdk egs seismic risk mask (#737)

Fix: git rm -r " " and delete the on-disk directory. Confirm first that the two diverging copies contain no unmerged work (diff " /solve_network.py" workflow/scripts/solve_network.py), then remove. Nothing in the repo imports or references these paths (git grep for them returns only the tree entries).

Risk: safe — provably no behaviour change

workflow/scripts/build_co2_storage.py:10 — build_co2_storage is called with logger=None but dereferences logger on its first statement

The module never defines a module-level logger, so the only call site passes the literal None. Line 10 runs unconditionally before any work, so rule build_co2_storage (workflow/rules/build_sector.smk:201, consumed by add_sectors via build_electricity.smk:932 when co2.storage is enabled) dies with an AttributeError that names nothing about CO2 storage. The defensive if logger is not None at line 59 shows someone knew None was possible and guarded the wrong (last) call instead of the first. The other three logger.info calls at lines 51/53 have the same defect.

Evidence
L8-10:
def build_co2_storage(regions_onshore_geojson, co2_storage_geojson, output_csv, logger):
    # get PyPSA-USA network nodes and CO2 storage information at a county level
    logger.info("Calculate CO2 storage potentials and costs")

L59-60:
    if logger is not None:
        logger.info(f"Save CO2 storage potentials and costs into CSV file '{output_csv}'")

L73-77:
    build_co2_storage(
        snakemake.input["regions_onshore"],
        snakemake.input["co2_storage"],
        snakemake.output["co2_storage"],
        None,
    )

Fix: Drop the logger parameter entirely and declare logger = logging.getLogger(__name__) at module scope, matching every other script in this domain (build_shapes.py, build_powerplants.py, build_fuel_prices.py). Delete the if logger is not None guard at line 59.

Risk: safe — provably no behaviour change

workflow/scripts/build_renewable_profiles.py:280 — renewable.dataset dispatch has no else branch; an unrecognised value yields NameError 250 lines later

renewable.dataset is a free-text user config key (workflow/config/config.common.yaml:6 dataset: godeeep #atlite or godeeep). Set it to Godeeep, atlite with a trailing space, or delete it, and neither block runs; profile, capacities, p_nom_max, potential, average_distance are all unbound and line 540 raises NameError: name 'profile' is not defined. Nothing in the message mentions the config key or the two legal values, so the user debugs an xarray merge instead of a one-character typo. The same .get("dataset", False) expression is re-evaluated at lines 551 and 561 — each an opportunity for the branches to drift out of sync.

Evidence
L280:
    if snakemake.params.renewable.get("dataset", False) == "atlite":

L430:
    if snakemake.params.renewable.get("dataset", False) == "godeeep":

L537-547:
    # ds of renewable data to be outputted
    ds = xr.merge(
        [
            profile.rename("profile"),
            capacities.rename("weight"),
            p_nom_max.rename("p_nom_max"),
            potential.rename("potential"),
            average_distance.rename("average_distance"),
        ],
        compat="override",
    )

Fix: Read dataset = snakemake.params.renewable.get("dataset") once near line 252, validate it immediately (if dataset not in ("atlite", "godeeep"): raise ValueError(...) naming the key and the legal values), then branch if dataset == "atlite": ... elif dataset == "godeeep": ... and reuse the local at lines 551/561.

Risk: safe — provably no behaviour change

workflow/scripts/plot_statistics_sector.py:200 — Three plot functions filter by module-global month_i instead of their own month parameter

The functions read as if they filter on the caller-supplied month; they actually read a loop variable that lives in another scope. Today it happens to be correct only because :1606 and :1678 do by_month_kwargs["month"] = month_i right before each call. Any other caller — a test, a notebook, a future rule that passes month=3 — gets whichever month the last __main__ loop happened to leave behind, or a bare NameError if the loop never ran. The figure is still written and titled with the requested month, so the failure mode is a correctly-labelled chart of the wrong month's data.

Evidence
plot_statistics_sector.py:196-200 (also identical pattern at :295-296 and :1027-1028):
for row, period in enumerate(investment_periods):
    df = df_all.loc[period]

    if month:
        df = df[df.index.get_level_values("timestep").month == month_i]
`month` is the declared parameter (line 167: `month: int | None = None,`). `month_i` is never assigned in any of the three functions; it resolves to the module-global bound by the `__main__` loops at :1584 and :1656:
    for month_i, month_name in months.items():
Ruff's F821 does not fire precisely because `month_i` is a real module-level binding.

Fix: Replace month_i with month at plot_statistics_sector.py:200, :296, and :1028. No other change is needed — the parameter is already threaded through and truthiness-checked one line above.

Risk: safe — provably no behaviour change

workflow/scripts/test/test_land.py:46 — test_land.py has zero test functions — only an unused fixture and a bare # Tests comment

CLAUDE.md states the unit suite "covers constraint/helper logic only — test_land.py, test_policy.py, test_reserves.py" and workflow/scripts/test/README.md:10 lists "Land Use Constraints tests in test_land.py". Both documents assert coverage that does not exist. pytest -m fast in CI collects this module, imports it, and reports nothing — a green run is read as "land-use constraints pass" when the land-use constraint code has no automated test at all. This is the single most dangerous thing in the test tree because it converts absence of coverage into apparent presence.

Evidence
line 13:  sys.path.append(os.path.join(os.path.dirname(__file__), ".."))
line 19:  @pytest.fixture
line 20:  def land_use_network(base_network):
...
line 43:      return n
line 44:
line 45:
line 46:  # Tests            <-- end of file (46 lines total)

Verified: `grep -c '^def test_'` over workflow/scripts/test/*.py returns 0 for test_land.py (all other files return 1-11). Repo-wide `grep -rn "land_use_network"` returns exactly one hit — the definition at test_land.py:20. The file is entirely unreachable.

Fix: Either restore the tests (the fixture's docstring and the wind3/region_a setup describe the intended cases: a limiting land-use constraint over two generators sharing land_region, and a non-limiting one) or delete the file and correct the two documents. Do not leave it as-is. If restoring, add_land_use_constraints in workflow/scripts/opts/ is the unit under test and the existing base_network fixture already supplies the land_region columns.

Risk: safe — provably no behaviour change


The rest (52)

Dead code (24)

  • tests/equivalence/report_sections/summary.py:181 — Signature-guessing branches that can never fire, plus four never-referenced definitions
    medium · whole modeling domain · Dead code · risk: safe
  • workflow/rules/build_electricity.smk:991rule prepare_network declares log: twice; the second block is silently discarded
    medium · power system model · Dead code · risk: safe
  • workflow/rules/common.smk:44config_provider's scenario branch calls an undefined global scenarios — unreachable and broken
    medium · whole modeling domain · Dead code · risk: safe
  • workflow/rules/common.smk:123 — Six unreferenced helpers in common.smk, one with a bare except:, one targeting a dead path
    medium · whole modeling domain · Dead code · risk: safe
  • workflow/scripts/_helpers.py:330 — ~200 unreachable lines in _helpers.py; one is broken and creates a circular import
    medium · whole modeling domain · Dead code · risk: safe
  • workflow/scripts/add_extra_components.py:187 — Dead code: 55-line function with its call site commented out, unused normed, unused cost load, 4 dead params
    medium · power system model · Dead code · risk: safe
  • workflow/scripts/aggregate_egs.py:132 — Dead specs_caps binding whose comment contradicts the pre_caps weights actually used
    medium · power system model · Dead code · risk: safe
  • workflow/scripts/aggregate_to_substations.py:87substations parameter is overwritten on first use, making sub.csv a phantom Snakemake input
    medium · power system model · Dead code · risk: safe
  • workflow/scripts/aggregate_to_substations.py:153 — Unreachable else in the cols2drop ladder; its guard's error message lists values it rejects
    medium · power system model · Dead code · risk: safe
  • workflow/scripts/build_demand.py:408 — get_growth_rate and _dissagregate_on_ba are unreachable; ReadStrategy.units cannot be called
    medium · power system model · Dead code · risk: safe
  • workflow/scripts/build_population_layouts.py:241 — A 57-line commented-out PyPSA-Eur algorithm sits at the end of the module as a bare string literal
    medium · sector coupling · Dead code · risk: safe
  • workflow/scripts/build_sector_costs.py:524 — EfsBuildingData reads self.lifetimes / self.fixed_cost, which exist only inside commented-out code
    medium · sector coupling · Dead code · risk: safe
  • workflow/scripts/build_stock_data.py:404 — _get_marginal_cost is dead and its except handler references a loop variable that may be unbound
    medium · sector coupling · Dead code · risk: safe
  • workflow/scripts/constants.py:46 — ~205 lines (17%) of constants.py are mapping tables no module imports
    medium · whole modeling domain · Dead code · risk: safe
  • workflow/scripts/nrel_exclusion/build_nrel_availability.py:57 — RASTER_NODATA is unreferenced and its comment describes behaviour that no longer exists
    medium · power system model · Dead code · risk: safe
  • workflow/scripts/opts/interchange.py:80 — Dead period assignment shadowed by the loop variable; direction branch written out three times
    medium · power system model · Dead code · risk: safe
  • workflow/scripts/plot_network_maps.py:855 — plot_lmp_map is dead and reads the module-global n instead of its own network parameter
    medium · power system model · Dead code · risk: safe
  • workflow/scripts/summary.py:58 — Five of thirteen public functions are unreferenced, and main loads a network then does nothing
    medium · power system model · Dead code · risk: safe
  • workflow/scripts/zenodo_downloader.py:118 — Roughly 78 of 270 lines are an unreferenced public API: four methods and three module wrappers
    medium · whole modeling domain · Dead code · risk: safe
  • workflow/Snakefile:123workflow/rules/collect.smk is a 0-byte file that no include: references
    low · whole modeling domain · Dead code · risk: safe
  • workflow/scripts/build_bus_regions.py:111 — Two unreachable special cases in the onshore-region loop: a pass stub and a MISO-0001 branch
    low · power system model · Dead code · risk: safe
  • workflow/scripts/log.py:1 — log.py is unreferenced and duplicates the (also unreferenced) _helpers.setup_custom_logger
    low · whole modeling domain · Dead code · risk: safe
  • workflow/scripts/opts/policy.py:156 — 32 lines of groupby scaffolding collapsed to a scalar, guarded by truthiness on a linopy object
    low · power system model · Dead code · risk: safe
  • workflow/scripts/summary_sector.py:401 — Debugger residue if sector == "res" and state == "VT": pass inside a duplicated else-branch
    low · sector coupling · Dead code · risk: safe

Duplication (11)

  • tests/equivalence/report_sections/maps.py:28 — Pass/fail tolerances are written twice — the gate in compare.py and the maps in maps.py can drift
    high · whole modeling domain · Duplication · risk: safe
  • workflow/rules/postprocess.smk:32plot_network_maps and plot_statistics declare the identical log path and will clobber each other
    medium · power system model · Duplication · risk: safe
  • workflow/scripts/add_extra_components.py:408 — The same 16-line "rename time-dependent columns and join them back" block is copy-pasted three times
    medium · power system model · Duplication · risk: safe
  • workflow/scripts/add_extra_components.py:641add_demand_response open-codes six near-identical n.madd blocks differing only by direction
    medium · power system model · Duplication · risk: safe
  • workflow/scripts/build_natural_gas.py:174 — filter_on_sate: misspelled abstract hook whose body is pasted verbatim into four subclasses
    medium · sector coupling · Duplication · risk: safe
  • workflow/scripts/build_shapes.py:230 — Four if/elif branches call filter_shapes with byte-identical arguments
    medium · whole modeling domain · Duplication · risk: safe
  • workflow/scripts/eia.py:1107 — Four AEO extractors duplicate vehicle_codes and format_data verbatim to vary only build_url
    medium · whole modeling domain · Duplication · risk: safe
  • workflow/scripts/eulp.py:275 — EulpTotals is a 109-line near-clone of Eulp and is never instantiated anywhere in the repo
    medium · sector coupling · Duplication · risk: safe
  • workflow/scripts/plot_statistics.py:215 — Two functions contain literally duplicated statement blocks — including a duplicated docstring mid-body
    medium · power system model · Duplication · risk: safe
  • workflow/rules/build_electricity.smk:192nrel_avail and nrel_caps input lambdas are 21-line verbatim clones differing by two tokens
    low · power system model · Duplication · risk: safe
  • workflow/scripts/summary.py:204 — get_capacity_brownfield's retirement_method selects between two byte-identical functions
    low · power system model · Duplication · risk: safe

Testability (7)

  • workflow/scripts/add_electricity.py:1099main(snakemake) takes snakemake as a parameter, but four callees read the module global instead
    medium · power system model · Testability · risk: safe
  • workflow/scripts/add_extra_components.py:1535 — No main(): 230 lines of orchestration live directly under if __name__ == "__main__":
    medium · power system model · Testability · risk: safe
  • workflow/scripts/build_base_network.py:112 — Functions depend on snakemake and logger, both bound only inside the main guard
    medium · power system model · Testability · risk: safe
  • workflow/scripts/cluster_network.py:830 — calibrate_tamu_transmission_capacity ignores its costs parameter, reads global hvac_overhead_cost
    medium · power system model · Testability · risk: safe
  • workflow/scripts/cluster_network.py:458 — convert_to_transport reads undeclared global agg_busmap and derives a dead non_agg_buses
    medium · power system model · Testability · risk: safe
  • workflow/scripts/solve_network.py:148 — Solve path reads the snakemake global inside two functions instead of taking it as a parameter
    medium · power system model · Testability · risk: safe
  • workflow/scripts/test/test_solve_network.py:71 — The only e2e test for solve_network patches a renamed-away function, so it can never pass
    medium · whole modeling domain · Testability · risk: safe

Error handling (5)

  • tests/equivalence/compare.py:73is_waived treats an absent waiver key as a wildcard, so one typo silently waives real findings
    high · whole modeling domain · Error handling · risk: safe
  • workflow/scripts/build_natural_gas.py:60 — StateGeometry memoisation tests a GeoDataFrame for truthiness; cache raises ValueError on reuse
    medium · sector coupling · Error handling · risk: safe
  • workflow/scripts/eia.py:260 — Production.data_creator raises InputPropertyError with wrong kwarg names, yielding a TypeError
    medium · whole modeling domain · Error handling · risk: safe
  • workflow/scripts/cluster_network.py:565 — Bare except: in cluster_regions swallows a missing name column as "names are not numeric"
    low · power system model · Error handling · risk: safe
  • workflow/scripts/eia.py:298 — Four else: raise InputPropertyError year checks are unreachable after an exhaustive if/elif
    low · whole modeling domain · Error handling · risk: safe

Config coupling (2)

  • tests/equivalence/report_sections/dag.py:38 — dag.py hardcodes config.equivalence.yaml, ignoring the interconnect-aware paths.CONFIGFILE
    medium · whole modeling domain · Config coupling · risk: safe
  • CLAUDE.md:36 — CLAUDE.md claims CI runs ./test.sh; main.yml was replaced and a dead .test_sh is tracked
    low · whole modeling domain · Config coupling · risk: safe

Naming (1)

  • workflow/rules/build_electricity.smk:380demand_dissagregate_data is misspelled and routes through a strategy var encoding no decision
    low · power system model · Naming · risk: safe

API surface (1)

  • workflow/scripts/build_demand.py:45 — Context's two setters are both named strategy; read_strategy is silently unsettable
    low · power system model · API surface · risk: safe

Magic values (1)

  • workflow/scripts/build_heat.py:14 — VALID_HEAT_SYSTEMS never used while its tuple is inlined six times; sector enums bypassed
    medium · sector coupling · Magic values · risk: safe

Suggested order

  1. The five criticals above.
  2. Deletions — the dead-code items. One PR, no behaviour change, and they make the remaining code readable.
    The largest blocks: plot_validation_sector.py (758 lines, zero references repo-wide), three
    nrel_exclusion modules (573 lines), ~290 lines of plot_validation_production.py reachable only via
    commented-out call sites, ~200 lines of _helpers.py, ~205 lines (17%) of constants.py.
  3. Duplication — extract the shared plotting scaffold. create_title is byte-identical across
    plot_network_maps.py:169 and plot_statistics.py:60; save_fig and the two Sankey helpers each exist twice.
  4. Everything else.

Note on the deletions

Some of these are blocked on intent rather than analysis — the code is provably unreachable, but deleting it
destroys information about why it was written. The review recorded 78 such questions; 18 of them are
straight "delete or keep?" calls that a maintainer can answer in a sentence each. Happy to post those as a
follow-up if useful.


How this list was produced

A ruff sweep with an expanded rule set (F,C90,ARG,ERA,SIM,RET,PLR,B,PIE,TRY,PTH,DTZ) established an
objective baseline, then twelve reviewers each read one domain of the codebase in full rather than sampling.
Eight of the twelve domains went through an adversarial verification pass instructed to refute: open the file
at the line, confirm the quoted evidence is verbatim, grep the whole repo before accepting any dead-code claim,
and diff both sites before accepting any duplication claim. That pass adjusted 64 findings and confirmed 32.
The remaining four domains self-verified under the same instructions, and the critical findings were re-checked
by hand.

Every item below was confirmed by opening the file at the line. Line numbers are against develop as of
2026-08-24. No code has been changed — this is a survey only.

Each finding carries a risk label:

  • safe — provably no behaviour change (mostly deletions)
  • low — small blast radius, no numerics
  • numerics — touches model results; should not land without a validation run

Filed from an automated clean-code review. Items are unchecked survey results, not agreed work — triage
before acting.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    choreRepository maintenance related taskgood first issueGood for newcomers

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions