Endogenous Industry Heat - #1943
LukasFrankenQ wants to merge 33 commits into
Conversation
Endogenous industry sector ratios: - Swap the Pulp/Paper PRODCOM mappings, which were crossed (pulp -> paper products, paper -> pulp products). - Fix the pharmaceuticals backup share: heat>500 was 1.0, making the row sum to 2.0; set to 0.0 so the bands sum to 1.0. - Override the Fleiter 2025 Mechanical pulp row (100% heat<100) with 100% heat100-200, reflecting TMP/CTMP refiner steam at 130-170 C (Rehfeldt et al. 2018). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
@fneum @Irieo @toniseibold I think this is now in a quite good state, and could maybe even be merged, some notes:
|
There was a problem hiding this comment.
Thanks for working on this and pushing it further!
Upon testing the PR code for an own project I ran in to the problem that options and spatial in the new functions in prepare_sector_network.py are global. My suggestion is to pass these as arguments to the functions (as is done for options in the other functions in prepare_sector_network.py and as I suggest in PR2232 for spatial).
Some further comments on a stray pass and a minor unnecessarily complex if condition.
bus2 and bus3 are not consistent throught out the newly added links:
Sometimes it is:
bus2="co2 atmosphere",
bus3=spatial.co2.nodes,
(e.g. ["industry_t"]["heat100-200"]["biomass"])
other times:
bus2=spatial.co2.nodes,
bus3="co2 atmosphere",
(e.g., "heat200-500" biomass and methane links)
| """ | ||
|
|
||
| logger.info("Add industry heat in the temperature band 100-200 °C.") | ||
| pass |
There was a problem hiding this comment.
Function stops here, remove pass.
| ) | ||
|
|
||
|
|
||
| def add_t_industry100_200(n, nodes, industrial_demand, costs, must_run): |
There was a problem hiding this comment.
Add options, spatial as parameters:
def add_t_industry100_200(n, nodes, industrial_demand, costs, must_run, options, spatial):
|
|
||
| if ( | ||
| options["industry_t"]["heat100-200"]["biomass"] | ||
| or not options["industry_t"]["endogen"] |
There was a problem hiding this comment.
Cleanup, or not is not required, as "endogen" is required for "heat100-200".
if options["industry_t"]["heat100-200"]["biomass"]:
| options["industry_t"]["heat100-200"]["biomass"] | ||
| or not options["industry_t"]["endogen"] | ||
| ): | ||
| logger.info( |
| ) | ||
|
|
||
|
|
||
| def add_t_industry200_500(n, nodes, industrial_demand, costs, must_run): |
There was a problem hiding this comment.
Add options, spatial as parameters:
def add_t_industry200_500(n, nodes, industrial_demand, costs, must_run, options, spatial):
| p_nom_extendable=True, | ||
| p_min_pu=must_run, | ||
| efficiency=costs.at["direct firing gas", "efficiency"], | ||
| ) |
There was a problem hiding this comment.
It seems like the lifetime is missing. (lifetime=costs.at["direct firing gas", "lifetime"],)
|
|
||
| must_run = options["industry_t"]["must_run"] | ||
|
|
||
| add_t_industry100_200(n, nodes, industrial_demand, costs, must_run) |
There was a problem hiding this comment.
Pass arguments options and spatial add_t_industry100_200(n, nodes, industrial_demand, costs, must_run, options, spatial)
| must_run = options["industry_t"]["must_run"] | ||
|
|
||
| add_t_industry100_200(n, nodes, industrial_demand, costs, must_run) | ||
| add_t_industry200_500(n, nodes, industrial_demand, costs, must_run) |
There was a problem hiding this comment.
Pass arguments options and spatial add_t_industry200_500(n, nodes, industrial_demand, costs, must_run, options, spatial)
|
|
||
| add_t_industry100_200(n, nodes, industrial_demand, costs, must_run) | ||
| add_t_industry200_500(n, nodes, industrial_demand, costs, must_run) | ||
| add_t_industry500(n, nodes, industrial_demand, costs, must_run) |
There was a problem hiding this comment.
Pass arguments options and spatial add_t_industry500(n, nodes, industrial_demand, costs, must_run, options, spatial)
|
I have been trying to integrate this PR in my workflow and I want to flag an issue I have encountered for the myopic runs. add_brownfield.py fixes every non-extendable link's capacity to its inherited p_nom_opt and sets p_nom_extendable=False going forward. For the industry_t supply links (gas/hydrogen/biomass/ etc..) , p_min_pu is set to a fixed must_run fraction of that fixed inherited capacity. so once an inherited capacity becomes brownfield, its must-run floor is a fixed absolute MW value, permanently. If the nodal industry-heat demand at that bus falls in a later horizon (e.g. demand for combustion plants due to industrial transformation), the inherited floor can end up exceeding the new demand. Since the fixed inherited plant can't ramp below its floor, and newer plant can only add supply, not subtract from an oversupplied older one. In this case the nodal balance becomes structurally infeasible. I want to suggest a fix, that we discussed with @martavp Links whose bus1 ends in " lowT industry"/" mediumT industry"/" highT industry", don't fix p_nom outright during brownfield carry-over. Instead, keep them extendable but capped:
Hopefully this can be useful, otherwise please disregard this comment. |
# Conflicts: # doc/configuration.md # scripts/prepare_sector_network.py
Single-country runs drop processes the country does not have via dropna(how='all'), so selecting them with idx[:, process] raised a KeyError. Guard both band-splitting loops with a column mask. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- pass options and spatial as parameters to add_t_industry* functions - remove stray pass and leftover review-question logging - drop dead 'or not endogen' condition (functions only run when endogen) - normalize CC links to bus2='co2 atmosphere', bus3=co2 stored ordering (efficiencies swapped alongside, no behavior change) - add missing lifetime on heat200-500 industry hydrogen link Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Hi @w-tusche , apologies for the long silence here! Thank you for the review and suggestions, I have adjusted the content of the PR accordingly |
|
@BertoGBG Thank you for this suggestion, very good point, this was easily reproduced! I fixed this issue in the implementation but set the @fneum I would flag this PR now as being ready to merge (or review) :) |
w-tusche
left a comment
There was a problem hiding this comment.
No worries, we all got other things to do.
Thanks for implementing the suggested changes!
One more bug I encountered:
| biomass: true | ||
| methane: true | ||
| hydrogen: true | ||
| "heat>500": |
There was a problem hiding this comment.
I encountered an "Invalid argument" bug when running post-processing steps (creating balance maps and timeseries for the new carrieres).
OSError: [Errno 22] Invalid argument: 'results/heat-test/graphs/balances-heat>500 industry.pdf'
The ">" in the carrier name "heat>500" is not filename-safe on all OS systems. In my case the bug occured when trying to store the post-processing output for the new carriers on a NTFS-mount on a Linux system.
Suggested fix: use "+" instead of ">" so the idea would be to then have "heat500+ industry"
There was a problem hiding this comment.
Absolutely fair, thank you, also adjusted :)
'>' is a reserved filename character on Windows/NTFS, so postprocessing outputs like 'balances-heat>500 industry.pdf' fail with OSError on those filesystems. Renamed the band key, carriers and bus names to 'heat500+' as suggested in review; config defaults and JSON schema regenerated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
I would like to bring two more things to the discussion: 1.
|
| source → sink | lift | COP_Lorenz | 55% of it |
|---|---|---|---|
| 20 °C → 150 °C | 130 K | 3.255 | 1.79 |
| 20 °C → 200 °C | 180 K | 2.629 | 1.45 |
| 80 °C → 150 °C | 70 K | 6.045 | 3.32 |
| 80 °C → 200 °C | 120 K | 3.943 | 2.17 |
snippet for the numbers above
COP = 3.2 # industrial heat pump high temperature, efficiency, costs_2050
ETA = 0.55 # DEA: "around 55% of the theoretical COP"
def lorenz(src, snk):
# inputs in °C simplified without glide temperatures
return (snk + 273.15) / (snk - src)
for src, snk in [(20, 150), (20, 200), (80, 150), (80, 200)]:
L = lorenz(src, snk)
print(f"{src} -> {snk} | lift {snk - src} K | Lorenz {round(L,3)} | 55% {round(ETA * L,2)}")
for snk in (150, 200): # coldest source for which COP is reachable
T = snk + 273.15
print(f"sink {snk} C | min source {round(T - T / COP - 273.15,1)} C (eta=100%) | {round(T - ETA * T / COP - 273.15,1)} C (eta=55%)")For steam generation, and using the simplified formula without glides, a COP of 3.2 requires a source warmer than 17.8 °C at a 150 °C sink and warmer than 52.1 °C at a 200 °C sink before any losses at all, and above 77.3 °C and 118.7 °C, respectively, to reach it at the catalogue's 55%. So with an ambient source the value is a warm-source COP used outside its range, and above the Lorenz limit in the upper part of the band.
Since the band is aggregated over 100-200 °C and the DEA sheet is for sink <= 150 °C, not covering the top of the band may be acceptable — but in my opinion the mismatch is in the source, not only in the sink.
One thing that speaks for considering changes: the heating-sector HPs have a variable COP depending on the ambient temperature, while the industry HP has a fixed COP from the cost file. This also affects the config sector: time_dep_hp_cop — turning it on only affects the heating-sector HPs (urban central, urban decentral and rural) and not the industrial one, which might be misleading, as one would think it changes from fixed to time-dependent COP for all heat pumps. Maybe rename to to time_dep_LT_hp_cop?
All in all I think this industry heat pump implementation is rather optimistic. My investigation here is far from complete but I wanted to share the thoughts for discussion and before a quick merge.
Side note: H2 industrial combustion cost
In some of my test runs (not purely the contents of this PR) the 500+ industry heat was provided to around 40% by the industrial H2 combustion, so the "TODO: research cost of industrial H2 combustion, here set to 10x methane combustion" might be quite important.
Resolves conflicts from upstream's "Streamline workflow" (PyPSA#1838): - rules/solve_myopic.smk: accept upstream deletion; the add_brownfield rule no longer exists and compose_network already receives tech_costs. - scripts/add_brownfield.py: adopt upstream's main() entry point and pass the costs DataFrame through it to add_brownfield(), which needs it for the industry_t FOM-only costs of inherited process-heat links. - scripts/compose_network.py: forward the already-loaded costs to apply_brownfield(). - rules/build_sector.smk: keep upstream's {horizon}-based build_industrial_energy_demand_per_node rule with the endogenous sector ratios as an extra input; drop the stale {planning_horizons} copy; use the scripts() helper and add a message for build_industry_sector_ratios_endogenous. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ug8xNAwre4wLU5H2h2KuQj


pypsa/pypsa-eur#611
Adds industry process heat demands
heat100-200,heat200-500andheat>500by subtracting them from industrybiomassandmethanedemandsif
config["sector"]["industry_t"]["endogen"] == True.Temperature splits are process-aware, based on Fraunhofer ISI research Fleiter et al. (2025), Hydrogen Infrastructure in the Future CO2-Neu....
The PR does not cover steelworks. Its endogenisation is in a separate PR.
Changes to the Model
With industry heat endogenous, all existing industry-related components remain, heat demands are purely subtracted from biomass and methane industry demands, and are added as separate loads with the respective supply opportunities.
The changes to
prepare_sector_networkare largely identical to the previous PR by @lisazeyen, and the heat generation technologies added are the same. The temperature bands are chosen to match potentially interesting alternative renewable heat supply such as solar thermal and geothermal, which could be added to the model if desired.The logic for exogenous heating remains untouched. There was a very sensible proposition by @koen-vg to set the exogenous heat supply mix in the config, as is done with industry production pathways. However, the volume of content this approach would add to the config was, in my estimation, too large.
Therefore, this PR introduces a new rule that builds (planning-horizons–unaware) heat-endogenous ratios and then simply infers endogenous industry demands (including the temperature bands) in
build_industry_energy_demand_per_node, following the same logic as exogenous demand usingratios_intermediate.As a result,
industry_energy_demands_per_nodeconsists essentially two concatenated dataframes. Level 0 in columns== "exogenous"contains the old dataframe and== "endogenous"is the new one.Based on the endogenisation-switch in config,
prepare_sector_networkpulls the respective columns.Towards Validation
Comparing exogenous and endogenous 2030 industry demands shows that the PR removes the vast majority of (industry)
solid biomassand most of (industry)methanedemand. Exogenous heat presumes electrification, likely explaining why the total energy demand is lower in the exogenous case.The PR uses Fleiter et al. (2025) to estimate temperature band distributions for each industry process, and uses alternative process-specific sources to fill data gaps in between. Again, no steelworks in this PR.
These are very distinct between industries; however, they still yield largely similar temperature distributions between countries:
Spatial heterogeneity might only manifest in higher spatial resolutions.
Here, and in the previous iterations, we are aiming to align with Agora's estimate of industry heating demand at a total of around 1850 TWh/a:
Considering we are not including steelworks, we are doing quite well:
ToDos and Food for Thought
process emissions, HVC disaggregation,current electricity, etc. are synergising correctly with current methods.heat100-200andlow-temperature heat, but in some sectors, electrification of higher-temperature demands is feasible.Happy about any feedback, and all the best,
Lukas
@lisazeyen @millingermarkus @Irieo @koen-vg @toniseibold @fneum @bobbyxng @p-glaum
PS: Are the pictures readable, or do they throw an error when clicked on?
Checklist
pixi.toml(usingpixi add <dependency-name>).config/config.default.yaml.doc/configtables/*.csv.these instructions <https://pypsa-eur.readthedocs.io/en/latest/data_sources.html>_ have been followed.doc/release_notes.rstis added.