Endogenize steel and cement production and add existing industry sites by toniseibold · Pull Request #1719 · PyPSA/pypsa-eur

toniseibold · 2025-06-25T14:39:47Z

Closes # (if applicable).
PyPSA/pypsa-de#8

Dependencies

relies on the latest master of technology-data which includes parameters for cement kilns and fossil steel routes
GEM Cement Tracker
Data for European industry sites with added relining dates and planned phase out of BF-BOFs from GEM Steel Tracker

Changes proposed in this Pull Request

This PR endogenizes the production of steel and cement and for brownfield optimizations adds the existing plants for steel, cement, ammonia and methanol.
This PR uses the dataset provided by Neuwirth et al. to add existing industry sites.
Where necessary this dataset is complemented with other datasets since it's limited to EU27 countries only.

Ammonia

As soon as the config setting config["sector"]["ammonia"] is either True or regional and the optimization runs with perfect or myopic foresight, ammonia plant capacities are added in add_existing_baseyear.

Switching existing Haber-Bosch capacities from natural gas to hydrogen is not permitted since it's associated with retrofitting costs and there are only few Examples.
Comparison of eurostat demand vs. database installed capacity shows a high degree of agreement.

Methanol

Existing methanol capacities are added as methanol plants. It is implemented as a green methanolisation, since grey and blue are already indirectly included with SMR and SMR CC hydrogen production.

Steel

There are two routes for steel making implemented. The BF-BOF uses coal to reduce iron ore and produce steel in one process. On the other hand, natural gas or hydrogen direct reduced iron processes reduce iron to hot briquetted iron (HBI). The last step is to produce iron via an electric arc furnace (EAF).
EAF are also necessary to recycle secondary steel however this is still hidden in the electricity for industry load.

Cement

Cement production consists of two processes, clinker production and cement production.
For clinker production, the raw material is fed into a cement kiln, and under high temperature heat the clinker is burnt.
The heat for clinker production can either be supplied via biomass, waste or natural gas - additional to those specific emissions, there is carbon from the raw material oxidizing which contributes to the specific emissions.
The clinker is then ground with gypsum to get cement.

Installed Capacities

In general, the production capacities mostly agree with the demand from the JRC-IDEES database. The methanol capacities do not match the naphtha for industry demand since that one is met via oil based routes currently.
In bus regions where the demand is higher than the brownfield capacities, new plants can be built from links added in prepare_sector_network.

Open ToDos

code testing
add sources to data_retrieval.rst
add grey methanol technology parameters to technology-data
Adding Grey Methanolisation technology-data#248
add colors to plotting.default.yaml that actually make sense
wait for Adding Clinker Process Emissions technology-data#266 to be merged

Checklist

I tested my contribution locally and it works as intended.
Code and workflow changes are sufficiently documented.
Changed dependencies are added to pixi.toml (using pixi add <dependency-name>). not applicable
Changes in configuration options are added in config/config.default.yaml.
Changes in configuration options are documented in doc/configtables/*.csv.
A release note doc/release_notes.rst is added.

…pacities

for more information, see https://pre-commit.ci

…nfig for existing cement and steel plants

for more information, see https://pre-commit.ci

bobbyxng · 2026-01-15T15:47:43Z

Updated branch to master, including removal of doctables, moved to schema setup, see #1912

…O: Create new TD release for final merge.

for more information, see https://pre-commit.ci

bobbyxng

Thanks @toniseibold for this super extensive PR! I have now been able to go through the code base and have primarily focused on consistency, there are still a few changes, proposals and some questions wrt. my personal understanding. In summary:

Allow for toggling endogenisation via an enable or empty list
Please double-check units, efficiency and cost calculations/normalisations, both in add_existing_baseyear and prepare_sector_network
Please double-check that all arguments, params are passed to all functions and add correct docstring/typing to all functions
Consider for some parameters to be moved to config or technolog-data
Please open a PR on data when the PR is ready @lkstrp

Could you also do a comparison plot for energy use and/or regional distribution with and without endogenisation, e.g. weekly/24-hourly resolution.

Thanks a lot :)

bobbyxng · 2026-03-11T13:29:15Z

Also please merge with latest master and double-check compatibility with the new schema

- `pixi run sync-locks` - `pixi run generate-config`

for more information, see https://pre-commit.ci

- `pixi run sync-locks` - `pixi run generate-config`

for more information, see https://pre-commit.ci

toniseibold · 2026-03-25T15:33:09Z

I did a couple of checks to make sure the extension works as intended (however low temporal resolution runs)

Cement and Steel Demand

Exogenous by multiplying sector_ratios with production levels for steel and cement
Δ Endogenous: new exogenous demands for industry

Carrier / Output	Exogenous (TWh/a)	Δ Endogenous (TWh/a)
Electricity	74.53	75.00
Coal	7.51	123.00*
Coke	84.46	*
Biomass	41.10	41.00
Methane / Gas	127.96	128.00
Hydrogen	20.73	21.00
Heat	2.12	2.0
Naphtha	20.12	20.00
Ammonia	0.00	0.0
Methanol	0.00	0.0
Process emissions	107.04	107.04

*coke: 1.366 x coal

Endogenous Demand for Steel and Cement Production

Carrier	Technology	Value (TWh/a)
Hydrogen	H₂ DRI	45.43
Gas	Gas DRI	19.98
Electricity	DRI	29.69
	EAF	18.43
	Cement kiln	7.78
	Cement finishing	29.66
Coal	BOF	171.71
Heat	Cement kiln (total)	105.83
	from Biomass	0.11
	from Gas	0.00
	from Waste	105.72
CO₂	Cement kiln (process)	56.83
	from Heat	30.20

Commodity Prices (low resolution excluding carbon price for meoh)

I spotted one weirdness:
industry_sector_ratios_2050 report negative process emissions for HVC
So far that was not a problem since the process emissions from other subsectors are >0 which leads to an overall load > 0 but we could start running into problems
Maybe @fneum knows more about that

Merges upstream/industry_plants (PyPSA#1719, toniseibold): adds fossil-vs-green route competition for steel (H2-DRI vs gas-DRI into a shared reduction bus, same competing-Links-into-one-bus pattern already used for shipping/industry heat) and cement, plus existing- plant brownfield handling via a new industry_plants resource. Also re-adds methanol-to-olefins/aromatics (add_meoh_to_oa, gated by methanol.meoh_to_oa) -- removed upstream in PyPSA#1299 for lacking a proper HVC bus; still routes into the shared spatial.oil.naphtha pool, so the original blend-accounting concern isn't actually fixed, just reinstated as-is from upstream. Conflicts (4 files, all mechanical -- independent insertions at the same anchor point, no shared logic rewritten by both sides): - rules/build_sector.smk, rules/solve_myopic.smk, rules/solve_perfect.smk: both sides added independent params/inputs to the same rule blocks; resolved by keeping both. - scripts/prepare_sector_network.py: this fork's add_rock_weathering and upstream's new add_meoh_to_oa both landed immediately before add_dac (pure coincidence); this fork's entire add_low/medium/ high_t_industry + add_exogen_t_industry block and upstream's new adjust_industry_demand both landed immediately before add_industry. Reconstructed both pairs intact rather than trusting git's interleaving. Kept this fork's add_dac(n, costs, spatial) signature (both branches were internally self-consistent; call site already passes spatial explicitly throughout this fork). No manual demand-routing fix was needed: add_industry already routes adjust_industry_demand()'s steel/cement-excluded remaining_demand into industrial_demand before it reaches add_low/medium/high_t_industry when endogenous_sectors.enable is true, auto-merged correctly since neither branch touched that exact span. sector.endogenous_sectors.enable defaults to false (upstream default, unchanged) -- steel/cement endogenization is opt-in. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KkHwmFFdgK5T1XFbFHA5SA

…ssignment np.searchsorted(grouping_years, valid_years, side="right") returns len(grouping_years) for any build_year past the last grouping boundary (e.g. built 2025-2029 with baseyear=2030 and grouping_years_industry ending at 2025) -- np.array(grouping_years)[indices] then raises IndexError. Clip into the last bucket instead. Found running the merged PR PyPSA#1719 steel/cement endogenization against this fork's data (a real existing plant built in that window), not caused by the merge itself. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KkHwmFFdgK5T1XFbFHA5SA

toniseibold changed the title ~~Add industry sites~~ Endogenize steel and cement production and add existing industry sites Oct 21, 2025

koen-vg mentioned this pull request Oct 22, 2025

Endogenise industry heat supply in steam, medium and high T segments #611

Open

toniseibold added 3 commits December 1, 2025 14:13

implementing existing ammonia plants

5e8d03c

refining existing Haber-Bosch plants + Documentation

243ef47

endogenizing steel and cement production and add existing industry ca…

643fcc9

…pacities

toniseibold force-pushed the industry_plants branch from d46d609 to 643fcc9 Compare December 1, 2025 13:39

[pre-commit.ci] auto fixes from pre-commit.com hooks

ef8e50f

for more information, see https://pre-commit.ci

toniseibold mentioned this pull request Dec 1, 2025

Adding Grey Methanolisation PyPSA/technology-data#248

Merged

5 tasks

toniseibold and others added 7 commits December 1, 2025 16:30

cleaning up plotting colors

5af80f0

[pre-commit.ci] auto fixes from pre-commit.com hooks

018cf57

for more information, see https://pre-commit.ci

own rule for industry plant data preparation, restructuring of the co…

870ca70

…nfig for existing cement and steel plants

adding data files

321cb4b

Merge branch 'master' into industry_plants. Updated to schema.

91977cc

Updated config.default from schema and schema.json.

221cbcc

[pre-commit.ci] auto fixes from pre-commit.com hooks

c921adb

for more information, see https://pre-commit.ci

github-code-quality Bot found potential problems Jan 15, 2026

View reviewed changes

Comment thread scripts/build_industry_plants.py Fixed

Comment thread scripts/build_industry_plants.py Fixed

Comment thread scripts/prepare_sector_network.py Fixed

Comment thread scripts/lib/validation/config/sector.py Fixed

bobbyxng self-requested a review January 15, 2026 14:03

bobbyxng added the enhancement label Jan 15, 2026

bobbyxng added 3 commits January 15, 2026 16:17

Merge branch 'master' into industry_plants

b75e18c

Added PR test config.

7f4cb06

Removed configtables, not used anymore in master.

74e5fdc

bobbyxng and others added 8 commits January 16, 2026 11:03

Merge branch 'master' into industry_plants

08a0053

Readded grouping_years_industry key and added respective schema entry.

f760f70

Added temporary dev technology-data which includes grey methanol. TOD…

01a9fb7

…O: Create new TD release for final merge.

Updated dev config.

63f8d7f

Updated dev config.

fcfb34f

[pre-commit.ci] auto fixes from pre-commit.com hooks

eb93737

for more information, see https://pre-commit.ci

Removed stray config from other PR.

2aabfc1

Updated temporary PR config.

2062ff0

bobbyxng requested changes Mar 11, 2026

View reviewed changes

toniseibold and others added 2 commits March 13, 2026 10:17

Merge remote-tracking branch 'origin/master' into industry_plants

806b54a

[pypsa-bot] run sync-locks & generate-config

47a7fd3

- `pixi run sync-locks` - `pixi run generate-config`

github-code-quality Bot found potential problems Mar 13, 2026

View reviewed changes

Comment thread scripts/prepare_sector_network.py Fixed

toniseibold and others added 2 commits March 16, 2026 14:25

Requested changes and improvements

9fe0a03

[pre-commit.ci] auto fixes from pre-commit.com hooks

b20a39a

for more information, see https://pre-commit.ci

github-code-quality Bot found potential problems Mar 16, 2026

View reviewed changes

Comment thread scripts/add_existing_baseyear.py Fixed

Comment thread scripts/prepare_sector_network.py Fixed

pypsa Bot and others added 7 commits March 16, 2026 13:29

[pypsa-bot] run sync-locks & generate-config

e9e4553

- `pixi run sync-locks` - `pixi run generate-config`

minor bugfixes

76b329d

[pre-commit.ci] auto fixes from pre-commit.com hooks

7706973

for more information, see https://pre-commit.ci

flatten keys

f8219bb

add unique plotting colors for all new carriers

093abd6

[pre-commit.ci] auto fixes from pre-commit.com hooks

39ec842

for more information, see https://pre-commit.ci

adding existing global cement tracker data

f0b4efa

toniseibold requested a review from bobbyxng March 25, 2026 15:35

Merge branch 'master' into industry_plants

81aa0aa

toniseibold marked this pull request as ready for review March 25, 2026 15:35

lkstrp requested a review from euronion March 26, 2026 13:39

Navigation Menu

Uh oh!

Endogenize steel and cement production and add existing industry sites - #1719

Endogenize steel and cement production and add existing industry sites#1719
toniseibold wants to merge 34 commits into
masterfrom
industry_plants

toniseibold commented Jun 25, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bobbyxng commented Jan 15, 2026

Uh oh!

bobbyxng left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bobbyxng commented Mar 11, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

toniseibold commented Mar 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

toniseibold commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dependencies

Changes proposed in this Pull Request

Ammonia

Methanol

Steel

Cement

Installed Capacities

Open ToDos

Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bobbyxng commented Jan 15, 2026

Uh oh!

bobbyxng left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bobbyxng commented Mar 11, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

toniseibold commented Mar 25, 2026

Cement and Steel Demand

Endogenous Demand for Steel and Cement Production

Commodity Prices (low resolution excluding carbon price for meoh)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

toniseibold commented Jun 25, 2025 •

edited

Loading