Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 5 additions & 6 deletions .copier-answers.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
# Changes here will be overwritten by Copier
# DO NOT MANUALLY MODIFY THIS FILE
# !!!!! DO NOT MANUALLY MODIFY THIS FILE !!!!!
_commit: latest
_src_path: https://github.com/calliope-project/data-module-template.git
_src_path: https://github.com/modelblocks-org/data-module-template.git
author_email: i.ruizmanuel@tudelft.nl
author_family_name: Ruiz Manuel
author_given_name: Ivan
github_org: calliope-project
github_org: modelblocks-org
license: Apache-2.0
module_description: A module that collects, harmonizes and aggregates global powerplant
capacities for energy systems modelling.
module_long_name: clio - Powerplants module
module_description: Aggregate global powerplant capacities into any resolution.
module_long_name: Modelblocks - Powerplants module
module_short_name: module_powerplants
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ A clear and concise description of what you expected to happen.
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
- OS: [e.g. Linux Fedora 41, Windows 10...]
- OS: [e.g. Linux Fedora 43, Windows 11...]
- Version [e.g. v0.1.1]

**Additional context**
Expand Down
5 changes: 5 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
blank_issues_enabled: false
contact_links:
- name: General information
url: https://www.modelblocks.org/
about: Please consult our website for general information on the Modelblocks methodology.
11 changes: 8 additions & 3 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@ Fixes #

## Reviewer checklist

* [ ] `INTERFACE.yaml` is up-to-date with all relevant user resources and results.
* [ ] The integration example is up-to-date with a minimal use-case of the module.
* [ ] Module documentation is up-to-date.
* [ ] There are no `pip` dependencies in the module's environment files (`workflow/envs/`).
* [ ] All rules use `pathvars` (e.g., `<results>`) in their inputs and outputs.
* [ ] The integration test-suite is successful, including:
* [ ] `pre-commit.ci` tests pass.
* [ ] tests pass for all relevant OS configurations (linux, osx, windows).
* [ ] Module documentation is up-to-date, including:
* [ ] `INTERFACE.yaml` mentions all relevant `pathvars` and `wildcards`.
* [ ] `README.md` describes how to use the module and has the necessary citations.
2 changes: 1 addition & 1 deletion .github/workflows/check-version.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ on:

jobs:
copier-update:
uses: calliope-project/data-module-template/.github/workflows/template-check-version.yml@latest
uses: modelblocks-org/data-module-template/.github/workflows/template-check-version.yml@latest
40 changes: 36 additions & 4 deletions .github/workflows/pr-ci.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,41 @@
name: CI
# !!!!! DO NOT MODIFY UNLESS ABSOLUTELY NECESSARY !!!!!
# This workflow helps standardise integration tests across data modules
name: Pull Request CI tests for Modelblocks data modules
on:
pull_request:
branches:
- "main"

jobs:
ci-tests:
uses: calliope-project/data-module-template/.github/workflows/template-pr-ci.yml@latest
build:
name: Build
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: ["3.12"]
steps:
- uses: actions/checkout@v4
- name: Setup pixi
uses: prefix-dev/setup-pixi@v0.8.3
- name: Run integration tests
id: tests
run: pixi run test-integration
continue-on-error: true
- name: Run snakemake linting
id: linting
run: |
pixi run snakemake --lint --snakefile workflow/Snakefile
pixi run snakemake --lint --snakefile tests/integration/Snakefile
continue-on-error: true
- name: Save integration logs
if: ${{ always() }}
uses: actions/upload-artifact@v4
with:
name: integration-test-logs-${{ matrix.os }}
path: tests/integration/resources/powerplants/logs
if-no-files-found: ignore
retention-days: 30
- name: Fail if integration or linting failed
if: ${{ steps.tests.outcome == 'failure' || steps.linting.outcome == 'failure' }}
run: exit 1
4 changes: 3 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,6 @@ on:

jobs:
release-workflow:
uses: calliope-project/data-module-template/.github/workflows/template-release.yml@latest
permissions:
contents: write
uses: modelblocks-org/data-module-template/.github/workflows/template-release.yml@latest
3 changes: 2 additions & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ repos:
# Repo quality
- id: check-added-large-files
args: [--enforce-all]
exclude: ^pixi\.lock$
- id: forbid-submodules
- id: check-case-conflict
- id: check-illegal-windows-names
Expand Down Expand Up @@ -44,4 +45,4 @@ repos:
files: .*\.(py|smk|md)$|^Snakefile$

ci: # https://pre-commit.ci/
autoupdate_schedule: monthly
autoupdate_schedule: quarterly
67 changes: 40 additions & 27 deletions INTERFACE.yaml
Original file line number Diff line number Diff line change
@@ -1,32 +1,45 @@
# Module Input-Output structure for automated doc. generation
resources:
user:
"{shape}/impute/{category}.parquet": |
Optional. Files with point-specific powerplant data to impute on top of the files produced by this module.
Useful when important powerplant facilities are missing in the source datasets.

Can also be used to replace data when combined with `excluded_ids` in the configuration.

"{shape}/proxies/rooftop_pv.tif": Proxy raster to use to estimate rooftop PV capacity.

"{shape}/shapes.parquet": |
GeoParquet shapefile with the desired regional disaggregation.

IMPORTANT: shapes must add up to whole countries to ensure proxied national statistics are correct.

results:
"{shape}/aggregated/{adjustment}/{category}.parquet": >
Aggregated powerplant capacity per polygon in the given shapefile.
All categories are available.

"{shape}/aggregated/proxies/rooftop_pv.tif": >
Generated proxy raster used to estimate rooftop PV capacity per country.

"{shape}/statistics/category_capacity.parquet": >
National-level capacity statistics per category, for all the countries in the shape file.

pathvars:
snakemake_defaults:
logs:
default: "<logs>"
description: location of rule log files.
resources:
default: "<resources>"
description: "location of module resource files."
results:
default: "<results>"
description: "location of module results."
user_resources:
shapes:
default: "<resources>/user/{shapes}/shapes.parquet"
description: |
GeoParquet shapefile with the desired regional disaggregation.
IMPORTANT: shapes must add up to whole countries to ensure proxied national statistics are correct.
proxy_rooftop_pv:
default: "<resources>/user/{shapes}/proxies/rooftop_pv.tif"
description: Proxy raster to use to estimate rooftop PV capacity.
imputed_powerplants:
default: "<resources>/user/{shapes}/impute/{category}.parquet"
description: |
Optional. Files with point-specific powerplant data to impute on top of the files produced by this module.
Useful when important powerplant facilities are missing in the source datasets.
Can also be used to replace data when combined with `excluded_ids` in the configuration.
wemi:
default: "<resources>/user/WEMI.xls"
description: |
Wind Energy Market Intelligence dataset.
This optional dataset will replace the GEM GWPT dataset if selected in the configuration.
results:
aggregated_capacity:
default: "<results>/{shapes}/aggregated/{adjustment}/{category}.parquet"
description: >
Aggregated powerplant capacity per polygon in the given shapefile.
powerplants:
default: "<results>/{shapes}/powerplants/{adjustment}/{category}.parquet"
description: Disaggregated powerplants within polygons in the given shapefile.
wildcards:
shape: name of the provided shapefile.
shapes: name of the provided shapefile.
adjustment: >
Whether or not total capacity per country has been re-scaled to fit national statistics.
Options: "adjusted" or "unadjusted".
Expand Down
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@
same "printed page" as the copyright notice for easier
identification within third-party archives.

Copyright 2025 AUTHORS
Copyright 2026 AUTHORS

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand Down
53 changes: 38 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,44 @@
# clio - Powerplants module
# Modelblocks - Powerplants module

A module that collects, harmonizes and aggregates global powerplant capacities for energy systems modelling.
Aggregate global powerplant capacities into any resolution.

![Example of European shapes](./docs/europe_example.png)
<!-- Place an attractive image of module outputs here -->

A modular `snakemake` workflow built for [`clio`](https://clio.readthedocs.io/) data modules.

## Using this module
## About
<!-- Please do not modify this templated section -->

This module can be imported directly into any `snakemake` workflow.
Please consult the integration example in `tests/integration/Snakefile` for more information.
This is a modular `snakemake` workflow created as part of the [Modelblocks project](https://www.modelblocks.org/). It can be imported directly into any `snakemake` workflow.

For more information, please consult the Modelblocks [documentation](https://modelblocks.readthedocs.io/en/latest/),
the [integration example](./tests/integration/Snakefile),
and the `snakemake` [documentation](https://snakemake.readthedocs.io/en/stable/snakefiles/modularization.html).

## Overview
<!-- Please describe the processing stages of this module here -->

Data processing steps:
1.
2.

## Configuration
<!-- Please describe how to configure this module below -->

Please consult the configuration [README](./config/README.md) and the [configuration example](./config/config.yaml) for a general overview on the configuration options of this module.

## Input / output structure
<!-- Please describe input / output file placement below -->

Please consult the [interface file](./INTERFACE.yaml) for more information.

## Development
<!-- Please do not modify this templated section -->

We use [`pixi`](https://pixi.sh/) as our package manager for development.
Once installed, run the following to clone this repo and install all dependencies.
Once installed, run the following to clone this repository and install all dependencies.

```shell
git clone git@github.com:calliope-project/module_powerplants.git
git clone git@github.com:modelblocks-org/module_powerplants.git
cd module_powerplants
pixi install --all
```
Expand All @@ -28,16 +49,18 @@ For testing, simply run:
pixi run test-integration
```

To view the documentation locally, use:

```shell
pixi run serve-docs
```

To test a minimal example of a workflow using this module:

```shell
pixi shell # activate this project's environment
cd tests/integration/ # navigate to the integration example
snakemake --use-conda --cores 2 # run the workflow!
```

## References
<!-- Please provide thorough referencing below -->

This module is based on the following research and datasets:

*
*
15 changes: 7 additions & 8 deletions config/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
# Configuration
We recommend consulting the following before using this module:
- `config/config.yaml`: a generic example configuration of this module.
- `workflow/internal/config.schema.yaml`: a schematic overview of all the configuration options of this module.
- `INTERFACE.yaml`: lists module input and output files, and their default locations.
- `tests/integration/Snakefile`: an example of how to call this module from another workflow.

This workflow is part of the [clio project](https://clio.readthedocs.io/en/latest/).
Please consult our documentation for more details.
Other useful resources are:

- `INTERFACE.yaml`: user input files (placed in `resources/user`) and module output files (placed in `results`).
- `workflow/internal/config.schema.yaml`: general configuration options.
- `tests/integration/`: a simple example of how to use this module.
This data module is part of the [Modelblocks](https://www.modelblocks.org/) project.
Please consult the [Modelblocks documentation](https://modelblocks.readthedocs.io/) for more details.
1 change: 1 addition & 0 deletions figures/.gitkeep
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# For module documentation figures
1 change: 1 addition & 0 deletions mypy.ini
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
[mypy]
disable_error_code = import-untyped
exclude = (^|/)\.(snakemake|pixi)(/|$)
exclude_gitignore = True
Loading
Loading