Skip to content

set ens members to one, this is about 3x faster for a single forecast#12

Open
JoachimKoenigslieb wants to merge 5 commits intodmidk:mainfrom
JoachimKoenigslieb:feature/ens-member-1
Open

set ens members to one, this is about 3x faster for a single forecast#12
JoachimKoenigslieb wants to merge 5 commits intodmidk:mainfrom
JoachimKoenigslieb:feature/ens-member-1

Conversation

@JoachimKoenigslieb
Copy link
Copy Markdown

Running with this change I get:

((venv) ) (base) jjk@nwp-archive2:~/run-sunflow> time ./venv/bin/sunflow --run_mode files --dataset DWD --bbox CUSTOM --custom-bbox="-10.75,47.25,20,63.5" --time "2026-02-01T09:00"
Pysteps configuration file found at: /home/jjk/run-sunflow/venv/lib/python3.12/site-packages/pysteps/pystepsrc
...
2026-05-06 13:42:11.480 | INFO     | sunflow.main:cli:478 - Script end time = 2026-05-06 13:42:11 UTC

real    0m4.343s
user    0m3.673s
sys     0m0.762s

And without I get:

real    0m12.015s
user    0m57.413s
sys     0m10.522s

So wall-clock is about 3x faster and the user a lot lower because it does not do multi-processing stuff to get the ens members I presume.

@KristianHMoller
Copy link
Copy Markdown
Contributor

KristianHMoller commented May 6, 2026

Thanks @JoachimKoenigslieb !

This is great!

But perhaps, we should do it a bit more future proof and use this variable from config.py: ens_members=int(os.getenv("ENS_MEMBERS", "1")),
It defaults to one, so there should be no issue

@JoachimKoenigslieb
Copy link
Copy Markdown
Author

Yes, that sounds like a great idea.

I've used the value from the config instead!

@KristianHMoller
Copy link
Copy Markdown
Contributor

Thanks a lot @JoachimKoenigslieb !

I can confirm the speed up:

With update:

real	0m8.143s
user	0m10.613s
sys	0m1.547s

Without update :

real	0m15.351s
user	1m32.068s
sys	0m11.578s

If you fix the linting (with pre-commit install; pre-commit run --all-files it should be done automatically) and update the CHANGELOG.md, I will approve and then we will see who has authorization to merge.

Copy link
Copy Markdown
Contributor

@KristianHMoller KristianHMoller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With an entry in CHANGELOG.md, this is ready for approval and merge!

Comment thread CHANGELOG.md
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]
- Pass down number of ensembles from configurations to `ProbabilisticAdvection`. This gives a roughly 3x speedup in the no ensembles case.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Pass down number of ensembles from configurations to `ProbabilisticAdvection`. This gives a roughly 3x speedup in the no ensembles case.
- Pass down number of ensembles from configurations to `ProbabilisticAdvection`. This gives a roughly 3x speedup in the no ensembles case [!12](https://github.com/dmidk/sunflow/pull/12), @JoachimKoenigslieb

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants