Skip to content

Ensure consistent units of the realization coordinate in RealizationClusterAndMatch - #2414

Open
gavinevans wants to merge 4 commits into
metoppv:masterfrom
gavinevans:mobt_783_realization_units
Open

Ensure consistent units of the realization coordinate in RealizationClusterAndMatch#2414
gavinevans wants to merge 4 commits into
metoppv:masterfrom
gavinevans:mobt_783_realization_units

Conversation

@gavinevans

@gavinevans gavinevans commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Related issues: #2259, #2305, #2332, #2340, #2359, #2361

Description
This PR makes a minor edit to ensure the the realization coordinate units are consistent prior to attempting to merge cubes together from different sources. These forecast sources should have the same units for the realization coordinate, however, this can sometimes not be true, with some forecast sources having a realization coordinate with units of "1" and some forecast sources having a realization coordinate with units of unknown. To avoid this resulting in merge failures, the units of the realization coordinate is manually set to "1".

Testing:

  • Ran tests and they passed OK
  • Added new tests for the new feature(s)

@mo-jbeaver mo-jbeaver left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a few comments to the code but the tests all ran successfully.

Comment thread improver/clustering/realization_clustering.py
Comment thread improver/clustering/realization_clustering.py
@mo-jbeaver mo-jbeaver assigned gavinevans and unassigned mo-jbeaver Jul 23, 2026
@gavinevans gavinevans assigned mo-jbeaver and unassigned gavinevans Jul 24, 2026
mo-jbeaver
mo-jbeaver previously approved these changes Jul 24, 2026

@mo-jbeaver mo-jbeaver left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the docstring and explaining the unit's context. Happy to approve.

@mo-jbeaver mo-jbeaver removed their assignment Jul 24, 2026
@brhooper brhooper self-assigned this Jul 27, 2026

@brhooper brhooper left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @gavinevans

The code here looks fine, but I don't understand why it is necessary or desirable. Do we understand how we can end up in a situation where the units of forecast cubes do not accurately reflect the actual units of the quantities within?

contains more realizations than the primary input.
contains more realizations than the primary input. It also demonstrates that
input cubes can carry realization units of "unknown" while processing still
succeeds because output realization units are normalised prior to merging.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
succeeds because output realization units are normalised prior to merging.
succeeds because output realization units are unified prior to merging.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Corrected. Regarding the reason that this change is necessary, I've put a bit more context in this comment as to why the realization coordinate units are currently mismatching.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Would it make more sense to modify improver/standardise.py (maybe this function

def _standardise_dtypes_and_units(cube: Cube) -> None:
) so that all cubes with a realisation coordinate have the units of that coordinate enforced to be 1?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean to enforce that the realization coordinate as a unit of 1 regardless of the input, similar to the way that the time coordinates have their units converted regardless after getting the centralised accepted units for the time coordinates (https://github.com/metoppv/improver/blob/master/improver/metadata/check_datatypes.py#L109)? I'd thought of modifying the standardise CLI/plugin to be able to set the units of a coordinate, but I thought that would be a fairly random addition as setting the units, rather than converting the units from one unit to another would be a bit odd in a lot of situations, so I didn't think this was best placed as a general fix, so I've just fixed it where it matters in this PR. If you're interested in enforcing that realization coordinate has a unit of 1 regardless of the input through standardise, as done for the time coordinates, perhaps that is something that could be done as a generalisation later (i.e. after this PR is merged), so as not to complicate things in the short-term.

@brhooper brhooper assigned gavinevans and unassigned brhooper Jul 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants