Parse the highest from newly retrieved upstream version

[nforro]

Some projects have multiple release streams and the latest version released upstream may not be the highest version ever captured. The version field always contains the highest recorded version, no matter what version actually triggered the event. Instead, read the version from the upstream_versions field that contains newly retrieved versions. Unfortunately, there can be more of those (depending on the configured backend), so choose the highest one.

For more context see fedora-infra/anitya#2009.

[centosinfra-prod-github-app]

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.

[gemini-code-assist]

Code Review

This pull request correctly changes the logic to parse the highest version from the upstream_versions field in the new hotness update event, which is more accurate than using the top-level version field. I've added one suggestion to make the implementation more robust against cases where upstream_versions might be missing or empty in the event payload, which would prevent potential runtime errors.

[JoseExposito]

Hey Nikola,

Thanks a lot for implementing this feature!!

I added a minor comment about the version path in the JSON, but other than that (and with no context of the rest of the code of the project) this change looks good to me.

[centosinfra-prod-github-app]

Build succeeded.
https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/packit-service/buildset/4a94c5d0c27546febf3f0352a247d59c

✔️ pre-commit SUCCESS in 1m 47s
✔️ packit-service-tests SUCCESS in 3m 48s

[lbarcziova]

the change looks good to me, is there any chance this could break existing users? @mfocko could you please also have a look, I recall you were dealing with this code for CentOS stream updates.

[nforro]

is there any chance this could break existing users

I don't think so, but I can't be 100% sure.

I recall you were dealing with this code for CentOS stream updates

Weren't those Anitya messages?

[nforro]

I don't think so, but I can't be 100% sure.

I mean, this could only break projects where the latest upstream version is not always the highest, but that can be mitigated with upstream_tag_include/upstream_tag_exclude, unless they for some reason rely on pull-from-upstream running for version X triggered by a release of version Y (why would someone do that though?). And "break" in this context means opening a PR or a set of PRs that can be ignored/closed.

[centosinfra-prod-github-app]

Build succeeded.
https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/packit-service/buildset/9ed6db046fc94ef7a789ee6f92796e86

✔️ pre-commit SUCCESS in 1m 49s
✔️ packit-service-tests SUCCESS in 3m 46s

[JoseExposito]

I mean, this could only break projects where the latest upstream version is not always the highest

If I understood correctly, monitoring status would need to be set to "monitoring all" as well. The recommended option in the docs is "monitoring". The former notifies about all releases, the later notifies only about the highest.

So that could be another way of mitigation/configuration.

[nforro]

If I understood correctly, monitoring status would need to be set to "monitoring all" as well. The recommended option in the docs is "monitoring". The former notifies about all releases, the later notifies only about the highest.

So that could be another way of mitigation/configuration.

Good point 👍

[grulja]

Am I right that this will not work in case upstream releases two versions at the same time? Which I think is not a rare case, for example when upstream issues a fix for CVE across multiple supported versions of the same project.

[nforro]

Am I right that this will not work in case upstream releases two versions at the same time?

That's still two releases, hence two separate events.

[nforro]

That's still two releases, hence two separate events.

Actually, thinking about it, it probably depends on the backend and how (and how often) it parses the releases.
@Zlopez Am I correct? If an upstream project does two releases shortly after each other, is it possible there will be just one event with upstream_versions containing both those releases?

If that's the case, we need to handle this on Packit side by cloning the event and setting proper version in each instance.

[Zlopez]

@nforro That was the reason why the upstream_versions even exists to be able to report multiple new versions retrieved at once, before that only if the latest changed we reported on it.

Just to show some example: Upstream project releases version 2.5.1 and also release version 1.9.2 with the same fix. In current state upstream_versions = ['2.5.1', '1.9.2'] in previous solution only 2.5.1 would be reported.

[nforro]

If that's the case, we need to handle this on Packit side by cloning the event and setting proper version in each instance.

After giving this more thought, it would be better to capture the list of versions in the event (like events.anitya.VersionUpdate does already) and process it in the handler, calling sync_release() for each tag inferred from each version in the list.
However, work has been done to make sync_release() accept multiple versions, so it seems the intent is to do the processing there and not in the handler. I suppose that makes sense from CLI PoV, but it will require more changes to existing code (all it does currently is take the first version from the list and ignore the rest) and passing multiple versions is not enough, we'd have to allow passing multiple tags as well.
@mfocko @lbarcziova WDYT?

[mfocko]

Wouldn't it make sense to move this, sometime in the future, to the specfile?

[nforro]

Perhaps, though this is targeted to compare Python version specifiers (compatible with SemVer), while specfile already provides comparable (N)EVR(A) objects that represent RPM versioning with its rules.

[mfocko]

I see… sorry for opening another can of worms…

[mfocko]

the change looks good to me, is there any chance this could break existing users? @mfocko could you please also have a look, I recall you were dealing with this code for CentOS stream updates.

@lbarcziova CentOS should consume different topic, Anitya, IIRC, produces either

• new hotness — latest greatest, which is great for Rawhide and Fedora, or
• version update — all new releases, which is great for stable Fedora (never got adopted…) or CentOS

[lbarcziova]

@nforro if we don't want to support multi-version sync from the CLI , then I agree that extending the handler to loop over the versions and create a separate sync run per version could be more straightforward. This could help with easier reporting/dashboard separation. Since the previous work was really initial, I think starting fresh at the handler level could be the cleaner path if we all agree.

[nforro]

Thanks, let's discuss this tomorrow on arch. To reiterate, the plan would be:

• Unify the Anitya events so that events.anitya.NewHotness also captures a list of versions
• Adjust GetSyncReleaseTagMixin to work with a list of versions
• Modify AbstractSyncReleaseHandler and run sync_release() for every tag (or every version for non-git upstreams)
• (optional) Revert the change in sync_release() so it accepts a single version again

Implications:

• Nothing should change for existing projects/packages with monitoring status set to Monitoring
• Packages with monitoring status set to Monitoring all will get pull-from-upsteam triggered on every release, no matter the version, and it's their responsibility to configure the job(s) to accept only the required version stream (using version_update_mask or upstream_tag_include/upstream_tag_exclude)
• For manual triggering it will be necessary to implement Support version argument when retriggering pull-from-upstream #3033
• We may also consider adding support for version specifiers/specifier sets to allow for more granular version filtering: Allow specifying version update mask using simplified syntax packit#2191 (comment)