bump cosign and add support for additional platforms

[xavier-dfns]

Summary

Prebuilt binaries are shipped by cosign for ppc64le, riscv64 and s390x.
Add support for these platforms in the action.

Bump cosign version at the same time

[cpanato]

@xavier-dfns those platforms are available in github actions runners?

[xavier-dfns]

@cpanato No it's not supported officially. We are building the action runner binary ourselves and self-hosting the runner. IBM also provide a hosted runner: https://community.ibm.com/community/user/blogs/mingmay-pang2/2024/02/05/github-actions-runner-for-ibm-z-and-linuxone
For riscv some cloud provider supports it: https://github.com/apps/rise-risc-v-runners

[Hayden-IO]

We are going to stop building for those platforms at the next major release. We have no way to test releases on these platforms, and I don't want to ship a pre-compiled binary that we haven't been able to validate is working as expected.

[Hayden-IO]

If official runners add support, we can add these platforms to our test matrix.

[eshattow]

@Hayden-IO writes:

We have no way to test releases on these platforms...If official runners add support, we can add these platforms to our test matrix.

RISE RISC-V runners are no-cost for open source community, and function the same as official runners provided the GitHub marketplace "app" has been configured by the GitHub account organization or personal account. They are professionally managed and run on real hardware e.g. for true representative test of releases. There is nothing exotic about riscv64 architecture it should not be any challenge to support and is massively growing in use for A.I. LLM workloads.

I don't know what to say for powerpc and S/390 more than those remain important in security-critical roles and it would be disappointing to gatekeep out these platforms that were already functional in previous cosign releases. That should depend on sponsorship and/or community involvement though. I do know at least all major hardware vendors designing new products to market have RISC-V included as functional portions of their engineering process.

Anyways the correct measure would be to turn up all the platforms supported by cosign checksums.txt and actually verify the checksums.txt itself is not tampered with, instead of this manual labor process. If you have concerns about specific platforms those should be addressed upstream of cosign-installer in cosign itself.

Meanwhile (at least until next major cosign release) I will maintain a fork for RISC-V users and continue to send patches upstream. I respect there is not much community spirit these days and maintainers are burned out on accepting contributions. I would like to know what tests need to be assessed for platforms to be considered able to be supported?

[Hayden-IO]

Ideally it's as simple as updating the test matrix for the RISC-V platform, or any new platform. Without CI testing, we can't commit to not accidentally breaking the platform with future changes. And even if an issue is found during testing, we would likely have to rely on the community for a fix, none of the maintainers develop on that platform.

In the meantime, if you do find bugs for this platform, fixes are appreciated.

[eshattow]

Ideally it's as simple as updating the test matrix for the RISC-V platform, or any new platform. Without CI testing, we can't commit to not accidentally breaking the platform with future changes. And even if an issue is found during testing, we would likely have to rely on the community for a fix, none of the maintainers develop on that platform.

In the meantime, if you do find bugs for this platform, fixes are appreciated.

Would it be appropriate to have an "allow-unsupported" or some such option, as testing coverage may be a requirement before a platform is considered Tier 1 supported?

Does passing the test coverage in CI with a hosted runner make a platform suitable for release or is there more assurance needed (community maintainers, build duration qualifiers, etc.)? I am reading this as you're having to take a position that if Microsoft-owned GitHub doesn't directly support a given platform then it is not possible, and/or kind of vauge on what community involvement looks like that would be acceptable for community support of a cosign platform; is this written down yet or codified so cosign-installer can track on changes to cosign (and by extension, cosign supported platforms) without the second layer of manual gatekeeping?

@xavier-dfns we should pivot to upstream cosign adding platforms to the testing matrix, and enforcing in a more automatic way then for cosign-installer to only allow by default platforms that cosign has in its test matrix, or, whatever metric is appropriate (allowlist / denylist); would you strip the platform additions here for the moment and make this just a cosign version bump? Thanks! -E

[Hayden-IO]

These are great questions! I've added this topic of which platforms should be officially supported to our community's TAC meeting this week.

[eshattow]

@Hayden-IO writes:

These are great questions! I've added this topic of which platforms should be officially supported to our community's TAC meeting this week.

Hi is there any news of this?

[Hayden-IO]

Hey, we did chat about this back in June and agreed that if we don't have automated testing for these additional platforms, we shouldn't ship prebuilt binaries. Users can of course build the binaries themselves and we could keep a list of manually tested platforms in documentation.

We can explore using the RISE RISC-V app for adding testing for RISC-V, though we need to confirm it requires no sensitive permissions. Many of these apps that run in CI can have access to secrets, which would be a hard blocker for us to use it.

cc @steiza since i know you looked into risc-v testing

[eshattow]

also cc @gounthar @luhenry who may know more authoritative information about the RISE RISC-V runners inner-workings. I don't know any contacts of the IBM team providing ppc64le s390x runners, @xavier-dfns do you have contacts that would know answers to those questions about secrets?

Adding testing coverage in ci/cd before committing to publishing builds is a similar approach as numpy, that's great!

[gounthar]

I use the RISE runners a lot for riscv64 work upstream, but I am a user of them, not an operator, so Ludovic is the one to trust on the internals.

On the permission question, do not take my word for it. The list is shown at install time and afterwards under Settings > Installations, so you can read exactly what it asks for before granting anything.

My understanding, and you work in this area far more than I do, is that the app's scope and what a job can reach are separate questions. A job on any self-hosted runner sees whatever secrets the workflow hands it, plus its own GITHUB_TOKEN, whoever operates the machine. Same for the IBM Z runners. If that is right, the riscv64 job can sit at permissions: contents: read with no secrets and off pull_request_target, and the app's scope stops being the deciding factor. A build-and-test job does not need any of them.

There is a second risk worth separating from the secrets one, and it runs the other way: GitHub's documented caveat for self-hosted runners is about the code, not the operator. "We recommend that you only use self-hosted runners with private repositories. This is because forks of your public repository can potentially run dangerous code on your self-hosted runner machine by creating a pull request that executes the code in a workflow." (https://docs.github.com/en/actions/how-tos/manage-runners/self-hosted-runners/manage-access). That applies to any third-party pool on a public repo.

As a precedent, k0s runs a nightly riscv64 build, unit tests and two smoke suites on ubuntu-24.04-riscv (https://github.com/k0sproject/k0s/blob/main/.github/workflows/riscv64.yml, added by Ludovic in April). The run I looked at this week is 5/5 green across build, airgap image bundle, unit tests and both smoketests: https://github.com/k0sproject/k0s/actions/runs/33144860413

If it would help to see whether cosign actually passes on riscv64 before you decide anything, I can send a PR wiring it into the test matrix and chase whatever breaks.

[luhenry]

We can explore using the RISE RISC-V app for adding testing for RISC-V, though we need to confirm it requires no sensitive permissions. Many of these apps that run in CI can have access to secrets, which would be a hard blocker for us to use it.

Hi, happy to answer any question about the RISE RISC-V Runners. The only permissions it needs are:

• Metadata (read): it's required for every GitHub Apps
• Self-hosted runners (read/write): it's to register/unregister runners

There is no reason to have any other credentials so we won't be requesting more :)

You can find documentation about the infrastructure and how it's implemented here.