Konflux operator integration test by AdamSaleh · Pull Request #105 · rh-gitops-midstream/catalog

AdamSaleh · 2026-03-09T13:41:11Z

This should be ready for review.

The code itself was produced mostly by claude but I reviewed and tested it extensively.

Intent of the pipeline:

The pipeline uses a specific test image, that is split into base, that should be rebuilt infrequently and a layer on top of it that contains all of the scripts used in the pipeline
the scripts are configured by env-vars, making them easy to use from pipelines or when testing locally
Pipeline provisions it's own konflux cluster - it is based on arm64 hypershift, version 4.14
It installs appropriate catalog and installs the latest operator.
runs the parallel test-suite from https://github.com/rh-gitops-release-qa/gitops-operator that houses the for of gitops-operator, to facilitate for fast test updates
after it finishes it pushes installation logs, test logs and assorted debug information as quay artefact
it sends a message to gitops-test-notification channel

Outstanding questions:

what openshift version should we test against?
should there be difference for running against master?
any part of the pipeline that looks wonky?

svghadi · 2026-03-09T14:35:10Z

/ok-to-test

AdamSaleh · 2026-03-09T16:17:49Z

/ok-to-test

AdamSaleh · 2026-03-10T06:48:39Z

/ok-to-test

AdamSaleh · 2026-03-11T08:22:23Z

/retest

AdamSaleh · 2026-04-03T16:25:13Z

/retest

red-hat-konflux · 2026-04-05T21:19:57Z

Caution

There are some errors in your PipelineRun template.

PipelineRun	Error
steps/prepare-and-push-logs.yaml	`yaml validation error: line 116: could not find expected ':'`

AdamSaleh · 2026-04-06T15:29:52Z

/retest

AdamSaleh · 2026-04-09T16:35:27Z

/retest

AdamSaleh · 2026-04-11T10:13:45Z

/retest

svghadi

LGTM.
Thanks @AdamSaleh . This is great work. It will definitely speed up release testing.

svghadi · 2026-04-14T09:41:33Z

+  params:
+    - description: Snapshot of the application
+      name: SNAPSHOT
+      default: '{"components": [{"name":"gitops-operator-bundle-main", "containerImage": "quay.io/redhat-user-workloads/rh-openshift-gitops-tenant/gitops-operator-bundle:latest"}]}'


Suggested change

default: '{"components": [{"name":"gitops-operator-bundle-main", "containerImage": "quay.io/redhat-user-workloads/rh-openshift-gitops-tenant/gitops-operator-bundle:latest"}]}'

default: '{"components": [{"name":"gitops-operator-bundle-main", "containerImage": "quay.io/redhat-user-workloads/rh-openshift-gitops-tenant/gitops-operator-bundle:main"}]}'

There is no latest tag published by the konflux pipeline

Fortunately I get that filled by the pipeline, but point taken, will ammend :D

svghadi · 2026-04-14T09:46:46Z

+              - name: imageContentSources
+                value: |


We need to make a note somewhere to update these values when https://github.com/rh-gitops-midstream/catalog/blob/main/.tekton/images-mirror-set.yaml changes

Ok, I will look into this and see if I can fetch the yaml instead. There should be a param in that step for that.

svghadi · 2026-04-14T09:55:03Z

+                value: "quay.io/devtools_gitops/test_image"
+            script: |
+              #!/bin/bash
+              /usr/local/bin/run-and-save-logs.sh /usr/local/bin/install-operator.sh


I presume we will have another PR to make this configurable to install the operator version to test.

Hm ... I think I could actually make this configurable from the pipeline parameters configurable in konflux.
Probably:

version or channel for the operator

version of the openshift to provision

With these two we would have single pipeline be able to cover reasonable amount of the testing ~matrix

svghadi · 2026-04-14T09:58:04Z

+              - name: BRANCH
+                value: "master"


Same as previous comment.

Will ammend

AdamSaleh · 2026-04-22T15:50:00Z

/retest

AdamSaleh · 2026-04-24T16:23:00Z

/retest

AdamSaleh · 2026-05-02T09:16:06Z

/retest

AdamSaleh · 2026-05-14T08:20:30Z

/retest

AdamSaleh · 2026-05-16T13:35:33Z

/retest

AdamSaleh · 2026-05-18T05:12:09Z

/retest

There are currently four test-suites being run: - gitops-operator's e2e ginkgo test-suite, sharded into 3 scripts - the rollouts e2e tests - gitops operator's ui test verifying login (more tests to come) - the argocd tests in a separate pipeline There is simple parametrized pipeline, where you can choose: - the openshift version - size of cluster nodes - the channel to be used in the catalog - the test-script to run Secont separate pipeline installs standalone argocd and runs the e2e tests All the tests are run from precompiled docker image, the pipeline will check at the start and build them if hte images were changed. The test and utility scripts always get copied. The logs get uploaded to quay. At the end of the pipeline, it will send a message to gitops-test-notification channel on slack The code is mostly authored by prompting claude and tested against the v1.20 branch of the catalog repo. Assisted-by: Claude <usersafety@anthropic.com> Signed-off-by: Adam Saleh <adam@asaleh.net>

AdamSaleh changed the title ~~Konflux integration dast integration test~~ WIP: Konflux integration dast integration test Mar 9, 2026

AdamSaleh marked this pull request as draft March 9, 2026 14:14

AdamSaleh force-pushed the konflux-integration branch from 41d18bd to 703a458 Compare March 9, 2026 16:04

AdamSaleh force-pushed the konflux-integration branch from 703a458 to 2fd5e19 Compare March 10, 2026 08:14

AdamSaleh force-pushed the konflux-integration branch from 82f03d4 to 89e6493 Compare April 3, 2026 16:32

AdamSaleh force-pushed the konflux-integration branch 5 times, most recently from 935fa2d to 74dbe55 Compare April 6, 2026 15:00

AdamSaleh force-pushed the konflux-integration branch 13 times, most recently from 762df94 to 9348d5f Compare April 8, 2026 14:43

AdamSaleh changed the title ~~WIP: Konflux integration dast integration test~~ WIP: Konflux operator integration test Apr 8, 2026

AdamSaleh force-pushed the konflux-integration branch from 9348d5f to 888e847 Compare April 9, 2026 14:09

AdamSaleh force-pushed the konflux-integration branch from 888e847 to 0184f23 Compare April 11, 2026 09:15

AdamSaleh force-pushed the konflux-integration branch 4 times, most recently from d3f83b9 to a42fb1c Compare April 12, 2026 14:36

AdamSaleh marked this pull request as ready for review April 13, 2026 08:12

AdamSaleh changed the title ~~WIP: Konflux operator integration test~~ Konflux operator integration test Apr 13, 2026

svghadi approved these changes Apr 14, 2026

View reviewed changes

AdamSaleh force-pushed the konflux-integration branch from b880081 to 8511fb9 Compare April 28, 2026 07:18

AdamSaleh force-pushed the konflux-integration branch 3 times, most recently from d0b8e3a to 0486ac2 Compare May 13, 2026 17:02

AdamSaleh force-pushed the konflux-integration branch from a1078f5 to a3ad998 Compare May 14, 2026 09:07

AdamSaleh force-pushed the konflux-integration branch 2 times, most recently from dba2d0f to 6757062 Compare May 21, 2026 11:59

AdamSaleh force-pushed the konflux-integration branch from 6757062 to e623277 Compare May 22, 2026 08:08

	default: '{"components": [{"name":"gitops-operator-bundle-main", "containerImage": "quay.io/redhat-user-workloads/rh-openshift-gitops-tenant/gitops-operator-bundle:latest"}]}'
	default: '{"components": [{"name":"gitops-operator-bundle-main", "containerImage": "quay.io/redhat-user-workloads/rh-openshift-gitops-tenant/gitops-operator-bundle:main"}]}'

Conversation

AdamSaleh commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

svghadi commented Mar 9, 2026

Uh oh!

AdamSaleh commented Mar 9, 2026

Uh oh!

AdamSaleh commented Mar 10, 2026

Uh oh!

AdamSaleh commented Mar 11, 2026

Uh oh!

AdamSaleh commented Apr 3, 2026

Uh oh!

red-hat-konflux Bot commented Apr 5, 2026

Uh oh!

AdamSaleh commented Apr 6, 2026

Uh oh!

AdamSaleh commented Apr 9, 2026

Uh oh!

AdamSaleh commented Apr 11, 2026

Uh oh!

svghadi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AdamSaleh commented Apr 22, 2026

Uh oh!

AdamSaleh commented Apr 24, 2026

Uh oh!

AdamSaleh commented May 2, 2026

Uh oh!

AdamSaleh commented May 14, 2026

Uh oh!

AdamSaleh commented May 16, 2026

Uh oh!

AdamSaleh commented May 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

AdamSaleh commented Mar 9, 2026 •

edited

Loading