[AI] OSAC-3593: Move e2e test suite (tests/) from osac-test-infra into the osac mono-repo - #409
Conversation
… osac mono-repo Co-authored-by: Ameya Sathe <asathe@redhat.com>
|
@osac-jira-ai-issue-solver[bot]: This pull request references OSAC-3593 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "5.1.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
AI Session Costs
|
|
🤖 Finished Review · ❌ Failure · Started 6:13 AM UTC · Completed 6:55 AM UTC Commit: |
-Upgrade pygments transitive dependency from 2.19.2 to >= 2.20.0 via uv lock --upgrade-package pygments in tests/e2e/uv.lock. -This resolves the low-severity ReDoS vulnerability (GHSA-5239-wwwm-4pmq) flagged during the e2e test suite migration from osac-test-infra. Signed-off-by: Ameya Sathe <asathe@redhat.com> Assisted-by: Chai Bot <ship-help-github@redhat.com> rh-pre-commit.version: 2.4.0 rh-pre-commit.check-secrets: ENABLED
|
🤖 Review · Commit: |
…dependabot at different runs [OSAC-3593] Signed-off-by: Ameya Sathe <asathe@redhat.com> Assisted-by: Chai Bot <ship-help-github@redhat.com> rh-pre-commit.version: 2.4.0 rh-pre-commit.check-secrets: ENABLED
|
🤖 Finished Review · ✅ Success · Started 9:04 AM UTC · Completed 9:46 AM UTC Commit: |
ReviewFindingsMedium
Low
Previous runReviewFindingsHigh
Medium
Low
Labels: PR adds Python e2e test suite with uv lock, modifies GitHub Actions workflows, and restructures repo infrastructure. Next steps:
|
![fullsend-ai-review[bot]](https://avatars.githubusercontent.com/in/3478501?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
fullsend-ai-review
Bot
added
python:uv
Auto-dismissed: only Prow labels gate merging
Update references in docs/CONVENTIONS.md, osac-csi-driver/AGENTS.md, and osac-operator/AGENTS.md to reflect that the e2e test suite has migrated from the external osac-test-infra repository into tests/e2e/ within the mono-repo. Signed-off-by: Ameya Sathe <asathe@redhat.com> Assisted-by: Claude Code <noreply@anthropic.com> rh-pre-commit.version: 2.4.0 rh-pre-commit.check-secrets: ENABLED
|
🤖 Review · ❌ Terminated · Started 10:42 AM UTC · Ended 11:24 AM UTC Commit: |
|
/fs-fix Address all Low findings: update CONVENTIONS.md cross-repo reference, fix component count in AGENTS.md, move unique_name to tests/core/helpers.py, fix default_networking return type annotation, remove duplicate --as system:admin in helpers.py:633, and remove ANN101 from per-file-ignores in pyproject.toml. |
There was a problem hiding this comment.
See the review comment for full details.
Note: The following inline comments could not be posted on the diff (GitHub returned 422) and are included here instead:
AGENTS.md(file-level): Line 3 · [medium] documentation-consistency
Prose says 'seven components, each with its own CLAUDE.md/AGENTS.md' but the Components table now has eight entries after adding tests/e2e/. The count is stale. Additionally, tests/e2e/ has no CLAUDE.md or AGENTS.md, making it the only listed component without component-level docs.
Suggested fix: Update the count to 'eight' or remove the specific count. Add a minimal tests/e2e/AGENTS.md covering test structure, fixtures, and local run instructions.
tests/e2e/Containerfile:11: [medium] supply-chain
OpenShift client (oc, kubectl) and grpcurl are downloaded via HTTPS without checksum verification. The OSAC CLI download in the same file uses SHA256 checksums, demonstrating the pattern but not applying it consistently.
Suggested fix: Add SHA256 checksum verification for both downloads. Pin the OCP version rather than using 'stable' to make checksums deterministic.
tests/e2e/Containerfile:14: [low] supply-chain-image-pinning
Base image (ubi9/ubi:latest) and uv (ghcr.io/astral-sh/uv:latest) use mutable :latest tags without digest pins.
Suggested fix: Pin both images by digest or specific version tag.
fullsend-ai-review
Bot
added
the
requires-manual-review
|
🤖 Finished Review · ✅ Success · Started 10:42 AM UTC · Completed 11:24 AM UTC Commit: |
sk-ilya
left a comment
sk-ilya
left a comment
There was a problem hiding this comment.
could we please preserve the original repos commit history when migrating to monorepo? 🙏
|
/hold |
openshift-ci
Bot
added
the
do-not-merge/hold
Auto-dismissed: only Prow labels gate merging
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: osac-jira-ai-issue-solver[bot] The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
Closing this PR based on review feedback from @sk-ilya — the migration should preserve the original A revised approach using AI-generated. Review for accuracy. |
…est-infra (1/5) (osac-project#488) ## Summary History-preserving migration of the **bmaas** e2e test suite from [osac-test-infra](https://github.com/osac-project/osac-test-infra) into this mono-repo. This is **PR 1 of 5** in a stacked series — one per test suite (bmaas → vmaas → caas → catalog → storage). Commit history from `osac-test-infra` is preserved via `git filter-repo`, so `git log tests/bmaas/` shows the original authors and messages. ### What this PR adds - **`tests/bmaas/`** — BMaaS lifecycle, inventory-exhaustion, and networking e2e tests with suite-level `conftest.py` - **`tests/core/`** — shared test utilities used by all suites (`grpc_client`, `k8s_client`, `keycloak`, `helpers`, `runner`, etc.) - **`tests/conftest.py`** — session-scoped fixtures (gRPC client, Keycloak setup, tenant provisioning) inherited by every suite - **`pyproject.toml`** — pytest configuration (`testpaths`, `pythonpath`, `pytest-xdist` parallelism, markers, logging) migrated from `osac-test-infra` ### Placement decision Tests live at repo-root `tests/` (not nested under components). This preserves the existing import chain (`from tests.core.*`) and pytest conftest hierarchy with zero refactoring. ### Stack overview | PR | Suite | Base | |----|-------|------| | **1/5 ([this PR](osac-project#488 (comment) | **bmaas** + shared infra | `([main](https://github.com/osac-project/osac/tree/main))` | | [2/5](osac-project#489) | vmaas | this branch | | 3/5 | caas | vmaas branch | | 4/5 | catalog | caas branch | | 5/5 | storage | catalog branch | > **Merge order matters** — this PR must land first since it contains `pyproject.toml`, `tests/core/`, and the shared `conftest.py` that all subsequent suites depend on. ## Context - Jira: [OSAC-3593](https://redhat.atlassian.net/browse/OSAC-3593) - Parent: [OSAC-4255](https://redhat.atlassian.net/browse/OSAC-4255) - Supersedes: osac-project#409 (abandoned — did not preserve commit history per reviewer @sk-ilya's [feedback](osac-project#409 (review))) ## Test plan - [ ] Verify `git log tests/bmaas/` shows original commit history from `osac-test-infra` - [ ] Verify `git log tests/core/` shows original commit history - [ ] Confirm pytest discovers all bmaas tests: `python -m pytest tests/bmaas/ --collect-only` - [ ] Full e2e validation after all 5 PRs are merged (per [OSAC-3593](https://redhat.atlassian.net/browse/OSAC-3593) acceptance criteria) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added a comprehensive end-to-end testing foundation for compute, networking, bare-metal, authentication, metering, and lifecycle workflows. - Added coverage for networking connectivity, tenant isolation, external IP behavior, NAT, and resource cleanup. - Added bare-metal lifecycle, restart, inventory exhaustion, provisioning, and recovery scenarios. - Added reusable command-line, API, Kubernetes, authentication, polling, and metering test utilities. - **Chores** - Added project configuration for testing, formatting, linting, and type checking. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Will Gordon <wgordon@redhat.com> Signed-off-by: Elad Tabak <etabak@redhat.com> Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com> Signed-off-by: Juan Hernandez <juan.hernandez@redhat.com> Signed-off-by: Alberto Losada <alosadag@redhat.com> Signed-off-by: Zoltan Szabo <zszabo@redhat.com> Signed-off-by: Dakota Crowder <dcrowder@redhat.com> Signed-off-by: Ygal Blum <ygal.blum@gmail.com> Signed-off-by: Haim Tayrie <htayrie@htayrie-thinkpadt14gen5.raanaii.csb> Signed-off-by: Nick Carboni <ncarboni@redhat.com> Signed-off-by: Vladik Romanovsky <vromanso@redhat.com> Signed-off-by: Tzif <tmorgens@redhat.com> Signed-off-by: MENNY ABOUSH <maboush@maboush-thinkpadt14gen5.raanaii.csb> Signed-off-by: omer-vishlitzky <omer.vishlitzky@gmail.com> Signed-off-by: Marc Sluiter <msluiter@redhat.com> Signed-off-by: Dan Manor <dmanor@redhat.com> Co-authored-by: omer-vishlitzky <omer.vishlitzky@gmail.com> Co-authored-by: openshift-merge-bot[bot] <148852131+openshift-merge-bot[bot]@users.noreply.github.com> Co-authored-by: Will Gordon <wgordon@redhat.com> Co-authored-by: Elad Tabak <etabak@redhat.com> Co-authored-by: Omer Vishlitzky <ovishlit@redhat.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com> Co-authored-by: Juan Hernandez <juan.hernandez@redhat.com> Co-authored-by: Omer Vishlitzky <22615781+omer-vishlitzky@users.noreply.github.com> Co-authored-by: Ori Amizur <60868946+ori-amizur@users.noreply.github.com> Co-authored-by: Dakota Crowder <dcrowder@redhat.com> Co-authored-by: Alberto Losada <alosadag@redhat.com> Co-authored-by: Ilya Skornyakov <iskornya@redhat.com> Co-authored-by: Zoltan Szabo <zszabo@redhat.com> Co-authored-by: CrystalChun <cchun@redhat.com> Co-authored-by: Elior Erez <eerez@redhat.com> Co-authored-by: Crystal <CrystalChun@users.noreply.github.com> Co-authored-by: Ygal Blum <ygal.blum@gmail.com> Co-authored-by: Haim Tayrie <htayrie@htayrie-thinkpadt14gen5.raanaii.csb> Co-authored-by: Ori Amizur <oamizur@redhat.com> Co-authored-by: Nick Carboni <ncarboni@redhat.com> Co-authored-by: Vladik Romanovsky <vromanso@redhat.com> Co-authored-by: Tzif <tmorgens@redhat.com> Co-authored-by: htayrie-rh <htayrie@redhat.com> Co-authored-by: Menny Aboush <mennyaboush@gmail.com> Co-authored-by: MENNY ABOUSH <maboush@maboush-thinkpadt14gen5.raanaii.csb> Co-authored-by: Daniel Erez <danielerez@gmail.com> Co-authored-by: Marc Sluiter <msluiter@redhat.com> Co-authored-by: Siddarth R <sroyapal@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Adrien Gentil <agentil@redhat.com> Co-authored-by: Deivanai Murugappan Rani <dmurugap@redhat.com> Co-authored-by: Dan Manor <dmanor@redhat.com> Co-authored-by: Chai Bot <ship-help-github@redhat.com>
…est-infra (1/5) (osac-project#488) ## Summary History-preserving migration of the **bmaas** e2e test suite from [osac-test-infra](https://github.com/osac-project/osac-test-infra) into this mono-repo. This is **PR 1 of 5** in a stacked series — one per test suite (bmaas → vmaas → caas → catalog → storage). Commit history from `osac-test-infra` is preserved via `git filter-repo`, so `git log tests/bmaas/` shows the original authors and messages. ### What this PR adds - **`tests/bmaas/`** — BMaaS lifecycle, inventory-exhaustion, and networking e2e tests with suite-level `conftest.py` - **`tests/core/`** — shared test utilities used by all suites (`grpc_client`, `k8s_client`, `keycloak`, `helpers`, `runner`, etc.) - **`tests/conftest.py`** — session-scoped fixtures (gRPC client, Keycloak setup, tenant provisioning) inherited by every suite - **`pyproject.toml`** — pytest configuration (`testpaths`, `pythonpath`, `pytest-xdist` parallelism, markers, logging) migrated from `osac-test-infra` ### Placement decision Tests live at repo-root `tests/` (not nested under components). This preserves the existing import chain (`from tests.core.*`) and pytest conftest hierarchy with zero refactoring. ### Stack overview | PR | Suite | Base | |----|-------|------| | **1/5 ([this PR](osac-project#488 (comment) | **bmaas** + shared infra | `([main](https://github.com/osac-project/osac/tree/main))` | | [2/5](osac-project#489) | vmaas | this branch | | 3/5 | caas | vmaas branch | | 4/5 | catalog | caas branch | | 5/5 | storage | catalog branch | > **Merge order matters** — this PR must land first since it contains `pyproject.toml`, `tests/core/`, and the shared `conftest.py` that all subsequent suites depend on. ## Context - Jira: [OSAC-3593](https://redhat.atlassian.net/browse/OSAC-3593) - Parent: [OSAC-4255](https://redhat.atlassian.net/browse/OSAC-4255) - Supersedes: osac-project#409 (abandoned — did not preserve commit history per reviewer @sk-ilya's [feedback](osac-project#409 (review))) ## Test plan - [ ] Verify `git log tests/bmaas/` shows original commit history from `osac-test-infra` - [ ] Verify `git log tests/core/` shows original commit history - [ ] Confirm pytest discovers all bmaas tests: `python -m pytest tests/bmaas/ --collect-only` - [ ] Full e2e validation after all 5 PRs are merged (per [OSAC-3593](https://redhat.atlassian.net/browse/OSAC-3593) acceptance criteria) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added a comprehensive end-to-end testing foundation for compute, networking, bare-metal, authentication, metering, and lifecycle workflows. - Added coverage for networking connectivity, tenant isolation, external IP behavior, NAT, and resource cleanup. - Added bare-metal lifecycle, restart, inventory exhaustion, provisioning, and recovery scenarios. - Added reusable command-line, API, Kubernetes, authentication, polling, and metering test utilities. - **Chores** - Added project configuration for testing, formatting, linting, and type checking. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Will Gordon <wgordon@redhat.com> Signed-off-by: Elad Tabak <etabak@redhat.com> Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com> Signed-off-by: Juan Hernandez <juan.hernandez@redhat.com> Signed-off-by: Alberto Losada <alosadag@redhat.com> Signed-off-by: Zoltan Szabo <zszabo@redhat.com> Signed-off-by: Dakota Crowder <dcrowder@redhat.com> Signed-off-by: Ygal Blum <ygal.blum@gmail.com> Signed-off-by: Haim Tayrie <htayrie@htayrie-thinkpadt14gen5.raanaii.csb> Signed-off-by: Nick Carboni <ncarboni@redhat.com> Signed-off-by: Vladik Romanovsky <vromanso@redhat.com> Signed-off-by: Tzif <tmorgens@redhat.com> Signed-off-by: MENNY ABOUSH <maboush@maboush-thinkpadt14gen5.raanaii.csb> Signed-off-by: omer-vishlitzky <omer.vishlitzky@gmail.com> Signed-off-by: Marc Sluiter <msluiter@redhat.com> Signed-off-by: Dan Manor <dmanor@redhat.com> Co-authored-by: omer-vishlitzky <omer.vishlitzky@gmail.com> Co-authored-by: openshift-merge-bot[bot] <148852131+openshift-merge-bot[bot]@users.noreply.github.com> Co-authored-by: Will Gordon <wgordon@redhat.com> Co-authored-by: Elad Tabak <etabak@redhat.com> Co-authored-by: Omer Vishlitzky <ovishlit@redhat.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com> Co-authored-by: Juan Hernandez <juan.hernandez@redhat.com> Co-authored-by: Omer Vishlitzky <22615781+omer-vishlitzky@users.noreply.github.com> Co-authored-by: Ori Amizur <60868946+ori-amizur@users.noreply.github.com> Co-authored-by: Dakota Crowder <dcrowder@redhat.com> Co-authored-by: Alberto Losada <alosadag@redhat.com> Co-authored-by: Ilya Skornyakov <iskornya@redhat.com> Co-authored-by: Zoltan Szabo <zszabo@redhat.com> Co-authored-by: CrystalChun <cchun@redhat.com> Co-authored-by: Elior Erez <eerez@redhat.com> Co-authored-by: Crystal <CrystalChun@users.noreply.github.com> Co-authored-by: Ygal Blum <ygal.blum@gmail.com> Co-authored-by: Haim Tayrie <htayrie@htayrie-thinkpadt14gen5.raanaii.csb> Co-authored-by: Ori Amizur <oamizur@redhat.com> Co-authored-by: Nick Carboni <ncarboni@redhat.com> Co-authored-by: Vladik Romanovsky <vromanso@redhat.com> Co-authored-by: Tzif <tmorgens@redhat.com> Co-authored-by: htayrie-rh <htayrie@redhat.com> Co-authored-by: Menny Aboush <mennyaboush@gmail.com> Co-authored-by: MENNY ABOUSH <maboush@maboush-thinkpadt14gen5.raanaii.csb> Co-authored-by: Daniel Erez <danielerez@gmail.com> Co-authored-by: Marc Sluiter <msluiter@redhat.com> Co-authored-by: Siddarth R <sroyapal@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Adrien Gentil <agentil@redhat.com> Co-authored-by: Deivanai Murugappan Rani <dmurugap@redhat.com> Co-authored-by: Dan Manor <dmanor@redhat.com> Co-authored-by: Chai Bot <ship-help-github@redhat.com>
4 participants
Resolves OSAC-3593
Summary
Move the e2e test suite from
osac-test-infra/tests/into this mono-repo attests/e2e/, so a PR that changes API behavior can update its own tests in the same PR.tests/e2e/as a component and clarify osac-test-infra now holds only workflow orchestrationTransition plan
The e2e workflows still reference osac-test-infra for the checkout (needed for orchestration actions/scripts). A companion PR in osac-test-infra will update the reusable workflows to source tests from the osac checkout instead, after which osac-test-infra's
tests/copy can be removed.Test plan
pytest --collect-only)from tests.core.grpc_client import GRPCClient)ruff checkandruff formatpass cleanlyuv lock --checkpasses (dependencies resolve)go work sync)