add some tests for the new harvester integration tests by Vicente-Cheng · Pull Request #2730 · harvester/tests

Vicente-Cheng · 2026-06-23T18:49:26Z

Which issue(s) this PR fixes:

Issue #

What this PR does / why we need it:

This PR aims for the following stuff:

reuse the apiclient
we can run tests in parallel
add volume/image basic tests
add pr-baseline to run vm/volume/image

Special notes for your reviewer:

Additional documentation or context

Here is the temporary output:

# ./run.sh -p 8 -i pr-baseline                                                                    
Loading .env file...
Environment variables loaded from .env
======================================
Harvester Robot Framework Test Runner
======================================
Output: ./results
Log level: DEBUG
======================================

Running tests...
Storing .pabotsuitenames file
2026-06-23 18:42:56.279426 [PID:3040124] [4] [ID:4] EXECUTING Tests.Regression.Test Vm
2026-06-23 18:42:56.279666 [PID:3040126] [5] [ID:6] EXECUTING Tests.Regression.Test Volume Expand
2026-06-23 18:42:56.279689 [PID:3040127] [0] [ID:0] EXECUTING Tests.Regression.Test Image
2026-06-23 18:42:56.279802 [PID:3040129] [1] [ID:2] EXECUTING Tests.Regression.Test Image Invalid Url
2026-06-23 18:42:56.279905 [PID:3040131] [6] [ID:5] EXECUTING Tests.Regression.Test Volume
2026-06-23 18:42:56.279933 [PID:3040133] [2] [ID:1] EXECUTING Tests.Regression.Test Image Invalid Checksum
2026-06-23 18:42:56.279973 [PID:3040135] [3] [ID:3] EXECUTING Tests.Regression.Test Image Valid Checksum
2026-06-23 18:42:56.280084 [PID:3040137] [7] [ID:7] EXECUTING Tests.Regression.Test Volume Snapshot
2026-06-23 18:42:57.382513 [PID:3040135] [3] [ID:3] PASSED Tests.Regression.Test Image Valid Checksum in 1.1 seconds
2026-06-23 18:43:00.684726 [PID:3040129] [1] [ID:2] PASSED Tests.Regression.Test Image Invalid Url in 4.4 seconds
2026-06-23 18:43:09.792474 [PID:3040131] [6] [ID:5] PASSED Tests.Regression.Test Volume in 13.5 seconds
2026-06-23 18:43:11.293790 [PID:3040137] [7] [ID:7] still running Tests.Regression.Test Volume Snapshot after 15.0 seconds
2026-06-23 18:43:11.293887 [PID:3040124] [4] [ID:4] still running Tests.Regression.Test Vm after 15.0 seconds
2026-06-23 18:43:11.293939 [PID:3040133] [2] [ID:1] still running Tests.Regression.Test Image Invalid Checksum after 15.0 seconds
2026-06-23 18:43:11.294800 [PID:3040127] [0] [ID:0] still running Tests.Regression.Test Image after 15.0 seconds
2026-06-23 18:43:11.295415 [PID:3040126] [5] [ID:6] still running Tests.Regression.Test Volume Expand after 15.0 seconds
2026-06-23 18:43:15.599436 [PID:3040126] [5] [ID:6] PASSED Tests.Regression.Test Volume Expand in 19.3 seconds
2026-06-23 18:43:22.005519 [PID:3040137] [7] [ID:7] PASSED Tests.Regression.Test Volume Snapshot in 25.7 seconds
2026-06-23 18:43:22.005888 [PID:3040127] [0] [ID:0] PASSED Tests.Regression.Test Image in 25.7 seconds
2026-06-23 18:43:31.315599 [PID:3040124] [4] [ID:4] still running Tests.Regression.Test Vm after 35.0 seconds
2026-06-23 18:43:31.315741 [PID:3040133] [2] [ID:1] still running Tests.Regression.Test Image Invalid Checksum after 35.0 seconds
2026-06-23 18:43:56.337180 [PID:3040124] [4] [ID:4] still running Tests.Regression.Test Vm after 60.0 seconds
2026-06-23 18:43:56.337330 [PID:3040133] [2] [ID:1] still running Tests.Regression.Test Image Invalid Checksum after 60.0 seconds
2026-06-23 18:44:26.366814 [PID:3040124] [4] [ID:4] still running Tests.Regression.Test Vm after 90.0 seconds
2026-06-23 18:44:26.366778 [PID:3040133] [2] [ID:1] still running Tests.Regression.Test Image Invalid Checksum after 90.0 seconds
2026-06-23 18:44:27.968486 [PID:3040133] [2] [ID:1] PASSED Tests.Regression.Test Image Invalid Checksum in 91.6 seconds
2026-06-23 18:44:55.394112 [PID:3040124] [4] [ID:4] PASSED Tests.Regression.Test Vm in 119.0 seconds

Signed-off-by: Vicente Cheng <vicente.cheng@suse.com>

Copilot

Pull request overview

This PR expands the Harvester Robot Framework regression coverage (image + volume lifecycle) and updates the test runner/libs to reuse the shared apiclient/harvester_api with optional suite-level parallel execution and a selectable operation strategy (CRD vs REST).

Changes:

Added new regression suites for volumes (basic/expand/snapshot) and images (basic + negative checksum/URL + optional valid checksum), and tagged a PR baseline set (pr-baseline).
Updated the runner (run.sh) to support pabot parallelism (-p) and strategy selection (-S), and documented these in the README.
Refactored VM/Image/Volume libraries to select CRD vs REST implementations via HARVESTER_OPERATION_STRATEGY, added snapshot readiness support, and replaced the standalone client with an adapter around the shared harvester_api.

Reviewed changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
harvester_robot_tests/tests/regression/test_volume.robot	New basic PVC lifecycle regression suite (PR baseline).
harvester_robot_tests/tests/regression/test_volume_expand.robot	New PVC expansion regression suite (PR baseline).
harvester_robot_tests/tests/regression/test_volume_snapshot.robot	New PVC snapshot/restore regression suite (PR baseline).
harvester_robot_tests/tests/regression/test_vm.robot	Adds `pr-baseline` tag to basic VM regression suite.
harvester_robot_tests/tests/regression/test_image.robot	New image lifecycle regression suite (PR baseline).
harvester_robot_tests/tests/regression/test_image_valid_checksum.robot	New optional “valid checksum” image suite (PR baseline).
harvester_robot_tests/tests/regression/test_image_invalid_url.robot	New negative image suite for invalid URL (PR baseline).
harvester_robot_tests/tests/regression/test_image_invalid_checksum.robot	New negative image suite for invalid checksum (PR baseline).
harvester_robot_tests/run.sh	Adds `-p` pabot parallelism and `-S` strategy selection; updates runner behavior/help.
harvester_robot_tests/requirements.in	Adds `robotframework-pabot` dependency for parallel execution.
harvester_robot_tests/README.md	Documents parallel runs, CRD vs REST strategy, and the `pr-baseline` tag set.
harvester_robot_tests/libs/volume/volume.py	Selects CRD vs REST implementation by env var; exposes snapshot-ready API.
harvester_robot_tests/libs/volume/base.py	Updates volume interface for snapshot class + snapshot readiness.
harvester_robot_tests/libs/volume/crd.py	Aligns PVC access mode/mode, adds snapshot class + readiness wait, labels snapshot/restore resources.
harvester_robot_tests/libs/volume/rest.py	Reworks REST volume operations to use shared `harvester_api` managers and adds snapshot wait/cleanup.
harvester_robot_tests/libs/keywords/volume_keywords.py	Updates volume keyword layer for snapshot class + snapshot readiness.
harvester_robot_tests/keywords/volume.resource	Adds higher-level volume lifecycle + snapshot keywords used by new suites.
harvester_robot_tests/libs/vm/vm.py	Selects CRD vs REST implementation by env var.
harvester_robot_tests/libs/vm/rest.py	Aligns REST VM operations with shared client endpoints/behaviors (image UID lookup, list/get, backup, etc.).
harvester_robot_tests/libs/image/image.py	Selects CRD vs REST implementation by env var.
harvester_robot_tests/libs/image/crd.py	Improves CRD image state derivation to correctly report `Failed` vs `Active/Importing`.
harvester_robot_tests/libs/image/rest.py	Updates REST image listing call to match shared client.
harvester_robot_tests/keywords/image.resource	Adds list/existence/deletion/state helper keywords used by new image suites.
harvester_robot_tests/libs/utility/utility.py	Initializes the shared API client via the new adapter.
harvester_robot_tests/libs/utility/api_client.py	New adapter subclass to provide tuple-returning generic HTTP verbs.
harvester_robot_tests/libs/constant.py	Adds default VolumeSnapshotClass constant.
harvester_robot_tests/libs/harvester_api.py	Removes the standalone Robot client (replaced by shared apiclient).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+        p) PROCESSES=$OPTARG ;;
+        S) STRATEGY=$OPTARG ;;


 Available Tags:
    Priority: p0, p1, p2
    Type: coretest, regression, negative, smoke, sanity
    Component: virtualmachines, images, volumes, networks, backup, ha


+# Parallel test execution (suite-level) via `pabot`
+robotframework-pabot>=2.16,<3


        """List all images"""
        api = get_harvester_api_client()
-        code, data = api.images.list(namespace)
+        code, data = api.images.get("", namespace)


Vicente-Cheng · 2026-06-23T18:56:32Z

cc @bk201

- include create/expand/snapshot/restore/delete - add parameter that we can parallel run the test Signed-off-by: Vicente Cheng <vicente.cheng@suse.com>

- reuse the apiclient and add a new parameter to select strategy Signed-off-by: Vicente Cheng <vicente.cheng@suse.com>

- include create/list/delete/checksum/url Signed-off-by: Vicente Cheng <vicente.cheng@suse.com>

- it would be used for harvester integration, include the VM/Image/Volume basic tests Signed-off-by: Vicente Cheng <vicente.cheng@suse.com>

Vicente-Cheng · 2026-06-23T19:11:03Z

Moved to draft since we might add some basic tests for the VM part.

But for the current PR, it should work.

robot_tests: use original apiclient

13e5668

Signed-off-by: Vicente Cheng <vicente.cheng@suse.com>

Copilot AI review requested due to automatic review settings June 23, 2026 18:49

Copilot started reviewing on behalf of Vicente-Cheng June 23, 2026 18:49 View session

github-actions Bot assigned Vicente-Cheng Jun 23, 2026

Vicente-Cheng requested a review from a team June 23, 2026 18:50

Copilot AI reviewed Jun 23, 2026

View reviewed changes

Vicente-Cheng force-pushed the misc-improvement branch from 1a7ad16 to 53a3c08 Compare June 23, 2026 18:55

Vicente-Cheng added 4 commits June 24, 2026 03:03

robot_tests: Add generic volume test

bf1a43a

- include create/expand/snapshot/restore/delete - add parameter that we can parallel run the test Signed-off-by: Vicente Cheng <vicente.cheng@suse.com>

robot_tests: add the rest part for the volume testing

4e471b6

- reuse the apiclient and add a new parameter to select strategy Signed-off-by: Vicente Cheng <vicente.cheng@suse.com>

robot_tests: add immage tests

401a9cb

- include create/list/delete/checksum/url Signed-off-by: Vicente Cheng <vicente.cheng@suse.com>

robot_tests: add pr-baseline

e652566

- it would be used for harvester integration, include the VM/Image/Volume basic tests Signed-off-by: Vicente Cheng <vicente.cheng@suse.com>

Vicente-Cheng force-pushed the misc-improvement branch from 53a3c08 to e652566 Compare June 23, 2026 19:04

Vicente-Cheng marked this pull request as draft June 23, 2026 19:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

add some tests for the new harvester integration tests#2730

add some tests for the new harvester integration tests#2730
Vicente-Cheng wants to merge 5 commits into
harvester:mainfrom
Vicente-Cheng:misc-improvement

Vicente-Cheng commented Jun 23, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Vicente-Cheng commented Jun 23, 2026

Uh oh!

Vicente-Cheng commented Jun 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		# Parallel test execution (suite-level) via `pabot`
		robotframework-pabot>=2.16,<3

Uh oh!

Conversation

Vicente-Cheng commented Jun 23, 2026

Which issue(s) this PR fixes:

What this PR does / why we need it:

Special notes for your reviewer:

Additional documentation or context

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Vicente-Cheng commented Jun 23, 2026

Uh oh!

Vicente-Cheng commented Jun 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants