feat: add charm-ci integration test workflow by javierdelapuente · Pull Request #687 · canonical/traefik-k8s-operator

javierdelapuente · 2026-05-25T09:10:57Z

Summary

Integrates canonical/charm-ci reusable workflows for building, testing, and publishing the charm, replacing the old observability release workflow.

CI workflows

File	Change	Purpose
`.github/workflows/integration-test.yaml`	Added	Calls `charm-ci/integration-test.yml` on PRs; builds charm and runs spread jobs
`.github/workflows/publish.yaml`	Added	Calls `charm-ci/publish-artifacts.yml` + lib release on push to main
`.github/workflows/pull-request.yaml`	Modified	Removed enable-integration flag (now handled by integration-test.yaml)
`.github/workflows/quality-gates.yaml`	Deleted	Superseded by charm-ci workflows
`.github/workflows/release.yaml`	Deleted	Superseded by publish.yaml

charm-ci configuration

File	Purpose
`artifacts.yaml`	Charm build manifest (generated by `opcli artifacts init`)
`spread.yaml`	Single `integration-suites` entry; `opcli` recursively auto-discovers all `test_*.py` files under `tests/integration/` including the jubilant subdirectory. Juju 4.0 variants for jubilant tests override `CONCIERGE` and set explicit `MODULE` file paths.
`concierge.yaml`	microk8s + juju 3.6/stable environment provisioning
`concierge-juju4.yaml`	Same but with juju 4.0/stable, used by jubilant test variants

Jubilant tests refactor

The two existing jubilant tests (test_dynamic_configs_jubilant.py, test_tls_multi_unit_jubilant.py) were moved into a tests/integration/jubilant/ subdirectory with a dedicated conftest.py:

tests/integration/jubilant/conftest.py (new): owns all jubilant-specific fixtures — traefik_charm (resolves CHARM_PATH to absolute path), juju fixture (honours --model/--keep-models), and no-op autouse shadows that prevent the outer setup_env and copy_traefik_library_into_tester_charms fixtures (which depend on ops_test) from running for jubilant tests.
tests/integration/conftest.py: cleaned up — all jubilant-awareness removed (no more ops_test override, no "jubilant" in module.__name__ guards, no import jubilant).
Files renamed: test_tls_multi_unit_jubilant.py → test_https_multi_unit.py (function also renamed to test_https_on_all_units) to avoid pytest -k test_tls substring collisions with the regular TLS test modules.

How it works

On PR: integration-test workflow builds the charm, then runs one spread job per test file (auto-discovered by opcli using rglob). Each job receives the test file path as a positional pytest argument — not -k — so there is no risk of substring-based test selection bleeding across modules.
Jubilant juju4 variants additionally swap in concierge-juju4.yaml to bootstrap juju 4.0/stable.
On push to main: publish workflow finds the last passing integration-test run, publishes the charm to latest/edge, then releases charm libraries.

Add files needed to run integration tests using canonical/charm-ci's reusable workflow alongside the existing observability workflow: - artifacts.yaml: charm build manifest (opcli artifacts init) - spread.yaml: integration-test backend with 14 MODULE variants - tests/integration/run/task.yaml: spread task bridging CHARM_PATH - concierge.yaml: microk8s + juju 3.6/stable provisioning - .github/workflows/integration-test-charm-ci.yaml: PR workflow The CHARM_PATH env var is extracted from opcli's --charm-file output since traefik's conftest uses CHARM_PATH rather than --charm-file. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

juju requires local charm paths to start with ./ or be absolute. opcli produces relative paths like ./built-charm-.../foo.charm, but Python's Path() strips the ./ prefix causing juju to error with 'charm is ambiguous'. Use realpath to convert to absolute path, matching what the observability workflow does. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Switch from microk8s to k8s provider with load balancer, add LXD, model defaults (test-mode, automatically-retry-hooks), and remove charmcraft (not needed - main charm is pre-built, testers are not used in this test setup). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Tester charms need charmcraft + LXD to build during integration tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Add pytest-invocation-mode: observability to spread.yaml (traefik uses CHARM_PATH env var, not --charm-file flags) - Replace custom grep-based task.yaml with standard opcli template - opcli pytest expand now handles CHARM_PATH automatically in observability mode Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add publish-charm-ci.yaml that: - Calls canonical/charm-ci publish-artifacts reusable workflow on push to main - Publishes to latest/edge channel - Releases charm libraries after successful publish using charming-actions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Replaced by publish-charm-ci.yaml which uses canonical/charm-ci publish-artifacts reusable workflow. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Also update the workflow name to 'Integration Tests' and fix the reference in publish-charm-ci.yaml. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Neither is needed for integration tests — charmcraft pack doesn't require a keyring, and the default channel is sufficient. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Not used by any automated integration test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Calling request.getfixturevalue('ops_test') from within an async fixture causes 'RuntimeError: This event loop is already running' because ops_test is itself async — initializing it from inside a running event loop fails. Instead, override ops_test in conftest.py to yield None for jubilant modules. This prevents python-libjuju from attempting a juju 4 connection (which it doesn't support), while keeping the original direct-parameter pattern for setup_env intact. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Move juju fixture into conftest.py, honouring --model (concierge) and --keep-models pytest-operator options; use name='juju' to avoid shadowing the juju module import - Remove redundant inline juju fixtures from both jubilant test modules - Add _all_settled() helper (all_active + all_agents_idle) for more robust waits in test_dynamic_configs_jubilant - Fix race in test_dynamic_config_removed_after_relation_removed: wait for the ingress relation to disappear from status before asserting the config file is gone Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…emplate charm-ci PR #45 replaced the pytest-invocation-mode enum with Jinja2 templates. Migrate from the removed 'observability' mode to the new pytest-environment-template key, which sets CHARM_PATH for the tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Add pytest-arguments-template: "" to suppress the default pfe-style --charm-file flag; traefik tests use CHARM_PATH env var instead, which is set by pytest-environment-template. Without this, both CHARM_PATH and --charm-file=... were passed, causing pytest to reject the unknown --charm-file argument. - Fix import order in conftest.py (ruff I001: jubilant before juju) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Path('./foo/bar.charm') strips the leading ./ so is_local_charm() in python-libjuju fails both startswith('.') and os.path.isabs() checks, causing it to treat the charm as a Charmhub URL and send empty string to Juju. Using .resolve() makes it absolute. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Same Path('./...') issue as in conftest.py: juju CLI also rejects relative paths without ./ prefix as ambiguous. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Create tests/integration/jubilant/ with its own conftest.py that owns traefik_charm, juju, and no-op shadows for the two autouse fixtures (copy_traefik_library_into_tester_charms, setup_env) - Move test_dynamic_configs_jubilant.py and test_tls_multi_unit_jubilant.py into the new subdirectory; remove their now-redundant local traefik_charm fixtures - Clean up tests/integration/conftest.py: remove ops_test override, remove jubilant name-guards, remove juju_fixture, remove ops_test is None check - Add a second integration-suite in spread.yaml for tests/integration/jubilant/ so charm-ci auto-discovers jubilant tests independently; move juju4 variant overrides into that suite Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Add norecursedirs = ["jubilant"] to pytest config so auto-discovery doesn't recurse into tests/integration/jubilant/ during regular runs - Set pytest-arguments-template to pass tests/integration/jubilant/ as an explicit path for the jubilant suite (explicit paths override norecursedirs, so jubilant tests are still collected) This prevents -k test_tls from substring-matching test_tls_multi_unit_jubilant. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

This reverts commit 670ea59.

test_dynamic_configs_jubilant → test_dynamic_configs (no regular counterpart) test_tls_multi_unit_jubilant → test_https_multi_unit + rename test function test_tls_on_all_units → test_https_on_all_units The regular suite has test_tls.py (MODULE test_tls). Pytest's -k does substring matching so -k test_tls would also collect test_tls_multi_unit_jubilant and its test_tls_on_all_units function. Renaming to test_https_multi_unit breaks this. Also drop the _jubilant suffix — the jubilant/ subdirectory already signals that. Update spread.yaml MODULE/CONCIERGE variant keys accordingly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

charm-ci@main changed _discover_modules_in to use rglob and return relative file paths (e.g. jubilant/test_dynamic_configs.py) instead of stems. MODULE values are now full paths relative to OPCLI_CWD, and pytest receives the file path directly (no -k) — exact file selection with no substring collision risk. This makes the two-suite workaround unnecessary. Revert to a single suite tests/integration/ whose auto-discover recursively finds all test files including those under jubilant/. Update juju4 variant keys to the new path-based naming convention: jubilant_test_dynamic_configs_juju4 / jubilant_test_https_multi_unit_juju4 with MODULE values as explicit file paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

seb4stien · 2026-06-02T08:43:36Z

+    branches:
+      - main
+
+permissions:


I think the recommendation is to have the permissions at the job level to be specific.

seb4stien · 2026-06-02T08:43:50Z

+    branches:
+      - main
+
+permissions:


Should be at the job level.

Which is already the case for publish

javierdelapuente · 2026-06-02T08:53:01Z

+- .*_cache
+integration-suites:
+  tests/integration/:
+    cwd: ./


working-dir

javierdelapuente · 2026-06-02T09:13:18Z

+    pytest-arguments-template: ""
+    pytest-environment-template: |
+      {% for build in artifacts.charms[0].builds if build.arch == arch %}
+      CHARM_PATH={{ build.path }}


what guardrails can we have in here?

javierdelapuente requested a review from a team as a code owner May 25, 2026 09:10

is-devops-bot added the Charm Libraries: Out of sync label May 25, 2026

javierdelapuente mentioned this pull request May 25, 2026

Spread CI prepare: /home/ubuntu/.local owned by root breaks charmcraft for ubuntu user canonical/charm-ci#20

Closed

javierdelapuente and others added 4 commits May 25, 2026 11:31

ci: retrigger CI

6301959

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

ci: retrigger CI (charm-ci fix merged)

55d2193

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

ci: retrigger CI (HOME=/root fix merged)

090ea1b

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

javierdelapuente force-pushed the feat/charm-ci-integration-tests branch from 77da5c5 to 090ea1b Compare May 25, 2026 11:32

javierdelapuente and others added 6 commits May 25, 2026 11:42

chore: remove accidentally committed generated files

8832f76

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

chore: gitignore spread/opcli generated files

eeaa964

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

fix: add charmcraft 3.x/candidate to concierge.yaml

642a365

Tester charms need charmcraft + LXD to build during integration tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

This was referenced May 26, 2026

feat: add charm-ci integration test workflow javierdelapuente/traefik-k8s-operator#1

Open

feat: add charm-ci integration test workflow javierdelapuente/javi-traefik-k8s-operator#1

Merged

javierdelapuente added 2 commits May 28, 2026 08:58

Disable integration tests

c1eb58e

Remove enable-integration from pull-request

b3ca1f2

javierdelapuente mentioned this pull request May 28, 2026

feat: support multi-track publishing (track/** branches) canonical/charm-ci#41

Open

javierdelapuente and others added 8 commits May 28, 2026 13:38

Merge branch 'main' into feat/charm-ci-integration-tests

8673374

chore: remove old observability release workflow

8a2cd46

Replaced by publish-charm-ci.yaml which uses canonical/charm-ci publish-artifacts reusable workflow. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

chore: rename integration-test-charm-ci to integration-test

35af59d

Also update the workflow name to 'Integration Tests' and fix the reference in publish-charm-ci.yaml. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

chore: rename publish-charm-ci to publish

6907257

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

chore: remove leftover publish-charm-ci.yaml

6843b53

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

chore: remove gnome-keyring and charmcraft channel pin from concierge

8a1a9b5

Neither is needed for integration tests — charmcraft pack doesn't require a keyring, and the default channel is sufficient. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

chore: remove unused jq and yq from concierge.yaml

ab4dc8a

Not used by any automated integration test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

javierdelapuente added the no-release-note This PR does not require a change artifact label May 28, 2026

javierdelapuente and others added 2 commits May 28, 2026 13:11

chore: pin charm-ci workflows to v0.0.1-alpha.1

2c17189

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

ci: retrigger workflows

7380f43

javierdelapuente marked this pull request as draft June 1, 2026 09:19

javierdelapuente and others added 6 commits June 1, 2026 09:22

chore: update charm-ci workflow refs to @main

fea2a49

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Merge branch 'main' into feat/charm-ci-integration-tests

74c50fa

chore: migrate to integration-suites for auto-discovery

e449d80

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

chore: remove committed task.yaml (now generated by opcli)

0666b2d

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

chore: add juju 4.0 variants for jubilant integration tests

988326d

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

fix: use underscores in spread variant names (hyphens not allowed)

6c0538b

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

javierdelapuente mentioned this pull request Jun 1, 2026

feat: add discover-path key to integration-suites to decouple suite identity from test directory canonical/charm-ci#46

Open

javierdelapuente and others added 14 commits June 1, 2026 11:49

fix: skip OpsTest setup for jubilant tests in autouse fixture

a06780d

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

fix: skip ops_test-dependent autouse fixtures for jubilant tests

1c29571

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

fix: resolve CHARM_PATH to absolute in jubilant test fixtures

75502b5

Same Path('./...') issue as in conftest.py: juju CLI also rejects relative paths without ./ prefix as ambiguous. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Revert "fix: prevent regular suite from collecting jubilant tests"

bf658d2

This reverts commit 670ea59.

ci: retrigger CI

ead9eaf

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

javierdelapuente marked this pull request as ready for review June 2, 2026 05:55

javierdelapuente requested a review from swetha1654 June 2, 2026 05:55

javierdelapuente and others added 2 commits June 2, 2026 07:30

ci: pin charm-ci to v0.0.1-alpha.1

a1007c4

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

ci: pin charm-ci publish workflow to v0.0.1-alpha.1

610a37e

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

seb4stien reviewed Jun 2, 2026

View reviewed changes

javierdelapuente commented Jun 2, 2026

View reviewed changes

Comment thread spread.yaml

- .*_cache

integration-suites:

tests/integration/:

cwd: ./

Copy link
Copy Markdown

Contributor Author

javierdelapuente Jun 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

working-dir

javierdelapuente commented Jun 2, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add charm-ci integration test workflow#687

feat: add charm-ci integration test workflow#687
javierdelapuente wants to merge 45 commits into
mainfrom
feat/charm-ci-integration-tests

javierdelapuente commented May 25, 2026 •

edited

Loading

Uh oh!

seb4stien Jun 2, 2026

Uh oh!

seb4stien Jun 2, 2026

Uh oh!

seb4stien Jun 2, 2026

Uh oh!

javierdelapuente Jun 2, 2026

Uh oh!

javierdelapuente Jun 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

javierdelapuente commented May 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

CI workflows

charm-ci configuration

Jubilant tests refactor

How it works

Uh oh!

seb4stien Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

seb4stien Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

seb4stien Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

javierdelapuente Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

javierdelapuente Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

javierdelapuente commented May 25, 2026 •

edited

Loading