Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion copier.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,23 @@ use_rust:

python_version:
type: str
default: "3.10"
default: "3.12"
help: Minimum supported Python version
validator: >-
{% if not (python_version | regex_search('^3[.][0-9]+$')) %}
Python version must use the 3.X format (for example, 3.12).
{% endif %}

# Computed: mutmut mutation testing needs a base interpreter of 3.13 or
# greater because the shared workflow's helper scripts require >= 3.13.

mutmut_supported:
type: bool
default: >-
{{ python_version is string
and (python_version | regex_search('^3[.][0-9]+$'))
and (python_version.split('.')[1] | int >= 13) }}
when: false
Comment thread
coderabbitai[bot] marked this conversation as resolved.

codescene_project_id:
type: str
Expand Down
9 changes: 9 additions & 0 deletions docs/developers-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,15 @@ Docker availability, and runs `make test WITH_ACT=1`.
Rust-enabled workflows pass `rust_extension/Cargo.toml` to the coverage action
because the generated Python project root does not contain a Rust manifest.

The mutation-testing workflow template is rendered when either mutation engine
is available. Its mutmut job requires a minimum Python version of 3.13 or newer;
the hidden `mutmut_supported` Copier answer enforces that gate. The cargo-mutants
job is gated independently by `use_rust`, so a Rust-enabled project with a
Python 3.12 baseline still receives Rust mutation testing. Both jobs delegate to
reusable `leynos/shared-actions` workflows. Dependabot owns their pinned commit
SHAs; contract tests and documentation must not duplicate those values because
Dependabot cannot update them outside the workflow template.

### Workflow pins and Dependabot

Dependabot owns the upgrade of GitHub Actions and reusable workflows,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
name: Mutation testing

# Thin caller of the shared mutation-testing reusable workflows; caller
# guides: leynos/shared-actions docs/mutation-mutmut-workflow.md and
# docs/mutation-cargo-workflow.md.
# Scheduled runs mutate only files changed within the detection window;
# manual dispatch runs mutate everything. To test a branch, select it in
# the Actions "Run workflow" control.

on:
schedule:
# Stagger this cron per repository so estate runs do not bunch up;
# existing repositories occupy the 03:05-09:05 UTC band.
- cron: "30 9 * * *" # Daily, 09:30 UTC
workflow_dispatch:

# Default the token to no scopes; jobs opt in explicitly.
permissions: {}

# Serialize runs per ref: a dispatch during a scheduled run (or vice
# versa) queues rather than racing. Runs are informational, so queued
# runs wait instead of cancelling.
concurrency:
group: mutation-testing-${{ "{{" }} github.ref {{ "}}" }}
cancel-in-progress: false

jobs:
{%- if mutmut_supported %}
mutation:
permissions:
contents: read
id-token: write # OIDC workflow-source resolution
uses: leynos/shared-actions/.github/workflows/mutation-mutmut.yml@859416a90eb3987b46a57682c5d6b8964ad3f0a6
with:
# Flat package layout: the package directory sits at the
# repository root, so no module prefix is stripped.
paths: "{{ package_name }}/"
module-prefix-strip: ""
# Always >= 3.13 here: this job renders only when the project's
# baseline interpreter satisfies the shared workflow's helper
# scripts.
python-version: "{{ python_version }}"
Comment thread
leynos marked this conversation as resolved.
{%- endif %}
{%- if use_rust %}

mutation-rust:
permissions:
contents: read
id-token: write # OIDC workflow-source resolution
uses: leynos/shared-actions/.github/workflows/mutation-cargo.yml@859416a90eb3987b46a57682c5d6b8964ad3f0a6
with:
# The extension crate is the only Cargo target and lives outside
# the repository root, so it is declared as an extra crate and the
# root path list is emptied.
paths: ""
extra-crate-dirs: "rust_extension"
# The shared workflow always fans the root target out on manual
# dispatch, and this repository has no root crate, so that leg
# fails; keep it to a single shard. Scheduled scoped runs and the
# rust_extension target are unaffected.
shard-count: 1
{%- endif %}
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,20 @@ actions under `.github/`.
- `.github/workflows/get-codescene-sha.yml` is manually dispatched. It fetches
the CodeScene coverage CLI installer, computes its SHA-256 digest, and writes
the result to the `CODESCENE_CLI_SHA256` repository variable.
{% if mutmut_supported or use_rust %}
- `.github/workflows/mutation-testing.yml` runs daily at 09:30 UTC and supports
manual dispatch. It delegates to reusable workflows from `leynos/shared-actions`;
stagger the generated cron schedule before adopting it alongside other
repositories.
{% if mutmut_supported %}
The mutmut job is generated only when the minimum Python version is 3.13 or
newer because the shared workflow helpers require Python 3.13 or newer.
{% endif %}
{% if use_rust %}
Rust-enabled projects also receive a cargo-mutants job for `rust_extension/`,
independently of the Python version baseline.
{% endif %}
{% endif %}
- `.github/actions/build-wheels` wraps `cibuildwheel` with `uvx` and uploads
architecture-specific wheel artefacts.
- `.github/actions/pure-python-wheel` builds a pure Python wheel with
Expand Down
19 changes: 19 additions & 0 deletions template/docs/users-guide.md.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,25 @@ Dependabot pull requests; a weekly scheduled audit on the default branch is the
compensating control. Rust-enabled projects also run `cargo audit` from the
`rust_extension` crate directory.

{% if mutmut_supported or use_rust %}
## Scheduled Mutation Testing

The `.github/workflows/mutation-testing.yml` workflow runs mutation testing
daily at 09:30 UTC and can also be started manually from GitHub Actions. Adjust
the generated cron schedule to stagger it against other repositories.

{% if mutmut_supported %}
For projects whose minimum Python version is 3.13 or newer, the workflow runs
mutmut against `{{ package_name }}/`. The Python mutation job is omitted for
older baselines because the shared workflow helpers require Python 3.13 or
newer.
{% endif %}
{% if use_rust %}
Rust-enabled projects also run cargo-mutants against the crate in
`rust_extension/`, independently of the Python version baseline.
{% endif %}
{% endif %}

## Rust Test Behaviour

Rust-enabled projects use `cargo nextest run` when `cargo-nextest` is available.
Expand Down
11 changes: 11 additions & 0 deletions template/pyproject.toml.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -294,6 +294,17 @@ enable = [
"too-many-statements",
]

{%- if mutmut_supported %}

[tool.mutmut]
# mutmut copies the tree before running the suite: tests that read
# repository-root paths outside tests/ must either add those paths to
# `also_copy` or guard themselves with a skipif. See the shared-actions
# caller guide (docs/mutation-mutmut-workflow.md).
source_paths = ["{{ package_name }}/"]
pytest_add_cli_args_test_selection = ["tests/"]
{%- endif %}

[tool.pytest.ini_options]
# Tests automatically killed after seconds elapsed
timeout = 30
Expand Down
8 changes: 6 additions & 2 deletions tests/helpers/pyproject_contracts.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,16 @@


def _assert_pyproject_contracts(
*, package_name: str, pyproject: dict[str, Any], use_rust: bool
*,
package_name: str,
pyproject: dict[str, Any],
use_rust: bool,
python_version: str,
) -> None:
"""Assert generated Python packaging contracts."""
project = require_mapping(pyproject, "project", "pyproject.toml")
assert project.get("name"), "expected generated project metadata to include a name"
assert project.get("requires-python") == ">=3.10", (
assert project.get("requires-python") == f">={python_version}", (
"expected generated pyproject.toml to use the requested Python version"
)
dependency_groups = require_mapping(
Expand Down
14 changes: 8 additions & 6 deletions tests/helpers/rendering.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,20 +43,22 @@ def run_quality_gates(project: CopierProject) -> None:


def render_project(
tmp_path: Path,
destination: Path,
copier: CopierFixture,
*,
project_name: str,
package_name: str,
use_rust: bool = False,
python_version: str = "3.10",
python_version: str = "3.12",
) -> CopierProject:
"""Render a generated Python project with explicit template answers.

Parameters
----------
tmp_path : pathlib.Path
Temporary directory used as the generated project destination.
destination : pathlib.Path
Destination directory for the generated project. Callers may pass a
subdirectory of their pytest temporary directory to distinguish
rendered variants.
copier : CopierFixture
``pytest-copier`` fixture bound to this template repository.
project_name : str
Expand All @@ -65,7 +67,7 @@ def render_project(
Python import package name answer passed to Copier.
use_rust : bool, default=False
Whether to include the optional PyO3 extension.
python_version : str, default="3.10"
python_version : str, default="3.12"
Minimum supported Python version answer passed to Copier.

Returns
Expand All @@ -79,7 +81,7 @@ def render_project(
Propagates rendering failures raised by Copier or ``pytest-copier``.
"""
return copier.copy(
tmp_path,
destination,
project_name=project_name,
package_name=package_name,
use_rust=use_rust,
Expand Down
5 changes: 5 additions & 0 deletions tests/helpers/tooling_contracts.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,13 +53,17 @@ def assert_generated_tooling_contracts(
build_wheels_action: str,
pure_wheel_action: str,
use_rust: bool,
python_version: str = "3.12",
) -> None:
"""Assert generated Python/Rust tooling contracts from one validator.

Parameters
----------
package_name : str
Generated Python import package name.
python_version : str, default="3.12"
Minimum supported Python version used to render the project, matched
against the generated ``requires-python`` constraint.
agents : str
UTF-8 text of the generated ``AGENTS.md`` file.
pyproject : dict[str, Any]
Expand Down Expand Up @@ -118,6 +122,7 @@ def assert_generated_tooling_contracts(
package_name=package_name,
pyproject=pyproject,
use_rust=use_rust,
python_version=python_version,
)
_assert_agents_contracts(agents)
_assert_agents_make_targets_mirror_makefile(
Expand Down
37 changes: 24 additions & 13 deletions tests/test_helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,19 @@
REPO_ROOT = Path(__file__).resolve().parent.parent


"""Validate direct helper-module error handling and edge cases.
The tests in this module exercise support helpers without rendering a full
Copier project. They keep helper fallibility contracts explicit by checking
``pytest.fail`` conversion paths, generated-file schema helpers, and tooling
contract assertions directly.
"""
if TYPE_CHECKING:
from pytest_copier.plugin import CopierProject
REPO_ROOT = Path(__file__).resolve().parent.parent
_TEST_ACTION_SHA = "0" * 40
_DEPENDABOT_UPDATED_TEST_ACTION_SHA = "1" * 40


@pytest.mark.parametrize(
("output", "expected"),
[
Expand Down Expand Up @@ -422,18 +435,16 @@ def test_ci_coverage_action_contract_accepts_any_pinned_sha_rejects_branch_ref()
)

dependabot_bumped_workflow = workflow.replace(
"927edd45ae77be4251a8a18ca9eb5613a2e32cbd",
"0123456789abcdef0123456789abcdef01234567",
_TEST_ACTION_SHA,
_DEPENDABOT_UPDATED_TEST_ACTION_SHA,
)
assert_ci_coverage_action_contract(
ci_workflow=dependabot_bumped_workflow,
package_name="helper_pkg",
use_rust=False,
)

branch_ref_workflow = workflow.replace(
"@927edd45ae77be4251a8a18ca9eb5613a2e32cbd", "@main"
)
branch_ref_workflow = workflow.replace(f"@{_TEST_ACTION_SHA}", "@main")
with pytest.raises(
AssertionError,
match="expected CI to use the shared coverage action pinned to a 40-hex",
Expand Down Expand Up @@ -614,15 +625,15 @@ def _coverage_main_workflow(*, guard: str, rust_setup: str = "") -> str:
steps:
{rust_setup}\
- name: Generate coverage
uses: leynos/shared-actions/.github/actions/generate-coverage@927edd45ae77be4251a8a18ca9eb5613a2e32cbd
uses: leynos/shared-actions/.github/actions/generate-coverage@{_TEST_ACTION_SHA}
with:
output-path: coverage.xml
format: cobertura
artefact-name-suffix: helper-pkg
with-ratchet: 'true'
- name: Upload coverage data to CodeScene
if: {guard}
uses: leynos/shared-actions/.github/actions/upload-codescene-coverage@927edd45ae77be4251a8a18ca9eb5613a2e32cbd
uses: leynos/shared-actions/.github/actions/upload-codescene-coverage@{_TEST_ACTION_SHA}
with:
format: cobertura
path: coverage.xml
Expand Down Expand Up @@ -691,7 +702,7 @@ def test_coverage_main_workflow_contract_requires_rust_setup() -> None:
rust_setup = (
" - name: Set up Rust\n"
" uses: leynos/shared-actions/.github/actions/setup-rust"
"@927edd45ae77be4251a8a18ca9eb5613a2e32cbd\n"
f"@{_TEST_ACTION_SHA}\n"
)
rust_manifest = " cargo-manifest: rust_extension/Cargo.toml\n"
workflow = _coverage_main_workflow(
Expand Down Expand Up @@ -727,8 +738,8 @@ def test_coverage_main_workflow_contract_accepts_any_pinned_sha_rejects_branch_r
workflow = _coverage_main_workflow(guard="env.CS_ACCESS_TOKEN != ''")

dependabot_bumped_workflow = workflow.replace(
"927edd45ae77be4251a8a18ca9eb5613a2e32cbd",
"0123456789abcdef0123456789abcdef01234567",
_TEST_ACTION_SHA,
_DEPENDABOT_UPDATED_TEST_ACTION_SHA,
)
assert_coverage_main_workflow_contract(
coverage_main_workflow=dependabot_bumped_workflow,
Expand All @@ -737,7 +748,7 @@ def test_coverage_main_workflow_contract_accepts_any_pinned_sha_rejects_branch_r
)

branch_ref_workflow = workflow.replace(
"generate-coverage@927edd45ae77be4251a8a18ca9eb5613a2e32cbd",
f"generate-coverage@{_TEST_ACTION_SHA}",
"generate-coverage@main",
)
with pytest.raises(
Expand All @@ -752,7 +763,7 @@ def test_coverage_main_workflow_contract_accepts_any_pinned_sha_rejects_branch_r
)

upload_branch_ref_workflow = workflow.replace(
"upload-codescene-coverage@927edd45ae77be4251a8a18ca9eb5613a2e32cbd",
f"upload-codescene-coverage@{_TEST_ACTION_SHA}",
"upload-codescene-coverage@main",
)
with pytest.raises(
Expand All @@ -779,7 +790,7 @@ def _ci_workflow(*, persist_credentials: str, coverage_inputs: str) -> str:
persist-credentials: {persist_credentials}
- name: Test and Measure Coverage
if: ${{{{ github.event_name == 'pull_request' }}}}
uses: leynos/shared-actions/.github/actions/generate-coverage@927edd45ae77be4251a8a18ca9eb5613a2e32cbd
uses: leynos/shared-actions/.github/actions/generate-coverage@{_TEST_ACTION_SHA}
with:
output-path: coverage.xml
format: cobertura
Expand Down
Loading
Loading