From 31e1b93666e447e56b919e7c38aa38f62a540490 Mon Sep 17 00:00:00 2001 From: leynos Date: Mon, 20 Jul 2026 00:46:57 +0200 Subject: [PATCH 1/4] Document mutation-testing workflow contract tests Add a section to docs/developers-guide.md covering the mutation-testing caller workflow and its contract test, tests/workflow_contracts/mutation_testing_test.py. Contributors changing the caller's exclude-globs, extra-args, permissions, concurrency, or triggers need to know why the pull request fails and how to run the check locally with `make test-workflow-contracts`, without reverse engineering it from the test source. --- docs/developers-guide.md | 65 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/docs/developers-guide.md b/docs/developers-guide.md index d5ede4b2..f7079d2e 100644 --- a/docs/developers-guide.md +++ b/docs/developers-guide.md @@ -144,6 +144,71 @@ If a workflow's behaviour genuinely depends on a feature only present from a particular commit onwards, express that as a comment or a changelog note, not as a test assertion on the SHA string. +## Mutation-testing workflow contract tests + +This repository runs scheduled, informational mutation testing through a thin +caller workflow, +[`.github/workflows/mutation-testing.yml`](../.github/workflows/mutation-testing.yml), +which delegates to the shared reusable workflow +`leynos/shared-actions/.github/workflows/mutation-cargo.yml`. The heavy +lifting — running `cargo-mutants` and summarizing survivors — lives in +`shared-actions`; this repository carries only declarative configuration. The +run is **informational only**: it never gates a pull request. Survivors are +reported through the job summary and downloadable artefacts so they can be +triaged into tests, not enforced as a blocking check. + +The workflow runs in two modes. A **daily schedule** (03:05 UTC) fires a +change-scoped run that mutates only the source files touched within the +detection window, so quiet days are cheap no-ops. A **manual dispatch** (the +Actions "Run workflow" control) mutates every target, fanned out across +shards; select a branch in that control to exercise a feature branch. + +The caller passes two configuration inputs, each carrying intent: + +- `exclude-globs` — `src/ir/cycle_verification.rs`, + `src/ir/from_manifest_verification.rs`, and `src/ir/graph_kani_map.rs`: + modules gated behind `#[cfg(kani)]` mod declarations. `cargo-mutants` does + not evaluate that cfg, so mutants inserted there would compile to nothing + and survive as noise rather than genuine test gaps. +- `extra-args` — `--all-features`, so the mutation run matches the `make + test` CI baseline; a mismatch would report feature-gated code (the + `legacy-digests` feature) as untested. + +The caller does not set `extra-crate-dirs`, the input reserved for crate +directories outside the Cargo workspace. `ambient_fs`, the repository's +sanctioned ambient-filesystem escape hatch, is a genuine workspace member +(`[workspace] members = ["ambient_fs"]`, with `default-members = [".", +"ambient_fs"]`), not a non-workspace companion crate, so it needs no separate +mutation invocation here. `test_support` is deliberately excluded from the +workspace (`exclude = ["test_support"]`) and this workflow does not mutate +it. + +The `uses:` reference pins the shared workflow to a full 40-character commit +SHA rather than a branch or tag, so a force-push upstream cannot silently +change what runs here. The contract test asserts only that the pin is a full +lowercase-hex commit SHA, not a particular value — the shape-only pinning +policy described above in "Workflow pins and Dependabot" — so Dependabot +bumps it automatically without any accompanying test edit. + +Because the caller is configuration rather than code, a contract test, +[`tests/workflow_contracts/mutation_testing_test.py`](../tests/workflow_contracts/mutation_testing_test.py), +pins the shape it must uphold, failing the pull request when the caller +drifts — repointing the pin at a branch, widening the token scope, or +dropping a configuration input — rather than letting the breakage surface +only in a scheduled run. Run it locally with `make test-workflow-contracts`. +The test validates: + +- the `uses:` reference targets `mutation-cargo.yml` pinned to a full, + lowercase-hex commit SHA; +- the `with:` block carries exactly the expected configuration (the + `#[cfg(kani)]` module excludes and `--all-features`); +- job permissions are least-privilege (`contents: read`, `id-token: write`) + and the workflow-level default token scope is empty; +- `concurrency` serializes runs per ref without cancelling one in progress; + and +- the triggers keep the daily schedule and a plain `workflow_dispatch` with + no legacy branch input. + ## Spelling enforcement `make markdownlint` enforces en-GB-oxendict (Oxford) spelling over the From 37139a3f87c33252dfd2165d3e543acc2e6621fa Mon Sep 17 00:00:00 2001 From: leynos Date: Wed, 22 Jul 2026 01:09:50 +0200 Subject: [PATCH 2/4] Normalize developer guide formatting Apply the repository Markdown formatter to the mutation-testing contract documentation and adjacent prose after rebasing onto `origin/main`. --- docs/developers-guide.md | 68 ++++++++++++++++++++-------------------- 1 file changed, 34 insertions(+), 34 deletions(-) diff --git a/docs/developers-guide.md b/docs/developers-guide.md index f7079d2e..9392bca2 100644 --- a/docs/developers-guide.md +++ b/docs/developers-guide.md @@ -93,8 +93,8 @@ continuous integration (CI); `make lint-clippy` runs the Clippy-only subset. Whitaker is configured by `dylint.toml` at the repository root. The `no_std_fs_operations` lint currently ignores in-source `allow`/`expect` attributes, so deliberately ambient filesystem access is confined to the -`ambient_fs` leaf crate and the `test_support` test-fixture crate, both of -which `dylint.toml` excludes from that lint with a documented rationale. +`ambient_fs` leaf crate and the `test_support` test-fixture crate, both of which +`dylint.toml` excludes from that lint with a documented rationale. When command output is long, preserve exit codes and logs: @@ -111,15 +111,15 @@ For documentation changes, also run: ### Workflow pins and Dependabot -Dependabot owns the upgrade of GitHub Actions and reusable workflows, -including calls into `leynos/shared-actions`. Contract tests that assert a -caller's exact commit SHA create a lockstep dependency: every time Dependabot -opens a bump PR, the test fails until a human edits the pinned constant to -match. That defeats the purpose of automated dependency updates and turns a -routine bump into a manual chore. +Dependabot owns the upgrade of GitHub Actions and reusable workflows, including +calls into `leynos/shared-actions`. Contract tests that assert a caller's exact +commit SHA create a lockstep dependency: every time Dependabot opens a bump PR, +the test fails until a human edits the pinned constant to match. That defeats +the purpose of automated dependency updates and turns a routine bump into a +manual chore. -Contract tests may still verify the *shape* of a reusable-workflow caller. -They must not verify the specific SHA value. +Contract tests may still verify the *shape* of a reusable-workflow caller. They +must not verify the specific SHA value. - Do assert the workflow references the correct reusable workflow path. - Do assert the ref is pinned to a full 40-character commit SHA, not a @@ -150,8 +150,8 @@ This repository runs scheduled, informational mutation testing through a thin caller workflow, [`.github/workflows/mutation-testing.yml`](../.github/workflows/mutation-testing.yml), which delegates to the shared reusable workflow -`leynos/shared-actions/.github/workflows/mutation-cargo.yml`. The heavy -lifting — running `cargo-mutants` and summarizing survivors — lives in +`leynos/shared-actions/.github/workflows/mutation-cargo.yml`. The heavy lifting +— running `cargo-mutants` and summarizing survivors — lives in `shared-actions`; this repository carries only declarative configuration. The run is **informational only**: it never gates a pull request. Survivors are reported through the job summary and downloadable artefacts so they can be @@ -160,43 +160,43 @@ triaged into tests, not enforced as a blocking check. The workflow runs in two modes. A **daily schedule** (03:05 UTC) fires a change-scoped run that mutates only the source files touched within the detection window, so quiet days are cheap no-ops. A **manual dispatch** (the -Actions "Run workflow" control) mutates every target, fanned out across -shards; select a branch in that control to exercise a feature branch. +Actions "Run workflow" control) mutates every target, fanned out across shards; +select a branch in that control to exercise a feature branch. The caller passes two configuration inputs, each carrying intent: - `exclude-globs` — `src/ir/cycle_verification.rs`, `src/ir/from_manifest_verification.rs`, and `src/ir/graph_kani_map.rs`: modules gated behind `#[cfg(kani)]` mod declarations. `cargo-mutants` does - not evaluate that cfg, so mutants inserted there would compile to nothing - and survive as noise rather than genuine test gaps. -- `extra-args` — `--all-features`, so the mutation run matches the `make - test` CI baseline; a mismatch would report feature-gated code (the + not evaluate that cfg, so mutants inserted there would compile to nothing and + survive as noise rather than genuine test gaps. +- `extra-args` — `--all-features`, so the mutation run matches the `make test` + CI baseline; a mismatch would report feature-gated code (the `legacy-digests` feature) as untested. The caller does not set `extra-crate-dirs`, the input reserved for crate directories outside the Cargo workspace. `ambient_fs`, the repository's sanctioned ambient-filesystem escape hatch, is a genuine workspace member -(`[workspace] members = ["ambient_fs"]`, with `default-members = [".", -"ambient_fs"]`), not a non-workspace companion crate, so it needs no separate -mutation invocation here. `test_support` is deliberately excluded from the -workspace (`exclude = ["test_support"]`) and this workflow does not mutate -it. +(`[workspace] members = ["ambient_fs"]`, with +`default-members = [".", "ambient_fs"]`), not a non-workspace companion crate, +so it needs no separate mutation invocation here. `test_support` is +deliberately excluded from the workspace (`exclude = ["test_support"]`) and +this workflow does not mutate it. The `uses:` reference pins the shared workflow to a full 40-character commit SHA rather than a branch or tag, so a force-push upstream cannot silently change what runs here. The contract test asserts only that the pin is a full lowercase-hex commit SHA, not a particular value — the shape-only pinning -policy described above in "Workflow pins and Dependabot" — so Dependabot -bumps it automatically without any accompanying test edit. +policy described above in "Workflow pins and Dependabot" — so Dependabot bumps +it automatically without any accompanying test edit. Because the caller is configuration rather than code, a contract test, [`tests/workflow_contracts/mutation_testing_test.py`](../tests/workflow_contracts/mutation_testing_test.py), -pins the shape it must uphold, failing the pull request when the caller -drifts — repointing the pin at a branch, widening the token scope, or -dropping a configuration input — rather than letting the breakage surface -only in a scheduled run. Run it locally with `make test-workflow-contracts`. -The test validates: +pins the shape it must uphold, failing the pull request when the caller drifts +— repointing the pin at a branch, widening the token scope, or dropping a +configuration input — rather than letting the breakage surface only in a +scheduled run. Run it locally with `make test-workflow-contracts`. The test +validates: - the `uses:` reference targets `mutation-cargo.yml` pinned to a full, lowercase-hex commit SHA; @@ -214,8 +214,8 @@ The test validates: `make markdownlint` enforces en-GB-oxendict (Oxford) spelling over the repository's Markdown prose with [`typos`](https://github.com/crate-ci/typos), as required by the [documentation style guide](documentation-style-guide.md). -The repository-root `typos.toml` is deterministically generated output assembled -from two policy layers: +The repository-root `typos.toml` is deterministically generated output +assembled from two policy layers: 1. The shared estate dictionary in `leynos/agent-helper-scripts` supplies generally valid Oxford forms, accepted technical terms, corrections, and @@ -908,8 +908,8 @@ Versioning and compatibility rules: ## BDD command helpers and environment handling -The BDD step module `tests/bdd/steps/manifest_command_helpers.rs` provides three -helpers that launch the netsuke binary in a controlled environment: +The BDD step module `tests/bdd/steps/manifest_command_helpers.rs` provides +three helpers that launch the netsuke binary in a controlled environment: - **`netsuke_executable()`** — locates the compiled netsuke binary using `assert_cmd::cargo::cargo_bin!("netsuke")`. Returns the resolved `PathBuf` or From c229e78f9ecafa32f1ef7826cf2306a09d4359fb Mon Sep 17 00:00:00 2001 From: leynos Date: Wed, 22 Jul 2026 01:26:08 +0200 Subject: [PATCH 3/4] Document sanctioned filesystem lint exclusions Describe every category excluded from `no_std_fs_operations` so the developer guide matches the live `dylint.toml` policy. --- docs/developers-guide.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/developers-guide.md b/docs/developers-guide.md index 9392bca2..7718d775 100644 --- a/docs/developers-guide.md +++ b/docs/developers-guide.md @@ -92,9 +92,10 @@ the standalone installer described in the continuous integration (CI); `make lint-clippy` runs the Clippy-only subset. Whitaker is configured by `dylint.toml` at the repository root. The `no_std_fs_operations` lint currently ignores in-source `allow`/`expect` -attributes, so deliberately ambient filesystem access is confined to the -`ambient_fs` leaf crate and the `test_support` test-fixture crate, both of which -`dylint.toml` excludes from that lint with a documented rationale. +attributes, so `dylint.toml` excludes each sanctioned ambient-filesystem scope +with a documented rationale: the `build_script_build` Cargo build-script crate, +the `ambient_fs` application leaf crate, the `test_support` test-fixture crate, +and the enumerated integration-test and workflow-contract crates. When command output is long, preserve exit codes and logs: From e9ed7eb5dcb4b638f020aaff0a46163adb0c8b94 Mon Sep 17 00:00:00 2001 From: leynos Date: Wed, 22 Jul 2026 12:48:39 +0200 Subject: [PATCH 4/4] Document mutation guide completion gates Add a section-specific checklist that combines the authoritative general and documentation gates and requires contributors to record each result. --- docs/developers-guide.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/developers-guide.md b/docs/developers-guide.md index 7718d775..07905f44 100644 --- a/docs/developers-guide.md +++ b/docs/developers-guide.md @@ -210,6 +210,17 @@ validates: - the triggers keep the daily schedule and a plain `workflow_dispatch` with no legacy branch input. +Before merging this mutation-testing workflow documentation change, follow the +authoritative [Quality gates](#quality-gates) guidance and record the output of +every command in this completion checklist: + +- `make fmt` +- `make markdownlint` +- `make nixie` +- `make check-fmt` +- `make lint` +- `make test` + ## Spelling enforcement `make markdownlint` enforces en-GB-oxendict (Oxford) spelling over the