Skip to content

Document mutation-testing workflow contract tests - #362

Open
leynos wants to merge 1 commit into
mainfrom
docs/mutation-contract-tests
Open

Document mutation-testing workflow contract tests#362
leynos wants to merge 1 commit into
mainfrom
docs/mutation-contract-tests

Conversation

@leynos

@leynos leynos commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Summary

Adds a ### Workflow contract tests subsection under the existing
## Mutation-Testing Reusable Workflows section of
docs/developers-guide.md. This host repository both defines the
reusable mutation workflows and, via workflow_scripts/, consumes
mutation-mutmut.yml, so its contract tests differ from a downstream
consumer's. The new subsection documents both drift guards:

  • workflow_scripts/tests/test_mutation_testing_caller.py — the caller
    contract test (analogous to a consumer repo's): shape-only USES_RE
    SHA pin, least-privilege permissions, empty default token scope,
    per-ref non-cancelling concurrency, schedule + plain
    workflow_dispatch triggers, and the exact with: block
    (paths: "workflow_scripts/", module-prefix-strip: ""). Self-skips
    when .github/ is absent (mutmut's mutants/ sandbox).
  • workflow_scripts/tests/test_mutation_workflow_shape.py — with no
    consumer analogue: pins the self-checkout relocation invariant in the
    reusable workflow definitions themselves (issue workflow-src checkout pollutes the caller's tree during mutation runs #343 regression
    guard).

It records the local run command and cross-links the helper-script unit
tests and the act integration tests (Workflow Test Harness).

Framing / permutation

Host-repo special case (per brief's shared-actions note). Not
permutations A–E. Combined single subsection, framing confirmed by the
orchestrator before opening this PR. The repository already had a
developer's guide and a mutation section, so this is a ### subsection
rather than a new ## section, and no new-guide work was needed.

Local run command documented

uv run --with pyyaml --with pytest \
  pytest workflow_scripts/tests/test_mutation_testing_caller.py \
         workflow_scripts/tests/test_mutation_workflow_shape.py -v

Verified: 10 passed against up-to-date origin/main.

Cross-link

No TOC/index exists near the top of the guide, so no TOC entry was
added. Added an in-section cross-link to the "Workflow Test Harness"
heading using the GitHub slug
#workflow-test-harness-testsworkflowsconftestpy.

Roadmap / execplan tracking

No roadmap/execplan tracking applies to this docs task. (The mutation
work's execplan, docs/execplans/add-mutation-testing-workflows.md,
covers the workflows themselves, not this developer-guide documentation.)

Docs lint

  • markdownlint-cli2 on the file: 0 errors.
  • typos (repo typos.toml): passes after correcting serialises
    serializes per en-GB Oxford (-ize) spelling.

Docs-only change; the Rust/Python test suites were not run.

Add a "Workflow contract tests" subsection under the existing
"Mutation-Testing Reusable Workflows" section of the developer's guide.
It documents the two drift guards specific to this host repository: the
caller contract test (test_mutation_testing_caller.py), analogous to a
downstream consumer's, and the workflow-shape test
(test_mutation_workflow_shape.py), which pins the self-checkout
relocation invariant in the reusable workflow definitions themselves. It
records the local run command and cross-links the helper-script unit
tests and the act integration tests.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @leynos, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@coderabbitai

coderabbitai Bot commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Summary

  • Added a Workflow contract tests subsection to docs/developers-guide.md.
  • Documented caller workflow contract checks and reusable workflow self-checkout relocation invariants.
  • Added the local pytest command, dependency notes, and links to helper-script unit tests and Workflow Test Harness integration tests.
  • Recorded that the documented tests, Markdown linting, and typo checks passed; Rust and Python test suites were not run.

Walkthrough

The developer guide adds a “Workflow contract tests” subsection documenting caller and reusable-workflow structural checks, local commands, dependencies, and separate execution-path coverage.

Changes

Workflow contract documentation

Layer / File(s) Summary
Add workflow contract test guidance
docs/developers-guide.md
Documents PyYAML-based caller workflow checks, reusable-workflow structure checks, checkout relocation via $RUNNER_TEMP and WORKFLOW_DIR, local test commands, dependencies, and separate unit/property and act-gated coverage.

Possibly related PRs

Suggested labels: Issue

Poem

Workflow truths in pages bright,
YAML guarded, paths set right.
Temp trees keep the source pristine,
Contract tests watch every seam.
Run the checks; let drift take flight.


Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Security And Privacy ❓ Inconclusive Placeholder while evidence is gathered. Need the new docs subsection and referenced tests to confirm no secrets, unsafe permissions, or privacy leaks are introduced.
✅ Passed checks (19 passed)
Check name Status Explanation
Title check ✅ Passed The title matches the docs change and clearly names the mutation-testing workflow contract tests.
Description check ✅ Passed The description is directly about the same documentation update and stays on topic.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Testing (Overall) ✅ Passed PASS: The PR is docs-only; no code or test behaviour changed, so the testing-coverage requirement is not applicable.
User-Facing Documentation ✅ Passed Keep this in developer docs: the PR only changes docs/developers-guide.md and adds internal workflow contract test notes, with no user-facing behaviour or users-guide updates needed.
Developer Documentation ✅ Passed docs/developers-guide.md now documents the workflow contract tests, run command, and cross-links; no translated guide variants exist.
Module-Level Documentation ✅ Passed PASS: This PR only changes docs/developers-guide.md; no Python modules are added or modified, so the module docstring rule is not applicable.
Testing (Unit And Behavioural) ✅ Passed PASS: The PR is docs-only; no test code changed. The documented tests already target workflow-contract invariants at the reusable-workflow boundary.
Testing (Property / Proof) ✅ Passed PASS: The docs already recommend property-based tests for the helper-script invariants; the workflow-shape checks are finite structural contracts, so no proof obligation arises.
Testing (Compile-Time / Ui) ✅ Passed Docs-only PR; no compile-time/UI code was changed, and the referenced tests are YAML-shape contract checks, not snapshot/trybuild cases.
Unit Architecture ✅ Passed Only docs/developers-guide.md changed; no query, command, dependency, or side-effect boundaries were altered.
Domain Architecture ✅ Passed Docs-only change in docs/developers-guide.md; no domain, adapter, or infrastructure code was modified.
Observability ✅ Passed Docs-only change in docs/developers-guide.md; no operational behaviour, logging, metrics, tracing, or alerts were introduced.
Performance And Resource Use ✅ Passed Pass: keep the PR docs-only; it adds no executable code, loops, I/O, allocations, or hot-path work.
Concurrency And State ✅ Passed PASS: This PR only documents existing workflow contract tests; it adds no shared mutable state, async tasks, or ordering logic, and concurrency is already asserted in tests.
Architectural Complexity And Maintainability ✅ Passed Doc-only change adds prose about existing workflow tests; it introduces no new abstractions, layers, registries, or indirection.
Rust Compiler Lint Integrity ✅ Passed Docs-only change: no Rust files were touched, so no lint suppressions, dead-code anchors, or clone regressions were introduced.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/mutation-contract-tests

Comment @coderabbitai help to get the list of available commands.

codescene-access[bot]

This comment was marked as outdated.

codescene-access[bot]

This comment was marked as outdated.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/developers-guide.md`:
- Around line 442-443: Update the sentence in the developers guide by removing
the comma immediately before “because” after the `module-prefix-strip: ""`
clause, leaving the surrounding wording unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 87a4a422-6e96-495a-b3c1-a4c0e6ed55d8

📥 Commits

Reviewing files that changed from the base of the PR and between c3141c3 and 63af5a9.

📒 Files selected for processing (1)
  • docs/developers-guide.md
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

Comment thread docs/developers-guide.md
Comment on lines +442 to +443
`module-prefix-strip: ""`, because the repository's own Python source
is mutated as a flat-layout package.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Remove the comma before “because”.

The clause is essential to the sentence, so remove the comma to follow the repository’s grammar standard.

Triage: [type:grammar]

-  module-prefix-strip: ""`, because the repository's own Python source
+  module-prefix-strip: ""` because the repository's own Python source
🧰 Tools
🪛 LanguageTool

[formatting] ~442-~442: If the ‘because’ clause is essential to the meaning, do not use a comma before the clause.
Context: ...ripts/"and module-prefix-strip: ""`, because the repository's own Python source is...

(COMMA_BEFORE_BECAUSE)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/developers-guide.md` around lines 442 - 443, Update the sentence in the
developers guide by removing the comma immediately before “because” after the
`module-prefix-strip: ""` clause, leaving the surrounding wording unchanged.

Sources: Path instructions, Linters/SAST tools

@codescene-access codescene-access Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No quality gates enabled for this code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant