Skip to content

Document mutation-testing workflow contract tests - #419

Merged
leynos merged 4 commits into
mainfrom
docs/mutation-contract-tests
Jul 22, 2026
Merged

Document mutation-testing workflow contract tests#419
leynos merged 4 commits into
mainfrom
docs/mutation-contract-tests

Conversation

@leynos

@leynos leynos commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Summary

  • Add a ## Mutation-testing workflow contract tests section to
    docs/developers-guide.md, placed immediately after the existing
    ### Workflow pins and Dependabot subsection (the general shape-only
    pinning policy this contract test follows). No mutation-testing-specific
    section already existed, so this is a new top-level section rather than a
    merge into one.
  • Documents .github/workflows/mutation-testing.yml, the thin caller that
    delegates to leynos/shared-actions/.github/workflows/mutation-cargo.yml,
    and its contract test,
    tests/workflow_contracts/mutation_testing_test.py.

Findings reconciled against the repo

  • Contract-test style: shape-only. The test asserts the uses: ref is
    the correct path pinned to a 40-character lowercase-hex commit SHA (no
    hard-coded PINNED_SHA constant), so Dependabot bumps freely with no
    test edit required. There is no pytestmark = skipif(...) guard — this
    is a Rust repo, so the test isn't run inside a mutmut sandbox that omits
    .github/.
  • with: inputs actually present: only exclude-globs (the three
    #[cfg(kani)]-gated modules under src/ir/, which cargo-mutants
    cannot evaluate and would otherwise report as noisy survivors) and
    extra-args: --all-features (matching the make test CI baseline).
    paths is not set (uses the shared workflow's default).
  • extra-crate-dirs is not set, and I deliberately did not document
    it as if it were. I re-read Cargo.toml's [workspace] block:
    ambient_fs is a genuine workspace member (members = ["ambient_fs"],
    default-members = [".", "ambient_fs"]), not a non-workspace companion
    crate — extra-crate-dirs is reserved (per
    shared-actions/docs/mutation-cargo-workflow.md) for crate directories
    outside the workspace, so it doesn't apply here. test_support is
    excluded from the workspace (exclude = ["test_support"]) and this
    workflow does not mutate it. This is a deviation from the "companion
    crate via extra-crate-dirs" framing I was initially briefed with; the
    section documents the repo's real configuration instead.
  • Local command: make test-workflow-contracts (exists in the
    Makefile; runs
    uv run --with 'pytest>=8' --with 'pyyaml>=6' pytest tests/workflow_contracts -q).
    Ran it locally: 6 passed.
  • Roadmap/execplan: no roadmap or execplan tracking applies. I checked
    docs/roadmap.md and docs/execplans/; the only "mutation" hits in the
    roadmap concern OrthoConfig's non-interactive/mutation-safety metadata
    and an unrelated --dry-run preview item, not this workflow.
  • TOC cross-link: not applicable. docs/developers-guide.md has no
    table of contents or index list to link into; the guide is a flat
    sequence of ##/### sections.

Docs-lint result

make markdownlint (spelling/typos + markdownlint-cli2): passed after
fixing two spelling misses the linter caught in my first draft
(summarisingsummarizing, serialisesserializes — this repo's
typos.toml enforces Oxford -ize spelling, consistent with
analyse/behaviour staying as-is elsewhere in the same paragraph). Final
run: 0 errors.

Per instructions, only the docs/markdown gates were run — not make test
or make lint.

File changed

Latest review validation

  • make fmt: passed.
  • make markdownlint: passed (34 checks, 0 Markdown errors).
  • make nixie: passed (all Mermaid diagrams valid).

References

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @leynos, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@coderabbitai

coderabbitai Bot commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Documentation updates clarify lint exceptions, reusable-workflow pinning guidance, mutation-testing workflow contracts, and BDD command-helper descriptions.

Changes

Developer workflow documentation

Layer / File(s) Summary
Workflow contract guidance
docs/developers-guide.md
Document Dependabot lockstep behaviour and describe mutation-testing workflow modes, inputs, pinning, permissions, concurrency, triggers, and contract-test assertions.
Contributor guidance clarifications
docs/developers-guide.md
Clarify filesystem-lint exclusions and introduce the three BDD helpers for controlled netsuke execution.

Possibly related PRs

Suggested reviewers: codescene-access

Poem

Pins hold fast, but not one sacred hash,
Mutation tests wake with the daily flash.
Lints guard paths where filesystem calls roam,
Three BDD helpers guide netsuke home.


Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (2 inconclusive)

Check name Status Explanation Resolution
Testing (Unit And Behavioural) ❓ Inconclusive Pending evidence from the changed workflow and contract-test files. Inspect the diff and test files to verify whether the added tests exercise real workflow behaviour or only internal shape.
Security And Privacy ❓ Inconclusive placeholder2 inspect2
✅ Passed checks (18 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Testing (Overall) ✅ Passed Pass: treat this as not applicable; the PR only changes docs/developers-guide.md and introduces no new behaviour to guard.
User-Facing Documentation ✅ Passed Treat this PR as non-user-facing: it only updates docs/developers-guide.md, so docs/users-guide.md is not required.
Developer Documentation ✅ Passed docs/developers-guide.md adds the mutation-testing section; no matching roadmap item or execplan needed updating, and no translated developer-guide copy exists.
Module-Level Documentation ✅ Passed PASS: The PR changes only docs/developers-guide.md; no Rust/Python modules were added or edited, so the module docstring rule is unaffected.
Testing (Property / Proof) ✅ Passed The change only documents a fixed workflow contract and exact config checks; it introduces no ranged invariant or proof obligation that would call for property testing or formal proof.
Testing (Compile-Time / Ui) ✅ Passed PASS: the PR only updates docs plus a Python/YAML workflow contract; no Rust/TS compile-time or UI-output behaviour is added, so trybuild/snapshot tests are not applicable.
Unit Architecture ✅ Passed Pass: the PR only edits docs/developers-guide.md; no code, workflows, or tests were changed, so the architecture separation check is not applicable.
Domain Architecture ✅ Passed PASS: The PR only changes docs/developers-guide.md; no domain, adapter, persistence, or transport code was touched.
Observability ✅ Passed Only docs/developers-guide.md changed in the PR range, so no production behaviour changed and no new observability was required.
Performance And Resource Use ✅ Passed Only docs/developers-guide.md changed; no runtime code, loops, I/O, allocations, or hot-path logic were introduced.
Concurrency And State ✅ Passed Approve: only docs changed; the workflow and contract test already pin per-ref concurrency and no shared mutable state is introduced.
Architectural Complexity And Maintainability ✅ Passed Docs-only change; add no new abstraction, dependency, layer, or coupling, so architectural complexity stays flat.
Rust Compiler Lint Integrity ✅ Passed Only docs/developers-guide.md changed; no Rust files, lint suppressions, or extra clones were added, so compiler lint integrity is unaffected.
Title check ✅ Passed Keep the title aligned with the new mutation-testing workflow contract-test documentation.
Description check ✅ Passed Keep the description aligned with the added mutation-testing workflow contract-test documentation.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/mutation-contract-tests

Comment @coderabbitai help to get the list of available commands.

codescene-access[bot]

This comment was marked as outdated.

codescene-access[bot]

This comment was marked as outdated.

@leynos
leynos marked this pull request as ready for review July 21, 2026 22:40
leynos added 2 commits July 22, 2026 01:00
Add a section to docs/developers-guide.md covering the mutation-testing
caller workflow and its contract test,
tests/workflow_contracts/mutation_testing_test.py. Contributors changing
the caller's exclude-globs, extra-args, permissions, concurrency, or
triggers need to know why the pull request fails and how to run the
check locally with `make test-workflow-contracts`, without reverse
engineering it from the test source.
Apply the repository Markdown formatter to the mutation-testing contract
documentation and adjacent prose after rebasing onto `origin/main`.
@lodyai
lodyai Bot force-pushed the docs/mutation-contract-tests branch from 596dbdc to 37139a3 Compare July 21, 2026 23:10
codescene-access[bot]

This comment was marked as outdated.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/developers-guide.md`:
- Around line 96-97: Update the filesystem-scope statement in the developers
guide to acknowledge every sanctioned exclusion configured in dylint.toml,
including build_script_build, integration-test crates, and the relevant
application crates, rather than implying access is limited to ambient_fs and
test_support.
- Line 147: Expand the “Mutation-testing workflow contract tests” section to
document running make fmt, make markdownlint, make nixie, make check-fmt, make
lint, and make test before merging, and instruct contributors to record each
command’s output.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 0cc0834a-1150-41f9-9b5d-7dfa161ed58e

📥 Commits

Reviewing files that changed from the base of the PR and between bd17072 and 37139a3.

📒 Files selected for processing (1)
  • docs/developers-guide.md
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

Comment thread docs/developers-guide.md Outdated
Comment thread docs/developers-guide.md
Describe every category excluded from `no_std_fs_operations` so the
developer guide matches the live `dylint.toml` policy.
codescene-access[bot]

This comment was marked as outdated.

codescene-access[bot]

This comment was marked as outdated.

codescene-access[bot]

This comment was marked as outdated.

@buzzybee-df12

Copy link
Copy Markdown
Collaborator

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Add a section-specific checklist that combines the authoritative general
and documentation gates and requires contributors to record each result.
codescene-access[bot]

This comment was marked as outdated.

@codescene-access codescene-access Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No quality gates enabled for this code.

@leynos
leynos merged commit 35dd0b0 into main Jul 22, 2026
20 checks passed
@leynos
leynos deleted the docs/mutation-contract-tests branch July 22, 2026 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants