Skip to content

Expose uv's libpython on LD_LIBRARY_PATH for PyO3 baselines (#372) - #379

Open
leynos wants to merge 1 commit into
mainfrom
fix/mutation-cargo-pyo3-libpython
Open

Expose uv's libpython on LD_LIBRARY_PATH for PyO3 baselines (#372)#379
leynos wants to merge 1 commit into
mainfrom
fix/mutation-cargo-pyo3-libpython

Conversation

@leynos

@leynos leynos commented Jul 21, 2026

Copy link
Copy Markdown
Owner

Summary

mutation-cargo.yml's unmutated baseline fails deterministically for callers
whose crates embed a Python interpreter via PyO3 (the auto-initialize
pattern), such as leynos/tei-rapporteur's tei-py and stilyagi. PyO3's build
script resolves the interpreter from PATH; Setup uv places a uv-managed
CPython ahead of the system python3, and its libpython3.NN.so.1.0 lives
under uv's private install directory, off the dynamic loader's default search
path. The baseline cargo test binary then aborts with "libpython3.13.so.1.0:
cannot open shared object file" (exit 127) and no mutant is ever scored.

This adds a step, after Python provisioning and before Run mutation testing,
that derives the resolved interpreter's LIBDIR via
sysconfig.get_config_var("LIBDIR") and prepends it to LD_LIBRARY_PATH for
the later steps. It is harmless for non-PyO3 callers: no test binary links
libpython, so nothing loads from the added directory.

Closes #372.

Supersedes the caller-side mitigation tracked in leynos/stilyagi#90 (which
worked around this with setup-commands); that issue can stay open for the
caller to drop its workaround once this lands — this PR does not close it.

Review walkthrough

  • .github/workflows/mutation-cargo.yml — new Expose libpython for PyO3 test binaries step in the mutants job (guarded env.ACT != 'true'), between
    Setup uv and Relocate workflow source. It skips cleanly when no LIBDIR
    resolves.
  • docs/mutation-cargo-workflow.md — documents the automatic handling.
  • workflow_scripts/tests/test_mutation_workflow_shape.py — new invariant
    test_mutants_job_exposes_libpython_before_running pins that the export
    precedes the run step.

Validation

  • pytest workflow_scripts/tests/test_mutation_workflow_shape.py — 5 passed.
  • ruff format --check / ruff check — clean; markdownlint / typos on the
    changed doc — clean.

PyO3 embedding crates (the auto-initialize pattern) link their test binaries
against the interpreter PyO3's build script resolves from PATH. Setup uv puts
a uv-managed CPython ahead of the system python3, and its libpython lives
under uv's private install directory, off the loader's default search path.
The unmutated baseline cargo test binary then aborts with "libpython3.NN.so:
cannot open shared object file" and no mutant is ever scored.

After provisioning Python, derive the interpreter's LIBDIR via sysconfig and
add it to LD_LIBRARY_PATH for the later steps, before Run mutation testing.
This is harmless for non-PyO3 callers: nothing links libpython, so nothing
loads from the added directory. Pin a shape-test invariant that the export
precedes the run step, and document the automatic handling in the caller
guide.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @leynos, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@coderabbitai

coderabbitai Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Summary

  • Updated the mutation workflow to expose uv-managed Python’s LIBDIR through LD_LIBRARY_PATH, enabling PyO3 embedding binaries to load libpython during baseline tests.
  • Documented the automatic handling for PyO3 auto-initialize crates.
  • Added a workflow-shape test confirming the library path is exported before mutation testing.

Walkthrough

The mutation workflow now exposes uv-managed Python’s LIBDIR through LD_LIBRARY_PATH before cargo-mutants, with safeguards for local emulation and missing paths. Documentation and workflow-shape tests cover the behaviour.

Changes

PyO3 libpython runtime support

Layer / File(s) Summary
Expose libpython before mutation testing
.github/workflows/mutation-cargo.yml
Resolve Python LIBDIR and append valid directories to LD_LIBRARY_PATH before running mutation testing, except under ACT=true.
Document and enforce the workflow contract
docs/mutation-cargo-workflow.md, workflow_scripts/tests/test_mutation_workflow_shape.py
Document automatic PyO3 embedding support and verify that the LIBDIR export precedes the mutation-testing step.

Sequence Diagram(s)

sequenceDiagram
  participant mutants_job
  participant python3
  participant LD_LIBRARY_PATH
  participant cargo_mutants
  mutants_job->>python3: Resolve Python LIBDIR
  python3-->>mutants_job: Return library directory
  mutants_job->>LD_LIBRARY_PATH: Append valid LIBDIR
  mutants_job->>cargo_mutants: Run mutation testing
Loading

Possibly related issues

  • leynos/stilyagi#90 — Both changes expose uv-provisioned Python’s LIBDIR through LD_LIBRARY_PATH for PyO3 mutation-test binaries.

Possibly related PRs

Suggested labels: Issue

Poem

Python’s library path now shines,
Before mutation testing lines.
PyO3 binaries find their home,
No workaround needs to roam.
Tests guard the ordering bright.


Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore

❌ Failed checks (1 error, 2 warnings)

Check name Status Explanation Resolution
Testing (Overall) ❌ Error The new workflow-shape test only matches strings and step order; it never asserts the GITHUB_ENV write that makes LD_LIBRARY_PATH persist. Require the test to assert the env-file append (GITHUB_ENV and >>) and, ideally, the exact export step content, so a non-persistent echo cannot pass.
Developer Documentation ⚠️ Warning The new libpython/LD_LIBRARY_PATH requirement lives only in caller-facing docs; docs/developers-guide.md and any ADR/execplan do not record the decision. Add a maintainer note in docs/developers-guide.md under Mutation-Testing Reusable Workflows, and record the runtime-search-path decision in the relevant execplan or ADR.
Testing (Unit And Behavioural) ⚠️ Warning The new test only inspects workflow YAML strings and step order; it never verifies the shell step writes to GITHUB_ENV or exercises the workflow boundary. Add an act-level or equivalent behavioural test that runs the workflow step and asserts LD_LIBRARY_PATH persists into Run mutation testing, including the GITHUB_ENV write and the LIBDIR-missing path.
✅ Passed checks (17 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately reflects the libpython LD_LIBRARY_PATH fix and includes #372.
Description check ✅ Passed The description matches the workflow fix and documents the PyO3/libpython behaviour.
Linked Issues check ✅ Passed The change exposes uv's libpython via LD_LIBRARY_PATH, which satisfies #372's baseline-failure fix.
Out of Scope Changes check ✅ Passed The doc update and workflow-shape test stay within the libpython exposure fix.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
User-Facing Documentation ✅ Passed Use the linked workflow guide: it documents the PyO3/libpython handling, and README points users to that guide.
Module-Level Documentation ✅ Passed The only Python module touched already has a top-level docstring describing the workflow tests and their relationship to the reusable workflows.
Testing (Property / Proof) ✅ Passed PASS: the PR already pins the fixed step-order invariant with an example-based workflow-shape test; no broad input range or proof assumption calls for Hypothesis or an exhaustive proof.
Testing (Compile-Time / Ui) ✅ Passed No Rust/TypeScript compile-time path is added; the new YAML-shape assertion already pins the behavioural invariant, so trybuild/snapshots are not required.
Unit Architecture ✅ Passed PASS: the query/write work stays at the workflow edge, missing LIBDIR is handled explicitly, and the new test/doc changes keep responsibilities separate.
Domain Architecture ✅ Passed PASS: The PR only changes CI workflow, docs, and workflow-shape tests; no domain model, repository, or business logic is touched, so domain boundaries stay intact.
Observability ✅ Passed The new workflow step logs both branches: it announces when libpython is added and warns when LIBDIR is missing, so the runtime decision boundary is visible.
Security And Privacy ✅ Passed PASS: Only runner-derived LIBDIR is written to GITHUB_ENV; no secrets, credentials, broad permissions, or unsafe input handling were introduced.
Performance And Resource Use ✅ Passed Approve: the new step is a single Python call plus one env write; no new loops, unbounded growth, or repeated I/O.
Concurrency And State ✅ Passed PASS: the only state is a job-local GITHUB_ENV write, done by one step and read later in order; no concurrent workers, locks, or async lifetimes are introduced.
Architectural Complexity And Maintainability ✅ Passed Keep it explicit: one shell step and one shape test, with no new abstraction, layer, or dependency added.
Rust Compiler Lint Integrity ✅ Passed No Rust sources changed, and the PR adds no lint suppressions, artificial usage anchors, or suspicious clone-heavy code.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/mutation-cargo-pyo3-libpython

Comment @coderabbitai help to get the list of available commands.

codescene-access[bot]

This comment was marked as outdated.

codescene-access[bot]

This comment was marked as outdated.

@leynos
leynos marked this pull request as ready for review July 26, 2026 22:46

@codescene-access codescene-access Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No quality gates enabled for this code.

@pandalump

Copy link
Copy Markdown
Collaborator

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot added the Issue label Jul 27, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/mutation-cargo-workflow.md`:
- Around line 102-106: Qualify the automatic PyO3 embedding-crate handling
description to apply only when the workflow resolves a valid interpreter LIBDIR
on hosted runners. Note that the LD_LIBRARY_PATH export is skipped when ACT=true
and remains unchanged when LIBDIR is unavailable, so avoid stating
unconditionally that no setup-commands workaround is needed.

In `@workflow_scripts/tests/test_mutation_workflow_shape.py`:
- Around line 72-78: Update the `exposers` predicate in the shape test to
require each matching run script to reference `GITHUB_ENV` and use `>>`, in
addition to the existing `LD_LIBRARY_PATH` and `get_config_var("LIBDIR")`
checks, ensuring the computed environment value is persisted for later steps.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 6e32f0e7-5822-4edf-ba84-614baa06cfac

📥 Commits

Reviewing files that changed from the base of the PR and between 074f7d8 and a074875.

📒 Files selected for processing (3)
  • .github/workflows/mutation-cargo.yml
  • docs/mutation-cargo-workflow.md
  • workflow_scripts/tests/test_mutation_workflow_shape.py
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • leynos/polythene (auto-detected)

Comment on lines +102 to +106
- Callers with PyO3 embedding crates (the `auto-initialize` pattern)
are handled automatically: the workflow adds the uv-provisioned
interpreter's `LIBDIR` to `LD_LIBRARY_PATH` before running
cargo-mutants, so test binaries linked against `libpython` load it at
run time. No `setup-commands` workaround is needed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Qualify the automatic-handling claim.

State that this applies when the workflow resolves a valid LIBDIR on hosted runners. The implementation skips the export under ACT=true and intentionally continues unchanged when LIBDIR is missing, so “No setup-commands workaround is needed” is not universally guaranteed.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/mutation-cargo-workflow.md` around lines 102 - 106, Qualify the
automatic PyO3 embedding-crate handling description to apply only when the
workflow resolves a valid interpreter LIBDIR on hosted runners. Note that the
LD_LIBRARY_PATH export is skipped when ACT=true and remains unchanged when
LIBDIR is unavailable, so avoid stating unconditionally that no setup-commands
workaround is needed.

Comment on lines +72 to +78
exposers = [
index
for index, step in enumerate(steps)
if isinstance(step.get("run"), str)
and "LD_LIBRARY_PATH=" in typ.cast("str", step.get("run"))
and 'get_config_var("LIBDIR")' in typ.cast("str", step.get("run"))
]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Assert persistence through GITHUB_ENV.

Require the shape test to verify that the export script writes to GITHUB_ENV, not merely that it contains the relevant strings. Otherwise, a command that computes or prints LD_LIBRARY_PATH could pass while Run mutation testing still receives no updated environment.

         and "LD_LIBRARY_PATH=" in typ.cast("str", step.get("run"))
         and 'get_config_var("LIBDIR")' in typ.cast("str", step.get("run"))
+        and "GITHUB_ENV" in typ.cast("str", step.get("run"))
+        and ">>" in typ.cast("str", step.get("run"))
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@workflow_scripts/tests/test_mutation_workflow_shape.py` around lines 72 - 78,
Update the `exposers` predicate in the shape test to require each matching run
script to reference `GITHUB_ENV` and use `>>`, in addition to the existing
`LD_LIBRARY_PATH` and `get_config_var("LIBDIR")` checks, ensuring the computed
environment value is persisted for later steps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mutation-cargo: baseline build fails for PyO3 embedding crates (libpython not on LD_LIBRARY_PATH)

2 participants