Support pyo3 abi3t features on Python3.15 and PyO3 0.29#3113
Conversation
|
@messense do you have any opinions about how tests for abi3t support should look? My first instinct would be to follow and generalize the existing tests for abi3. Does that sound like a good plan, or do you have large test refactorings in mind separate from this? |
|
Sorry missed the reply in the issue, follow and generalize the existing tests for abi3 sounds good to me. |
Sorry for the delay over here. I just added some new tests. Since abi3t support requires Python 3.15 or newer, I needed to also add some utilities to introspect the Python used by the test runner. |
|
I don't think there are any tests for Python 3.15 yet. @messense what do you think about replacing |
|
I should also note that the new test crates depend on my PR branch for PyO3/pyo3#5807 and I'll need to update them after that PyO3 PR is merged and PyO3 does a release. Not quite sure how to sequence landing everything. |
| /// minimum version. Version‑specific fallback builds (e.g. Python 3.10 when | ||
| /// abi3 targets ≥ 3.11) return `false` so that `Py_LIMITED_API` is not | ||
| /// defined and interpreter‑specific linker names are used. | ||
| pub fn is_abi3_for_interpreter(&self, interpreter: &PythonInterpreter) -> bool { |
There was a problem hiding this comment.
Where does this function end up being needed over is_stable_abi_for_interpreter?
There was a problem hiding this comment.
Good call - I think I needed it in an earlier version of this PR but I can delete it now and simplify things a lot.
There was a problem hiding this comment.
Oh no, that's wrong. It's load-bearing in the implementation of configure_pyo3_env, which ends up generating a pyo3 configuration file using the abi3 key. I probably need to detect the pyo3 version and generate an appropriate config file with a target_abi key if I find a new-enough pyo3.
There was a problem hiding this comment.
I added a new use_target_abi argument to configure_pyo3_env and set up maturin to generate a configuration file with an abi3 or target_abi argument based on the detected PyO3 version. This let me delete is_abi3_for_interpreter.
|
OK, I think CI is stabilized. Not sure what's up with the remaining Mac pypy3.11 failures, but I think they're caused by a change in PyO3 unrelated to abi3t support (PyO3/pyo3#6041). |
|
Also I should say that I had to drop 3.13t from the interpreters considered for interpreter discovery, since PyO3 dropped 3.13t support. |
|
PyO3/pyo3#6078 fixes the PyPy linker issue causing the remaining test failures. |
|
Sorry for all the noise the past few days and the substantial growth of this PR. I realized that I wasn't actually running the integration tests locally on Python 3.15, but once I realized that the maturin CI uses That said, I think the implementation is ready to review now and is only waiting for PyO3 to do a 0.29 release include abi3t support. There probably need to be docs as well, I will look at that next week I think. |
|
I'm holding off review this PR until it's out of draft state, ping me if I missed any notifications, thanks! |
…eters (#3206) This is partially extracted from #3113. Currently, free-threaded interpreters are not always considered for `--find-interpreters`. It is included on Unix but not on Windows. This means that users need to explicitly do another build to produce 3.14t wheels. This also contributes to PyO3/maturin-action#368 and I'm planning to do a followup over there to fix that once this is merged. With this PR, `--find-interpreters` always picks up 3.14t and newer. I intentionally excluded 3.13t because PyO3 0.29 will refuse to build wheels for 3.13t. This also follows manylinux, which recently dropped 3.13t from the images. --------- Co-authored-by: messense <messense@icloud.com>
|
I think I understand the cause of the random Windows failures, see pypa/pip#9034. In short, pip's cache isn't safe to concurrently write to on Windows like maturin's CI does. I pushed a commit disabling pip caching, which hopefully doesn't negatively impact CI times too much. |
|
This is still not ready to merge because PyO3 hasn't yet done a release and I'll need to update the test crates one final time. That said, I just added some documentation and marked this ready for review because it's otherwise totally ready. IMO it'd be a good idea to coordinate merging this and quickly cutting a release with the PyO3 0.29 release. |
Per [the CFFI docs](https://cffi.readthedocs.io/en/latest/installation.html#installation-and-status), PyPy vendors CFFI and installing it and requiring it is unnecessary. This is particularly annoying because cffi doesn't ship PyPy wheels, so it triggers an unnecessary C build at install time. Also see, for example, the way [cryptography handles this](https://github.com/pyca/cryptography/blob/ba936e527bc4e3948c338a7bde54c3bd81957c61/pyproject.toml#L8). I noticed this was an issue in the CI for #3113, which saw failures related to the unnecessary CFFI builds failing.
messense
left a comment
messense
left a comment
There was a problem hiding this comment.
LGTM, thanks, let's wait for pyo3 0.29 release then update and merge this.
ngoldbaum
changed the title
|
Updated to use PyO3 0.29 in the test crates. Everything passes locally but let's see what CI says. |
3 participants
Adds support for the new abi3t ABI in Python 3.15. See https://pyo3.rs/main/building-and-distribution.html?highlight=abi3t#py_limited_apiabi3abi3t for more info from the pyo3 side of things.