Skip to content

ofiuco collapse cpXY-abi3 wheels to cp3x-abi3 and ignores more specific python_tag #137

Description

@azryve

Hi. While upgrading to the latest version of the ofiuco I encountered the following regression in some tests:

self = <temporalio.worker.workflow_sandbox._importer.Importer object at 0x7f03eed25f40>
name = 'cryptography.hazmat.bindings._rust'
globals = {'__builtins__': {'ArithmeticError': <class 'ArithmeticError'>, 'AssertionError': <class 'AssertionError'>, 'Attribute...files/ofiuco++parse+cryptography-42.0.8-cp37-abi3-manylinux_2_28_x86_64/cryptography/hazmat/primitives/hashes.py', ...}
locals = {'__builtins__': {'ArithmeticError': <class 'ArithmeticError'>, 'AssertionError': <class 'AssertionError'>, 'Attribute...files/ofiuco++parse+cryptography-42.0.8-cp37-abi3-manylinux_2_28_x86_64/cryptography/hazmat/primitives/hashes.py', ...}
fromlist = ('openssl',), level = 0
...
>       mod = importlib.__import__(name, globals, locals, fromlist, level)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E       ImportError: PyO3 modules compiled for CPython 3.8 or older may only be initialized once per interpreter process

Researching this specific regression it appears that between 2 possible versions of the cryptography wheel, ofiuco prefers the cp37 one.

% grep cryptography poetry.lock | grep cp | grep manylinux_2_28_x86
    {file = "cryptography-42.0.8-cp37-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:6b7c4f03ce01afd3b76cf69a5455caa9cfa3de8c8f493e0d3ab7d20611c8dae9"},
    {file = "cryptography-42.0.8-cp39-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:9c0c1716c8447ee7dbf08d6db2e5c41c688544c61074b54fc4564196f55c25a7"},

This also does not match the behaviour of pip which sorts abi3 versions based on python_tag and prefers the newer one:

$ pip3 debug --verbose | grep abi3-manylinux_2_28 | head
  cp312-abi3-manylinux_2_28_x86_64
  cp311-abi3-manylinux_2_28_x86_64
  cp310-abi3-manylinux_2_28_x86_64
  cp39-abi3-manylinux_2_28_x86_64
  cp38-abi3-manylinux_2_28_x86_64
  cp37-abi3-manylinux_2_28_x86_64


$ pip3 install --dry-run cryptography==42.0.8
Collecting cryptography==42.0.8
  Downloading cryptography-42.0.8-cp39-abi3-manylinux_2_28_x86_64.whl.metadata (5.3 kB)
...

So while abi3 builds are compatible, there are some extensions introduced between cpython versions which make them not exactly equivalent.

But I dont see currently a way to express this with a single cp3x-abi3-... condition for all versions of interpreter.
Would you be open to an idea of revisiting this specific case and having multiple cp3{...}-abi3-... conditions for different interpreters to more closely match with pip selection behaviour here?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions