Conversation
Reviewer's GuideThe PR adds an isolated, opt-in static OpenSSL installation for packages such as cryptography, compiling providers—including legacy algorithms—into libcrypto.a so embedded wheels remain self-contained and auditwheel-independent, while keeping the existing shared OpenSSL installation synchronized and unaffected. Sequence diagram for building an embedded cryptography wheelsequenceDiagram
participant Build as Builder image
participant Static as build-openssl-static.sh
participant Crypto as cryptography build
participant Wheel as Repaired wheel
participant Runtime as Python runtime
Build->>Static: Configure no-shared no-module -fPIC
Static->>Static: make install_sw
Static->>Static: Check ossl_legacy_provider_init in libcrypto.a
Static-->>Build: /opt/_internal/static-openssl
Build->>Crypto: Set OPENSSL_DIR and OPENSSL_STATIC
Crypto->>Crypto: Link static libcrypto.a into _rust.abi3.so
Crypto-->>Wheel: Self-contained extension without bundled libcrypto/libssl
Wheel->>Runtime: OSSL_PROVIDER_load(NULL, legacy)
Runtime-->>Wheel: Legacy ciphers available
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Contributor
|
LGTM |
smatula
force-pushed
the
builder-openssl-static-for-cryptography
branch
2 times, most recently
from
September 15, 2026 20:30
f948576 to
4ff9962
Compare
Gives cryptography wheels the same shape as the official PyPI wheels and
restores six ciphers that are currently unusable, without changing the
shared OpenSSL that CPython, curl and Arrow link against.
Change:
- build-openssl-static.sh: new, builds OpenSSL no-shared no-module -fPIC
into /opt/_internal/static-openssl-3.5 plus a version-independent
static-openssl symlink
- Containerfile: run it in build_cpython from the same RUN as the shared
build, so the two cannot drift apart on version or hash, and propagate
the prefix into the final image
Why:
Linked dynamically, cryptography gets libcrypto and libssl bundled into
the wheel by auditwheel, but not ossl-modules/legacy.so -- OpenSSL
dlopens that, and auditwheel only follows DT_NEEDED. The bundled
libcrypto's MODULESDIR is also compiled to a builder path that does not
exist at wheel runtime. OSSL_PROVIDER_load(NULL, "legacy") fails at
import and Blowfish, CAST5, IDEA, SEED, ARC4 and RC2 all raise
UnsupportedAlgorithm. Since cryptography 50.x that failure is only a
warning, so the import succeeds, wheel-check passes and the broken wheel
ships.
Upstream avoids this by embedding OpenSSL in _rust.abi3.so instead.
This provides an OpenSSL that can be embedded the same way. Consuming it
is opt-in from the index side, via OPENSSL_DIR and OPENSSL_STATIC in
overrides/settings/cryptography.yaml.
Measured. cryptography 50.0.1 built from sdist, repaired with auditwheel,
exercised on python:3.12-slim:
_rust.abi3.so bundled libs legacy ciphers
index today 6.32 MB libcrypto, 1 of 7
libssl
this change 13.66 MB none 7 of 7
PyPI wheel 14.39 MB none 7 of 7
the 1 that always works is TripleDES, which is in the default provider
DT_NEEDED on _rust.abi3.so is identical to the PyPI wheel's: libgcc_s,
libpthread, libdl, libc, ld-linux
Isolation checks on the same image: the shared libcrypto.so.3 and
libssl.so.3 are byte-identical to before, its ossl-modules/legacy.so is
still in place, the static prefix is absent from PKG_CONFIG_PATH and from
the ldconfig cache, and builder CPython still does TLS 1.3 to pypi.org
with curl returning 200. No other wheel is affected.
Notes:
- no-shared does NOT imply no-module. Without no-module the providers
stay separate .so files that a static link cannot reach, and the
resulting wheel is broken in exactly the same way -- libcrypto.a then
contains no ossl_legacy_provider_init at all. The script asserts that
symbol is present so this cannot regress silently.
- The prefix is named so it does not match the /opt/_internal/openssl*
globs the final stage uses to publish the shared build into
/usr/local. If it matched, its pkgconfig files would be symlinked over
the shared ones and every package would silently switch to the static
build.
- Adds 17 MB to the builder image (14 MB archives, 2.4 MB headers) and
nothing to any wheel that does not opt in. The 7.8 MB openssl CLI that
install_sw also lays down is removed; rebuilding cryptography without
it gave the same 13.66 MB extension module, no bundled libs and 7 of 7
ciphers, so nothing reads it.
- Does not close the OpenSSL version gap: the builder is on 3.5.4 while
PyPI's cryptography 50.0.1 ships 4.0.2. Separate change.
Validated by rebuilding OpenSSL inside the existing builder image and
committing it, not by a full image build, so the compiler was in-image
gcc rather than the static_clang cross-compiler. The PR's own image build
is the real confirmation.
Assisted-by: Claude Opus 5 <noreply@anthropic.com>
smatula
force-pushed
the
builder-openssl-static-for-cryptography
branch
from
September 16, 2026 12:21
4ff9962 to
d44e958
Compare
smatula
marked this pull request as ready for review
September 16, 2026 12:22
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've reviewed your changes and they look great!
Sourcery assessment
Needs a human reviewer. This changes the OpenSSL code embedded in published wheels and changes which legacy crypto providers are available, so a build or version mistake could cause cryptographic failures or security-relevant behavior in installed packages. Reverting would stop future builds but would not remove already published wheels; those would require replacement or yanking.
smatula
force-pushed
the
builder-openssl-static-for-cryptography
branch
from
September 16, 2026 14:21
d44e958 to
0ced5cf
Compare
jvulgan
reviewed
Sep 16, 2026
Parity review compares the OpenSSL version embedded in our wheels against
the PyPI ones, and 3.5.4 (30 Sep 2025) is roughly eleven months behind.
cryptography 50.0.1 on PyPI statically links OpenSSL 4.0.2, read out of
the shipped binary:
$ strings -a cryptography/hazmat/bindings/_rust.abi3.so \
| grep -E "^OpenSSL [0-9]"
OpenSSL 4.0.2 25 Aug 2026
An earlier revision of this commit moved both builds to 4.0.2 on the
assumption that nothing in this stage depended on what 4.0 removed. That
was wrong, and CI proved it: OpenSSL 4.0 made struct asn1_string_st
opaque, and CPython 3.12's Modules/_ssl.c still reaches into it.
./Modules/_ssl.c:1342:31: error: incomplete definition of type
'ASN1_OCTET_STRING' (aka 'struct asn1_string_st')
1342 | if (name->d.ip->length == 4) {
6 warnings and 8 errors generated.
make: *** [Makefile:3134: Modules/_ssl.o] Error 1
Eight errors across lines 1342-1548, all direct ->data and ->length
accesses. There is no flag for it; the shared build cannot move to 4.x
until CPython goes through ASN1_STRING_get0_data() upstream. The check
that missed this looked for ENGINE_ references and the
PY_PROTO_MINIMUM_AVAILABLE guard and found both clear, but never looked
at struct member access.
So the two builds get separate pins. They already had separate prefixes
and separate consumers; only the version was shared.
The static build is reachable only by a package that sets OPENSSL_DIR and
OPENSSL_STATIC, which today means cryptography alone, and that path does
not go through CPython's _ssl. It tracks 4.x. The failing run got past
both OpenSSL builds before dying in CPython, so 4.0.2 with no-shared
no-module is already known to compile and to keep ossl_legacy_provider_init
in libcrypto.a -- the static build's assertion passed on the real
cross-compiler.
The shared build tracks 3.x, at 3.6.4, the newest release on that line
and published the same day as 4.0.2. Being behind 4.x costs nothing here,
because PyPI has no single OpenSSL version to match: every project builds
its own. pypa/manylinux pins 3.5.8 in its own Dockerfile, which is what
cmake 4.4.3 embeds; snowflake-connector-python 5.0.0b2 vendors 3.6.2 via
the openssl-src crate; cryptography ships its own image built on 4.0.2.
3.6.4 is at or above every consumer of the shared build:
package ours (3.6.4) PyPI
cmake 3.6.4 3.5.8 ahead
snowflake 3.6.4 3.6.2 ahead
Both tarball hashes were taken from the release artifacts and
cross-checked against the .sha256 published beside them.
The shared prefix moves from /opt/_internal/openssl-3.5 to openssl-3.6
and the static one lands at static-openssl-4.0. The final stage globs
openssl* and static-openssl* separately, so the two stay isolated, and
the version-independent static-openssl symlink means consumers setting
OPENSSL_DIR need no change.
Assisted-by: Claude Opus 5 <noreply@anthropic.com>
smatula
force-pushed
the
builder-openssl-static-for-cryptography
branch
from
September 16, 2026 18:50
0ced5cf to
e0ad777
Compare
The static OpenSSL was building correctly and then being deleted before it reached the final image. Pulled from the PR build of 0ced5cf: /opt/_internal/static-openssl -> /opt/_internal/static-openssl-4.0 /opt/_internal/static-openssl-4.0/include/openssl/*.h present /opt/_internal/static-openssl-4.0/lib/ cmake/ ossl-modules/ pkgconfig/ present no libcrypto.a no libssl.a $ find /opt/_internal -name '*.a' | wc -l 0 finalize.sh drops every archive under /opt/_internal: find /opt/_internal -name '*.a' -print0 | xargs -0 rm -f That line is original to the builder (044d5ad) and is aimed at libpython*.a, which the lines above it tar up first. It is unscoped, and the Containerfile invokes it in the same RUN that copies static-openssl* in, one instruction later, so the archives are deleted a few seconds after they arrive. Nothing before this branch needed an .a to survive into the final image, so nothing noticed. What is left behind is worse than an empty directory: the headers, the .pc files and lib/ itself all survive, so the prefix looks populated. cryptography would set OPENSSL_DIR and OPENSSL_STATIC, openssl-sys would go looking for lib/libcrypto.a, and the failure would land on whoever re-onboarded it rather than here. So exempt that one prefix. Everything else still goes, including the shared openssl-3.x archives and libpython*.a; verified in the image by seeding the pre-purge state and running the new predicate. The second half of this is the check. build-openssl-static.sh already asserts ossl_legacy_provider_init is in libcrypto.a, and that assertion passed on the real cross-compiler in the failing build -- it runs in build_cpython, two stages before anything here, and cannot see what happens to the archive afterwards. A build-stage check on an artifact that ships from the final stage cannot do the job on its own, so the same assertion now also runs on the finished filesystem. Against the 0ced5cf image it fails on the missing archive, which is what it is for. Assisted-by: Claude Opus 5 <noreply@anthropic.com>
jvulgan
approved these changes
Sep 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Gives cryptography wheels the same shape as the official PyPI wheels and restores six ciphers that are currently unusable. The shared OpenSSL that CPython, curl and Arrow link against stays on the 3.x line and keeps its own pin.
Change:
Why:
Linked dynamically, cryptography gets libcrypto and libssl bundled into the wheel by auditwheel, but not ossl-modules/legacy.so -- OpenSSL dlopens that, and auditwheel only follows DT_NEEDED. The bundled libcrypto's MODULESDIR is also compiled to a builder path that does not exist at wheel runtime. OSSL_PROVIDER_load(NULL, "legacy") fails at import and Blowfish, CAST5, IDEA, SEED, ARC4 and RC2 all raise UnsupportedAlgorithm. Since cryptography 50.x that failure is only a warning, so the import succeeds, wheel-check passes and the broken wheel ships.
Upstream avoids this by embedding OpenSSL in _rust.abi3.so instead. This provides an OpenSSL that can be embedded the same way. Consuming it is opt-in from the index side, via OPENSSL_DIR and OPENSSL_STATIC in overrides/settings/cryptography.yaml.
Measured. cryptography 50.0.1 built from sdist, repaired with auditwheel, exercised on python:3.12-slim:
the 1 that always works is TripleDES, which is in the default provider
DT_NEEDED on _rust.abi3.so is identical to the PyPI wheel's: libgcc_s,
libpthread, libdl, libc, ld-linux
Taken against a 3.5.4 static build, before the pin split below. What is
being measured is the mechanism, which does not change with the version;
the exact MB on 4.0.2 will differ slightly.
Two OpenSSL pins, not one:
An earlier revision moved both builds to 4.0.2. CI rejected it. OpenSSL 4.0 made struct asn1_string_st opaque and CPython 3.12's Modules/_ssl.c still reaches into it directly, so 3.12.12 fails to compile with eight "incomplete definition of type 'ASN1_OCTET_STRING'" errors across lines 1342-1548. There is no flag for it; the shared build cannot move to 4.x until CPython goes through ASN1_STRING_get0_data() upstream.
So the two builds get separate pins, which they can have because they already had separate prefixes and separate consumers:
This does close the version gap for the static build: PyPI's cryptography 50.0.1 ships OpenSSL 4.0.2 and so do we now. The shared build being on 3.x costs nothing, because there is no single "PyPI OpenSSL version" to match -- every project builds its own. pypa/manylinux pins 3.5.8 in its own Dockerfile, which is what cmake embeds; snowflake-connector-python 5.0.0b2 vendors 3.6.2 via the openssl-src crate. 3.6.4 is at or above both.
Isolation checks, on a full image build of this branch: the two prefixes do not overlap, the static prefix is absent from PKG_CONFIG_PATH and from the ldconfig cache, the shared build still ships its ossl-modules/legacy.so, and no wheel that does not set OPENSSL_DIR can reach the static one. The shared libcrypto.so.3 and libssl.so.3 are no longer byte-identical to main, because the shared pin moves 3.5.4 -> 3.6.4 -- a deliberate part of this change rather than a side effect of the static build.
Notes:
On the finalize.sh change, because it is the least obvious part:
The first full image build of this branch produced a static-openssl prefix with headers, .pc files and an empty lib/ -- and no libcrypto.a anywhere in the image. finalize.sh drops every archive under /opt/_internal:
That line is original to the builder (044d5ad) and is aimed at libpython*.a, which the lines above it tar up first. It is unscoped, and the Containerfile invokes it in the same RUN that copies static-openssl* in, one instruction later. Nothing before this branch needed an .a to survive into the final image, so nothing noticed. What is left behind looks populated, so cryptography would have failed to link at re-onboarding time rather than here.
build-openssl-static.sh already asserted ossl_legacy_provider_init was present, and that assertion passed -- it runs in build_cpython, two stages earlier, and cannot see what happens to the archive afterwards. A build-stage check on an artifact that ships from the final stage cannot do the job alone, so the same assertion now also runs on the finished filesystem.
Validated by a full image build, which is what found both of the problems above; the earlier in-image rebuild of OpenSSL alone would not have caught either.
Assisted-by: Claude Opus 5 noreply@anthropic.com