From 06c1cc23bd826478c0af589204e52d250afcf39b Mon Sep 17 00:00:00 2001 From: Teodor Calin Date: Mon, 27 Jul 2026 17:39:11 +0300 Subject: [PATCH 1/2] fix(publish): stop shipping Linux x86-64 binaries to every platform MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `pip install pilotprotocol` was broken for most users on 1.13.6. Three independent packaging defects compounded: 1. The sdist carried the native runtime. MANIFEST.in recursive-included pilotprotocol/bin/, and the publish workflow stages the freshly built binaries there *before* `python -m build --sdist` runs on the Linux runner. So the "platform-neutral" sdist on PyPI contained Linux x86-64 ELF binaries. Anyone who fell through to it — see (2) and (3) — got a successful install and then, at first use: macOS: FileNotFoundError: Cannot find libpilot.dylib linux/arm64: OSError: .../libpilot.so: cannot open shared object file MANIFEST.in now prunes pilotprotocol/bin. The wheel still carries it via [tool.setuptools.package-data]; only the sdist is pruned, so an sdist install degrades to a clear "libpilot not found" instead of silently installing the wrong architecture. 2. The wheels were tagged cp311. The payload is prebuilt Go binaries plus a CGO shared library loaded through ctypes — no C extension, nothing linked against libpython — but BinaryDistribution.has_ext_modules() makes setuptools stamp the build host's interpreter tag, and this workflow builds on Python 3.11. Result: Python 3.12/3.13/3.14 could not see the wheel at all and silently fell back to the broken sdist. Retag to py3-none so any CPython 3.x resolves it. 3. Only two wheels were built, and one of them lied. The matrix was {ubuntu-latest, macos-latest}, so there was no linux/arm64 wheel at all, and the macOS wheel was labelled macosx_26_0_universal2 — universal2 because the runner's Python is, macOS 26 because that is what the runner now runs. The Go toolchain had produced a single-arch arm64 libpilot, so the universal2 claim was false (broken on Intel) and the 26.0 floor excluded everyone not yet on Tahoe. Build all four of linux/{x86_64,aarch64} and macos/{arm64,x86_64} on native runners, and force the real platform tag with an 11.0 deployment floor. Also adds a post-build assertion that the wheel actually contains libpilot and pilotctl, so a regression here fails the release instead of publishing an empty package. Co-Authored-By: Claude Opus 5 --- .github/workflows/publish.yml | 72 +++++++++++++++++++++++++++++++---- MANIFEST.in | 21 +++++++--- 2 files changed, 80 insertions(+), 13 deletions(-) diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 8501593..b0f470b 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -61,15 +61,23 @@ jobs: build-wheels: needs: prep - name: Build wheel (${{ matrix.platform }}) + name: Build wheel (${{ matrix.platform }}/${{ matrix.arch }}) strategy: fail-fast: false matrix: include: - os: ubuntu-latest platform: linux - - os: macos-latest + arch: x86_64 + - os: ubuntu-24.04-arm + platform: linux + arch: aarch64 + - os: macos-14 # Apple Silicon + platform: macos + arch: arm64 + - os: macos-13 # Intel platform: macos + arch: x86_64 runs-on: ${{ matrix.os }} steps: - name: Checkout sdk-python @@ -235,27 +243,75 @@ jobs: working-directory: sdk-python run: | pip install auditwheel patchelf - for plat in manylinux_2_35_x86_64 manylinux_2_31_x86_64 manylinux_2_28_x86_64; do - if auditwheel repair dist/*-linux_x86_64.whl --plat "$plat" -w dist/ 2>/dev/null; then + A="${{ matrix.arch }}" + for plat in "manylinux_2_35_$A" "manylinux_2_31_$A" "manylinux_2_28_$A"; do + if auditwheel repair dist/*-linux_$A.whl --plat "$plat" -w dist/ 2>/dev/null; then echo "repaired to $plat"; break fi done - rm -f dist/*-linux_x86_64.whl || true + rm -f dist/*-linux_$A.whl || true + + # The wheel payload is pure data — prebuilt Go binaries and a CGO shared + # library loaded via ctypes. Nothing links against libpython and there is + # no C extension, so the interpreter tag setuptools infers from the build + # host (cp311, because this workflow builds on Python 3.11) is wrong and + # needlessly narrow: it made the wheel invisible to every Python except + # 3.11, sending 3.12/3.13/3.14 users to the sdist instead. + # + # Retag to py3-none so any CPython 3.x resolves it. The platform tag is + # also forced on macOS: setuptools labels the wheel "universal2" because + # the runner's Python is universal2, but the Go toolchain produced a + # single-arch libpilot, so the universal2 claim was a lie that broke + # Intel Macs. Pin the real arch and a 11.0 floor instead of the runner's + # macOS 26, which excluded everyone not yet on Tahoe. + - name: Normalize wheel tags + shell: bash + working-directory: sdk-python + run: | + set -euo pipefail + python -m pip install --upgrade wheel + if [ "${{ matrix.platform }}" = "macos" ]; then + python -m wheel tags --python-tag py3 --abi-tag none \ + --platform-tag "macosx_11_0_${{ matrix.arch }}" --remove dist/*.whl + else + python -m wheel tags --python-tag py3 --abi-tag none --remove dist/*.whl + fi + echo "Final wheel(s):"; ls -1 dist/*.whl - name: Verify shell: bash working-directory: sdk-python run: python -m twine check dist/* + # Guard against shipping an empty wheel: the native runtime is the whole + # point of this package, and MANIFEST.in now prunes it from the sdist. + - name: Assert wheel carries the native runtime + shell: bash + working-directory: sdk-python + run: | + python - <<'PY' + import glob, sys, zipfile + w = glob.glob("dist/*.whl")[0] + names = zipfile.ZipFile(w).namelist() + need = ["pilotprotocol/bin/pilot-daemon", "pilotctl"] + have = [n for n in names if "/bin/" in n] + print(w, "->", have) + if not any("libpilot" in n for n in have) or not any("pilotctl" in n for n in have): + sys.exit("::error::wheel is missing the native runtime") + PY + - name: Upload wheel uses: actions/upload-artifact@v4 with: - name: dist-${{ matrix.platform }} + name: dist-${{ matrix.platform }}-${{ matrix.arch }} path: sdk-python/dist/*.whl retention-days: 7 - - name: Upload sdist (Linux only) - if: matrix.platform == 'linux' + # Exactly one job may publish the sdist, otherwise the artifact name + # collides. It is platform-neutral now (MANIFEST.in prunes bin/), so any + # single job will do. + - name: Upload sdist (Linux x86_64 only) + if: matrix.platform == 'linux' && matrix.arch == 'x86_64' uses: actions/upload-artifact@v4 with: name: dist-sdist diff --git a/MANIFEST.in b/MANIFEST.in index fe134ae..136c7a1 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -3,11 +3,22 @@ include README.md include LICENSE include CHANGELOG.md -# Include all binaries in bin/ directory (the seed cache). -# Dotfiles like .pilot-version need an explicit pattern because some -# setuptools versions skip them under recursive-include. -recursive-include pilotprotocol/bin * -include pilotprotocol/bin/.pilot-version +# The native runtime (pilot-daemon, pilotctl, pilot-updater, libpilot.*) is +# deliberately NOT shipped in the sdist. +# +# Those binaries are platform-specific and are staged into pilotprotocol/bin/ +# by the publish workflow just before the wheel is built, so whatever host +# builds the sdist would otherwise bake ITS OWN os/arch binaries into a +# platform-neutral artifact. That is exactly what shipped in <=1.13.6: the +# sdist was cut on the Linux runner, so every platform without a matching +# wheel (macOS on any Python != 3.11, and all arm64 Linux) silently installed +# Linux x86-64 ELF binaries and then failed at run time with a confusing +# "cannot open shared object file" / missing-libpilot.dylib error. +# +# The wheel still carries them via [tool.setuptools.package-data]; only the +# sdist is pruned. Installing from sdist now yields a clearly-diagnosable +# "libpilot not found" instead of a wrong-architecture payload. +prune pilotprotocol/bin # Include type stubs if any recursive-include pilotprotocol *.pyi From 41c9724aa37a26aa0b20690276d87c54b04f4556 Mon Sep 17 00:00:00 2001 From: Teodor Calin Date: Mon, 27 Jul 2026 17:41:37 +0300 Subject: [PATCH 2/2] fix(ci): install pytest-cov in the ffi-smoke job MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The ffi-smoke job — the one that proves libpilot actually exports every symbol the SDK binds, i.e. the regression test for C1 — has failed on every run since it was added in b814fdc, including on main. pyproject's [tool.pytest.ini_options] addopts unconditionally passes --cov/--cov-report, and the job passes --no-cov to turn that off. The job installed only `pytest`, so both flags were unrecognized and pytest exited 4 during argument parsing, before collecting anything: pytest: error: unrecognized arguments: --cov=pilotprotocol --cov-report=term-missing ... --no-cov Install pytest-cov so --no-cov is understood and the smoke test actually runs. Co-Authored-By: Claude Opus 5 --- .github/workflows/ci.yml | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b53032b..58b4eb3 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -79,7 +79,13 @@ jobs: working-directory: sdk-python run: | pip install --upgrade pip - pip install -e . pytest + # pytest-cov is required even though this job measures no coverage: + # pyproject's [tool.pytest.ini_options] addopts always passes --cov, + # and the run below passes --no-cov to switch it off. Without the + # plugin installed, pytest rejects both as unrecognized arguments and + # exits 4 before collecting a single test — which is why ffi-smoke + # has failed on every run since it was introduced. + pip install -e . pytest pytest-cov - name: Resolve FFI symbols against the freshly built library working-directory: sdk-python