feat(bindings): build, test and publish the Swift package, the Android library and the Python wheels from a release - #496
Merged
Conversation
…d library and the Python wheels from a release A release builds each from its own binaries and tests it before anything publishes: the Swift package on a simulator with an application built against it, the Android library with all four ABIs and a minified application, each wheel installed by pip and run through the suite on its platform. The wheels, the Swift package and the XCFramework its manifest names go on the GitHub release. Maven Central and PyPI publish behind repository variables; the Swift package is pulled by its distribution repository's own workflow, which verifies the release's attestations and tags itself with its own token, so no credential for it exists here. Two defects on the way: every wheel was py3-none-any (four files, one name, four libraries), and a release candidate would have taken the release's number on PyPI. The Linux tag is now read from the library's glibc floor and the wheel is numbered from the tag in Python's spelling.
A bare repository's HEAD points at git's configured default branch, which is master where nothing set it, as on the runner. The test pushed the workflow to main, and the clone that edits it then checked out nothing.
pyproject.toml has never installed bless on Windows, where it has no backend, and ble_peripheral imported it at module level on the package's way in, so the import failed on every Windows install. Guard the import; is_available() is False without it, as start() already requires. Found by the first run of the Python suite against a Windows wheel.
… one line, and the suite states two Windows differences local_api_client.py --socket raised the NotImplementedError asyncio uses for a Unix socket on Windows and printed a traceback where a user should see one JSON line; it now does, pointing at --tcp. Two tests state what the first run of the suite on Windows found. After a retried stop(), the core, not the manager, still holds the file-store lock there, and the next manager over the same directories is refused; close() releases the stores and passes, so the stop variant is xfail on Windows until it is reproduced off a runner. The stalled-peer bound test wedges on a single 1 MiB frame, which Windows loopback absorbs despite 8 KiB socket buffers; the 2 MiB stall its sibling forms does wedge, so it is skipped there and the bound stays covered.
The lifecycle test built the real AsyncZeroconf from synchronous code, which binds it to a thread of its own, and closed it inside asyncio.run, on another loop: a wait that timed out once on a Windows runner. The bridge builds it in start(), on the running loop; the test now does the same.
The dry run of the release built them manylinux_2_34: the floor is glibc 2.34, which the old manylinux_2_17 tag never checked.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
A release now builds, tests and publishes the three bindings that were built and tested on every pull request but shipped nowhere: the Swift package, the Android library and the Python wheels.
Built and tested from the release's own binaries, before anything publishes. A failure in any of these stops npm and crates.io too.
swift-package: the XCFramework the package links is built inbuild-iosfrom the gated device and simulator slices and zipped once; the package is assembled around it and runs every suite, the bridge's suites and the consumer check on a simulator (scripts/test-swift-package.sh, the same script the pull-request job now runs). The manifest a consumer will resolve (real url, real checksum) is parsed byswift package dump-package.android-library: built from the release's four ABIs and the regenerated Kotlin, with the pull-request checks plus--require-natives, lint, the test-count check (scripts/check_android_test_results.py, shared with ci.yml) and the minified consumer build. Writes an unsigned Maven repository.python-package+test-python-wheels: one wheel per desktop library, installed by pip and run through the whole suite on macOS, Linux x86_64, Linux aarch64 and Windows.On the GitHub release: the wheels,
offline-protocol-X.Y.Z-swiftpm-xcframework.zipandoffline-protocol-X.Y.Z-swift-package.tar.gz, all underSHA256SUMS.txtand the attestation.Published, each behind a repository variable:
com.offlineprotocol:offline-protocol-android)MAVEN_CENTRAL_PUBLISHMAVEN_CENTRAL_USERNAME/PASSWORD,MAVEN_SIGNING_KEY[_PASSWORD]offline-protocol-sdk)PYPI_PUBLISHpypiThe Swift package is pulled, not pushed. The enterprise policy disables deploy keys on
Offline-Protocol/offline-protocol-swift, and nothing in this repository should hold a credential for another one anyway. So the release attaches the assembled package and the archive its manifest names, and the distribution repository's own workflow (already in place, currently disabled) downloads the two, verifies their attestations against this repository'srelease.yml, checks outscripts/publish-swift-package.shfrom the release tag and tags itself with its own token.gh workflow run publish.yml -R Offline-Protocol/offline-protocol-swift -f version=X.Y.Zafter a release, or its six-hourly schedule.Three bugs found on the way
py3-none-any. Four wheels with one file name and a different native library inside; pip would have installed a Windows DLL on Linux without a word. Each wheel is now retagged for its platform. The Linux tag is read from the library (manylinux_2_Nfor the newest glibc symbol it needs) rather than written down: the oldmanylinux_2_17claim was never checked, and the library carries the floor of the image that built it.import offline_protocol_sdkfailed on every Windows install.pyproject.tomlhas never installedblesson Windows, where it has no backend, andble_peripheralimported it at module level on the package's way in. The import is guarded;BlePeripheral.is_available()is False there, whichstart()already required. Caught by the first run of the suite against a Windows wheel, in this PR's CI.pyproject.tomlcarries the release core, sov0.28.0-rc.1builtoffline_protocol_sdk-0.28.0, which PyPI never lets that release have afterwards. The wheel is now numbered from the tag in Python's spelling (scripts/pep440-version.sh:0.28.0rc1); a suffix with no agreed spelling fails the version gate before anything builds.What cannot be undone, and what refuses
publish-swift-package.shaccepts a version already tagged only when the tag holds exactly what it would publish (a re-run), and refuses one that differs. The workflow directory survives each tree replacement and is excluded from that comparison.scripts/maven-central-bundle.sh), and the bundle refuses a file it does not expect, a missing javadoc or sources jar, or a second version.Verification
actionlintclean onci.yml,release.ymland the distribution workflow; shellcheck clean on every script.scripts/tests/test-package-release-assets.sh(now 12 assets, 15 negative controls),test-publish-swift-package.sh(first publish, re-run, moved-tag refusal, rehearsal, six refusals each asserting its reason),test-pep440-version.sh,test-assemble-swift-package.sh,test-generate-bindings.sh,test_check_android_aar.py: all green locally.scripts/tests/test-maven-central.sh(throwaway gpg key, stand-in Portal playing every deployment state): green on Ubuntu 24.04.build-wheel.shinstalled in a clean venv and passed the whole Python suite (801 tests) from insidetests/.GITHUB_REPOSITORY, which inside the distribution repository names that repository, so its workflow would have refused every manifest (publish-swift-package.shnow takes--repository, and its test runs under the distribution repository's environment); and the Python suite's local-API tests use Unix sockets, which Windows lacks, so the Windows wheel test could never pass (the Unix-socket server skips on Windows, the TCP tests run there).ci.ymlgainsPython Wheel (macos-arm64, windows-x86_64): the library built, the wheel built and installed, the suite run from insidetests/, so the release's wheel test is not its first execution on either platform. This PR's CI is that first execution; expect a round on it.test-build-wheel.sh(Linux-only: gcc and readelf), the refactored Swift simulator script, and theubuntu-24.04-armwheel leg.Windows, after the first run of the suite there
767 of 770 tests passed on the Windows wheel the first time. Of the three that did not, one was the example:
local_api_client.py --socketraised theNotImplementedErrorasyncio uses for a Unix socket on Windows and printed a traceback where a user should see one JSON line; it now does. Two are platform differences the suite now states:test_a_retry_that_never_suspends_still_frees_the_manager[stop]isxfailon Windows: after a retriedstop(), the manager is freed (its weakref is None) but the core still holds the file-store lock, so the next manager over the same directories is refused.close()releases the stores whatever still holds the core, and its variant passes. Not reproduced off a runner yet; a Windows caller that wants the stores back at once shouldclose().test_the_real_responder_is_built_lazilybuilt the real zeroconf from synchronous code (so it bound to a thread of its own) and closed it on another loop, a wait that timed out once on the Windows runner and passed the time before. It now builds and closes on one loop, as the bridge does instart().test_a_stalled_peer_queue_is_boundedis skipped on Windows: it wedges on a single 1 MiB frame, which Windows loopback absorbs despite the 8 KiB socket buffers, while the 2 MiB stall the restart test forms does wedge there. The queue bound it checks is the same code on every platform.Not in this PR
offline-protocol-servicecommand defaults to a Unix socket, which asyncio does not provide on Windows; a Windows service needs--tcptoday, and a one-line refusal for the default there is a small follow-up.com.offlineprotocolnamespace, the signing key onkeys.openpgp.org, the fourMAVEN_*secrets, and the PyPI trusted publisher once the organisation exists.CONTRIBUTING.mdlists it.workflow_dispatchdry run ofrelease.ymlfrom this branch, withversion=0.27.0, is the rehearsal to run once the three new jobs are green here.