diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index a87c6932..1d2545e2 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -1,5 +1,14 @@ name: Release +# CONVENTION: before merging any PR that touches this file, run this workflow +# once from the PR's branch with `dry_run: true` (Actions -> Release -> Run +# workflow -> pick the branch) and confirm `test`, `self-interop`, and all four +# `package-libs` legs go green. `publish` is skipped on a dry run, so nothing is +# tagged or released. Several `package-libs` steps only ever execute in this +# workflow (never in `ci.yml`), so a plain PR check does not exercise them -- +# the v0.3.0 attempt failed on a shell bug in `package-libs` that had shipped +# unrun for a week. + on: workflow_dispatch: inputs: @@ -317,22 +326,31 @@ jobs: with: version: ${{ needs.prepare.outputs.zig_version }} + # --summary all leaves a full step tree in the log -- cheap, and it made + # the "verify" false-negative below (a shell bug, not a build bug) + # obvious once package-libs finally ran for real. - name: Build install tree (C + C++ bindings) - run: zig build -Dc-binding=true -Dcpp-binding=true install + run: zig build -Dc-binding=true -Dcpp-binding=true install --summary all - name: Verify install tree is complete shell: bash run: | set -euo pipefail fail=0 - # Dynamic libzzdds -- extension and directory differ per platform - # (.dll lands in bin/ on Windows, since Zig's InstallArtifact - # treats a .dll as isDll()). - if ! ls zig-out/lib/libzzdds.so zig-out/lib/libzzdds.dylib \ - zig-out/bin/zzdds.dll zig-out/lib/zzdds.dll 2>/dev/null | grep -q .; then - echo "::error::no dynamic libzzdds found under zig-out/" - fail=1 - fi + # Dynamic libzzdds -- name/dir differ per platform (.so / .dylib, and + # a .dll lands in bin/ on Windows since Zig's InstallArtifact treats + # it as isDll()). Probe each candidate with `test -f`, NOT + # `ls a b c d | grep`: under `set -o pipefail` a partial `ls` (some + # operands absent -> exit 2) wins over grep's success, so `if ! ls + # ... | grep -q .` inverted to a bogus "not found" even with the + # library sitting right there -- which is exactly how the first + # real release run "failed" on all four platforms. + shlib= + for cand in zig-out/lib/libzzdds.so zig-out/lib/libzzdds.dylib \ + zig-out/bin/zzdds.dll zig-out/lib/zzdds.dll; do + if [ -f "$cand" ]; then shlib="$cand"; break; fi + done + [ -n "$shlib" ] || { echo "::error::no dynamic libzzdds found under zig-out/"; fail=1; } for f in zig-out/include/dcps.h zig-out/include/zzdds.h zig-out/include/zidl_cdr.h \ zig-out/lib/pkgconfig/zzdds.pc zig-out/lib/cmake/ZZDDS/zzdds-config.cmake; do [ -f "$f" ] || { echo "::error::missing $f"; fail=1; } diff --git a/CHANGELOG.md b/CHANGELOG.md index 4652221d..8224173d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,45 @@ see [`docs/implementation_status.md`](docs/implementation_status.md); for planne Dated entries (no release tags past `v0.2.1-zig.0.16.0`; `build.zig.zon` is `0.2.1-zig.0.16.0-dev`). +## 2026-09-03 + +- **Release workflow — first real run shook out three `package-libs` bugs.** That job + (`release.yml`-only, never exercised by `ci.yml`) had shipped unrun since 2026-08-28. + 1. **`Verify install tree is complete` false negative.** The dynamic-lib probe was + `if ! ls <4 candidate paths> 2>/dev/null | grep -q .` under `set -o pipefail`; on any + one platform 3 of the 4 paths are absent so `ls` exits non-zero, `pipefail` propagates + that over grep's success, and `!` inverts it to a bogus "no dynamic libzzdds found" — + with `libzzdds.{so,dylib,dll}` sitting right there. Replaced with a `test -f` loop. + Also added `--summary all` to the install build. + 2. **macOS static archive not linkable by Apple `ld64`.** The `libzidl_cdr.a` that + `Step.Compile`'s GNU-format archiver writes has member offsets Apple's linker rejects + (`64-bit mach-o member 'zidl_cdr.o' not 8-byte aligned`). `zig cc` (LLD) tolerates it — + so `test-bindings` never saw it — but the new prebuilt-bundle consume check, which + links with Apple clang/ld, did. `build.zig` now installs the macOS `libzidl_cdr.a` by + re-packing the object with the `zig ar` subcommand (`--format=darwin`, 8-byte + aligned) instead of `b.installArtifact`; the internally-linked `zidl_cdr` static lib + is unchanged. Fixes a local `zig build install` on macOS and a macOS bundle + cross-compiled from another host, not just the release runner. (ziglang/zig#1981; + Linux `ar` output is fine.) + 3. **macOS dylib not linkable by Apple ld-prime C++ consumers.** Zig 0.16 publishes its + Mach-O linker-synthesized `___dso_handle` in `libzzdds.dylib`'s export trie. When a + C++ consumer registers a destructible static with `__cxa_atexit`, Apple ld 1267 binds + the reference to that dylib export and fails with `target '___dso_handle' does not + have address`. `zig c++`/LLD do not, so `test-bindings` was green; the new + prebuilt-bundle consume check building `examples/cpp/hello_world` with Apple clang++ + caught it. The macOS install step now post-processes the dylib + (`scripts/fix_macos_dylib_exports.sh`) to filter only that private runtime symbol + from its export trie (`strip -s` with the kept-symbol list; `strip -R` alone leaves + `LC_DYLD_EXPORTS_TRIE` untouched); the raw compile artifact is unchanged for Zig's + in-tree links. macOS builds also now default to Zig 0.16's supported deployment floor + (13.0) instead of stamping the build host's current OS version (which produced an + `LC_BUILD_VERSION` skew warning). `verify_release_bundle.py` gained a `check_structure` + guard that fails if the dylib re-exports `___dso_handle`. (ziglang/zig#24370.) +- **Convention — dry-run `release.yml` before merging any change to it.** Documented in the + workflow header and `docs/binding-release-plan.md`: run it from the PR branch with + `dry_run: true` (skips `publish`) and confirm `test`, `self-interop`, and all four + `package-libs` legs pass, since `package-libs` is release-only. + ## 2026-09-02 - **Pinned `zidl` v0.3.12-zig.0.16.0** (`build.zig.zon`) — the selective-parse family diff --git a/build.zig b/build.zig index 05f76d21..31a4c9f5 100644 --- a/build.zig +++ b/build.zig @@ -50,7 +50,17 @@ fn findJniIncludeDir(b: *std.Build, java_path: []const u8) ?JniIncludeDirs { } pub fn build(b: *std.Build) void { - const target = b.standardTargetOptions(.{}); + // A native macOS target otherwise inherits the build host's current OS + // version (for example 26.6.2), making release dylibs unusable to consumers + // targeting an older supported macOS. Zig 0.16 itself requires macOS 13, + // so use that as the default deployment floor unless the caller supplied + // an explicit minimum in -Dtarget. + var target_query = b.standardTargetOptionsQueryOnly(.{}); + const initially_resolved_target = b.resolveTargetQuery(target_query); + if (initially_resolved_target.result.os.tag == .macos and target_query.os_version_min == null) { + target_query.os_version_min = .{ .semver = .{ .major = 13, .minor = 0, .patch = 0 } }; + } + const target = b.resolveTargetQuery(target_query); const optimize = b.standardOptimizeOption(.{}); const sanitize_thread = b.option(bool, "sanitize-thread", "Enable ThreadSanitizer") orelse false; const debug_allocator = b.option(bool, "debug-allocator", "Route the default (allocator=NULL) factory allocation path through std.heap.DebugAllocator instead of std.heap.c_allocator, for fast attributable double-free/UAF diagnostics") orelse false; @@ -495,7 +505,30 @@ pub fn build(b: *std.Build) void { // without this, and was caught immediately with it. .use_llvm = if (sanitize_thread) true else null, }); - b.installArtifact(zidl_cdr_lib); + + // Install libzidl_cdr.a. On macOS the archive that `Step.Compile`'s + // own (GNU-format) archiver writes has member offsets Apple's ld64 + // rejects -- "64-bit mach-o member 'zidl_cdr.o' not 8-byte aligned". + // zig cc / LLD tolerate it (so linking it into libzzdds and the + // binding smoke tests below is fine), but a downstream C/C++ consumer + // building against the installed tree with Apple clang/ld cannot link + // it. Re-pack the same object with the `zig ar` subcommand, whose + // Darwin-format output is 8-byte aligned. Works when cross-compiling a + // macOS bundle from a non-macOS host too. See ziglang/zig#1981. + if (target.result.os.tag == .macos) { + const zidl_cdr_obj = b.addObject(.{ + .name = "zidl_cdr", + .root_module = zidl_cdr_mod, + .use_llvm = if (sanitize_thread) true else null, + }); + const repack = b.addSystemCommand(&.{ b.graph.zig_exe, "ar", "-rcs", "--format=darwin" }); + const fixed_a = repack.addOutputFileArg("libzidl_cdr.a"); + repack.addArtifactArg(zidl_cdr_obj); + const install_fixed_a = b.addInstallFileWithDir(fixed_a, .lib, "libzidl_cdr.a"); + b.getInstallStep().dependOn(&install_fixed_a.step); + } else { + b.installArtifact(zidl_cdr_lib); + } // Build libzzdds as a shared library exposing the C ABI surface. const zzdds_lib = b.addLibrary(.{ @@ -548,10 +581,25 @@ pub fn build(b: *std.Build) void { zzdds_lib.root_module.addIncludePath(b.path("include")); zzdds_lib.root_module.linkLibrary(zidl_cdr_lib); - const install_zzdds_lib = b.addInstallArtifact(zzdds_lib, .{}); - b.getInstallStep().dependOn(&install_zzdds_lib.step); + const install_zzdds_lib_step: *std.Build.Step = if (target.result.os.tag == .macos) blk: { + // Zig 0.16 incorrectly publishes the Mach-O linker-synthesized + // ___dso_handle in a dylib's export trie. An Apple-clang C++ + // consumer with a destructible static then fails in ld-prime with + // "target '___dso_handle' does not have address". Restrict the + // installed dylib to its existing exports minus that private + // runtime symbol. Keep the compile artifact unchanged for Zig's + // own in-tree linking. + const fix_exports = b.addSystemCommand(&.{ + "sh", + b.pathFromRoot("scripts/fix_macos_dylib_exports.sh"), + }); + fix_exports.addArtifactArg(zzdds_lib); + const fixed_dylib = fix_exports.addOutputFileArg("libzzdds.dylib"); + break :blk &b.addInstallFileWithDir(fixed_dylib, .lib, "libzzdds.dylib").step; + } else &b.addInstallArtifact(zzdds_lib, .{}).step; + b.getInstallStep().dependOn(install_zzdds_lib_step); zzdds_lib_for_reuse = zzdds_lib; - zzdds_lib_install_step = &install_zzdds_lib.step; + zzdds_lib_install_step = install_zzdds_lib_step; const gen_smoke_c = b.addRunArtifact(zidl_exe); gen_smoke_c.addArgs(&.{ "-b", "c", "--generate-zzdds-wrappers", "-o" }); diff --git a/build.zig.zon b/build.zig.zon index d284b45f..0ba97c21 100644 --- a/build.zig.zon +++ b/build.zig.zon @@ -16,5 +16,8 @@ "src", "idl", "test", + // build.zig runs scripts/fix_macos_dylib_exports.sh during a macOS + // `-Dc-binding` install, so it must travel with the package. + "scripts", }, } diff --git a/docs/binding-release-plan.md b/docs/binding-release-plan.md index 96ae50cc..98aab5b0 100644 --- a/docs/binding-release-plan.md +++ b/docs/binding-release-plan.md @@ -16,6 +16,12 @@ zzdds client surfaces. generated DDS/zzdds IDL bindings as the user-facing API. - Gate release candidates with `zig build test` and `zig build test-bindings -Dc-binding=true -Dcpp-binding=true`. +- Before merging any PR that modifies `.github/workflows/release.yml`, run the + `Release` workflow from that PR's branch with `dry_run: true` and confirm + `test`, `self-interop`, and all four `package-libs` legs pass. `publish` is + skipped on a dry run. Much of `package-libs` (the install-tree checks, the + prebuilt-bundle consume check) runs *only* in `release.yml`, so ordinary CI + never exercises it. ## Current Smoke Surface diff --git a/docs/roadmap.md b/docs/roadmap.md index 1299eec7..7c5edba1 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -416,6 +416,15 @@ release notes). builds the GitHub-release body from the curated, date-headed `CHANGELOG.md` sections written since the previous release tag (`scripts/extract_changelog.py`), falling back to raw commit subjects only if that yields nothing, and always appending a `compare` link. +- **macOS bundle links with the Apple toolchain** (2026-09-03) — the prebuilt-bundle consume + check building `examples/{c,cpp}/hello_world` with Apple clang/clang++ shook out two macOS + packaging defects, both worked around in `build.zig` (details in `CHANGELOG.md`): the + static `libzidl_cdr.a` re-packed with `zig ar --format=darwin` for `ld64` 8-byte + alignment (ziglang/zig#1981), and `libzzdds.dylib`'s export trie post-processed to drop a + spuriously-exported `___dso_handle` that broke Apple ld-prime C++ consumers + (`scripts/fix_macos_dylib_exports.sh`; ziglang/zig#24370). macOS builds also default to a + 13.0 deployment floor instead of the build host's OS version. **Both workarounds are + upstream Zig bugs — revisit deleting them at a Zig bump.** - **`ReleaseSmall` lane** (2026-08-29) — new `zig build test-release-small` step runs the whole unit suite at `-OReleaseSmall`, wired into `run_deterministic_matrix.py` (so `ci.yml`'s `test-linux` covers it) and `release.yml`'s `test` job (Linux x86_64 only). diff --git a/scripts/fix_macos_dylib_exports.sh b/scripts/fix_macos_dylib_exports.sh new file mode 100755 index 00000000..912b0c7f --- /dev/null +++ b/scripts/fix_macos_dylib_exports.sh @@ -0,0 +1,27 @@ +#!/bin/sh +set -eu + +input=$1 +output=$2 +exports="${output}.exports" + +# Zig 0.16 exports its Mach-O linker-synthesized ___dso_handle from dylibs. +# Apple ld then binds a C++ consumer's __cxa_atexit registration to that +# export instead of synthesizing an image-local handle, and ld-prime fails +# with "target '___dso_handle' does not have address". Keep the dylib's +# existing public interface while removing only that implementation detail +# from its export trie. `strip -R` is insufficient: it edits LC_SYMTAB but +# leaves LC_DYLD_EXPORTS_TRIE unchanged. +if ! /usr/bin/nm -gjU "$input" | /usr/bin/grep -qx '___dso_handle'; then + /bin/cp "$input" "$output" + exit 0 +fi + +/usr/bin/nm -gjU "$input" | /usr/bin/grep -vx '___dso_handle' > "$exports" +/bin/cp "$input" "$output" +/usr/bin/strip -s "$exports" -u "$output" + +if /usr/bin/nm -gjU "$output" | /usr/bin/grep -qx '___dso_handle'; then + echo "error: failed to remove ___dso_handle from $output" >&2 + exit 1 +fi diff --git a/scripts/verify_release_bundle.py b/scripts/verify_release_bundle.py index 2b6fedd0..168f4dfe 100755 --- a/scripts/verify_release_bundle.py +++ b/scripts/verify_release_bundle.py @@ -18,8 +18,9 @@ 3. The bundled `bin/zidl` code generator runs on this platform. 4. `find_package(ZZDDS)` resolves from the relocated prefix and its imported targets are usable -- `test/release-bundle/cmake_consumer` (fast, no - codegen) then the real `examples/c/hello_world` + `examples/cpp/hello_world` - downstream CMake projects, compiled and linked against the bundle. + codegen) then the real `examples//hello_world` downstream CMake + projects (`--example-langs`, default `c,cpp`), compiled and linked against + the bundle. 5. The linked binaries actually run -- a hello_world publisher/subscriber pair exchanges its 10 samples (loads `libzzdds` from the relocated prefix: catches broken rpath / install-name). @@ -103,6 +104,19 @@ def check_structure(prefix: Path) -> None: if not any((prefix / rel).is_file() for rel in shared): fail("bundle has no dynamic libzzdds (looked for: " + ", ".join(shared) + ")") + # Zig 0.16 incorrectly puts its Mach-O linker-synthesized ___dso_handle in + # a dylib's export trie. Apple ld-prime may then bind a C++ consumer's + # image-local reference to the dylib and fail with "target + # '___dso_handle' does not have address". The macOS packaging workaround + # must keep that implementation symbol private. + dylib = prefix / "lib/libzzdds.dylib" + if sys.platform == "darwin" and dylib.is_file(): + proc = run(["nm", "-gjU", str(dylib)], capture=True, timeout=60) + if proc.returncode != 0: + fail(f"could not inspect exports from {dylib}:\n{proc.stdout}") + if "___dso_handle" in proc.stdout.splitlines(): + fail("lib/libzzdds.dylib incorrectly exports ___dso_handle") + log("structure: all required files present") @@ -255,6 +269,9 @@ def main() -> int: help="build examples/{c,cpp}/hello_world against the bundle but do not run the " "pub/sub pair (link coverage only; use where live UDP DDS discovery is flaky, " "e.g. hosted macOS runners -- cmake_consumer still runs and loads libzzdds)") + ap.add_argument("--example-langs", default="c,cpp", + help="comma-separated hello_world ports to build against the bundle " + "(default 'c,cpp'; the useful narrower value is 'c').") ap.add_argument("--work", type=Path, default=None, help="working directory (default: a fresh temp dir)") ap.add_argument("--keep", action="store_true", help="do not delete the working directory on exit") ap.add_argument("--domain-base", type=int, default=58, @@ -300,7 +317,10 @@ def main() -> int: fail(f"cmake_consumer binary exited {proc.returncode}:\n{proc.stdout}") log(f"cmake_consumer: ran -- {proc.stdout.strip()}") - for i, name in enumerate(("c", "cpp")): + langs = [s.strip() for s in args.example_langs.split(",") if s.strip()] + if bad := [l for l in langs if l not in ("c", "cpp")]: + fail(f"--example-langs: unknown {bad} (want c and/or cpp)") + for i, name in enumerate(langs): example = args.examples_dir / name / "hello_world" if not (example / "CMakeLists.txt").is_file(): fail(f"example project not found: {example}")