Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
146 changes: 142 additions & 4 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,42 @@ jobs:
- name: Test (ReleaseFast)
run: zig build test -Doptimize=ReleaseFast

# Binding smoke tests, mirroring ci.yml's test-other job: the release
# gate previously ran only `zig build test` (+ReleaseFast +TSan) and
# never built a single binding, so a PR breaking the C/C++/Java
# generated bindings could still cut a release. `test-bindings` is
# pure Zig-build-system (zig cc / zig c++, no external CMake), so it
# runs on every platform; Java/JNI on Windows stays deferred (see
# ci.yml's test-other for the full CFG-hypothesis writeup).
- name: Install JDK (java-binding needs jni.h, not just a JRE)
if: runner.os != 'Windows'
uses: actions/setup-java@v4
with:
distribution: temurin
java-version: "21"

- name: C/C++/Java binding smoke tests
if: runner.os != 'Windows'
shell: bash
run: zig build test-bindings -Dc-binding=true -Dcpp-binding=true -Djava-binding=true 2>&1 | tee "$RUNNER_TEMP/binding-smoke.log"

- name: C/C++ binding smoke tests (Windows -- Java deferred, see ci.yml)
if: runner.os == 'Windows'
shell: bash
run: zig build test-bindings -Dc-binding=true -Dcpp-binding=true 2>&1 | tee "$RUNNER_TEMP/binding-smoke.log"

# build.zig downgrades missing JDK/JNI headers or javac to a
# std.log.warn + skip rather than a build failure -- without this
# check a broken setup-java wiring would silently pass instead of
# actually testing the Java binding.
- name: Verify no binding was silently skipped
shell: bash
run: |
if grep -qE "jni\.h not found|javac not found" "$RUNNER_TEMP/binding-smoke.log"; then
echo "::error::A binding was silently skipped instead of tested — see build.zig's std.log.warn fallbacks"
exit 1
fi

# ThreadSanitizer (Clang/LLVM) has no real Windows support. macOS is
# deliberately NOT included here either, despite being TSan-capable in
# principle -- see ci.yml's test-other job for why: even the minimal
Expand Down Expand Up @@ -142,7 +178,7 @@ jobs:
needs: [prepare, test]
runs-on: ubuntu-latest
env:
INTEROP_RTPS_REF: zenzen-a9f61ee
INTEROP_RTPS_REF: zenzen-c052430

steps:
- name: Check out zzdds
Expand Down Expand Up @@ -179,8 +215,13 @@ jobs:
with:
version: ${{ needs.prepare.outputs.zig_version }}

- name: Install Python (match omg-dds/dds-rtps interop workflow)
uses: actions/setup-python@v5
with:
python-version: "3.11.4"

- name: Install Python dependencies
run: pip install pexpect junitparser
run: pip install --requirement dds-rtps/requirements.txt

- name: Build shape_main (ReleaseSafe)
working-directory: dds-rtps/srcZig/zzdds
Expand All @@ -204,10 +245,92 @@ jobs:
name: release-interop-results
path: /tmp/interop-results.xml

# ── Prebuilt C/C++ library bundles ───────────────────────────────────────
#
# release.yml previously published only a git tag + a `zig fetch` URL --
# nothing for C/C++ consumers who don't build from source. This job builds
# the install tree (dynamic libzzdds + static libzidl_cdr + headers +
# pkgconfig + CMake package files) on each release platform and uploads it
# as a per-platform tarball; `publish` attaches them to the GitHub release.
#
# The *functional* smoke test of these libraries is the `test` job's
# `test-bindings` step, which compiles and runs real C/C++/Java programs
# against a freshly built libzzdds on every platform -- this job only has
# to confirm the install tree is complete before packaging it.
package-libs:
name: package-libs (${{ matrix.name }})
needs: [prepare, test]
strategy:
fail-fast: false
matrix:
include:
- name: linux-x86_64
os: ubuntu-latest
- name: linux-arm64
os: ubuntu-24.04-arm
- name: macos-arm64
os: macos-latest
- name: windows-x86_64
os: windows-latest
runs-on: ${{ matrix.os }}

steps:
- uses: actions/checkout@v4

- name: Install Zig
uses: mlugg/setup-zig@v2
with:
version: ${{ needs.prepare.outputs.zig_version }}

- name: Build install tree (C + C++ bindings)
run: zig build -Dc-binding=true -Dcpp-binding=true install

- name: Verify install tree is complete
shell: bash
run: |
set -euo pipefail
fail=0
# Dynamic libzzdds -- extension and directory differ per platform
# (.dll lands in bin/ on Windows, since Zig's InstallArtifact
# treats a .dll as isDll()).
if ! ls zig-out/lib/libzzdds.so zig-out/lib/libzzdds.dylib \
zig-out/bin/zzdds.dll zig-out/lib/zzdds.dll 2>/dev/null | grep -q .; then
echo "::error::no dynamic libzzdds found under zig-out/"
fail=1
fi
for f in zig-out/include/dcps.h zig-out/include/zzdds.h zig-out/include/zidl_cdr.h \
zig-out/lib/pkgconfig/zzdds.pc zig-out/lib/cmake/ZZDDS/zzdds-config.cmake; do
[ -f "$f" ] || { echo "::error::missing $f"; fail=1; }
done
# The bundle is renamed and extracted elsewhere, so zzdds.pc must be
# relocatable (prefix derived from ${pcfiledir}) rather than carrying
# a baked-in absolute build path.
if ! grep -qE '^prefix=\$\{pcfiledir\}' zig-out/lib/pkgconfig/zzdds.pc; then
echo "::error::zzdds.pc prefix= is not \${pcfiledir}-relative — bundle would not be relocatable"
grep '^prefix=' zig-out/lib/pkgconfig/zzdds.pc
fail=1
fi
[ "$fail" -eq 0 ]

- name: Package
shell: bash
run: |
set -euo pipefail
dir="zzdds-${{ needs.prepare.outputs.full_version }}-${{ matrix.name }}"
mv zig-out "$dir"
Comment thread
greptile-apps[bot] marked this conversation as resolved.
tar -czf "${dir}.tar.gz" "$dir"

- name: Upload bundle
uses: actions/upload-artifact@v4
with:
name: libbundle-${{ matrix.name }}
path: "*.tar.gz"
retention-days: 7

# ── Version bump, tag, and GitHub release ─────────────────────────────────
publish:
name: Publish
needs: [prepare, self-interop]
needs: [prepare, self-interop, package-libs]
runs-on: ubuntu-latest
if: ${{ !inputs.dry_run }}
permissions:
Expand All @@ -217,6 +340,13 @@ jobs:
with:
fetch-depth: 0

- name: Download prebuilt library bundles
uses: actions/download-artifact@v4
with:
pattern: libbundle-*
path: dist
merge-multiple: true

- name: Bump version in build.zig.zon
env:
FULL_VERSION: ${{ needs.prepare.outputs.full_version }}
Expand Down Expand Up @@ -275,7 +405,15 @@ jobs:
## Using zzdds as a Zig package
\`\`\`
zig fetch --save https://github.com/${{ github.repository }}/archive/refs/tags/${TAG}.tar.gz
\`\`\`"
\`\`\`

## Prebuilt C/C++ library bundles
\`zzdds-<version>-<platform>.tar.gz\` below each unpack to a
relocatable install prefix (\`include/\`, \`lib/\` with pkg-config +
CMake package files). Point \`CMAKE_PREFIX_PATH\` at the unpacked
directory, or \`PKG_CONFIG_PATH\` at its \`lib/pkgconfig\`
subdirectory." \
dist/*.tar.gz

- name: Bump to post-release dev version
env:
Expand Down
11 changes: 8 additions & 3 deletions build.zig
Original file line number Diff line number Diff line change
Expand Up @@ -717,9 +717,14 @@ pub fn build(b: *std.Build) void {
}

// Generate and install lib/pkgconfig/zzdds.pc.
// The prefix is baked in at install time (matches --prefix, defaulting to zig-out/).
// prefix is derived at pkg-config time from ${pcfiledir} (the directory
// holding this .pc file, i.e. <prefix>/lib/pkgconfig), so the install
// tree stays relocatable — it can be renamed, moved, or extracted from
// a release bundle without the header/library paths going stale. This
// matches the CMake package config below, which does the same via
// get_filename_component.
const pc_content = b.fmt(
\\prefix={s}
\\prefix=${{pcfiledir}}/../..
\\libdir=${{prefix}}/lib
\\includedir=${{prefix}}/include
\\
Expand All @@ -729,7 +734,7 @@ pub fn build(b: *std.Build) void {
\\Libs: -L${{libdir}} -lzzdds -lzidl_cdr
\\Cflags: -I${{includedir}}
\\
, .{ b.install_prefix, zzdds_version });
, .{zzdds_version});
const pc_wf = b.addWriteFiles();
const pc_lp = pc_wf.add("zzdds.pc", pc_content);
const install_pc = b.addInstallFileWithDir(pc_lp, .{ .custom = "lib/pkgconfig" }, "zzdds.pc");
Expand Down
21 changes: 21 additions & 0 deletions docs/design/ci-platform-coverage-expansion.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,9 @@ platform-conditional).
plausible new finding is a genuinely platform-specific allocator interaction, which is the
point.

**Outcome (PR #65, 2026-08-18): landed on all three platforms** — the
`DebugAllocator lane` step is in `ci.yml`'s `test-other` job, no platform-specific findings.

## Item 2 — C/C++/Java bindings on the 3-platform matrix

**Change:** extend `test-other` to also build+run the binding smoke tests, mirroring
Expand Down Expand Up @@ -212,6 +215,24 @@ wiring it into CI, specifically to find any such test and either fix it (make th
`Debug`/`ReleaseSafe`-only, or rewrite it to not depend on safety-check panics) rather than
discover it as a confusing CI failure.

**Outcome — `ReleaseFast` landed (PR #65, 2026-08-18):** the `release-fast` step is in
`run_deterministic_matrix.py` (covering Linux x86_64 via `test-linux`) and a
`Test (ReleaseFast)` step is in `release.yml`'s `test` job on all four platforms. No
safety-panic-dependent tests were found; the suite passes clean.

**Outcome — `ReleaseSmall` NOT landed; root-caused to an upstream Zig bug (2026-08-29):**
`zig build test -Doptimize=ReleaseSmall` produces 37 `panic: incorrect alignment` crashes
(`bootstrap_test` / `typesupport_test`, via `zidl-rt`'s `entity_box.zig` `unboxAsView`
`@alignCast(box.vtable)`). Traced to **Zig 0.16.0's self-hosted x86_64 backend emitting
read-only global constants with no alignment, only at `-OReleaseSmall`**: `&SomeImpl.views`
(an `extern struct` with `@alignOf` 8) and every `*_vtable` global land at odd addresses,
byte-packed in `.rodata`; `unboxAsView`'s `@alignCast` correctly traps it. Not a zzdds/zidl
defect — reproduces in ~10 lines with a bare `const val: u32` (`-OReleaseSmall -fno-llvm
-fno-lld` → 1-mod-4; `-OReleaseFast`, the LLVM backend, and Debug/ReleaseSafe all fine).
Full trail + minimal repro: `zz-dev/releasesmall-misaligned-rodata-investigation.md`.
Follow-ups: (1) file against `ziglang/zig`; (2) a ReleaseSmall lane, if wanted before the
fix lands, must force `.use_llvm = true` (as `emit-tests-llvm` already does for Valgrind).

## Cross-cutting implementation notes

- **CI runtime cost.** `test-other`'s current 20-minute timeout (ci.yml:98, sized off a past
Expand Down
86 changes: 54 additions & 32 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,8 +177,7 @@ Forward-looking only: known gaps, planned features, and open design questions.
participant-config); the `wait_for_historical_data`-should-time-out negative case
(catchup); `assert_liveliness()` + AUTOMATIC/MANUAL_BY_PARTICIPANT + `on_liveliness_lost`
(presence); `*_w_timestamp` symmetry + batch instance ops (registry); Java's
`instance_state` on the batch-take family; the `c`/`cpp`/`java` waitset ports don't exist
yet.
`instance_state` on the batch-take family.
- **Non-goal (recorded, not planned):** a spec-conformance harness, network simulation
(ns-3 / CORE), and formal verification / safety certification (DO-178C, IEC 61508, ISO
26262) — long-term concerns, not built. `design/testing-strategy.md`.
Expand Down Expand Up @@ -315,38 +314,61 @@ and shape are open — see `zidl/docs/roadmap.md` "Plugin architecture".
## CI / Release Platform Coverage

Audit of `build.zig` options, `scripts/run_deterministic_matrix.py`, `ci.yml`, and
`release.yml` against the platform/build-type matrix they exercise (re-reviewed 2026-08-16).
Ranked, lowest-effort first:

1. **Extend the DebugAllocator lane** to `test-other`'s Linux ARM64 / macOS / Windows matrix
(a one-line copy of `test-linux`'s `-Ddebug-allocator=true` step). Lowest effort;
suggested first move.
2. **Java/JNI binding smoke test on Windows** — deferred, not achieved. `java.exe` exits
code 9 with no crash file at the first JNI call; leading hypothesis is a Control Flow
Guard mismatch between `jvm.dll` and the zig-cc-built zzdds DLLs. Needs WinDbg on real
Windows hardware. Investigation trail: `zz-dev/windows-jni-crash-investigation.md`.
3. **TSan lane on macOS ARM64** — deferred. Even `test-tsan-self-check` segfaults before app
code; likely an upstream Zig/LLVM `libtsan` gap (`pthread_introspection_hook_install`
private-API drift). Revisit when Zig bundles a newer LLVM. Trail:
`zz-dev/macos-tsan-crash-investigation.md`. (TSan on Windows: Clang/LLVM has no supported
target. Extending `examples-tsan` to macOS is a separate follow-up.)
4. **`ReleaseFast` is never built or run** anywhere (CI or `release.yml`); `ReleaseSmall`
likewise. Run locally first to find tests that depend on a safety-check panic.
5. **Real vendor/self RTPS interop** (Connext / Cyclone / CoreDX / self) runs only on Linux
`release.yml` against the platform/build-type matrix they exercise. Original ranking
2026-08-16; progress notes below from PR #65 (2026-08-18) and the 2026-08-28 CI pass.

### Landed

- **DebugAllocator lane on `test-other`** (PR #65) — `zig build test -Ddebug-allocator=true`
now runs on Linux ARM64, macOS ARM64, and Windows x86_64, additive to `test-linux`'s
existing step.
- **`ReleaseFast` built and tested** (PR #65) — `run_deterministic_matrix.py` gained a
`release-fast` step (so `test-linux` covers it on Linux x86_64) and `release.yml`'s `test`
job runs `zig build test -Doptimize=ReleaseFast` on all four platforms.
- **C/C++ binding smoke tests everywhere** (PR #65 for `ci.yml`; 2026-08-28 for `release.yml`)
— `zig build test-bindings -Dc-binding -Dcpp-binding` runs on all `test-other` /
`release.yml` `test` platforms (Java added on Linux ARM64 + macOS; Java-on-Windows
deferred, see below).
- **Prebuilt library bundles** (2026-08-28) — `release.yml`'s new `package-libs` job builds
the C/C++ install tree (dynamic `libzzdds` + static `libzidl_cdr` + headers + pkgconfig +
CMake package files) on each of the four release platforms, verifies completeness, and
uploads a per-platform tarball that `publish` attaches to the GitHub release. Functional
coverage of the bundled libraries is the `test` job's `test-bindings` step.

### Deferred (investigation trails exist)

- **Java/JNI binding smoke test on Windows** — `java.exe` exits code 9 with no crash file at
the first JNI call; leading hypothesis is a Control Flow Guard mismatch between `jvm.dll`
and the zig-cc-built zzdds DLLs. Needs WinDbg on real Windows hardware. Trail:
`zz-dev/windows-jni-crash-investigation.md`.
- **TSan lane on macOS ARM64** — even `test-tsan-self-check` segfaults before app code;
likely an upstream Zig/LLVM `libtsan` gap (`pthread_introspection_hook_install` private-API
drift). Revisit when Zig bundles a newer LLVM. Trail:
`zz-dev/macos-tsan-crash-investigation.md`. (TSan on Windows: Clang/LLVM has no supported
target. Extending `examples-tsan` to macOS is a separate follow-up.)
- **`ReleaseSmall` gate — blocked on an upstream Zig codegen bug (root-caused 2026-08-29).**
`zig build test -Doptimize=ReleaseSmall` produces 37 `panic: incorrect alignment` crashes
(in `bootstrap_test` / `typesupport_test`, via `zidl-rt`'s `entity_box.zig` `unboxAsView`
`@alignCast(box.vtable)`). Root cause: **Zig 0.16.0's self-hosted x86_64 backend, only at
`-OReleaseSmall`, emits read-only global constants with no alignment** — `&SomeImpl.views`
(an `extern struct` `CAbiViews`, `@alignOf` 8) and every `*_vtable` global land at odd
addresses, packed byte-to-byte in `.rodata`. `unboxAsView`'s `@alignCast` correctly traps
it. Not a zzdds or zidl defect. Minimal repro (deterministic, ~10 lines): a bare
`const val: u32 = …` preceded by a 1-byte `const` lands 1-mod-4 under
`-OReleaseSmall -fno-llvm -fno-lld`; fine under `-OReleaseFast`, fine with the LLVM
backend, fine under Debug/ReleaseSafe. Full trail + repro in
`zz-dev/releasesmall-misaligned-rodata-investigation.md`. Next: file against `ziglang/zig`;
a ReleaseSmall CI lane would need `.use_llvm = true` (like `emit-tests-llvm`) until fixed.

### Still open, ranked

1. **Real vendor/self RTPS interop** (Connext / Cyclone / CoreDX / self) runs only on Linux
x86_64 — no wire / discovery / CDR coverage on Windows, macOS, or ARM64.
6. **No Intel macOS coverage** — `macos-latest` is Apple Silicon only.
7. **`release.yml` never builds a binding** — the release gate only requires a Linux x86_64
ReleaseSafe self-interop pass.
8. **No musl / static Linux target** — always glibc, always the native triple; `-Dtarget`
2. **No Intel macOS coverage** — `macos-latest` is Apple Silicon only.
3. **No musl / static Linux target** — always glibc, always the native triple; `-Dtarget`
is never actually cross-compiled.
9. **No prebuilt release binaries** — `release.yml` publishes only a tag, changelog, and
`zig fetch` URL; no compiled `libzzdds.{so,dll,dylib}` is built, uploaded, or
smoke-tested.
10. **Valgrind has no viable non-Linux equivalent** — treat as Linux-only unless a specific
non-Linux memory bug motivates revisiting.

The `test-other` job's 20-minute CI timeout needs re-budgeting once it also runs
DebugAllocator + bindings (+ JDK) + conditional TSan.
4. **Valgrind has no viable non-Linux equivalent** — treat as Linux-only unless a specific
non-Linux memory bug motivates revisiting.

---

Expand Down
5 changes: 3 additions & 2 deletions examples/docs/design/waitset-reference-app.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,5 +98,6 @@ interface struct itself, unlike every other generated operation. Both
Worth fixing in zidl at some point — tracked as a small follow-up, not
blocking here.

See each language's own README for build and run instructions once that
port exists; today only `zig/waitset` exists.
See each language's own README for build and run instructions. All four
ports (`zig`, `c`, `cpp`, `java`) exist; `zig`, `cpp`, and `c` are
additionally built and run under ThreadSanitizer in CI (`examples-tsan`).
Loading