Repository navigation
Temporarily disable ts-integration tests (revisit after modernization) - #22
Merged
Merged
Conversation
ts-integration-test, ts-end-to-end-test and ts-multiple-messages-test are disabled with a note: their install step builds the entire streamr-dev/network TS monorepo at a 2024 commit, which no longer works on current CI runners — and compiling the whole repository is not necessary anymore. They will be revisited against a current, slimmer TS setup after the toolchain/modules modernization completes. These were the only failing tests on the Linux CI legs, so validate.yml Linux legs are expected to go fully green with this change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collaborator
Author
|
CI outcome: the TS-test disable worked — linux-arm64 is fully green (install ✓ lint ✓ test ✓, first fully green leg in the repo's recent history). The ubuntu-24.04 leg failed on a single different test: Suggest: re-run the failed job from the Checks tab (I can't trigger reruns with my access) — or merge as-is if you're comfortable, since the flake predates this PR's change. Happy to dig into the test's timing assumptions as a separate task if it keeps recurring. 🤖 Generated with Claude Code |
ptesavol
added a commit
that referenced
this pull request
Jul 2, 2026
The networking integration tests (ConnectionLockingTest.*) are timing-sensitive on shared CI runners and have flaked on both the old toolchain (LockingBothWays, ubuntu, PR #22) and the new one (CanLockConnections, macos, PR #23) while passing on adjacent runs of the same configuration. A retry keeps the signal — a genuinely broken test fails both attempts — without burning CI round-trips on scheduler jitter. Proper timing-assumption fixes for these tests remain a separate workstream. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ptesavol
added a commit
that referenced
this pull request
Jul 2, 2026
…iosbuild] (#23) * Temporarily disable ts-integration tests ts-integration-test, ts-end-to-end-test and ts-multiple-messages-test are disabled with a note: their install step builds the entire streamr-dev/network TS monorepo at a 2024 commit, which no longer works on current CI runners — and compiling the whole repository is not necessary anymore. They will be revisited against a current, slimmer TS setup after the toolchain/modules modernization completes. These were the only failing tests on the Linux CI legs, so validate.yml Linux legs are expected to go fully green with this change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Phase 1.2: upgrade compilers to LLVM 22 on all platforms - macOS: Homebrew llvm@17 (dead against current Xcode SDKs — its baked-in DEFAULT_SYSROOT points at the removed MacOSX14.sdk) is replaced by the latest keg-only llvm formula (22.x). The build now locates the toolchain through the LLVM_PREFIX environment variable exported by install-prerequisities.sh, with a ${HOMEBREW_PREFIX}/opt/llvm fallback; the previously hardcoded /opt/homebrew/{include/c++/v1,lib/c++} paths in the toolchain file and the osx/ios overlay triplets are parameterized on LLVM_PREFIX. - Linux: gcc-14 is replaced by clang-22 + libc++ from apt.llvm.org (llvm-toolchain-noble-22). libc++ keeps the standard library uniform across macOS/iOS/Android/Linux — one C++26 feature matrix and a single C++ modules implementation. The x64-linux overlay triplet builds vcpkg deps with the same compiler/stdlib, and a new arm64-linux overlay gives the self-hosted arm64 runner the same treatment (it previously fell back to vcpkg defaults). - clang-format/clangd remain at 18 (lint toolchain is upgraded in a later phase, to keep lint churn out of this one). - CI modernization: validate.yml matrix drops macos-13/macos-14 in favor of macos-latest, ubuntu-24.04 becomes ubuntu-latest; validateios.yml/validateandroid.yml runners become macos-latest. Cache keys are salted (v1->v2) because caches built with the old compilers must not be reused. - arm64-ios.cmake also normalized from CRLF to LF line endings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add fmt overlay port (11.2.0 + cstdlib patch) for clang 22 The baseline-pinned fmt 11.0.2 no longer compiles under clang 22 with -std=gnu++26 (consteval/constant-expression failure in FMT_STRING). fmt 12.x is out because folly 2024.08 still relies on the fmt/core.h -> fmt/format.h shim that fmt 12 removed. fmt 11.2.0 works, with one small patch: it defines _LIBCPP_REMOVE_TRANSITIVE_INCLUDES but misses <cstdlib>, which libc++ 22 no longer provides transitively (fixed upstream in fmt 12). TEMPORARY: this overlay disappears with the Phase 1.3 vcpkg baseline bump, which brings current fmt and folly together. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Lint toolchain to LLVM 22 (clangd + clang-format), suppress new checks clangd 18 cannot parse libc++ 22 headers (unknown builtins like __builtin_clzg), so clangd moves to 22 with the compilers: on macOS via the keg's bin dir prepended to PATH, on Linux via the clangd-22 package. clang-format moves to 22 on both platforms as well — macOS lost its old force-linked copy with the keg-only switch, and format versions must not diverge between platforms or the check flip-flops. The resulting mechanical reformat (clang-format 22 line-breaking changes) touches 20 files. clang-tidy checks added/extended after version 18 fire on existing code; they are suppressed in .clang-tidy (root + the two test configs that shadow it) with a note, to be triaged in the lint-modernization phase. One targeted NOLINTNEXTLINE for a namespace alias that clangd 22 newly checks against naming rules. Verified locally: full build, 307/307 tests, full lint green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * iOS: preset folly try_run results in arm64-ios triplet (iosbuild) folly's configure runs try_run() checks that cannot execute when cross-compiling. The arm64-android triplet has always preset the *_EXITCODE results for them; the arm64-ios triplet never did, so folly's iOS configure errored out. Preset them with the values the same checks produce when they actually run on an arm64 Apple host (observed in the arm64-osx build): unaligned access ok, wchar ok, vsnprintf-errors/weak-symbols/linux-vdso not available. Verified locally: full arm64-ios dependency set builds with LLVM 22 (folly compiles against libc++ 22 headers with the existing availability workaround), all packages cross-compile, and the XCFramework is produced with platform=iOS, minos=13.0 and no references to runtime symbols newer than the deployment target. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * CI: allow triggering iOS/Android workflows from PR titles The keyword gates checked github.event.head_commit.message, which is only populated on push events — and these workflows' push trigger is main-only, so the keywords could never fire on a pull request. The gates now also match the PR title, making pre-merge iOS/Android validation possible. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * CI: retrigger PR checks after title change (iosbuild) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * CI: fix iOS/Android workflow submodule handling (iosbuild) The iOS workflow failed in 18s: its checkout did not fetch submodules (unlike validate.yml), leaving install-prerequisities.sh to clone them raw — and the wrappers/go submodule used an SSH URL, which fails on runners without SSH keys. goproxyclient is public, so .gitmodules now uses the https URL (run `git submodule sync` in existing checkouts), and the iOS/Android workflows check out submodules with the token like validate.yml does. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * CI: surface vcpkg buildtree logs as annotations on install failure (iosbuild) Replaces the hardcoded libdatachannel-only log dump with a generic handler: on install failure the most recently written vcpkg buildtree logs are printed (grouped) and their tails emitted as ::error:: annotations, so failures are diagnosable from the Checks UI without authenticated access to raw logs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Ports: set CMAKE_POLICY_VERSION_MINIMUM=3.5 on Apple triplets (iosbuild) CMake 4 (now on the macOS runners and current Homebrew) removed compatibility with cmake_minimum_required(<3.5); several ports at the pinned 2024 vcpkg baseline (first casualty: libevent) still declare such minimums and fail to configure. CMAKE_POLICY_VERSION_MINIMUM=3.5 is the documented escape hatch. Linux stays on CMake 3.28 (stock ubuntu-24.04/noble), which still accepts the old minimums. TEMPORARY: remove with the Phase 1.3 vcpkg baseline bump, which brings ports that declare modern CMake minimums. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Ports: set VCPKG_OSX_SYSROOT=macosx in osx triplets (iosbuild) CMake 4 no longer defaults CMAKE_OSX_SYSROOT to the SDK path, and the pinned 2024 vcpkg scripts compose "-isysroot ${CMAKE_OSX_SYSROOT}" unconditionally — with an empty value the flag swallowed the next argument and C system headers vanished (first casualty: openssl, 'assert.h' file not found). Naming the SDK explicitly restores the old behavior. The iOS triplet is unaffected: ios.toolchain.cmake sets its own iphoneos sysroot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * CI: annotate logs that contain error text, not just newest (iosbuild) The newest buildtree logs at failure time are often config/cache dumps without the actual compiler error; prefer recent logs that grep for error text and annotate the error context. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * CI: robust install failure annotations (iosbuild) The previous handler died under bash -e (ls on an empty glob exits 2) before emitting anything when vcpkg buildtrees were empty — which also revealed the failure can occur outside port builds. Capture install.sh output with tee and always annotate its tail (covers vcpkg, package and root build failures alike); buildtree log annotations remain for port failures. set +e inside the handler. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * CI: disarm errexit/pipefail around install pipeline (iosbuild) Composite run steps execute under bash -e -o pipefail, which aborted the step at the failing install pipeline before the annotation handler could run. Handler now verified locally under the same shell flags. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Linux ports: build at C++26 like every other platform (iosbuild) The Linux triplets never passed CMAKE_CXX_STANDARD=26 to ports (all other platforms' triplets do), so folly built at its default C++17. Under clang, C++17 means no coroutines: folly's coro translation units compile to empty objects (FOLLY_HAS_COROUTINES off), and the SDK's C++26 build then fails to link with undefined folly::coro symbols (observed on both Linux CI legs). The old gcc toolchain masked this because folly adds -fcoroutines for GNU compilers only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * libdatachannel: fix parallel-configure race in overlay patch (iosbuild) The overlay's fix-cmakelists.patch replaced upstream's configure_file with a REMOVE/COPY/RENAME dance that mutates the shared source tree. vcpkg configures debug and release in parallel, so the two configures raced on include/rtc/version.h(.in) — reliably breaking the x64-linux and arm64-linux builds ("file COPY cannot set modification time", "file RENAME failed"), while macOS kept winning the race by luck. Replaced with configure_file(... COPYONLY): same no-substitution behavior, idempotent, nothing consumed. Patch verified to apply cleanly to pristine v0.21.2 together with the other three patches. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add MODERNIZATION.md: the modernization plan referenced by the PRs The working plan for the toolchain update + C++ modules migration, kept current as phases land. Status as of this commit: Phases 1.0/1.1 and the ts-test interim merged (PRs #20-#22), Phase 1.2 in review (PR #23). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test.sh: retry failed tests once (ctest --repeat until-pass:2) The networking integration tests (ConnectionLockingTest.*) are timing-sensitive on shared CI runners and have flaked on both the old toolchain (LockingBothWays, ubuntu, PR #22) and the new one (CanLockConnections, macos, PR #23) while passing on adjacent runs of the same configuration. A retry keeps the signal — a genuinely broken test fails both attempts — without burning CI round-trips on scheduler jitter. Proper timing-assumption fixes for these tests remain a separate workstream. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * CI: include overlay triplets/ports in vcpkg cache keys (iosbuild) vcpkg's port ABI hashes depend on the overlay triplet and port files, not just vcpkg.json. Keying the caches on vcpkg.json alone meant any triplet/port change collided with a stale saved cache (save skips existing keys), forcing full dependency rebuilds on every retry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * iostest.sh: add --device option and -allowProvisioningUpdates ./iostest.sh runs on the Mac as before; ./iostest.sh --device [name] runs the same suite on a connected iPhone — the real check that the compiler output is compatible with the device's fixed libc++ runtime. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test.sh: add per-test timeout so hung tests fail and retry (iosbuild) ConnectionLockingTest hung indefinitely on a macOS runner (test step 30+ min for a ~35 s suite); --repeat until-pass only fires on failure, so a hang ate the job. --timeout 300 turns a hang into a failure, which then gets the single retry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * MODERNIZATION.md: record macOS CI networking-test flakiness decision Merged-as-is per review decision; the socket-based integration tests' timing/port assumptions are their own workstream. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 3, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Disables the three failing TS-interop tests (
ts-integration-test,ts-end-to-end-test,ts-multiple-messages-test) with an explanatory note in both CMakeLists, per discussion.Why: their install step (
ts-integration/install.sh) clones and builds the entire streamr-dev/network TS monorepo at a 2024 commit, which no longer works on current CI runners — and compiling the whole repository is not necessary anymore given how much the network monorepo has changed. They'll be revisited against a current, slimmer TS setup after the modernization completes.Ruled out during diagnosis: these tests don't depend on any external server at runtime — all three run against a locally spawned TS subscriber with
--local(127.0.0.1:44211on both the TS and C++ sides). The hardcoded global entry point (95.216.15.80:44211) insubscriber.tsis only used without--local. The clone target repo and its pinned commit also still exist.Expected CI effect: these were the only failing tests on the Linux legs, so
validate.yml's ubuntu-24.04 and linux-arm64 legs should now go fully green — giving Phase 1.2 a clean gate. (macOS legs stay red until Phase 1.2's compiler upgrade, unchanged.)🤖 Generated with Claude Code