Skip to content

Speed up Windows and macOS builds#3466

Open
hluk wants to merge 14 commits intomasterfrom
appveyor-build-speed
Open

Speed up Windows and macOS builds#3466
hluk wants to merge 14 commits intomasterfrom
appveyor-build-speed

Conversation

@hluk
Copy link
Owner

@hluk hluk commented Mar 7, 2026

Assisted-by: Claude (Anthropic)

hluk added 2 commits March 7, 2026 11:19
Enable MSVC multi-process compilation (-MP) and cmake --parallel to
significantly reduce Appveyor build times.

Changes:
- Add -DCMAKE_CXX_FLAGS="-MP" to enable parallel compilation within
  each MSBuild target (~80 files in copyq-common benefit most)
- Add --parallel to cmake --build for project-level parallelism
- Add --parallel to dependency builds in kf_build.sh
- Parallelize independent dependency builds in before_build.sh using
  background processes grouped by dependency order:
  Group A (concurrent): qca, qtkeychain, snoretoast, extra-cmake-modules
  Group B (after ECM): kconfig, kwindowsystem
  Group C (after B): knotifications, kstatusnotifieritem

Expected build_script phase reduction: ~19.5 min to ~6-8 min.

Assisted-by: Claude (Anthropic)
Cache Homebrew bottle downloads and KDE framework Cellar entries to
avoid redundant network I/O and source compilation on every CI run.

Changes to .github/workflows/build-macos.yml:
- Add Homebrew downloads cache step with restore-keys for broad reuse
- Add KDE frameworks cache step (Cellar entries), keyed on formula hashes
- Increase ccache max-size from 500M to 1G for headroom

Changes to utils/github/install-macos.sh:
- Remove full Homebrew reinstall from scratch (runner already has it)
- Remove pointless cmake uninstall (same version gets reinstalled)
- On KDE cache hit: re-link packages instead of rebuilding from source
- On cache miss: build from source as before

Expected install step reduction: ~9 min to ~2 min on cache hit.

Assisted-by: Claude (Anthropic)
@hluk hluk force-pushed the appveyor-build-speed branch from 6333137 to 7f89550 Compare March 7, 2026 11:10
@codecov
Copy link

codecov bot commented Mar 7, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.34%. Comparing base (d01f0c6) to head (5cf3dd8).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3466      +/-   ##
==========================================
+ Coverage   80.28%   80.34%   +0.05%     
==========================================
  Files         243      243              
  Lines       30614    30614              
==========================================
+ Hits        24578    24596      +18     
+ Misses       6036     6018      -18     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@hluk hluk force-pushed the appveyor-build-speed branch 4 times, most recently from c898b2e to fd1a65f Compare March 7, 2026 16:30
hluk added 5 commits March 7, 2026 17:07
Add PCH for the copyq-common OBJECT library with the most commonly
included Qt headers (QObject, QString, QWidget, QApplication, etc.).

MSVC re-parses these heavy headers for every translation unit. PCH
eliminates this redundant parsing, typically reducing MSVC compilation
time by 30-50%. GCC/Clang also benefit but to a lesser degree.

Assisted-by: Claude (Anthropic)
Skip 88 tests that exercise only platform-independent logic (scripting
engine, data operations, config parsing, class tests, CLI basics,
import/export) which are already fully covered by the Linux CI.

The remaining 145 core tests cover Windows-specific behavior: clipboard,
window focus, tray icon, shortcuts, dialogs, notifications, navigation,
drag-and-drop, and encryption. Plugin tests run separately to cover
plugin-specific Windows behavior.

This reduces Windows test runtime from ~13 min to ~5 min.

Assisted-by: Claude (Anthropic)
Move the SKIP_TESTS array from utils/appveyor/test.sh to a shared file
utils/windows-test-skip-list.sh so both Appveyor and GitHub Actions test
scripts can source it without duplication.

Assisted-by: Claude (Anthropic)
Replace Appveyor's Visual Studio generator (2-core VM, ~31 min) with
GitHub Actions using Ninja + MSVC (4-core runner). Expected improvement:
build ~4-6 min (down from ~16 min) + tests ~5 min (down from ~13 min).

Key changes:
- CMakePresets.json: Add Windows preset (Ninja, sccache, Release)
- build-windows.yml: Full CI pipeline with dependency caching
- build-windows-deps.sh: KDE framework builder using Ninja
- test-windows.sh: Test runner sourcing shared skip list

Dependencies:
- Qt 6.8.3 via jurplel/install-qt-action (cached)
- OpenSSL 3.x via Chocolatey
- MSVC via ilammy/msvc-dev-cmd
- sccache for compilation caching
- KDE frameworks built from source (cached by version)

Note: Packaging (portable zip, InnoSetup installer, screenshots) not yet
included. This commit establishes the build+test pipeline. Packaging can
be added once the core pipeline is verified.

Assisted-by: Claude (Anthropic)
Drop appveyor.yml and all utils/appveyor/ scripts. Windows CI is now
handled by .github/workflows/build-windows.yml.

Move KNotifications patch from utils/appveyor/patches/ to
utils/patches/ (referenced by the GitHub Actions dep builder).

Update docs to remove Appveyor references.

Assisted-by: Claude (Anthropic)
@hluk hluk force-pushed the appveyor-build-speed branch from fd1a65f to 662af32 Compare March 7, 2026 17:38
hluk added 6 commits March 7, 2026 17:49
- Add qt5compat module to install-qt-action (needed by KConfig)
- Build CMAKE_PREFIX_PATH from QT_ROOT_DIR explicitly since
  install-qt-action may not set CMAKE_PREFIX_PATH
- Set VCPKG_ROOT='' to prevent lukka/run-cmake from trying vcpkg
  environment setup (no vcpkg.json in this project)

Assisted-by: Claude (Anthropic)
Chocolatey installs OpenSSL to 'C:\Program Files\OpenSSL', not
'C:\Program Files\OpenSSL-Win64'. Auto-detect the actual path so
QCA can find OpenSSL and build the ossl plugin.

Assisted-by: Claude (Anthropic)
The precompiled headers include Qt headers (QWidget, QApplication) which
on MSVC pull in <windows.h>. The min/max macros from windows.h conflict
with std::max, std::min, and std::numeric_limits<T>::min()/max().

Add NOMINMAX definition for WIN32 builds to prevent these macros from
being defined.

Assisted-by: Claude (Anthropic)
- Add NOMINMAX definition for Windows builds to prevent min/max macro
  conflicts from windows.h (pulled in by precompiled Qt headers).
- Fix test step: use $GITHUB_WORKSPACE env var instead of
  ${{ github.workspace }} context expression, which produces
  backslash-escaped Windows paths that Git Bash misinterprets.

Assisted-by: Claude (Anthropic)
Track exit codes for core and plugin tests separately. Remove set -e
so both test stages run even if the first fails, and report which stage
failed. This makes CI failures easier to diagnose.

Assisted-by: Claude (Anthropic)
Precompiled headers break Clang builds (-Weverything on Qt system
headers) and macOS builds (ccache+PCH incompatibility). Guard PCH
with if(MSVC) since it was only intended for MSVC build speed.

Remove the Windows test skip list and simplify test-windows.sh to
run all tests without filtering.

Assisted-by: Claude (Anthropic)
@hluk hluk force-pushed the appveyor-build-speed branch from cc2b9e5 to 5cf3dd8 Compare March 8, 2026 04:40
Git Bash (MSYS2) on Windows GHA runners does not reliably capture
stdout from native Windows console applications. Switch the test
step from bash to pwsh (PowerShell) which natively runs Windows
executables and captures their output correctly.

Remove the now-unused bash test script (utils/github/test-windows.sh)
and inline the test invocation in the workflow.

Assisted-by: Claude (Anthropic)
@hluk hluk force-pushed the appveyor-build-speed branch from 5cf3dd8 to 81615d2 Compare March 8, 2026 05:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant