Skip to content

Replace sccache with ccache across CI workflows and build system - #3418

Draft
Lestropie wants to merge 1 commit into
devfrom
sccache_to_ccache
Draft

Replace sccache with ccache across CI workflows and build system#3418
Lestropie wants to merge 1 commit into
devfrom
sccache_to_ccache

Conversation

@Lestropie

Copy link
Copy Markdown
Member

Hoping to catch an opinion from @daljit46.

I've had multiple PRs where I've had to wipe the whole GitHub Action cache because of compilation failures where the build cache and the precompiled headers get tangled:

Example CI build failure from #3414
FAILED: [code=1] cpp/gui/CMakeFiles/mrtrix-gui.dir/mrview/tool/screen_capture/screen_capture.cpp.o 
/opt/hostedtoolcache/sccache/0.16.0/x64/sccache /usr/bin/c++ -DMRTRIX_BASE_VERSION=\"3.0.8\" -DMRTRIX_BUILD_TYPE=\"Release\" -DMRTRIX_HAVE_LGAMMA_R -DMRTRIX_HAVE_STRERROR_R -DMRTRIX_LINUX -DMRTRIX_PNG_SUPPORT -DQT_CORE_LIB -DQT_GUI_LIB -DQT_NETWORK_LIB -DQT_NO_DEBUG -DQT_OPENGLWIDGETS_LIB -DQT_OPENGL_LIB -DQT_WIDGETS_LIB -DSLANG_DYNAMIC -D_GLIBCXX_DEBUG -D_GLIBCXX_DEBUG_PEDANTIC -I/home/runner/work/mrtrix3/mrtrix3/build/cpp/gui/mrtrix-gui_autogen/include -I/home/runner/work/mrtrix3/mrtrix3/cpp/gui -I/home/runner/work/mrtrix3/mrtrix3/cpp/core -I/home/runner/work/mrtrix3/mrtrix3/build/_deps/json-src/include -I/home/runner/work/mrtrix3/mrtrix3/build/_deps/nifti1-src -I/home/runner/work/mrtrix3/mrtrix3/build/_deps/nifti2-src -I/home/runner/work/mrtrix3/mrtrix3/build/_deps/tcb_span-src/include -I/home/runner/work/mrtrix3/mrtrix3/build/_deps/magic_enum-src/include -isystem /home/runner/work/mrtrix3/mrtrix3/build/_deps/eigen3-src -isystem /home/runner/work/mrtrix3/mrtrix3/build/_deps/dawn-src/include -isystem /home/runner/work/mrtrix3/mrtrix3/build/_deps/slang-src/include -isystem /usr/include/x86_64-linux-gnu/qt6/QtCore -isystem /usr/include/x86_64-linux-gnu/qt6 -isystem /usr/lib/x86_64-linux-gnu/qt6/mkspecs/linux-g++ -isystem /usr/include/x86_64-linux-gnu/qt6/QtGui -isystem /usr/include/x86_64-linux-gnu/qt6/QtWidgets -isystem /usr/include/x86_64-linux-gnu/qt6/QtOpenGL -isystem /usr/include/x86_64-linux-gnu/qt6/QtNetwork -isystem /usr/include/x86_64-linux-gnu/qt6/QtOpenGLWidgets -O3 -DNDEBUG -fPIC -Werror -Wno-ignored-attributes -fPIC -std=gnu++17 -Winvalid-pch -include /home/runner/work/mrtrix3/mrtrix3/build/cpp/gui/CMakeFiles/mrtrix-gui.dir/cmake_pch.hxx -MD -MT cpp/gui/CMakeFiles/mrtrix-gui.dir/mrview/tool/screen_capture/screen_capture.cpp.o -MF cpp/gui/CMakeFiles/mrtrix-gui.dir/mrview/tool/screen_capture/screen_capture.cpp.o.d -o cpp/gui/CMakeFiles/mrtrix-gui.dir/mrview/tool/screen_capture/screen_capture.cpp.o -c /home/runner/work/mrtrix3/mrtrix3/cpp/gui/mrview/tool/screen_capture/screen_capture.cpp
In file included from /home/runner/work/mrtrix3/mrtrix3/cpp/gui/dialog/file.h:20,
                 from /home/runner/work/mrtrix3/mrtrix3/cpp/gui/mrview/tool/screen_capture/screen_capture.cpp:24:
/home/runner/work/mrtrix3/mrtrix3/cpp/gui/opengl/glutils.h:82:13: error: redefinition of ‘void MR::GUI::GL::check_error(const char*, int)’
   82 | inline void check_error(const char *filename, int line) { // check_syntax off (input is __FILE__)
      |             ^~~~~~~~~~~
In file included from /home/runner/work/mrtrix3/mrtrix3/cpp/gui/gui_pch.h:19,
                 from /home/runner/work/mrtrix3/mrtrix3/build/cpp/gui/CMakeFiles/mrtrix-gui.dir/cmake_pch.hxx:5,
                 from <command-line>:
/home/runner/work/mrtrix3/mrtrix3/cpp/gui/opengl/glutils.h:82:13: note: ‘void MR::GUI::GL::check_error(const char*, int)’ previously defined here
   82 | inline void check_error(const char *filename, int line) { // check_syntax off (input is __FILE__)
      |             ^~~~~~~~~~~

Looking online, it seems that sccache does not officially support PCH, and they are known to not play together particularly well. Indeed a55d6fd in #2877 disabled PCH on the Mac compilation check because this issue had already manifested in that environment.

A primary selling point of sccache is cloud-based synchronisation of build artifacts across machines. However if ccache is embedded in a dedicated GitHub Action, and we are only interested in sharing of build artifacts across CI runs, then that advantage seems to mostly disappear. sccache supports more than C++ but we don't need that. Finally, sccache stores per-compiled-object artifacts, whereas with ccache one is restricted to the whole tarball; it is not yet clear to me the extent to which this may be problematic, potentially to a reasonable extent when there's a large number of PRs open. Could pay for a little more Action artifact storage if it would help.

If the loss of per-compiled-object caching on GitHub is not a deal-breaker, then it would seem overall that ccache is a better choice than sccache. But curious to know if this was already factored in to the original decision process, whether I'm overlooking something, or getting the proportional weighting of priorities wrong.

Would an alternative option be to rely exclusively on sccache for CI Actions, disabling PCH, and use PCH exclusively as a local compilation speedup mechanism?

The project's CI previously used Mozilla's sccache for compiler caching.
All workflows and the CMake compiler-cache module have been migrated to
ccache instead, using the hendrikmuhs/ccache-action on Linux and macOS,
and a manual actions/cache restore on Windows where ccache is installed
via the MSYS2 package manager. Corresponding environment variables were
updated from SCCACHE_* to CCACHE_* equivalents. The CMake helper was
also fixed to search only the explicitly requested tool rather than the
full list of candidates, preventing a stray sccache binary on a runner
from silently taking precedence over ccache. The README was updated to
drop the sccache recommendation.

Generated-by: Claude Opus 4.8 <noreply@anthropic.com>
@Lestropie
Lestropie requested a review from daljit46 July 3, 2026 03:04
@Lestropie Lestropie self-assigned this Jul 3, 2026
@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown

clang-tidy review says "All clean, LGTM! 👍"

@daljit46

daljit46 commented Jul 5, 2026

Copy link
Copy Markdown
Member

My initial implementation of compiler caching for the CI was indeed using ccache, but I remember that I chose the mozilla-actions/sccache-action because it supports per-object caching (thus limiting the storage space consumed). I was also aware of lack of PCH support in sccache (I came across it when debugging a caching issue raised by @bjeurissen ). However even ccache) has limitations when used with PCHs, so it's not exactly trouble-free in that sense.

I think switching to ccache is not a bad idea. As you pointed out, the major disavantage is that it will lead to an "all or nothing" caching strategy, but given that GitHub provides 10Gb (for 7 days) I don't think that limit will be hit easily (you'll probably need dozens of PRs or more to get there). I think before deciding it might be worth clearing the cache, trigger this PR and monitor the overall cache usage (perhaps running a small post build step that calls ccache --show-stats). That will give you a rough estimate of how much cache a single PR consumes (another would be to use the GitHub API).

One additional benefit of using ccache in CI is that it should be faster (see here).

Comment thread cmake/CompilerCache.cmake
find_program(CACHE_BINARY NAMES ${CACHE_OPTION_VALUES})
# Honour the requested tool only, so selection is deterministic and a stray
# sccache on a runner image can never be picked in preference to ccache.
find_program(CACHE_BINARY NAMES ${CACHE_OPTION})

@daljit46 daljit46 Jul 5, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed this when writing this code, but I think CACHE_BINARY here should use NO_CACHE because if the developer changes CACHE_OPTION, this also needs updating.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants