partial c++20 module support by TroyKomodo · Pull Request #2516 · mozilla/sccache

TroyKomodo · 2025-12-18T01:19:52Z

Adds partial support for c++20 modules when using clang.

Also detects when modules are used in MSVC / GCC and disable cache for those calls.

partially addresses #2095

the clang c++20 module implementation (as well as msvc) is rather easy to parse and understand so adding support for it is fairly trivial. GCC however is a mess. An entirely different approach will be needed to get GCC module support which is far out side of the scope for this initial PR.

I will submit future PRs to get MSVC / GCC.

Test with nix

(pkgs.sccache.overrideAttrs (final: old: {
  version = "troy/cxx20-modules";
  cargoHash = "sha256-tCQrZ6cPGJrxHgeSJ4ZqZs6otWvVu6Ys3svAiwvmSDc=";
  src = fetchFromGitHub {
    owner = "troykomodo";
    repo = "sccache";
    rev = "troy/cxx20-modules";
    sha256 = "sha256-A0n/nsNPTHRC0Z6ERwCpoR1w3O/xl6d1yCw/5GeqNd4=";
  };
  cargoDeps = pkgs.rustPlatform.fetchCargoVendor {
    inherit (final) pname src version;
    hash = final.cargoHash;
  };
}))

codecov-commenter · 2025-12-18T08:20:50Z

Codecov Report

❌ Patch coverage is 87.70764% with 37 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.39%. Comparing base (81954de) to head (10fc4f6).

Files with missing lines	Patch %	Lines
src/compiler/clang.rs	84.45%	23 Missing ⚠️
src/compiler/gcc.rs	85.54%	12 Missing ⚠️
src/compiler/compiler.rs	33.33%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2516      +/-   ##
==========================================
+ Coverage   73.16%   73.39%   +0.22%     
==========================================
  Files          68       68              
  Lines       36743    37208     +465     
==========================================
+ Hits        26884    27309     +425     
- Misses       9859     9899      +40

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

ChuanqiXu9 · 2025-12-18T09:47:30Z

FYI ccache/ccache#1523

mathstuf

The code looks good to me. I'll probably try to test it some time next week.

I can help out with GCC module map file parsing too (it will need to actually be parsed due to the $root meta-descriptor). However, the output path is also given by it…that may prove more difficult to "know".

TroyKomodo · 2025-12-18T16:08:41Z

The code looks good to me. I'll probably try to test it some time next week.

I can help out with GCC module map file parsing too (it will need to actually be parsed due to the $root meta-descriptor). However, the output path is also given by it…that may prove more difficult to "know".

Thanks! It was my first time contributing to sccache so I'm not too familiar with the process.

I will fix up the clippy issues + windows test issues :p

mathstuf · 2025-12-21T03:59:26Z

It looks like BMI-only compilations are not properly considered:

467:    actual-stdout> [2-1->6/8@76.9] Building CXX object 'CMakeFiles/CXXModules__export_from_ninja@synth_cf4e03cd8b1e.dir/35a7889ff75f.bmi'          
467:    actual-stdout> FAILED: [code=254] CMakeFiles/CXXModules__export_from_ninja@synth_cf4e03cd8b1e.dir/35a7889ff75f.bmi 
467:    actual-stdout> /home/boeckb/code/depot/group-devel/sccache/src/target/debug/sccache  /usr/lib64/ccache/clang++   -g -std=gnu++20 --precompile -MD -MT 'CMakeFiles/CXXModules__
export_from_ninja@synth_cf4e03cd8b1e.dir/35a7889ff75f.bmi' -MF CMakeFiles/CXXModules__export_from_ninja@synth_cf4e03cd8b1e.dir/35a7889ff75f.bmi.d @CMakeFiles/CXXModules__export_from_
ninja@synth_cf4e03cd8b1e.dir/35a7889ff75f.bmi.modmap -o 'CMakeFiles/CXXModules__export_from_ninja@synth_cf4e03cd8b1e.dir/35a7889ff75f.bmi' -c /home/boeckb/misc/builds/cmake/build-cla
ng/Tests/RunCMake/CXXModulesCompile/exp-mods-from-ninja-ninja-install/lib/cxx/miu/importable.cxx
467:    actual-stdout> sccache: encountered fatal error
467:    actual-stdout> sccache: error: Failed to generate compile commands
467:    actual-stdout> sccache: caused by: Failed to generate compile commands
467:    actual-stdout> sccache: caused by: Missing object file output

This is from running CMake's CXXModules tests:

export SCCACHE_SERVER_PORT=23448 # unique to avoid ambient re-usage
export CXX="$(which sccache) $(which clang++)"
ctest -R CXXModules

mathstuf · 2025-12-21T04:00:32Z

You can ignore the tests failing about JSON argument list mismatches; it is not robust against mutli-argument compilers like here. If you do the "symlink trick" (ln -s $(which sccache) clang++ && export CXX="$PWD/clang++", that probably avoids the issue).

TroyKomodo · 2025-12-21T04:40:06Z

Hmm, i think this is just me forgetting to change it somewhere. I will fix

TroyKomodo · 2025-12-21T04:42:26Z

Okay so it seems i think we should always treat it as an object even if it really isnt...

sylvestre · 2025-12-21T09:08:01Z

could you please add an integration test ? ie high level which integrates into cmake and uses C++ module.
see
tests/cmake/ tests/sccache_args.rs tests/sccache_cargo.rs tests/sccache_rustc.rs

mathstuf · 2025-12-21T12:43:12Z

I tried the cmake tests and they ran fine for me but there were only 2...

There are dozens of test cases within each CTest-level test. These are the relevant ones, thanks!

Can you inspect the cache state (--show-stats) on an initial run and a subsequent run to see how well caching itself is working?

TroyKomodo · 2025-12-21T14:42:33Z

could you please add an integration test ? ie high level which integrates into cmake and uses C++ module.

see

tests/cmake/ tests/sccache_args.rs tests/sccache_cargo.rs tests/sccache_rustc.rs

Yes!

TroyKomodo · 2025-12-21T18:52:31Z

https://github.com/mozilla/sccache/actions/runs/20413181927/job/58653103431 test results look good!

sylvestre · 2025-12-21T18:55:17Z

are you sure ?

Run ${SCCACHE_PATH} --show-stats
Compile requests                      4
Compile requests executed             4
Cache hits                            2
Cache hits (C/C++)                    2
Cache misses                          2
Cache misses (C/C++)                  2
Cache hits rate                   50.00 %
Cache hits rate (C/C++)           50.00 %

TroyKomodo · 2025-12-21T18:57:13Z

That seems right no? the first time we invoke the compiler not cached yet then we invoke again and its cached.

There are only 2 files to compile, so i expected it to be 4 requests 2 hit 2 misses.

sylvestre · 2025-12-21T20:30:22Z

oh yeah, my bad, i thought we were doing a reset

TroyKomodo · 2025-12-21T21:04:18Z

oh yeah, my bad, i thought we were doing a reset

I just copied the other cmake test example, but I can reset the stats if that makes it easier to follow

mathstuf · 2025-12-22T04:43:03Z

Gah, module compilation in the tests doesn't really happen without a setting. For clang, this should work:

-DCMake_TEST_MODULE_COMPILATION=named,compile_commands,collation,partitions,internal_partitions,export_bmi,install_bmi,shared,bmionly,build_database,import_std23

src/compiler/gcc.rs

sylvestre · 2025-12-22T13:01:58Z

src/compiler/gcc.rs

    }

+    if !module_only_flag {
+        if let Some(module_output_path) = module_output_path {


could you please move this into a function?

I can, however this function is ~550 lines, I think it would be better if in a followup PR we break up this function rather than doing a half job here by breaking up just this one conditional.

i still want this to be a new function, sorry

tests/cmake-modules/CMakeLists.txt

tests/cmake-modules/mymodule.cppm

TroyKomodo · 2025-12-22T17:01:11Z

Gah, module compilation in the tests doesn't really happen without a setting. For clang, this should work:

-DCMake_TEST_MODULE_COMPILATION=named,compile_commands,collation,partitions,internal_partitions,export_bmi,install_bmi,shared,bmionly,build_database,import_std23

Ah, I see. Okay I will try again i think i need to setup my env to have support for import std.

TroyKomodo · 2025-12-22T19:47:37Z

Gah, module compilation in the tests doesn't really happen without a setting. For clang, this should work:
-DCMake_TEST_MODULE_COMPILATION=named,compile_commands,collation,partitions,internal_partitions,export_bmi,install_bmi,shared,bmionly,build_database,import_std23

Ah, I see. Okay I will try again i think i need to setup my env to have support for import std.

let
  pkgs = import <nixpkgs> {
    overlays = [
      (final: prev: {
        llvmPackages_21 = prev.llvmPackages_21 // {
          libcxx = prev.llvmPackages_21.libcxx.overrideAttrs (old: {
            postInstall = (old.postInstall or "") + ''
              substituteInPlace $out/lib/libc++.modules.json \
                --replace-fail '"../share' "\"$out/share"
            '';
          });
        };
      })
    ];
  };

  llvm = pkgs.llvmPackages_21;

  wrappedClangScanDeps = pkgs.writeShellScriptBin "clang-scan-deps" ''
    newargs=()
    inject_next=false
    for arg in "$@"; do
      if $inject_next; then
        newargs+=("$arg")
        newargs+=("-isystem" "${llvm.libcxx.dev}/include/c++/v1")
        inject_next=false
      elif [[ "$arg" == "--" ]]; then
        newargs+=("$arg")
        inject_next=true
      else
        newargs+=("$arg")
      fi
    done
    exec ${llvm.clang-tools}/bin/clang-scan-deps "''${newargs[@]}"
  '';

  sccache = "/home/troy/github/troy/sccache/result/bin/sccache";

  clangWrapper = pkgs.writeShellScriptBin "clang" ''
    exec ${sccache} "${llvm.libcxxClang}/bin/clang" "''${@}"
  '';
  clangxxWrapper = pkgs.writeShellScriptBin "clang++" ''
    exec ${sccache} "${llvm.libcxxClang}/bin/clang++" "''${@}"
  '';
in
pkgs.mkShell {
  hardeningDisable = [ "all" ];

  buildInputs = [
    pkgs.cmake
    pkgs.mold
    pkgs.ninja
    pkgs.openssl
    llvm.libcxxClang
    llvm.clang-tools
    wrappedClangScanDeps
  ];

  shellHook = ''
    export CC="${clangWrapper}/bin/clang"
    export CXX="${clangxxWrapper}/bin/clang++"
    export LD_LIBRARY_PATH="${llvm.libcxx.out}/lib:${pkgs.openssl.out}/lib:''${LD_LIBRARY_PATH:-}"
    export NIX_CFLAGS_COMPILE="-B${llvm.libcxx.out}/lib -isystem ${llvm.libcxx.dev}/include/c++/v1"
    export PATH="${wrappedClangScanDeps}/bin:''${PATH:-}"
  '';
}

A huge pain...

troy@troy-thinkpad ~/g/t/CMake (master) [SIGINT]> ctest -R CXXModules -j$(nproc) --test-dir=build   nix-shell 
Test project /home/troy/github/troy/CMake/build
Guessing configuration NoConfig
    Start 536: RunCMake.CXXModulesCompile
    Start 535: RunCMake.CXXModules
1/2 Test #535: RunCMake.CXXModules ..............   Passed   18.90 sec
2/2 Test #536: RunCMake.CXXModulesCompile .......   Passed  116.80 sec

100% tests passed, 0 tests failed out of 2

Label Time Summary:
CXXModules    = 135.70 sec*proc (2 tests)

Total Test time (real) = 116.81 sec
troy@troy-thinkpad ~/g/t/CMake (master)> ctest -R CXXModules -j$(nproc) --test-dir=build            nix-shell 
Test project /home/troy/github/troy/CMake/build
Guessing configuration NoConfig
    Start 536: RunCMake.CXXModulesCompile
    Start 535: RunCMake.CXXModules
1/2 Test #535: RunCMake.CXXModules ..............   Passed   19.94 sec
2/2 Test #536: RunCMake.CXXModulesCompile .......   Passed  104.62 sec

100% tests passed, 0 tests failed out of 2

Label Time Summary:
CXXModules    = 124.55 sec*proc (2 tests)

Total Test time (real) = 104.63 sec
troy@troy-thinkpad ~/g/t/CMake (master)>                                                            nix-shell

Every 2.0s: /home/troy/github/troy/sccache/result/bin/sccache -s       troy-thinkpad: Mon Dec 22 12:47:11 2025

Compile requests                   1976
Compile requests executed           830
Cache hits                          756
Cache hits (C/C++)                  756
Cache misses                         72
Cache misses (C/C++)                 72
Cache hits rate                   91.30 %
Cache hits rate (C/C++)           91.30 %
Cache timeouts                        0
Cache read errors                     0
Forced recaches                       0
Cache write errors                    0
Cache errors                          0
Compilations                         72
Compilation failures                  2
Non-cacheable compilations            0
Non-cacheable calls                 244
Non-compilation calls               902
Unsupported compiler calls            0
Average cache write               0.001 s
Average compiler                  0.217 s
Average cache read hit            0.000 s
Failed distributed compilations       0

Non-cacheable reasons:
-E                                  244

Cache location                  Local disk: "/home/troy/github/troy/CMake/build/Tests/CMakeFiles/TestHome/.cac
he/sccache"
Use direct/preprocessor mode?   yes
Version (client)                0.12.0
Cache size                          155 MiB
Max cache size                       10 GiB

mathstuf · 2026-01-13T01:39:47Z

If you could point me to where the code for the distributed cc lives so I can see how it works that would be great!

Look at src/dist/mod.rs. BuildResult would need built with the outputs. Inputs seem to be built into a tar file shipped via InputsReader. Chasing this down, it looks like src/compiler/c.rs's CInputsPackager needs to learn to ship module files and munge module mapper arguments/files as necessary.

avikivity · 2026-01-17T17:39:23Z

Isn't this needed both for caching and for distributed compilation?

The inputs are needed for caching to check timestamps/sizes/hashes and for distribution to ship over the files. Note hashing the preprocessed output is insufficient.

The outputs are needed for caching to stash in the cache and for distribution so ship back.

For distribution, we might need to massage the command line, adding -fmodule-file for each input and output module to redirect them to out place in the filesystem.

AJIOB · 2026-01-17T18:27:19Z

Isn't this needed both for caching and for distributed compilation?

The inputs are needed for caching to check timestamps/sizes/hashes and for distribution to ship over the files. Note hashing the preprocessed output is insufficient.

The outputs are needed for caching to stash in the cache and for distribution so ship back.

For distribution, we might need to massage the command line, adding -fmodule-file for each input and output module to redirect them to out place in the filesystem.

IMO, we can think about distributed compiling support in additional PR

sccache [1] combines ccache and distcc, and promises [2] to support modules early. Support it via a more general --compiler-cache option, replacing the current --ccache. [1] https://github.com/mozilla/sccache [2] mozilla/sccache#2516

sccache [1] combines ccache and distcc, and promises [2] to support modules early. Support it via a more general --compiler-cache option, replacing the current --ccache. The test workflow is adjusted for the new option name. [1] https://github.com/mozilla/sccache [2] mozilla/sccache#2516

mathstuf · 2026-01-20T02:26:45Z

Note that an initial investigation to at least drop "unsupported" errors and/or TODO comments in places would be useful for further work. It could also help avoid confusion by anyone trying modules and distributed builds in the interim.

TroyKomodo · 2026-01-20T06:57:56Z

Note that an initial investigation to at least drop "unsupported" errors and/or TODO comments in places would be useful for further work. It could also help avoid confusion by anyone trying modules and distributed builds in the interim.

I am still traveling so I haven't had time to try out the distributed compiler. I wish there were better tests than to setup a docker container and try run it. I wasn't even aware the distributed compiler was a thing until someone pointed it out in this PRs comments. Since all the unit tests passed... If I don't do the full impl, I will at minimum add unsupported errors for modules when using distributed compiler mode.

Sorry I haven't had time yet I should be fully settled down by Wednesday.

mathstuf · 2026-01-20T18:34:43Z

There's no rush on my account at least; take your time.

AJIOB · 2026-02-04T05:02:12Z

Hi @TroyKomodo,

Do you have any updates?

Looks like we have conflicts.

codspeed-hq · 2026-02-15T12:32:19Z

Merging this PR will not alter performance

✅ 62 untouched benchmarks
⏩ 2 skipped benchmarks¹

_{Comparing TroyKomodo:troy/cxx20-modules (10fc4f6) with main (81954de)}

2 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

TroyKomodo · 2026-02-15T12:50:24Z

Okay! So i finally have gotten the time to look through this PR again. I think, this automatically works with dist mode since we add the PCMs to the extra_hash_files which are sent to the dist server automatically?

Could someone please provide me with a way to test this or someone test this for me, using the integration test in the PR.

Integration tests fail because this branch is from a fork.

sylvestre · 2026-02-22T20:41:33Z

Integration tests fail because this branch is from a fork.

why do you think that ?

mathstuf

Seems fine as a stepping stone towards actual support. Just one minor request.

mathstuf · 2026-02-23T18:36:35Z

src/compiler/msvc.rs

    msvc_flag!("openmp:experimental", PassThrough),
    msvc_flag!("permissive", PassThrough),
    msvc_flag!("permissive-", PassThrough),
+    msvc_take_arg!("reference", OsString, Separated, TooHard),


The MSVC flags are very similar to Clang's. -interface and -internalPartition just need hashed like any other.

Can issues be opened and referenced from the comments for GCC and MSVC support?

TroyKomodo force-pushed the troy/cxx20-modules branch from 8f46985 to 930732c Compare December 18, 2025 01:24

ChuanqiXu9 mentioned this pull request Dec 18, 2025

Implement support for C++20 modules ccache/ccache#1523

Draft

12 tasks

mathstuf approved these changes Dec 18, 2025

View reviewed changes

TroyKomodo mentioned this pull request Dec 18, 2025

sccache partial support kelteseth/arewemodulesyet#78

Closed

TroyKomodo force-pushed the troy/cxx20-modules branch from 489600b to 71e9190 Compare December 19, 2025 15:25

TroyKomodo force-pushed the troy/cxx20-modules branch from c0a57b6 to 8841d54 Compare December 21, 2025 04:44

This comment was marked as outdated.

Sign in to view

sylvestre reviewed Dec 22, 2025

View reviewed changes

src/compiler/gcc.rs Outdated Show resolved Hide resolved

sylvestre reviewed Dec 22, 2025

View reviewed changes

tests/cmake-modules/CMakeLists.txt Outdated Show resolved Hide resolved

sylvestre reviewed Dec 22, 2025

View reviewed changes

tests/cmake-modules/mymodule.cppm Outdated Show resolved Hide resolved

sylvestre force-pushed the troy/cxx20-modules branch from 0aec08a to 64a1a01 Compare January 12, 2026 10:27

TroyKomodo force-pushed the troy/cxx20-modules branch from 64a1a01 to 864b5b0 Compare February 15, 2026 12:21

TroyKomodo requested review from mathstuf and sylvestre February 21, 2026 09:00

TroyKomodo added 12 commits February 22, 2026 17:56

partial c++20 module support

68d2959

remove unused conditional for multi input

35b089c

fix MSRV issue

971763d

ignore vscode settings

9b712cc

always output object even if its a module

2202ccb

fix tests

f0747ad

add cmake modules test

62ae559

fix test

30f98a7

fix newlines

50c54dc

add extra info to cannot cache call

0846553

fix fmt

bcddc25

split out module artifact code

10fc4f6

sylvestre force-pushed the troy/cxx20-modules branch from cf216c5 to 10fc4f6 Compare February 22, 2026 16:56

mathstuf reviewed Feb 23, 2026

View reviewed changes

Conversation

TroyKomodo commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-commenter commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ChuanqiXu9 commented Dec 18, 2025

Uh oh!

mathstuf left a comment

Choose a reason for hiding this comment

Uh oh!

TroyKomodo commented Dec 18, 2025

Uh oh!

mathstuf commented Dec 21, 2025

Uh oh!

mathstuf commented Dec 21, 2025

Uh oh!

TroyKomodo commented Dec 21, 2025

Uh oh!

TroyKomodo commented Dec 21, 2025

Uh oh!

This comment was marked as outdated.

sylvestre commented Dec 21, 2025

Uh oh!

mathstuf commented Dec 21, 2025

Uh oh!

TroyKomodo commented Dec 21, 2025

Uh oh!

This comment was marked as outdated.

TroyKomodo commented Dec 21, 2025

Uh oh!

sylvestre commented Dec 21, 2025

Uh oh!

TroyKomodo commented Dec 21, 2025

Uh oh!

sylvestre commented Dec 21, 2025

Uh oh!

TroyKomodo commented Dec 21, 2025

Uh oh!

mathstuf commented Dec 22, 2025

Uh oh!

Uh oh!

sylvestre Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

TroyKomodo Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

sylvestre Feb 15, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

TroyKomodo commented Dec 22, 2025

Uh oh!

TroyKomodo commented Dec 22, 2025

Uh oh!

mathstuf commented Jan 13, 2026

Uh oh!

avikivity commented Jan 17, 2026

Uh oh!

AJIOB commented Jan 17, 2026

Uh oh!

mathstuf commented Jan 20, 2026

Uh oh!

TroyKomodo commented Jan 20, 2026

Uh oh!

mathstuf commented Jan 20, 2026

Uh oh!

AJIOB commented Feb 4, 2026

Uh oh!

codspeed-hq bot commented Feb 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

TroyKomodo commented Feb 15, 2026

Uh oh!

TroyKomodo commented Dec 18, 2025 •

edited

Loading

codecov-commenter commented Dec 18, 2025 •

edited

Loading

codspeed-hq bot commented Feb 15, 2026 •

edited

Loading