Incremental Systems Rethought

[tomassedovic]

Metadata
Point of contact	@blyxyas
Team(s)	compiler
Goal document	2026/incremental-system-rethought

Summary

Design, propose, and implement a redesign of the incremental system to allow for shared common bases between different Rustc invocations with different arguments (i.e. cargo build and cargo check)

There's already an [on-going RFC], based on [this blog post and talk] by the same author (and this project goal's point of contact)
As my other project goals, this is a formalization of an already existing effort.

Tasks and status

---

Note: we have updated the body to match the 2026 goal. Your original text is preserved below.

Details

Metadata
Point of contact	@blyxyas
Team(s)	compiler
Goal document	2026/incremental-system-rethought

Summary

Design, propose, and implement a redesign of the incremental system to allow for shared common bases between different Rustc invocations with different arguments (i.e. cargo build and cargo check)

There's already an [on-going RFC], based on [this blog post and talk] by the same author (and this project goal's point of contact)
As my other project goals, this is a formalization of an already existing effort.

Tasks and status

[tomassedovic]

This issue is intended for status updates only.

For general questions or comments, please contact the owner(s) directly.

[blyxyas]

2026-07-07 Project Update

• Partial funding has been expressed, thus starting the effort.
• We are currently in the research phase, improving documentation and refactoring the rustc_metadata and rustc_incremental modules to better allow for the big changes down the line.
• There will be a meeting between the point of contact (@blyxyas) and the compiler team champion (@jackh726) next week (2026-07-13 17:00 UTC).

---

As part of those refactorings, pull requests rust-lang/rust#158859, rust-lang/rust#158695 and rust-lang/rust#158616 have been opened.

The current goal is to have some experimental experience with the incremental system, probably in the form of Build Over Check (having build not re-do check work if the conditions allow it).

---

Parallel to this experimental work, the noted RFC (rust-lang/rfcs#3881) still has concerns to-be-resolved. These will be worked on at the same time.

[blyxyas]

2026-08-18 Goal Update

This update has been ready for about 10 days, just waited to publish it until we had more consolidated results.

---

• A tracking issue is open, [Tracking] Incremental Systems Rethought rust#160799, with a few parallelizable issues to work on.
• On the track of parallelizing work, a few companies have offered their help in design and implementation work. Conversations are being held with these companies and it might evolve into a coordinated effort if possible.
• I've been experimenting with tracking name resolution information in this branch. This has evolved into a full fledged tool for metadata analysis (allowing for reducing information). I might publish this tool into a perm-unstable feature similar to -Zls.
• Jack Huey and I have changed the meeting schedule to biweekly.

Blockers

• None

[blyxyas]

2026-08-26 Meeting notes

On 2026-08-26, a meeting was held between @jackh726, @celinval, @jlizen and me. Here are the meeting notes

Notes

Alejandra:

• Research phase winding down/finished
• A few experiments on fork, not yet merged
• Three companies offering help
  • Niko also offered help in form of other people helping him
  • epage@ offered with cargo help (Editor's note: Currently paused)
  • julian from wyeworks
  • predrag from cargo-semver-checks/openai
  • Jack as compiler team champion

Celina: What have you learned so far from the experiments? Anything you could share?

Alejandra:

• metadata anlyzer tool, similar to name resolution tool, actually turned out to be pretty useful to get number of items per dependency that you depend on as in many cases you very minimally use dependencies' interfaces
  • might be a nice finished product as a CLI
• Have been discussing whether to decide which dependencies to compile from point of name resolution, monomorphization, or some other point. So far leaning towards name resolution which still tree shakes ~70% of work
  • Jack: We are trying to only compile the things we need, we could start only (maybe test) binary, do name resolution, only find the things it needs, do an interative cycle of name resolution from that to find out what you actually need to typecheck and do codegen for
    • Current plan is to keep the same scheme where you name resolution the dependents first but defer codegen until you know it is needed in leaf for typechecking + codegen

Celina:

• How does that tie up with sharing artifacts between cargo check + build? Or are we taking a different direction?

Jack:

• TLDR from me is that currently the reason they aren't shared is that with cargo build, you need to generate the MiR, and the format for rmeta is different from cargo check

Celina:

• Could have cargo check do more things to generate build-compatible rmeta, if you made the build rmeta leaner maybe it would be viable

Jack:

• Will probably need some new format that is more fine-grained, which captures name resolution and allows cross-crate incrementalism
• At least for the start, this would only take for debug builds + testing since this would be slower than release. But eventually it might be easier to use more generally
• I saw fairly recently somebody was talking about this too for having a better, more in-place way of doing incremental things in context of build directory sizes, where every new incremental artifact is a new file, but in theory if you split it out into multiple files you could have better reuse

Celina:

• Definitely there are some big problems with incremental file bloat, where things take 100s of gigs because of all the incremental files
• Curious about cacheability of some of these things once Ed starts to review designs
• Also curious about how much of this sharing relies on incremental compiler information vs eg "rmeta_2", since rmeta tends to be more cachable and shared by cargo but incremental files tends to be more opaque to cargo

Alejandra:

• Several routes, have thought about using the incremental compiler pretty exclusively, there is a route where all artifacts are shared by default to pretty much everybody can understand them without much of a burden of interpretation on compiler
• There is also a route where rmeta files are pretty complete and any compiler can understand them
• If there are requirements one way or a direction we can adjust but we would like to know fairly early if we rely on the query system caching things for our implementation

Celina:

• One use case to think about is, a lot of people use sccache, remote cache, to cache rmeta files. That relies on a smaller set of files, one of the first things you do is incremental artifacts to avoid blowing up your bandwidth to download things becomes more expensive than actual compiles

Jack:

• There is room for some "do both", this is essentially about cross-crate incremental analysis, if you want to share artifacts, you likely aren't rebuilding those artifacts else why are you sharing them, so you could definitely avoid incremental artifacts for the parts of your dependency graph that you aren't editing
• But for crates that you are editing, you likely want the fine-grained incremental artifacts to avoid doing extra work
• For that case, you do want a rmeta file that gives you all the information that you need for a cargo build rather than a cargo check rmeta separate

Celina:

• Another way to think about it is you have a cargo check rmeta that cargo build rmeta builds on top of

Jack:

• Yeah if you do the fine-grained rmeta split across multiple files, then that could work

Celina:

• Main question is if you can restore portions from cargo check rmeta is that actually faster

Alejandra:

• Yeah, codegen is actually pretty slow, cargo
• Only step that is performed cargo build that is not in cargo check is only codegen + calling llvm

Celina:

• Main question is can compiler restart a new session from rmeta, today it can't

Alejandra:

• Yep, that's the core project goal, it should be able to
• Today the compiler is way too focused on one session, each section assumes the previous is called, today there is not good isolation policy between stages
• If you have everything until name resolution, you should be able (and by end of goal, you WILL be able), to just call the codegen only

Celina:

• Will you be able to restart from rmeta, without incremental compiler?

Jack:

• I think the idea of an incremental compiler will mean something different, today it means something different than cross-crate incrementalization, the short is that yes you should be able to take an rmeta that was build with cargo check, download these for all crates that you are not rebuilding, and then just codegen / type check the things that you need
• There is going to be a tradeoff where you want to disable storing some artifacts for the current crate so you can build as fast as possible
• Aiming to do debug and testing is the right first step
• Eventually you should be able to do this for release too

Celina:

• I think by default cargo already turns off incremental compiler for non-workspace crates
  • Jack: I think that's true

Alejandra:

• There is a cool future where we have a remote cache with the rmeta files and call them on demand

Jack:

• You could imagine that for crates wtihin a workspace, you are only working on one crate, and you could download the CI artifacts from crates you aren't using, technically it is all incremental on, and if you did want to edit something you could generate incremental artifacts, but if you are downloading something you wouldn't have incremental artifacts to start with

Celina:

• There is work happening on cross-workspace incremental caching too
• I looked into the on-demand last year, whenever I talked to people, the first thing that would be "what about cache"
• If you compile things on-demand with one thing, once you compile everybody else, you anyway need to re-compile everything once you touch dependencies

Jack:

• You could imagine that ideally, if you build everything in CI, you shouldn't need to rebuild everything since you have artifacts
• But for workspace you might need to only rebuild things you need

Celina:

• But now cargo and the other build systems need to be aware of which has been compiled on demand

Alejandra:

• Yeah that's a bit complicated, we are talking to Ed Page to get help wtih this questions
• In theory once we nail down these details and merge patches, it shouldn't be that hard to implement a good remote cache
• Everything pretty much until HiR/MiR/THiR is very reproducible between architectures if they don't have target/cpu feature

Jess:

• Do you know fasterthanlime / Amos? He is working on a nicer remote cache, might be interested

Alejandra:

• Yeah I've chatted with them, having a real remote cache could be nice to test against

[blyxyas]

2026-09-09 Goal Update

• We're currently organizing a team, and holding calls with relevant people (for organizational adivce and ideas)
• A new repository with design thoughts is now active, at blyxyas/incr-systems
• The Cargo part of the goal is taking more shape, and it's currently working on a local branch. We should see published results soon
  • Being that every part of the goal is very interconnected, this is good news for all!

Help wanted

• I'm currently looking for help by a Cargo expert. Having help in the Cargo part would accelerate development greatly.
• If you or your company can offer some man-hours, please get in contact with me! I have a bunch of tasks which helping with would be greatly apprciated.