Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
263beff
docs: plan the module interface/implementation split, with the mechan…
Sunrisepeak Aug 13, 2026
826cc21
tools: a splitter for the interface/implementation migration, and the…
Sunrisepeak Aug 13, 2026
8365df0
refactor: separate module interface from implementation across 81 mod…
Sunrisepeak Aug 13, 2026
b87f49b
refactor: out-of-line class members, and keep module-private declarat…
Sunrisepeak Aug 13, 2026
b8e1566
fix(tools): outline.py must be idempotent, and out-of-line bodies nee…
Sunrisepeak Aug 13, 2026
1b515c1
test: the CLI parity guard has to scan implementation units too
Sunrisepeak Aug 13, 2026
7de587d
fix(tools): a dropped declaration must not take its doc comment with it
Sunrisepeak Aug 13, 2026
76765fe
tools: an export-surface invariant check for the split
Sunrisepeak Aug 13, 2026
3bd0e41
build: move the musl release target to gcc 16, and regenerate with co…
Sunrisepeak Aug 13, 2026
c94e199
fix: a module whose interface keeps a template keeps its declarations…
Sunrisepeak Aug 13, 2026
4b44fe3
fix: an in-class body holding a stream-less std::print is the instant…
Sunrisepeak Aug 13, 2026
d88d223
fix: name the stream in interface templates, and give implementation …
Sunrisepeak Aug 13, 2026
40b42e9
docs+tools: the split report, and drop a dead parameter from classify()
Sunrisepeak Aug 13, 2026
4c254fd
refactor: move the imports the bodies took with them, and fix three t…
Sunrisepeak Aug 13, 2026
95ca3ae
docs: the measurements, including the one where the split loses
Sunrisepeak Aug 13, 2026
29d1902
docs: the trim reverses the test regression, and falsifies my explana…
Sunrisepeak Aug 13, 2026
8fd028b
release: 2026.8.13.1
Sunrisepeak Aug 13, 2026
122a073
docs: release notes for 2026.8.13.1
Sunrisepeak Aug 13, 2026
2b76ef8
docs: say which tree each control ratio comes from
Sunrisepeak Aug 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
483 changes: 483 additions & 0 deletions .agents/docs/2026-08-13-module-impl-split-report.md

Large diffs are not rendered by default.

104 changes: 104 additions & 0 deletions .agents/docs/2026-08-13-release-2026.8.13.1-notes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# 2026.8.13.1 —— 「接口和实现分开」

> 配套:PR #545。报告:`.agents/docs/2026-08-13-module-impl-split-report.md`
> 工具:`.agents/tools/module-split/`(`regen.sh` 可逐字节复现本次提交树)

## 1. 一句话

110 个 `.cppm` 同时装着模块接口和它自己的实现,于是**任何函数体的改动都会改变 BMI,
所有 importer 全部重编**。这一版把它们分成标准的一对:`X.cppm`(`export module M;`,
产出 BMI)+ `X.cpp`(`module M;`,**不产出 BMI**)。

接口从 46,253 行降到 14,666 行。但这次真正值钱的不是那个 2.16×,而是**唯一一个
测回来是"变慢"的数,以及它最后被证明不是我说的那个原因。**

## 2. 先说三条被自己推翻的前提

都是我写下来之后被测量或编译器否掉的,记在这里而不是悄悄改掉。

### 2.1 「TU 翻倍,冷构建会变慢」→ **反了,快 2.16×**

110 个接口变成 110 接口 + 92 实现单元,TU 数量差不多翻倍,所以我在计划里写的是
"冷构建方向不确定,可能变慢"。实测 56.40s → 26.09s。**更小的 BMI 带来的收益超过
更多 TU 的成本**,而这只有测了才知道。

### 2.2 「`mcpp test` 变慢是链接成本」→ **被 phase 3 证伪**

单测套件一开始**慢 1.16×**(953.81s → 1104.83s),是唯一朝错方向走的数。
我的论证是:每个测试二进制的开销近乎恒定(+3.74s)、与测试文件自身大小无关,
而每个二进制现在要链接约 202 个目标文件而不是 112 个 —— 所以是链接。

**这个论证站不住。** BMI 加载**同样**是"每 TU 近乎恒定",因为它取决于导入闭包
而不是被编译的那个文件。我的证据无法区分两个候选原因。

phase 3(裁剪残留 import)成了对照实验:它**只**改接口的 import 边,每个二进制
链接的目标文件数前后完全一样。如果病因是链接,惩罚会留下来。它消失了 ——
**每二进制 +3.74s 变成 −0.11s**,套件反过来比 main 快 1.03×。

真因是:每个测试 TU 都在加载接口里根本没提到的模块的 BMI。

### 2.3 「对照探针必须不改善」→ **改善 2.9×,而这才是它有用的地方**

`compare_segment` 在两个分支上都是 `semver.cppm` 里的 `inline`,所以我写"它必须
不变"。它变快了。原因是:编辑接口时两侧重编的**模块集合**相同,但分支上那些下游
**接口本身小得多**。

正确的表述是它分离了两件事 —— **重编集合**(实现改动不再让 BMI 失效,只有搬移过的
探针拿到这个)和**重编成本**(仍要重编的东西现在更小,对照组也拿到)。对照组
2.9×,搬移过的探针 4.6–12.4×,**差距才是搬移的贡献**。

## 3. 数字

| | main | 本版 | |
|---|---|---|---|
| 接口行数 | 46,253 | **14,666** | 32% |
| 冷构建(3 次中位) | 56.40s | **26.09s** | **2.16×** |
| 改一个实现(66 下游) | 54.04s | **4.35s** | **12.4×** |
| 改一个类成员(41 下游) | 49.00s | **6.31s** | 7.8× |
| 对照(两侧都 `inline`) | 43.58s | 15.24s | 2.9× |
| `mcpp test`(38 个二进制) | 953.81s | **921.73s** | 1.03× |
| dev 二进制 `-O0` | 124,238,424 B | +0.10% | |

同一 worktree 内切换,只换 `src/`,路径/文件系统/工具链/依赖缓存全同。

**main 上改一个函数体的代价约等于一次完整冷构建** —— 因为分离之前,每次改函数体
其实都是在改接口。

## 4. 四套工具链,四个 dev 构建看不见的失败

`mcpp build`(`gcc@16.1.0`)是本项目四个编译器之一,另外三个各否掉了它接受的东西。

| 工具链 | 否掉了什么 |
|---|---|
| `gcc@15.1.0-musl` | 搬进实现单元的 `std::views::transform` 管道。**修法是把 musl 目标升到 `gcc@16.1.0-musl`** —— 反方向的同类问题此前咬过本项目(`views::split \| ranges::to` 在 15 上过、在 16 上让整个模块以 "Bad file data" 失败),所以两个目标现在共用一个编译器主版本,而不是用一个破口换另一个 |
| `llvm@20.1.7` | 接口 template 里**不带流参数的 `std::print`**。libc++ 把 `print(fmt,…)` 实现为 `print(stdout, fmt, …)`,所以 `FILE*` 重载必须在**实例化点**赢得重载决议 —— 而对接口 template 来说那个点在 importer 里。修法:4 个接口的 11 处显式写出流,外加 8 个实现单元补 `<cstdio>` |
| `gcc@16.1.0` | 两个 ICE,各自报在**另一个模块的另一个类型**上。`Config::instance_()` 是二分 config.cppm 的 85 个成员到 #34 才定位的 |
| 一个残留的 `.o` | 把第一个问题也藏过了 dev 工具链:清 `target/*/gcm.cache` 会留下 `.o`,该失败的 TU 根本没重编。**只有 `rm -rf target` 才算证明构建过** |

`llvm@20.1.7` 在 **Linux** 上就能逐字复现 macOS 的失败,所以那本是一个 40 秒的回路,
而我在此之前用掉了三轮 CI 去猜。**最小复现不行** —— 它需要整个项目,这正是那个
clang 可见的 bug 一开始看起来像 macOS libc 特有问题的原因。

## 5. 不变量:验过,不是断言

| 不变量 | 结果 |
|---|---|
| 函数定义 | **1,262 → 1,262**,零丢失零重复 |
| 导出标识符 | **961 → 961**,0 丢失 0 新增 |
| 模块名 | 110 个逐字节一致 |
| 动态初始化顺序 | 唯一跨单元分裂的模块里留下的是 `std::atomic<bool>{false}`(常量初始化),没有顺序可破坏 |
| 重建的 `#if` 守卫 | 两处与原文核对一致;macOS / Windows CI 实证 |

## 6. 刻意没做

**残留的 `inline` 一行未动 —— 接口里仍有 1,953 行在 `inline` 函数体内。**
在 main 的树上量到的是**同一个数字**,所以它们不是本次漏搬的,而是本来就 `inline`
的原样带过来。在模块接口里 `inline` 的含义是"把函数体放进 BMI 供 importer 内联",
去掉它是性能可见性决策,该由维护者定,不该由一次机械迁移悄悄做。这也是剩下最大的
一根杠杆(约再压 13%)。

## 7. 顺手记一笔(与本版无关)

`mcpp.toml` 里 `ftxui = "6.1.9"` 在 `[dependencies]` 下,走的是**已废弃的裸名回退**,
每次构建都会 warn,而该回退**在 mcpp 2026.9 移除**。一行的事(移到
`[dependencies.compat]`),也是 `mcpp.lock` 反复变动的原因。本轮没动,留给维护者定。
136 changes: 136 additions & 0 deletions .agents/plans/2026-08-13-module-interface-impl-separation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
# Module interface / implementation separation

**Branch:** `refactor/module-impl-separation`
**Date:** 2026-08-13

## Goal

Split every C++23 module in `src/` into a standard-conforming pair:

- `X.cppm` — **module interface unit** (`export module M;`): types, declarations,
templates, `constexpr`. Produces the BMI.
- `X.cpp` — **module implementation unit** (`module M;`): the function bodies.
Produces **no BMI**.

No directory moves, no behaviour changes, no new product code.

## Why: the BMI is the recompile trigger

Today every function body lives in the interface unit, so **every body edit
changes the BMI**, and everything that imports the module recompiles.

Measured fan-out over the 99 local modules (`build/bench/analyze.py` +
import-graph scan):

| module | direct importers | transitive downstream |
|---|---|---|
| `xlings.platform` | 45 | **63** |
| `xlings.core.palette` | 8 | 53 |
| `xlings.libs.json` | 31 | 52 |
| `xlings.core.log` | 39 | 47 |
| `xlings.core.config` | 34 | 35 |

Average transitive downstream: **10.7 modules**. Editing one line in
`platform.cppm`'s implementation rebuilds 64 translation units. After the
split it rebuilds one.

## Mechanism verified before writing any code

`build/bench/probe2` (a throwaway 3-file mcpp project) established that mcpp
and gcc@16.1.0 support the standard shape, and that the four rules the
migration depends on actually hold:

1. mcpp scans `.cpp` implementation units with P1689 dyndep and orders them
after their interface's BMI. `thing.cppm` → `thing.m.o` + `p2.thing.gcm`;
`thing.cpp` → `thing.o` and **no `.gcm`**.
2. A **non-exported** helper declared in the interface and defined in the
implementation unit is callable from an **exported template** that gets
instantiated in a downstream TU. It links and runs.
3. An `extern` module-linkage global declared in the interface and defined in
the implementation unit is readable from such a template.
4. The `export` keyword must be omitted in the implementation unit; a default
argument must appear only in the interface; `constexpr` + `static_assert`
stay in the interface.

Incremental behaviour on that probe:

| edit | recompiled | time |
|---|---|---|
| implementation body (`thing.cpp`) | `thing.cpp` only | 0.29s |
| interface (`thing.cppm`) | `.cppm` + `.cpp` + every importer | 0.79s |

mcpp additionally preserves a BMI's mtime when a recompile produces
byte-identical content, so an interface edit that does not change the BMI
already avoids downstream work. That is why the bodies are the thing to move.

## Classification rules

Applied per namespace-scope (or class-scope) entity.

**STAY in the interface — whole:**
- type definitions (`struct` / `class` / `union` / `enum`), `using`, `typedef`,
namespace aliases, `concept`, `static_assert`
- anything `template<...>` (11 sites, all variadic log/format wrappers)
- `constexpr` / `consteval` functions and variables, `inline` variables
- preprocessor conditionals that select declarations

**SPLIT — declaration in the interface, definition in the implementation unit:**
- non-template, non-`constexpr` function definitions at namespace scope,
exported or not (a non-exported helper still needs its declaration in the
interface when a template or another staying entity calls it)
- out-of-line-able member functions of non-template classes, including
`static` member functions (defined as `T C::f(...)`, no `static` keyword)
- namespace-scope variable definitions with dynamic initialisation

**MOVE WHOLE to the implementation unit:**
- anonymous-namespace blocks (4 files, all under `src/core/mirror/`)
- namespace-scope `static` free functions **not** referenced by a staying
entity (internal linkage cannot span two units)

## Ordering invariant

All namespace-scope variable *definitions* of a module move to the same unit
(the implementation unit), so their relative dynamic-initialisation order is
preserved. Never split a module's globals across the two units.

## Scope sizing

| | lines | share |
|---|---|---|
| outside class bodies (free functions) | 39,258 | 84.9% |
| inside class/struct bodies (need out-of-line members) | 6,995 | 15.1% |
| total across 110 `.cppm` | 46,253 | |

71 `static ... (...) {` definitions; indent 0 = namespace-scope internal
linkage, indent 4 = static member functions (ordinary out-of-line definitions).

Both groups are in scope. The class-heavy files are the high-fan-out ones
(`config.cppm` is 93% class body **and** has 34 direct importers), so
skipping them would forfeit much of the benefit.

## Known trade-off, to be measured not assumed

Moving a body out of the interface drops its implicit `inline`. Without LTO
the release build loses those cross-TU inlining opportunities. The dev build
is `-O0`; release is `-O2`. The report must carry binary size and, where
cheap, a runtime check — not a claim that this is free.

Cold-build direction is genuinely uncertain: TU count roughly doubles
(110 → ~220), which costs, while every BMI gets smaller, which pays. Measure
both cold and incremental.

## Verification

1. `mcpp build` succeeds.
2. `mcpp test` — full unit suite green (must run `mcpp build` first;
`test_interface_protocol` drives the real binary).
3. e2e suite via `tests/e2e/run_all.sh`.
4. Benchmark main vs branch in this one worktree by switching branches, so
path, filesystem and toolchain fingerprint are identical and only source
content differs.

## Baseline captured (main, this worktree)

Cold `mcpp build` after `rm -rf target`, warm global dependency cache, 32 cores:
**68.85s / 55.14s / 64.12s**. Variance is ~25%, so the comparison needs
repeats and a median, not a single pair of numbers.
93 changes: 93 additions & 0 deletions .agents/tools/module-split/analyze.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
#!/usr/bin/env python3
"""Size the interface/implementation split: how much of each .cppm is a
namespace-scope function body (movable) vs inside a class body (needs
out-of-line member syntax) vs must-stay (template/constexpr/type)."""
import re, glob, sys, collections

def strip_for_scan(s):
"""Blank out string/char literals and comments so brace matching is sane.
Keeps byte offsets identical."""
out = list(s)
i, n = 0, len(s)
while i < n:
c = s[i]
if c == '/' and i + 1 < n and s[i+1] == '/':
j = s.find('\n', i)
j = n if j < 0 else j
for k in range(i, j): out[k] = ' '
i = j
elif c == '/' and i + 1 < n and s[i+1] == '*':
j = s.find('*/', i + 2)
j = n if j < 0 else j + 2
for k in range(i, j):
if s[k] != '\n': out[k] = ' '
i = j
elif c in '"\'':
q = c; j = i + 1
# raw strings R"(...)"
if q == '"' and i > 0 and s[i-1] == 'R':
m = re.match(r'"([^(]*)\(', s[i:])
if m:
delim = m.group(1)
end = s.find(')' + delim + '"', i)
j = n if end < 0 else end + len(delim) + 2
for k in range(i, j):
if s[k] != '\n': out[k] = ' '
i = j; continue
while j < n:
if s[j] == '\\': j += 2; continue
if s[j] == q: j += 1; break
if s[j] == '\n': break
j += 1
for k in range(i, j):
if s[k] != '\n': out[k] = ' '
i = j
else:
i += 1
return ''.join(out)

tot = collections.Counter()
per_file = []
for f in sorted(glob.glob('src/**/*.cppm', recursive=True)):
src = open(f).read()
scan = strip_for_scan(src)
lines = src.count('\n')
# find class/struct/union bodies at any depth: `struct X ... {` ... matching `}`
cls_lines = 0
for m in re.finditer(r'\b(?:struct|class|union)\s+(\w+)[^;{]*\{', scan):
# skip forward declarations (handled by the `{` requirement)
start = scan.index('{', m.start())
depth, i = 0, start
while i < len(scan):
if scan[i] == '{': depth += 1
elif scan[i] == '}':
depth -= 1
if depth == 0: break
i += 1
cls_lines += src.count('\n', start, i)
tot['lines'] += lines
tot['class_body_lines'] += cls_lines
per_file.append((lines, cls_lines, f))

print(f"total .cppm lines : {tot['lines']}")
print(f"inside class/struct : {tot['class_body_lines']} "
f"({100*tot['class_body_lines']/tot['lines']:.1f}%)")
print(f"outside class/struct : {tot['lines']-tot['class_body_lines']} "
f"({100*(tot['lines']-tot['class_body_lines'])/tot['lines']:.1f}%)")
print()
print("files with the most class-body code:")
per_file.sort(key=lambda r: -r[1])
for lines, cls, f in per_file[:12]:
print(f" {cls:5}/{lines:5} ({100*cls/max(lines,1):4.0f}%) {f}")

# namespace-scope `static` free functions (internal linkage -> cannot be
# declared in one unit and defined in another)
print()
stat_fns = []
for f in sorted(glob.glob('src/**/*.cppm', recursive=True)):
scan = strip_for_scan(open(f).read())
for m in re.finditer(r'^([ \t]*)static\s+(?!.*\b(?:constexpr|inline)\b)([\w:<>,& *]+?)\s+(\w+)\s*\([^;]*?\)\s*(?:const\s*)?\{', scan, re.M):
stat_fns.append((f, m.group(3), len(m.group(1))))
print(f"namespace/class-scope `static ... (...) {{` definitions: {len(stat_fns)}")
for f, name, ind in stat_fns[:15]:
print(f" indent={ind:2} {name:28} {f}")
32 changes: 32 additions & 0 deletions .agents/tools/module-split/bisect-member.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#!/usr/bin/env bash
# Binary-search the smallest number of outlined members of ONE file that makes
# the build ICE. Prints the boundary; member N (1-based) is the trigger.
#
# Usage: bisect_member.sh <file.cppm> <hi>
set -u
FILE="${1:?file}"; HI="${2:?upper bound}"
lo=0 # known good
hi=$HI # known bad
while [ $((hi - lo)) -gt 1 ]; do
mid=$(( (lo + hi) / 2 ))
git checkout b1563fe -- src/
git clean -fq src/
python3 .agents/tools/module-split/split.py --all --write >/dev/null
python3 .agents/tools/module-split/outline.py --write --limit "$mid" "$FILE" >/dev/null
rm -rf target/x86_64-linux-gnu/*/gcm.cache
mcpp build >build/bench/bm.log 2>&1
ice=$(grep -ac 'internal compiler error' build/bench/bm.log)
err=$(grep -ac ' error: ' build/bench/bm.log)
if [ "$ice" -gt 0 ]; then
echo " limit=$mid ICE -> bad"
hi=$mid
elif [ "$err" -gt 0 ]; then
echo " limit=$mid errors=$err (not an ICE) -> treating as bad"
grep -a ' error: ' build/bench/bm.log | head -2
hi=$mid
else
echo " limit=$mid clean -> good"
lo=$mid
fi
done
echo "BOUNDARY: $lo good, $hi bad -> member #$hi is the trigger"
24 changes: 24 additions & 0 deletions .agents/tools/module-split/clang-bisect.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/usr/bin/env bash
# Binary-search the smallest number of outlined members of ONE file that makes
# the clang build fail. Member N (1-based, in outline order) is the trigger.
#
# clang_bisect.sh <file.cppm> <hi>
set -u
FILE="${1:?file}"; HI="${2:?upper bound}"
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "$HERE/../../.." && pwd)"
cd "$ROOT" || exit 1
lo=0
hi=$HI
while [ $((hi - lo)) -gt 1 ]; do
mid=$(( (lo + hi) / 2 ))
out=$(LIMIT=$mid bash "$HERE/clang-variant.sh" "b$mid" "$FILE" 2>&1 | tail -1)
if echo "$out" | grep -q 'rc=0 errors=0'; then
echo " limit=$mid clean -> good"
lo=$mid
else
echo " limit=$mid FAIL -> bad ${out#*errors=}"
hi=$mid
fi
done
echo "BOUNDARY: $lo good, $hi bad -> member #$hi is the trigger"
Loading
Loading