bug(perf): SessionEnd hook spawns N concurrent full-rescan pipelines — load 162 on 8-core M1

[LiuShiyuMath]

Symptom

After #459 was filed, the user ran their toohot monitor and saw loadavg jump from 88 → 162.31 / 130.28 / 98.34 on the same 8-core M1 — a 20× oversubscription. Top processes were no longer claude.exe --bg-spare; they were 6 concurrent ~/.teamagent/hooks/bin-session-end.cjs invocations at 17-37% CPU each.

This is a second, distinct TeamBrain-side bug layered on top of the upstream bg-spare leak. Unlike #459 this one is in our code.

Root cause

bin-session-end.cjs (compiled from packages/cli/src/bin-session-end.ts):

1. Foreground path receives a SessionEnd event, writes the payload to a /var/folders/.../teamagent-session-end-<ts>-<rand>.json tmp file, unconditionally spawns a detached child re-execing itself with TEAMAGENT_SESSION_END_PIPELINE=1, then returns.
2. Detached child path calls runFullRescanPipeline(input) (packages/cli/src/bin-stop.ts:853) → runStopPipeline(input, { fullRescan: true, modeTag: "full" }). This runs the full embedder rescan + analyze/calibrate/compile chain — described as unbounded in bin-session-end.ts:84-85 ("Pipeline is unbounded — the harness kills us at its own ~300s timeout if needed").
3. No concurrency gate on the spawn. A STOP_PIPELINE_LOCKS_DIR per-cwd singleton lock exists in bin-stop.ts:270 ("per-project singleton lock for detached pipeline child") with full stale-lock handling, but bin-session-end.ts:133 does not consult it before spawning. The only lock that fires inside the child is writeStopLock(cwd) at bin-stop.ts:421 which is comment-marked "best-effort — statusline will simply not show the indicator", i.e. a status flag, not a serializing lock.
4. Result: when a single Claude Code daemon shuts down N worktree sessions at once, or /clear is issued in N tabs, N detached children all win the race and start full rescans in parallel.

Reproduction

# Have N ≥ 4 worktree sessions of any teamagent-initialized project open
# (cwd in this report: /Users/m1/projects/metrixMarkets).
# Trigger SessionEnd in all of them within a short window:
#   - close the windows, or
#   - run /clear in each, or
#   - kill the parent claude.exe daemon
#
# Watch:
ps -axwwwo pid,etime,pcpu,command | grep "bin-session-end.cjs" | grep -v grep
sysctl -n vm.loadavg
ls -lat /var/folders/*/T/teamagent-session-end-*.json 2>/dev/null | head

Expected: N concurrent detached children, each 17-36% CPU, loadavg climbs through 100+ on 8-core hardware, UI starvation (WindowServer 30%+).

Evidence captured 2026-05-14 11:40 local

loadavg                   = 162.31 / 130.28 / 98.34   # 8-core M1 → ~20× oversubscription
pmset -g therm            = normal (NOT thermal)
ncpu                      = 8

8 SessionEnd fires within a 223 ms window (timestamps from tmpFile names):
  ...1778729968306-d5xnxnrrsko.json
  ...1778729968333-i5vyjb0ltkq.json
  ...1778729968435-oybwca4fbkd.json
  ...1778729968460-jpqw8ndqwla.json
  ...1778729968477-b5ptxz93wi5.json
  ...1778729968503-32cr4rmql4l.json
  ...1778729968508-trn8wwy6jph.json
  ...1778729968529-53k07tflaia.json

A few seconds later, 9 concurrent children still running:
  PID    PPID    %CPU   COMMAND
  26429    1     25.9%  node bin-session-end.cjs <tmpFile-1>.json
  26440    1     35.9%  node bin-session-end.cjs <tmpFile-2>.json
  26441    1     30.0%  node bin-session-end.cjs <tmpFile-3>.json
  26442    1     31.0%  node bin-session-end.cjs <tmpFile-4>.json
  26443    1     32.4%  node bin-session-end.cjs <tmpFile-5>.json
  26444    1     17.3%  node bin-session-end.cjs <tmpFile-6>.json
  27345    1     19.0%  node bin-session-end.cjs <tmpFile-7>.json
  (+ 2 more detached children)

All cwds in the payloads = /Users/m1/projects/metrixMarkets
(so they would all collide on the SAME per-cwd lock if the lock were checked)

Hook binary size: bin-session-end.cjs = 7.94 MB / 52,602 lines (tsup bundle).
Each fire pays the full ~8 MB JS parse cost in a fresh node process.

Why the lock isn't gating

bin-session-end.ts:69-147 foreground handler:

// ❌ No lock check before this spawn
const child = spawn(process.execPath, [selfPath, tmpFile], {
  detached: true,
  stdio: "ignore",
  cwd: ctx.cwd,
  env: { ...process.env, [SESSION_END_ENV_KEY]: "1" },
  windowsHide: true,
});

The infrastructure for the gate already exists — bin-stop.ts:295 has ${cwdLockKey(cwd)}.stop-pipeline.lock with stale-lock handling at bin-stop.ts:304+. It just isn't called here.

Proposed scope (needs grill before driver dispatch)

• Gate: have bin-session-end.ts foreground check STOP_PIPELINE_LOCKS_DIR/<cwdLockKey>.stop-pipeline.lock (existing, in bin-stop.ts) before spawning. If a live (non-stale) entry exists for this cwd, drop the new SessionEnd instead of spawning. This is the per-cwd singleton the code already documents wanting.
• Cross-cwd backpressure: the per-cwd lock doesn't help when N different worktrees fire at once. Add a global semaphore (~/.teamagent/locks/global-pipeline.sem with N ≤ 2) that the foreground checks before spawn; if over the cap, drop or queue (drop preferred — incremental rescan from the next session covers the gap).
• Bundle slim-down: bin-session-end.cjs is 7.94 MB. Lazy-load runFullRescanPipeline so the foreground spawn path doesn't have to parse the entire embedder/analyze/calibrate dep graph. Foreground path only needs spawn + 30 lines of envelope.
• Observability: emit hook-session-end.dropped AttributionEvent when the lock gates a fire, so the user can see how often the storm is being absorbed.
• Regression test: packages/cli/src/__tests__/bin-session-end.concurrency.test.ts fires N SessionEnds within 100ms in a temp project and asserts no more than 1 (or N ≤ semaphore cap) child enters runFullRescanPipeline.

Related

• Issue #459 — the bg-spare worker leak from upstream @anthropic-ai/claude-code. Independent root cause; both can fire on the same machine and compound. (bug(perf): Mac overheats — 8 claude.exe bg-spare worker pairs leak (loadavg 88 on 8-core M1) #459 contributes baseline load 60-90; this issue spikes it to 160+.)
• docs/adr/0013-inner-loop-on-ci.md — same scheduler-overload pathology (loadavg ≫ ncpu, thermal flag normal), different trigger (vitest fork burst). 3rd documented variant in this repo.
• docs/debugging/toohot-many-bg-spare-workers.md — needs follow-up entry for "session-end storm" as a 4th trigger of the same pathology.
• packages/cli/src/bin-session-end.ts:84-85 — code comment explicitly notes the pipeline is unbounded.
• packages/cli/src/bin-stop.ts:247-300 — STOP_PIPELINE_LOCKS_DIR infrastructure already in place, just not wired into the SessionEnd foreground path.

Notification

User asked for @libz Slack mention when this is fixed. Tracked in the slack DM linked from #459.

---

Filed via /investigate skill (continuation) on 2026-05-14.

[LiuShiyuMath]

为什么 Mac 会变烫 —— 非技术版诊断说明

这条评论是写给不写代码的人看的过热问题诊断。本机也生成了一份排版精致的 HTML 版（/talk-html，思源宋体 + SVG 图，已在维护者的 Chrome 打开）；下面是同等内容的 markdown 版，方便直接在 issue 里读。

01 · 它根本没"热"

机器发烫、风扇狂转、卡顿 —— 但这不是温度问题。整个过程内核的「热压力」指标一直是 normal，CPU 实际使用率也不高。炉子没烧红，厨房只是乱成一团。

打个比方 —— 餐厅厨房：你的电脑有 8 个 CPU 核心 = 厨房 8 位厨师。正常时订单一桌一桌来。但这次两百桌同时催菜 —— 厨师光是听吼声、转身、被打断就耗光了精力，一道菜都端不出来。厨房没着火，只是彻底瘫了。技术名叫「调度过载」。

02 · 一种病，三个触发源

三次表面症状一样（系统负载远超核心数、CPU 没满、温度正常），原因各不同：

触发源	时间	原因	峰值负载	状态
A	2026-05-10	4 个窗口同时跑全套测试	274（8 核，超载 34×）	✅ 已修：测试搬到 CI
B	2026-05-12	20 个用完没回收的后台 AI 助手进程堆积	61	◑ #459（上游工具问题，非 TeamBrain 代码）
C	2026-05-14	关窗口的「会话收尾」任务一拥而上	162–204	◑ #460（本 issue）+ 本机已修

03 · 这次（触发源 C）到底哪里出了岔子

TeamBrain 里有三处代码会启动「重型收尾任务」：

• bin-stop.ts —— 装了正经的锁（STOP_PIPELINE_LOCKS_DIR，同一时间只准跑一个），见 bin-stop.ts:1016 / :1090 / :986。
• bin-session-end.ts:133 —— 没接这道锁，无条件 spawn。
• bin-pre-compact.ts:97 —— 同样没接这道锁（探查代码时新发现的「孪生」bug，和 session-end 一模一样）。

打个比方 —— 没装红绿灯的路口：bin-stop 那个路口装了灯，车流有序；session-end 和 pre-compact 这两个路口压根没装灯 —— 一口气关好几个窗口，等于好几个方向的车同时冲进没灯的路口，谁也走不动。

04 · 本机已应用的修复

给没灯的路口装一道红绿灯（原子锁闸门，bin-session-end-gated.cjs，详见 #461）。「原子」= 哪怕十几个请求同一毫秒一起冲进来，也只有一个能拿到通行权，没有缝隙。

两条规则：

• 每个项目文件夹，同一时间最多 1 个收尾任务（掐死「一口气关好几个同项目窗口」的风暴）。
• 全机器加起来，同一时间最多 2 个。

其余请求不报错、不卡住关窗口 —— 安静放掉，下次会话自然补上。

压力重测：18 个请求一起冲 → 闸门挡下 15 个，只放行 1–2 个，系统负载纹丝不动。

05 · 修好了吗 —— 修好了

修复前	修复后
系统负载 204 —— 8 核机器严重超载	系统负载 4–5 —— 健康，从容运转

闸门已装在本机且重启后依然有效。永久修复在 #459 / #460 / #461 追踪 —— 正式版本合并后所有 TeamBrain 用户自动拿到。

06 · 下次怎么自己看出来（不用懂代码）

• 打开「活动监视器」看 CPU 栏 —— 如果 CPU 其实没跑满，但机器又烫又卡，十有八九是「调度过载」，不是真过热。
• 真过热的修法是降温；调度过载的修法是关掉失控进程 —— 方向完全不同。
• 看到一长串名字相同、数量异常多的进程，基本就是某个「没装红绿灯」的地方在发车。

---

给本 issue 的补充

探查代码时确认：bin-pre-compact.ts:97 有和 bin-session-end.ts:133 完全相同的「缺锁」bug —— in-source 正式修法应当同时覆盖这两条路径，复用 bin-stop.ts 已有的 pipelineLockPath() 锁基础设施，而不只是修 session-end 一条。本机的 bin-session-end-gated.cjs 目前只 gate 了 session-end；pre-compact 仍是裸的。

本评论由 /internal-comms + /talk-html 生成 · 关联 #459 #461

[github-actions]

⚠️ FIXEDFLOW (warn-only): 24 小时内未加 grill-ready label。当前为 7 天软提醒期；切到 enforce 后会自动 close。详见 docs/FIXEDFLOW.md。

[github-actions]

⚠️ FIXEDFLOW (warn-only): 24 小时内未加 grill-ready label。当前为 7 天软提醒期；切到 enforce 后会自动 close。详见 docs/FIXEDFLOW.md。

[github-actions]

⚠️ FIXEDFLOW (warn-only): 24 小时内未加 grill-ready label。当前为 7 天软提醒期；切到 enforce 后会自动 close。详见 docs/FIXEDFLOW.md。

[github-actions]

⚠️ FIXEDFLOW (warn-only): 24 小时内未加 grill-ready label。当前为 7 天软提醒期；切到 enforce 后会自动 close。详见 docs/FIXEDFLOW.md。

[github-actions]

⚠️ FIXEDFLOW (warn-only): 24 小时内未加 grill-ready label。当前为 7 天软提醒期；切到 enforce 后会自动 close。详见 docs/FIXEDFLOW.md。

[github-actions]

⚠️ FIXEDFLOW (warn-only): 24 小时内未加 grill-ready label。当前为 7 天软提醒期；切到 enforce 后会自动 close。详见 docs/FIXEDFLOW.md。

[github-actions]

⚠️ FIXEDFLOW (warn-only): 24 小时内未加 grill-ready label。当前为 7 天软提醒期；切到 enforce 后会自动 close。详见 docs/FIXEDFLOW.md。