Tree-sitter as the language-extractor upgrade path

[razakadam74]

Today gitmsg uses regex per language to pull symbols from the diff. It works, but has a ceiling — the C# delegate row took three rewrites and tree-sitter would have made that free.

Filing now so the upgrade path lives somewhere public.

Keep:

• zero runtime deps
• <100ms cold start
• single-binary build via bun build --compile

Open questions:

• native tree-sitter vs WASM web-tree-sitter — which one survives the constraints above?
• gitmsg only sees the diff hunk today; tree-sitter wants full files. Do we git show both sides and diff symbol tables, or is there a hunk-friendly approach I'm missing?
• migrate one language at a time, or all at once? Keep regex as fallback when no grammar ships?

Non-goals: not blocking 1.0, not removing the regex extractors.

Credit to u/lurking_bishop on r/commandline for the nudge and to https://mergiraf.org/ for the prior art.

[razakadam74]

Quick spike report before this gets cold.

Ran two probes in a sandbox.

1. tree-sitter-gitcommit via web-tree-sitter@0.25 — runtime loaded, but external C scanner ABI mismatch with the vendored .wasm gave back (source (subject (overflow))) instead of a parsed prefix tree. The _conventional_type external scanner silently no-matched. Vendoring third-party .wasm is fragile when the grammar uses external scanners — real migration would mean rebuilding grammars against a pinned runtime.

2. tree-sitter-typescript via tree-sitter-wasms@0.1.13 — parsed src/analyze/breaking.ts cleanly. 5-line query pulled only the exported symbols (detectBreaking, not the un-exported formatList). Numbers:

• Parser.init(): 7.3 ms
• Language.load (2.3 MB wasm): 6.7 ms
• first parse: 8.86 ms · warm: 0.42 ms
• cold path: 23.8 ms — well under the 100 ms budget
• stress on full src/ (11 files, 21 KB): 34.6 ms total, 3.15 ms/file

Bonus: exported is a built-in tree property. No per-language visibility heuristic — current regex extractors each encode it differently (^public, ^[A-Z], !startsWith('_')).

Net call: not migrating yet. Spike proved feasibility, not necessity. Audited recent banked extractor bugs — only the C# delegate row would have been free with tree-sitter. The scope-detection issues (#46, #54), the NEUTRAL_PATTERN fix (#64) — all unrelated to extractor quality. One banked example doesn't justify ~1–2 weeks of migration + ~10 MB binary growth from 5 grammars + new build-pipeline failure modes.

Parking it. Triggers to revisit:

• Detect signature changes of exported functions as breaking changes #61 needs full ASTs to detect signature changes, or
• 2+ wonky-message reports trace back to extractor limits, or
• a new-language regex ladder grows past ~60 lines.

Any one flips this from "tracked option" to "do it." Until then regex stays — and #61 is the real test of whether it hits its ceiling.