Roadmap to v2.0: Deep mode is a coverage feature, and R1 is shipping the backlog

[ojassug]

Docs only. No code changes, no behavioural change, and no test reads any of these files from disk (the filename matches under test/ are comment citations), so the suite is unaffected.

Three commits: the design spec, the roadmap rewritten to match it, and the decision entry plus a CLAUDE.md sync.

What this decides

v2.0.0 is tokendamper-deep — an opt-in tree-sitter backend that makes the supported-language list stop being hand-written. It was "Enterprise Gateway, Remote MCP & Proxy Guardrails".

Elision reduces 4 of 17 probed languages; every other bucket is 0.00%. Each of the four cost a hand-written lexer, a symbol extractor and a region scanner — in that order for the reason §56 measured — and roughly 1,400 lines apiece. One grammar answers all three questions: declaration nodes are the symbols, ERROR/MISSING nodes are the validity check, body node byte ranges are the regions.

Why not the Gateway: that section already contained the argument against itself. A Prometheus endpoint on a pass-through that saves nothing cross-turn instruments nothing, and invariant 8 has not moved. M7 (§54) fixed the half that was fixable. The three ecosystem items are held and listed, not deleted — MCP-over-Streamable-HTTP is named as having no premise problem and being the strongest candidate for the release after 2.0.

Two payoffs rejected, with the measurements that rejected them

Recorded in §75 so the next session doesn't re-derive them and reach the opposite answer.

• "Deep reduces more on the four we already have." The lexer is not the binding constraint. Go's fallbacks are 18 of 20 CONSTRAINT_DIRECTIVE_LOST; TypeScript's are 15 of 62, the same gate at the same rate. Neither is a parse failure. Built on this argument it would be BM25 and MMR a third time.
• "Deep makes validation a real syntax guarantee." §46's refusal to wire ts.createSourceFile is not reversed. The Fast path's claim stays bracket/quote integrity and validator-guarantee.test.ts stays as written.

The ordering

R1 → R2 → R3 → v2.0.0, named rather than numbered (§53).

R1 blocks everything, and it is where the project actually is. npm view tokendamper version returns 1.7.2 while tags run to v1.7.3, and [Unreleased] holds the whole 2026-08-30 security remediation including S-04's behavioural change. That is the failure the release skill exists to prevent, live again.

R2 precedes the feature because R4's whole claim is a number. The instrument producing it has a known bias — the constraint gate discards ~24% of files on both measured languages for a reason unrelated to any grammar — and no time axis at all, since stageDurationsMs is per stage and there is no per-file wall clock anywhere. This is §56's ordering argument pointed at measurement instead of safety.

R3 exists because a backend has to be checked where checking is still possible. Its control is staged, and that is a correction to the obvious design: byte-identity is right for the symbol and validator steps and wrong for regions, because a parser legitimately finds better spans than a lexer and demanding identity there forbids the improvement the feature exists for.

Stale facts fixed while auditing

• CLAUDE.md said "v1.7.3 shipped 2026-09-01". It was tagged and never published — the file carries "a tag in this repo does not imply a registry version" twenty lines below.
• ROADMAP.md's max_audit.md section still read "Unreleased — holds no number until it ships" for work that shipped as v1.6.0 on 2026-08-16.
• The "candidates whose preconditions hold" list still named per-item drift (closed unbuilt, §51) and sub-region elision (shipped as v1.4.0, §50).
• cache_control injection was moved to Milestone 8, not dropped with the section that housed it — it shares that milestone's exact-tokenizer precondition and was being tracked in two places.

What is not established

Stated in §9 of the spec and §75's closing section. The load-bearing ones: no language beyond the four has had its elidable ceiling measured; web-tree-sitter's per-process init cost is unmeasured and could make Deep unusable at the CLI while fine at the Gateway and MCP; and regions are the uncertain seam — that one grammar supplies all three is an argument from node types, not a measurement.

🤖 Generated with Claude Code