fix(markdown): hide math delimiters from comrak instead of patching characters

[PathGao]

Fixes #174, #197, #177 — one bug reported three times.

Stacked on #389 (fix/line-count-preserving-transforms). Merge that first; this branch registers its new step in the line-contract registry #389 introduces.

The root cause

comrak keeps running CommonMark inline rules inside math delimiters, so the formula KaTeX finally receives has already been rewritten. Verified against the current baseline:

issue	input	what comrak hands KaTeX
#174	$\bar{b}_{1} + \bar{b}_{2}$	$\bar{b}<em>{1} + \bar{b}</em>{2}$
#197	a &= b \\ in an aligned block	a &amp;= b \ — every row separator eaten, block collapses to one line
#177	$$\mathbf{100\%}$$	$$\mathbf{100%}$$ — bare % opens a TeX comment

#174's reporter noticed "only the first subscript can use {}" (a lone _ cannot pair) and "a space before it fixes it" (a space makes the _ non-left-flanking). Both observations are exactly this mechanism.

protect_display_math_underscores covered the intersection of two narrow cases — only _, only $$ — and settled none of the three. Patching one character class at a time cannot win, because Markdown has no business parsing TeX. The whole span is now masked before comrak runs and restored afterwards, and that function is deleted.

The inline $…$ rule is not invented here

KaTeX's own auto-render refuses to make this call. contrib/auto-render.js ships $$…$$, \(…\), \[…\] and the AMS environments, with $…$ commented out in the source and a note that LaTeX uses it but it ruins the display of ordinary $ in text. Upstream's position is that single-$ needs product-specific disambiguation.

Markpad already made that decision, in the frontend — convertInlineMathDelimiters / findInlineMathEnd in markdown.ts decide what a user actually sees rendered. The backend now uses that rule:

• an opening $ may not be escaped, may not follow an unconsumed $, and may not be followed by whitespace;
• the first candidate closer decides — a candidate preceded by whitespace or followed by a digit abandons the span rather than being skipped. That is the currency guard: in $100 and $200 the only candidate closer is preceded by a space, so nothing is math;
• $$ is tried before $, with an adjacent-span flag so $a$$b$ works.

No frontend change was needed — the backend moved to the frontend's rule, not the other way round. That direction is deliberate: the frontend's rule is already shipped and existing documents depend on it; the backend's was new and nothing depended on it. Move the side with no installed base.

It is also aligned to what the frontend can physically see, not just what it computes:

Frontend fact	Backend consequence
hardbreaks puts each source line in its own text node	a span never crosses a line break
processInlineMath skips CODE/PRE subtrees	a span never crosses an inline code span
processDisplayMathBlocks only renders the block form	multi-line $$ pairs only when both delimiters sit alone on their line

The mask

MPMATHMASK<n>E, one token per line of a span.

• Plain ASCII, not a private-use character. The previous \u{E000} sentinel is unsafe in a link destination — comrak percent-encodes it, so [t](http://x/$a$) came back as http://x/%EE%80%80…. [A-Z0-9] survives text nodes, alt values and hrefs verbatim and carries no CommonMark meaning.
• Unique by construction: the prefix grows until it does not occur in the document, checked case-insensitively.
• Line count preserved (fix(markdown): keep every preprocessing step on the same source line #389's contract): one token per line, indentation left outside the token so a formula in a list item keeps belonging to it.
• Heading anchors: comrak derives heading ids from rendered text, so a masked span also appears lowercased inside id= and href="#…". Restore puts the anchorized source there instead — without this, [[#A heading with $x_1$]] broke silently.
• Restore re-escapes exactly the four characters comrak escapes in a text node (&<>" — established empirically; comrak does not escape '), which the frontend undoes via textContent.

Code regions are excluded from both openers and closers, and a multi-line $$ search aborts on any code region. The old parity hazard — a lone $$ inside a fence flipping the odd/even split for the rest of the document — is gone by construction, because there is no parity any more.

The invariant is pinned, not remembered

Correctness here depends on backend and frontend recognising the same spans, and nothing enforced that. scripts/mathDelimiterCorpus.json — 24 cases of {name, markdown, html, math[]} — is what both are now asserted against:

• markdown and math are hand-authored: they state what should happen, so a corpus that merely records current behaviour cannot occur. Neither implementation is the reference.
• html is a live capture of real convert_markdown output, and a Rust test asserts it still matches — that is what stops the fixture from rotting.
• The TS half runs the real processMarkdownHtml, and its extractor re-implements nothing: it reads the frontend's verdict only from artifacts the frontend alone produces — \(…\), emitted only by convertInlineMathDelimiters, and data-math-source, set only by processDisplayMathBlocks. An extractor that parsed $…$ itself could agree with a broken implementation.
• A further test requires ≥10 must-be-math and ≥10 must-not-be-math cases, so neither "recognise nothing" nor "recognise everything" can pass.

Both directions proven red:

Mutation	Result
frontend: drop the "closer may not be followed by a digit" guard	TS test fails — prices run together: the frontend and the contract disagree about what is math
backend: drop the same guard	Rust test fails — left: [("inline", "100")] right: []
any backend change that alters rendering	live-capture test fails, with instructions to replace html and leave markdown/math alone

Misjudgement guards

Treating prose as math is worse than not protecting it, so that was the primary constraint. Every one of these asserts the input is not math and renders normally: $100 and $200 · $5 · $5–$10 · $100$200 · a lone $ · a trailing $ · $ 5 · \$100 · $$ inside a fence · `$a$` · a span reaching across an inline code span · a span reaching across a line break · prices on consecutive lines · an unpaired $$ in one paragraph vs another two paragraphs later.

Falsification of the three issues (implementation reverted, tests kept): 117 pass / 5 fail → 123 pass / 0 fail.

npm run check   432 files, 0 errors, 0 warnings
npm test        403 / 403
cargo test      123 / 123
cargo clippy    3 warnings, identical to baseline

Not covered

• A same-line $$x$$ mixed into prose is passed through verbatim by convertInlineMathDelimiters, so the frontend's verdict on it is not observable from the test without re-implementing the rule — which would defeat the point. Display cases in the corpus use the block form, whose verdict surfaces as data-math-source. That sub-case is guarded on the Rust side only, and the corpus header says so.
• \(…\) / \[…\] are deliberately not handled. Same root cause, but CommonMark eats the backslash before the frontend sees a delimiter, so they have never worked at all; masking them would start claiming text no released version treated as math. Marked in the code as a decision, not an oversight.
• A paired pair of backticks inside a span still wins as a code span — the frontend cannot pair delimiters across a <code> element either, so both sides decline together.
• A blank line inside a $$ block ends the CommonMark block and ends the search; the frontend cannot render across paragraphs either.

🤖 Generated with Claude Code

[PathGao]

Rebased onto 72d63db; the conflict is gone.

Cause: this branch was stacked on #389, which was squash-merged. The squash produced a new SHA, so the branch still carried #389's original commit and git saw the same changes twice. git rebase dropped it by patch-id (one commit in, one commit out), leaving just this change on top of current master.

Its dependency on #389 is satisfied now — the line-contract registry this PR registers mask_math_spans in is on master.

npm run check   432 files, 0 errors, 0 warnings
npm test        421 / 421
cargo test      123 / 123