fix(markdown): remove a fence's indentation before scanning its body

[P4suta]

Summary

A fenced code block indented under a list item is now scanned the way CommonMark reads it.
Before, the page below failed with unterminated shell heredoc and exit status 2, while the same block at the top level passed:

1. Step:

   ```sh
   cat <<'EOF'
   hello
   EOF
   ```

scan_markdown accepts a fence opener indented up to three spaces, but scan_markdown_fence handed the body to the embedded scanner with that indentation still on, so the child saw EOF.
CommonMark 4.5 takes up to as many spaces as the opener is indented off each line of the content.
The same hit Ruby's <<EOF, whose terminator also has to start its line.

• Rust: an indented fence's body is scanned over a de-indented MappedBytes copy.
  The child keeps its offset, so classification is unchanged, and merge_mapped now takes the copy's origin and maps every comment and diagnostic span back to the bytes on the page.
  A fence at column 0 takes the old path and makes no copy.
• OCaml: without_fence_indentation builds its own mapped_bytes, and the fence maps spans back through mapped_span.
• Tabs follow CommonMark 2.2: a tab that crosses the opener's column is taken off and the rest of its width comes back as spaces, each mapped to the tab.
  commonmark 0.31.2 and markdown-it 15.0.2 both read \tEOF under a three-space opener as EOF.

Three fixtures in hazards.json record the external facts, each checked against commonmark 0.31.2, markdown-it 15.0.2 and bash:

• markdown-indented-fence-takes-its-indentation-off-each-line: a line indented more than the opener keeps the rest, and one indented less loses all it has.
• markdown-fence-under-a-list-item-loses-the-item-indentation: the reported case.
• markdown-indented-fence-counts-a-tab-to-its-stop: \tEOF under a three-space opener is EOF, which <<- does not strip.

Compatibility and safety

Spans still name bytes on the page, so fix edits the same places it did.
The tests show that a comment in an indented fence is removed where it stands, and that a block comment across fence lines comes back as one span with its inner indentation and is removed as it would be from a C file of its own.
Only fences indented one to three spaces change behaviour, and only for languages where leading whitespace is lexically significant.
The scanner still does not model list items, so a fence under 10. sits four columns in and stays an opaque indented code block, as it was before.

Verification

• cargo test --manifest-path rust/Cargo.toml --workspace: all passed.
• cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --locked -- -D warnings: clean.
• cargo fmt --all --manifest-path rust/Cargo.toml -- --check: clean.
• opam exec -- dune runtest --root ocaml: 30 tests passed, including the new markdown-indented-fence.
• opam exec -- cargo xtask differential: 599 fixtures passed, 599 against a recorded expectation.
• python3 tools/validate_schemas.py (run in a venv with jsonschema), check_embedded_specs.py, check_directives.py --binary, gen_docs.py --binary --check: all passed.
• ocomment over this repository: 0 findings.
• cargo xtask preflight (full): 30 of 30 steps passed.
• An ad-hoc Rust/OCaml sweep of 32,000 random Markdown documents with indented fences, tabs, CR and CRLF across 25 fence languages: no disagreement.

Two pre-existing issues this change does not touch:

• With # -*- coding: utf-8 -*- on the first line of a python or ruby fence, OCaml classifies it as encoding and Rust as line.
  That happens at column 0 as well, because the OCaml child scans its body from offset 0.
• a_reflowed_source_is_still_the_same_comments failed once on a random seed: Rust ////: |\n//// reflows into a /// doc comment.
  It reproduces on origin/main, and it passed on the reruns.

Checklist

• Observable behavior has tests or fixtures.
• Rust and OCaml expectations agree when the shared scanner contract changes.
• Documentation and generated embedded assets are updated where needed.
• The change does not include credentials, build output, or unrelated edits.
• Breaking behavior and migration steps are called out explicitly.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: d9348939-7d0b-425f-9b01-ea92c42b1f2e

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}