Skip to content

README.md

Latest commit

 

History

History
284 lines (206 loc) · 37 KB

README.md

File metadata and controls

284 lines (206 loc) · 37 KB
Raw

markdown-codec

GitHub npm npm version CI

Hand-written CommonMark+GFM ⇄ DocumentTree codec, built on document-schema.js.

The same "hand-write the format instead of wrapping a third-party library" bet as pdf-codec, aimed at CommonMark and GFM. No micromark/remark/marked/markdown-it/commonmark/mdast/unified/turndown/showdown dependency (enforced by eslint no-restricted-imports). Runtime dependencies: document-schema.js (the shared pivot) and zod. readMarkdown/writeMarkdown read and write that pivot's tree-form DocumentTree; readMarkdownContent/writeMarkdownContent read and write the flat ContentDocument underneath it — the same model documents.js builds docx/pptx/odt/odp conversions around. See Two encodings.

graph TD
    schema("document-schema.js")
    ooxml("ooxml.js")
    odf("odf.js")
    pdfcodec("pdf-codec")
    bytecodec("byte-codec")
    mdcodec("markdown-codec")
    documents("documents.js")
    mcp("document-mcp")
    cli("document-cli")

    schema --> ooxml
    schema --> odf
    schema --> pdfcodec
    schema --> bytecodec
    schema --> mdcodec
    schema --> documents
    bytecodec --> pdfcodec
    ooxml --> documents
    odf --> documents
    pdfcodec --> documents
    bytecodec --> documents
    mdcodec --> documents
    documents --> mcp
    pdfcodec --> mcp
    documents --> cli
    odf --> cli
    pdfcodec --> cli

    click schema "https://github.com/ExaDev/documents.js/tree/main/packages/document-schema.js" "document-schema.js"
    click ooxml "https://github.com/ExaDev/documents.js/tree/main/packages/ooxml.js" "ooxml.js"
    click odf "https://github.com/ExaDev/documents.js/tree/main/packages/odf.js" "odf.js"
    click pdfcodec "https://github.com/ExaDev/documents.js/tree/main/packages/pdf-codec" "pdf-codec"
    click bytecodec "https://github.com/ExaDev/documents.js/tree/main/packages/byte-codec" "byte-codec"
    click mdcodec "https://github.com/ExaDev/documents.js/tree/main/packages/markdown-codec" "markdown-codec"
    click documents "https://github.com/ExaDev/documents.js" "documents.js"
    click mcp "https://github.com/ExaDev/documents.js/tree/main/packages/document-mcp" "document-mcp"
    click cli "https://github.com/ExaDev/documents.js/tree/main/packages/document-cli" "document-cli"

    style mdcodec fill:#f9a825,stroke:#333,stroke-width:3px
Loading

Status

The scanner, block parser, and inline parser are complete hand-written implementations of CommonMark 0.31.2's two-phase algorithm plus GFM's table/strikethrough/autolink/task-list-item extensions and GitHub's footnotes (see Footnotes). Both encodings' read/write pairs and both z.codec() pairs are wired and real. Conformance suites measure the full public surface (readMarkdownContent → writeMarkdownContent → reparse → render to HTML) against the vendored CommonMark/GFM corpora — see Fidelity for why the rate is below 100% (dominated by what ContentDocument can represent, not parsing gaps).

Getting started

Requires Node.js >=20 and pnpm 11.6.0 (pinned via packageManager in package.json).

pnpm install

Install as a dependency in another project:

pnpm add markdown-codec
# or
npm install markdown-codec

Published to npmjs.org via OIDC trusted publishing. dist/ is gitignored like every other package's build output: the published tarball carries the release build, and a workspace consumer gets dist/ from turbo's ^_build ordering, never from the repository.

Usage

Reading and writing markdown text:

import { readMarkdown, writeMarkdown } from "markdown-codec";

const { documentPackage, diagnostics } = readMarkdown(
  "# Title\n\nSome **bold** text with a [link](https://example.com).",
  {
    frontMatter: true, // parse a leading YAML front matter block into the package's metadata
    footnotes: true, // recognise [^label] markers and [^label]: definitions (default; see Footnotes)
    images: (destination) => undefined, // a synchronous MarkdownImageResolver port for non-data: URI images
  },
);

const markdown = writeMarkdown(documentPackage, {
  bulletListMarker: "-",
  emphasisMarker: "_",
  frontMatter: true, // emit the package's metadata back out as a leading front matter block
});

documentPackage is a DocumentTree — document-schema.js's tree form, with a minted styles table (see Two encodings). The field is named documentPackage rather than package because package is a reserved word in strict mode, so const { package } = readMarkdown(src) would not parse.

Both accept an optional signal (AbortSignal) and sink (MarkdownDiagnosticSink, called once per recoverable issue or construct-mapping gap — see Gotchas). writeMarkdown throws MarkdownUnsupportedDocumentKindError for a package whose kind is not 'wordprocessing', checked before flattening so every non-'wordprocessing' package reaches it the same way regardless of what else about that package would have failed document-schema.js's own flattenTree. A 'wordprocessing' package can still fail to flatten — a group carrying a style reference the package's own styles table has no entry for — and that failure surfaces as MarkdownPackageFlattenError, not a bare Error from the dependency. A DocumentTree's own layers/attachments/destinations/pages tables have no flat-ContentDocument home to land in; writeMarkdown reports one PACKAGE_TABLE_DROPPED diagnostic per non-empty table it finds rather than dropping them without a trace. The definitions table is the one exemption: this package's own link-tenant entries (what readMarkdown splices there from the source's reference definitions) render back out as [label]: destination "title" lines, so only a table holding foreign tenants reports.

The same round trip as a schema-validated z.codec() pair, mirroring pdf-codec's pdfCodec:

import { z } from "zod";
import { markdownCodec, MarkdownBytesSchema } from "markdown-codec";

const documentPackage = z.decode(markdownCodec, bytes); // throws if bytes are not well-formed UTF-8
const bytes2 = z.encode(markdownCodec, documentPackage);

MarkdownBytesSchema checks for well-formed UTF-8. The no-options form only; readMarkdown/writeMarkdown remain the entry points for an AbortSignal or diagnostic sink. Every construct-mapping gap reports through the sink as a stable code (e.g. md/nested-emphasis-flattened) — see MarkdownDiagnosticCodes and Gotchas.

Two encodings: DocumentTree and ContentDocument

document-schema.js states one document in two shapes, and owns the transform between them: the flat ContentDocument every codec's lowering pipeline actually builds, and the tree-form DocumentTree a serialised artefact carries — sections, headings, lists, and construct boundaries as real nested groups, plus a styles table minted over repeated property tuples. assembleTree goes flat → tree (decompose then factorStyles), flattenTree goes tree → flat. Only one direction is a genuine round trip: flattenTree(assembleTree(document)) reproduces document exactly, for any ContentDocument this package's own read side produces (checked against the full CommonMark and GFM conformance corpora, not just a hand-picked fixture — see src/conformance.test.ts/src/gfm-conformance.test.ts's own "tree pair matches the flat pair" suite). assembleTree(flattenTree(documentPackage)) does not, in general, reproduce documentPackage — a package carrying definitions/layers/attachments/destinations/pages loses all of them on the way through flattenTree, which carries forward only metadata and symbolTable (see Gotchas).

This package exposes a read/write pair and a codec at each level. The unsuffixed names are the tree-form ones and are what to reach for by default — a codec is a construction site, so the tree is what a caller gets unless they ask for otherwise. The Content-suffixed names are the flat pair one level down, mirroring the readXlsx/readXlsxContent naming already in ooxml.js:

Level Read Write Codec Value type
Tree (default) readMarkdown writeMarkdown markdownCodec DocumentTree
Flat readMarkdownContent writeMarkdownContent markdownContentCodec ContentDocument

The tree pair is the flat pair with the transform composed on — readMarkdown is assembleTree over readMarkdownContent, writeMarkdown is flattenTree before writeMarkdownContent — plus the two tree-only carries the flat form has no root for: the source's reference definitions splice into documentPackage.definitions (link tenant, keyed by normalised label) and the verbatim front-matter block into documentPackage.source.frontmatter, both rendered back out by writeMarkdown ([label]: dest "title" lines after the body; the original front matter verbatim in place of the regenerated block). Sources carrying neither render identically to the flat pair, pinned in src/package.test.ts; a source carrying either renders its extra block, which is the point of reaching for the tree. Options, diagnostics, and error behaviour are identical at both levels.

Reach for the flat pair when composing a package boundary by hand (decompose/flattenTree directly, or factorStyles with your own minting policy), when feeding a ContentDocument-consuming builder such as documents.js's conversion pipeline, or when a layout stage needs to stamp frames onto content before it is decomposed. Everything else wants the tree.

import { readMarkdownContent, writeMarkdownContent } from "markdown-codec";

const { document } = readMarkdownContent(source); // a ContentDocument: kind, metadata, sections
const markdown = writeMarkdownContent(document);

Architecture

Modelled on pdf-codec's own layering, aimed at CommonMark+GFM instead of PDF:

  • src/diagnostics/ — three-tier diagnostic policy (throw/recover/degrade); MarkdownDiagnosticCodes names every code.
  • src/ast/ — markdown AST node types (document/block/inline union), Zod-first.
  • src/options/ / src/defaults/ — read/write options (GFM toggles, sink, AbortSignal, write-side style) and defaults.
  • src/scan/ — CommonMark line/character scanner, plus entity-table.ts (generated from assets/html-entities/entities.json).
  • src/block/ — CommonMark block-structure algorithm (open-block stack, continuation matching): paragraphs, headings, code blocks, block quotes, lists (incl. GFM task-list-item), thematic breaks, link references, footnote definitions, GFM tables.
  • src/inline/ — emphasis, code spans, links, autolinks, raw HTML, GFM strikethrough, footnote references, line breaks. link.ts and footnote.ts hold the label grammars the block phase shares.
  • src/html/ — raw HTML recognition (bounded rules, not a general parser) plus render.ts (conformance oracle; internal only). html-table.ts is the one exception to "never parsed as markup": a block-level HTML that is, in full, one well-formed <table> recognises straight to a real ContentTable instead of opaque text — see HTML-table fallback.
  • src/image/ — PNG/JPEG dimension reader and base64 codec, shared by src/lower/ and src/emit/.
  • src/shared/ — string-shape conventions src/lower/src/emit agree on (style-constants.ts, list-id.ts's opaque numId). Re-exported so documents.js's MarkdownEditor reuses the identical grammar.
  • src/lower/ — AST → ContentDocument lowering (thin adapter, not a second parser); top-of-file table maps each construct to its diagnostic gap.
  • src/emit/ — ContentDocument → markdown text emission, the structural inverse of src/lower. html-table.ts is src/html/html-table.ts's own write-side counterpart.
  • src/read.ts / src/write.ts / src/codec.ts — the public entry points at both levels: readMarkdown/writeMarkdown/markdownCodec over DocumentTree, and readMarkdownContent/writeMarkdownContent/markdownContentCodec over ContentDocument. The tree-form functions are thin compositions of document-schema.js's assembleTree/flattenTree onto the flat ones; no conversion logic of their own lives here.

Vendored assets

assets/ holds real, unmodified conformance corpora (each with a NOTICE.md recording source, version, licence). None is read at runtime: assets/html-entities/entities.json is compiled into src/scan/entity-table.ts, and the spec corpora are test-only. So package.json's "files": ["dist"] is correct.

  • assets/commonmark/ — CommonMark spec + corpus (652 examples), tag 0.31.2 (CC-BY-SA 4.0).
  • assets/gfm/ — GitHub Flavored Markdown Spec (CC-BY-SA 4.0).
  • assets/html-entities/ — WHATWG HTML5 named character reference table (BSD 3-Clause).

Build, test, and lint

pnpm build         # turbo run _build (tsdown -> dist/, ESM + CJS + .d.ts)
pnpm typecheck     # turbo run _typecheck _typecheck:node (dual tsconfig)
pnpm lint          # turbo run _lint (eslint . --fix --cache --max-warnings 0)
pnpm test          # turbo run _test (vitest run --project unit, incl. CommonMark/GFM conformance)
pnpm test:workers  # turbo run _test:workers (unit suite under the real Cloudflare Workers/workerd runtime)
pnpm test:watch    # vitest --project unit
pnpm test:coverage # turbo run _test:coverage (vitest run --project unit --coverage)
pnpm test:smoke    # turbo run _test:smoke (rebuilds dist/, verifies ESM/CJS parity + a real round trip per bundle)
pnpm test:corpus   # turbo run _test:corpus (optional, gitignored real-world sanity check -- see Fidelity)

To run a single test file: pnpm vitest run src/path/to/file.test.ts.

Conventions

  • Zod-first schema/type/guard, matching pdf-codec/documents.js: every model type inferred from its Zod schema.
  • No type assertions. Every loosely-typed value narrowed through a type guard or Zod parse at the boundary.
  • No markdown-parsing library dependency, enforced by eslint no-restricted-imports.
  • z.codec() for the round trip (markdownCodec, markdownContentCodec), matching pdf-codec's pdfCodec: each wraps the independently-tested read/write pair at its own level with automatic two-way schema validation (no-options form only).
  • Shrink-only conformance exclusion list. Every spec example the read → write → reparse → render pipeline does not reproduce byte for byte is named in src/test-support/conformance-exclusions.ts, with a test asserting it genuinely still fails — the list shrinks as gaps close, never quietly grows.
  • Conventional commits, enforced via commitlint + husky.

Gotchas and quirks

Every construct src/lower/src/emit cannot represent losslessly is a documented MarkdownDiagnosticCodes entry:

  • md/invented-page-geometry — no page concept in markdown; one ContentSection with A4 + 1in defaults (overridable). Fires once.
  • md/nested-emphasis-flattened — same-kind nested emphasis flattens to one run.
  • md/link-title-dropped — the one titled shape still dropping: a nested image (inside a link or emphasis) or an unresolved image. Every other title rides a link construct's title field — a run-level extent for an inline or reference link, a block-scoped marker pair around a resolved image (which also restores the image's original destination on the way out).
  • md/blockquote-container-skipped — a blockquote containing a heading anywhere in its subtree cannot carry its division construct (a marker extent may not open a heading scope), so that quote degrades to indent-only structure while the heading keeps its fidelity. Every other quote carries a division construct pair — exact container boundary and exact nesting depth, with the indent and Quote styleId kept as the materialised formatting.
  • md/list-item-block-unlisted — a table, resolved image, or display-math block in a list item cannot carry ContentListMembership (paragraphs only).
  • md/list-marker-type-conflict — a nested list whose marker type disagrees with its enclosing list's minted numId keeps the enclosing type (first-wins).
  • md/math-inline-preserved-as-text — inline \( \) math stays a Cambria-Math-marked raw-LaTeX run; display $$ math is a real embedded formula carrying the presentation layer.
  • md/image-unresolved — no resolver, undefined return, or non-PNG/JPEG bytes degrades to alt-text run.
  • md/raw-html-preserved-as-text / md/raw-html-dropped — raw HTML kept as literal text (default) or dropped; never interpreted. The preserved text's verbatim original quarantines as markdown residue on its node, and this package's own writer re-emits that residue as-is.
  • md/front-matter-key-unmapped — no YAML/TOML engine; only LayoutMetadata's own string/array/enum fields are recognised (every one of them except producer, which is PDF-writer-only), each as a flat key: value (or keywords: [a, b]) line. The verbatim original block rides the package-level residue table (readMarkdown's documentPackage.source.frontmatter), which writeMarkdown re-emits as-is.
  • md/heading-level-clamped — styleId beyond Heading6 (from another format) clamps to level 6 via document-schema.js's shared clampHeadingLevel().
  • md/heading-line-break-collapsed — a hard or soft break embedded in a heading's own runs (most commonly a setext heading read with more than one physical line) is promoted to setext output whenever the level admits one (1 or 2) and the break's own placement is safe to promote, the only markdown grammar that can hold a genuine line break inside heading text; a level 3-6 heading with the same shape has no such fallback and always collapses the break to a single space instead, as does a level 1/2 heading whose break placement is unsafe to promote (see md/heading-line-break-unsafe-for-setext below).
  • md/heading-line-break-unsafe-for-setext — a level 1/2 heading's own embedded break sits where promoting to setext would violate one of the three clauses packed into setext's own grammar sentence (spec 0.31.2, "Setext headings"): "one or more lines of text, not interrupted by a blank line, of which the first line does not have more than 3 spaces of indentation, followed by a setext heading underline. The lines of text must be such that, were they not followed by the setext heading underline, they would be interpreted as a paragraph: they cannot be interpretable as a code fence, ATX heading, block quote, thematic break, list item, or HTML block." Reported when promoting would leave a blank line immediately before the underline (a trailing break) or in the middle of the heading's own text (an interior break between two embedded breaks, or a break immediately followed by incidental whitespace-only content); when the heading's own first content line opens with 4 or more columns of space/tab indentation — CommonMark's own tab-stop rule counts a tab by the same 4-column stop, so a single leading tab is exactly as unsafe as 4 leading spaces — since that line would read back as an indented code block rather than heading text; or when a line AFTER the first would itself be read as one of the spec's six interrupting constructs (a code fence, ATX heading, block quote, thematic break, list item, or HTML block) instead of ordinary paragraph continuation text — reachable from a bold or strikethrough run that becomes empty immediately after a break, whose bare pair of emphasis markers spells as a thematic break (____/****) or a code-fence opener (~~~~). ATX collapses the break to a single space instead of promoting into any of these corrupt reparses. A break at the very START of the heading's own text stays eligible for setext, but that exemption covers only a single leading blank line, and only the blank line itself: it sits before the text-and-underline run even begins, so it reparses as ordinary inter-block whitespace ahead of the heading rather than corrupting anything. A second consecutive leading blank line is not exempt — CommonMark's own list-item grammar allows at most one, so a second closes a list item as empty and spills the heading out with its list membership lost, even though it would be harmless at top level or inside a blockquote — and the indentation clause above still applies in full to the line that follows the leading blank line, the heading's own first real content: a leading break immediately followed by 4 or more columns of indentation is refused exactly as an unbroken heading in the same shape would be, since promoting would read that following line back as an indented code block regardless of the leading break ahead of it.
  • md/heading-style-overridden-for-line-break — the setext promotion above overriding the effective headingStyle: 'atx' (this option's own default just as much as an explicit caller choice); that one heading renders as setext regardless because ATX cannot hold the embedded break at all.
  • md/adjacent-links-merged / md/code-span-as-monospace-run — same-destination adjacent links merge; monospace runs emit as code spans.
  • md/paragraph-indent-dropped — indentLeftPt without a recognised styleId; indent dropped, paragraph renders.
  • md/list-numid-fallback — a foreign or absent numId (depth-only ContentListMembership) falls back to a plain bullet list.
  • md/raw-html-preserved-as-text — see the raw-HTML entry above.
  • md/table-html-fallback — a cell's own colSpan/rowSpan/background, or a block a plain GFM cell cannot represent at all (most commonly a nested table), sends the WHOLE table through the HTML-table fallback instead of plain pipe syntax — see HTML-table fallback. Fires once per table that falls back.
  • md/table-cell-formatting-dropped — the residual gap even inside the HTML-table fallback: a cell's own pattern background fill (only a solid fill maps onto CSS background-color), a nested table mixed with sibling content in the same cell (a cell's own nested table must be its entire content), or any other block kind (embeddedObject, pageBreak) — none of these have a representation this bounded recogniser attempts; the cell still renders, unmerged/unstyled, with that content dropped.
  • md/table-cell-multi-paragraph-joined — a cell carrying more than one block joins their rendered content with a literal <br>, real inline HTML every GFM table renderer treats as a genuine line break — markdown syntax in the plain pipe-syntax writer, a real <br> HTML tag in the HTML-table fallback.
  • md/table-cell-image-degraded — plain pipe-syntax writer only: an image-kind cell block emits inline (![alt](data:...)) rather than being dropped; on read-back it degrades to a run carrying the alt text with the image's own data URI as that run's hyperlink, the same "nested image" shape an image inside emphasis/a link already degrades to elsewhere in this package. The HTML-table fallback represents an image losslessly instead, as a real <img> tag.
  • md/duplicate-footnote-definition — two definitions share a label; every reference resolves to the first, both are kept as written.
  • md/footnote-body-heading-flattened — a heading inside a definition body is carried as literal ATX text, since a construct extent may not open or close a heading scope.
  • md/construct-unrepresented — a construct kind markdown has no syntax for renders transparently: its extent still appears, the construct itself does not.
  • md/package-table-dropped — writeMarkdown only, ahead of flattening: a DocumentTree's own definitions/layers/attachments/destinations/pages table has no flat-ContentDocument home (flattenTree's own envelope carries forward only metadata and symbolTable); fires once per non-empty table present.

Footnotes

GitHub's footnote extension ([^label] markers, [^label]: body definitions) is on by default, alongside the four GFM toggles — switch it off with footnotes: false. Neither CommonMark nor the GFM spec document defines footnotes, so both spellings are ordinary text with it off.

The two halves of a footnote map onto the same anchor construct at two different scopes, and that split is structural rather than a choice:

  • A definition becomes an anchor construct. Lowering emits document-schema.js's construct boundary markers — a constructStart carrying { kind: 'anchor', anchorType: 'footnote', name }, the definition's own lowered body blocks, and a constructEnd — which is what readMarkdownContent returns in its block flow, and what decompose promotes to a construct group of its own in the DocumentTree readMarkdown returns (the descriptor rides the group's node, the body blocks its children). The body rides the construct's extent rather than AnchorDescriptor.definition, which names a key in a package-level definitions table: DocumentTree does carry that table as a root (unlike the flat ContentDocument), but a table entry there is a flat descriptor record, not a container for block content, so a body that is genuinely several paragraphs, a code block, or a list still has nowhere to live as a table value either way — the construct's own bracketed extent is the one shape in this schema built to hold real block content. A bodyless [^1]: lowers to the point anchor the same descriptor describes: a pair with nothing between it.
  • A reference site becomes a point run-level anchor extent. A reference sits between two runs inside a paragraph, so no block-level boundary marker can bracket it without splitting the paragraph in two — but a run-level construct extent (RunConstructExtent on ContentParagraph.constructs, document-schema.js 4.5.0) names exactly that shape. Lowering emits an ordinary text run keeping the reference's own [^label] spelling (the materialised rendering, so a consumer that ignores constructs still shows [^1]) plus a point extent — { kind: 'anchor', anchorType: 'footnote', name } at startRun === endRun naming that run, the same wiring ooxml.js's docx reader mints for a w:footnoteReference — carried through decompose/flattenTree verbatim, exactly the way a table cell's own markers are. The writer spells [^label] back out from the covering extent rather than from anything about the run's text (which is what still distinguishes a genuine reference from a deliberately-escaped literal \[^1\]), gated by the same label grammar as the definition marker: a foreign name that grammar cannot spell degrades to the run's own escaped text plus md/construct-unrepresented.

Definitions are recognised at the document's own top level, directly inside a block quote, or directly inside a list item (ExaDev/markdown-codec#957) — a blockquote's own division construct threads its enclosing container's context straight through the definition's anchor pair the same way it already does for a nested blockquote, and a list item whose own first block is the anchor pair picks up the same placeholder-paragraph carry a nested blockquote already relies on. Not recognised inside another definition's own body: a footnote-in-a-footnote has no cross-format shape, so a [^2]: line there stays ordinary body text. A heading inside a definition body is still flattened to literal ATX text, and a blockquote containing a heading anywhere in its own subtree still degrades to indent-only structure — both because a construct boundary marker's extent may not open or close a heading-group scope, a document-schema.js-level limitation this package cannot route around from its own side.

Emission is the inverse and validates first: a section's markers must pair as balanced brackets (checked through document-schema.js's own findConstructMarkerImbalance, the shared definition every codec and decompose agree on) or writeMarkdownContent throws MarkdownUnbalancedConstructMarkersError. A tree already satisfies that balance by construction — decompose refuses to build one from an unbalanced stream — so writeMarkdown reaches this check only on a hand-built package flattened back to an unbalanced flow. A construct kind with no markdown syntax — a bookmark, a division, a tracked change — renders transparently: its extent still appears in place, only the construct's own identity is lost.

HTML-table fallback

GFM's own pipe-table syntax (github.github.com/gfm, "Tables (extension)") holds inline content only — no cell can carry colSpan/rowSpan/a background, and no cell can hold a block that isn't inlinable (a nested table, most notably). writeMarkdownContent detects this per table (tableNeedsHtmlFallback, src/emit/html-table.ts): the moment ANY cell needs one of these, the WHOLE table renders as a raw HTML <table> block instead of the plain | a | b | writer — GFM's own cell grammar cannot mix an HTML sub-block into one cell of an otherwise pipe-syntax table, so a partial, per-cell fallback is not an option. A raw HTML <table> block is legal CommonMark regardless (spec 0.31.2, "HTML blocks" start condition 6 names table directly — https://spec.commonmark.org/0.31.2/#html-blocks). A table with none of these needs still emits the existing, human-readable pipe syntax, unaffected.

readMarkdownContent/readMarkdown recognise this same shape on the way back in (parseHtmlTable, src/html/html-table.ts) — the one exception to this package's own "raw HTML is never parsed as markup" rule (see the raw-HTML gotcha above and src/html/html.ts's own top comment): a block-level HTML that is, in full, one well-formed <table> lowers straight to a real ContentTable, so a table written through this fallback reads back as an equal (or equivalent) ContentTable rather than degrading to opaque preserved text. This recognition is gated on GFM's table extension being enabled (gfmTables, default true) exactly the way plain pipe-table promotion already is — with it disabled, a <table> block stays exactly the opaque raw HTML it always was, since a ContentTable is a GFM-table-extension construct either way it is spelled in the source.

Both halves are bounded recognisers, not a general HTML/DOM parser — matching src/html/html.ts's own deliberate scope. They recognise <table>/<tr>/<td>/<th> and, on each cell, colspan/rowspan (positive integers) and a style="background-color:#rrggbb" declaration (a ContentCellFill's solid variant only — a pattern fill has no plain-CSS equivalent this writer attempts, and degrades with TABLE_CELL_FORMATTING_DROPPED), plus style="text-align:..." for the cell's own paragraph alignment — read per cell rather than only from the header row, a strictly richer fidelity than plain GFM's single per-column delimiter marker can ever express, so a table already falling back for another reason never silently loses alignment plain GFM would have kept. Attribute values must be double-quoted, matching exactly what the writer itself emits; a single-quoted or unquoted value is a real HTML shape neither side attempts.

A cell's own inline formatting renders as real HTML tags — <strong>/<em>/<del>/<code>/<a href>, the identical bold/italic/strike/hyperlink/Courier-New-as-code-span vocabulary the plain writer already spells in markdown syntax — because raw HTML block content is never reprocessed as markdown, so writing markdown punctuation inside this block would render as its own literal characters rather than formatting. The reader also recognises the common <b>/<i>/<s>/<strike> synonyms real hand-authored HTML tends to use, in any nesting order or combination.

A cell's own nested-block content is bounded to two shapes: several paragraph/image blocks join with a literal <br>, exactly as the plain writer already joins a multi-paragraph cell (TABLE_CELL_MULTI_PARAGRAPH_JOINED); or, when a cell's ENTIRE content is one nested table and nothing else, that table recurses through the identical fallback, unconditionally, since a table nested inside another table's cell can never itself be pipe syntax. A nested table mixed with sibling content in the same cell, or any other block kind (embeddedObject, pageBreak), has no representation either side attempts and is dropped with TABLE_CELL_FORMATTING_DROPPED, exactly as it always was before this fallback existed.

Fidelity

Markdown → ContentDocument is dominated by target-schema limits, not parsing gaps. The parser recognises every construct CommonMark and GFM define; the limiting factor is what ContentDocument can hold — a cross-format pivot shaped around docx/pptx/odt/odp/ods/odg, not markdown's richer model. Each gap is a permanent structural mismatch.

Round-trip conformance rate (read → write → reparse → render to HTML, compared byte for byte against expected HTML):

Corpus Examples Passing round trip Rate
CommonMark 0.31.2 (assets/commonmark/spec.json) 652 575 88.2%
GFM tagged extensions (table/strikethrough/autolink/task-list, assets/gfm/spec.txt) 23 23 100%
Combined 675 598 88.6%

Every non-passing example is named individually in src/test-support/conformance-exclusions.ts, attributed to a closed set of causes (shrink-only — see Conventions): most commonly an emphasis-span collision, an image with no data: URI source for the harness to embed, or a blockquote whose heading content skips its container pair.

Optional real-world corpus. test/corpus/ (gitignored) holds a pnpm test:corpus project for a manual sanity check against sibling READMEs on disk — asserts no throw and real content on reparse, not byte fidelity. Not part of pnpm test; run locally before significant parser/lower/emit changes.

Release and publishing

Release, CI, and commit-message conventions are all workspace-wide, not package-local — see the monorepo root README for the mechanism (topological per-package semantic-release via @exadev/semantic-release-workspace, OIDC trusted npm publishing, automatic sibling dependency-range rewriting) and its post-release republishing and attestation note on the restored GitHub Packages mirrors, npm aliases, and SBOM/provenance signing.

Contributing

Conventional Commits, enforced workspace-wide by commitlint through a root commit-msg hook. Work inside packages/markdown-codec/; see CONTRIBUTING.md for the shared git hooks and history conventions.

References

  • document-schema.js — owns both shared encodings (ContentDocument, DocumentTree) and the assembleTree/flattenTree transform between them.
  • pdf-codec — the sibling whose scaffold, tooling, and "hand-write the format" philosophy this project mirrors.
  • documents.js — bridges markdown to docx/odt/PDF via this package's ContentDocument (the flat pair; its own conversion pipeline assembles the package itself). Markdown has no presentation/spreadsheet/drawing variant, so pptx/odp/ods/odg are structurally out of reach.
  • CommonMark Spec — the base specification targeted.
  • GitHub Flavored Markdown Spec — GFM extensions layered on top.
  • WHATWG HTML § named character references — the entity table assets/html-entities/ vendors.

npm aliases

This package also published under an alternate name from the pre-monorepo pipeline:

Republished automatically — the alias's trusted publisher is registered against this repository and workflow (2026-09-10), so every release from the backfill run onward publishes under this name too; the registration evidence is on ExaDev/documents.js#728.

License

MIT