feat: add MHTML support - #149

Open

marcellmanfrin wants to merge 58 commits into

firecrawl:mainfrom

marcellmanfrin:feature/mhtml-support-pr

Commits on Aug 28, 2026

feat: add standalone HTML support
github-actions[bot]
committed
8b83460
fix: address HTML charset review feedback
marcellmanfrin
committed
b20d2d9
fix: harden HTML content detection
marcellmanfrin
committed
8dfa216
test: cover HTML detection review regressions
marcellmanfrin
committed
341be1b
fix: address HTML review regressions
marcellmanfrin
committed
50b9629
fix: address second HTML review regressions
marcellmanfrin
committed
9920d30
fix: match HTML5 preflight depth semantics
marcellmanfrin
committed
df61b31
fix: address final HTML review findings
marcellmanfrin
committed
076e77d

Commits on Aug 29, 2026

test: consolidate HTML review regressions
marcellmanfrin
committed
fbe5339
test: add HTML corpus integration regressions
marcellmanfrin
committed
c21198b
test: add controlled HTML corpus fixtures
marcellmanfrin
committed
0c4c81c
test: add LibreOffice HTML corpus regressions
marcellmanfrin
committed
adaf984
test: add LibreOffice HTML corpus fixtures
marcellmanfrin
committed
529aaa2
test: cover structural children in nested list wrappers
marcellmanfrin
committed
2bca8ee
fix: preserve structural children in HTML lists
marcellmanfrin
committed
ff5820f
style: format HTML corpus changes
marcellmanfrin
committed
6372cf0
test: refine LibreOffice numbering invariants
marcellmanfrin
committed
bbff29a
test: include HTML fixtures in corpus detection
marcellmanfrin
committed
fe53a0a
test: add HTML corpus snapshots
marcellmanfrin
committed
b6c0889
test: keep corpus walk helper local
marcellmanfrin
committed
5bd85ba
style: format corpus walk helper
marcellmanfrin
committed
6c78e2c
test: isolate HTML corpus fixture helper
marcellmanfrin
committed
ab653fb
test: cover non-rendering children inside lists
marcellmanfrin
committed
6c70669
fix: ignore non-rendering children when grouping lists
marcellmanfrin
committed
75cc2d2

Commits on Aug 30, 2026

Commits on Aug 31, 2026

Merge reconciled MHTML hardening candidate into PR 149 head
Marcell Manfrin
committed
c1c854c
fix: require multipart media type for MIME preflight nesting
Marcell Manfrin
committed
108180e
fix: preserve collected assets for frameset documents
Marcell Manfrin
committed
da61872
refactor: share default link/anchor resolution across HTML contexts
Marcell Manfrin
committed
bfebd99

Commits on Sep 1, 2026

fix: close implied paragraphs and preserve bare-hash links in HTML
Marcell Manfrin
committed
b50bc1b
fix: bound quoted-printable parts early and resolve relative bases
Marcell Manfrin
committed
b67f51e
fix: complete the paragraph-closing start tag set in the HTML preflight
Marcell Manfrin
committed
274f465
test: make the relative-base regression discriminate the fix
Marcell Manfrin
committed
de11157
fix: model foreign-content breakout, select/table implied closes, and stray list text
Marcell Manfrin
committed
4987b8b

Commits on Sep 4, 2026

fix: tighten MHTML preflight against encapsulated messages, oversized headers, and encoding problems
Marcell Manfrin
committed
932bf18

Commits on Sep 5, 2026

fix: delimiter-aware MIME scanning, capped root headers, transcoded rfc822 walk
Marcell Manfrin
committed
f439bc4
fix: keep raw-text tokenizer states out of foreign content
Marcell Manfrin
committed
f9ec62b
fix: accept MIME transport padding in boundary delimiter matching
Marcell Manfrin
committed
7382046
fix: keep the script tokenizer state out of foreign content
Marcell Manfrin
committed
1b54c2d
fix: reject linked stylesheets with broken transfer encoding
Marcell Manfrin
committed
beb415b
fix: scope HTML implied closes to HTML content and exit nested foreign roots
Marcell Manfrin
committed
0eae6b5
fix: strict closing delimiters and query-safe authority parsing
Marcell Manfrin
committed
50db484
fix: align preflight element accounting with the tree builder
Marcell Manfrin
committed
5448390
fix: walk encapsulated message chains and move image bytes into assets
Marcell Manfrin
committed
94deeea
fix: model html5ever end-tag scopes and ignored duplicate wrappers
Marcell Manfrin
committed
0f1b607
fix: keep body/html end tags open, scope wrappers out of foreign content
Marcell Manfrin
committed
5edd3a4
fix: stop rejecting base64 bodies by encoded size
Marcell Manfrin
committed
340618b
fix: model end-tag special stops and context-gated implied closes
Marcell Manfrin
committed
6c82715
fix: decode transfer-encoded rfc822 bodies during the preflight walk
Marcell Manfrin
committed
fa9c10b
test: pin foreign-root stop for scope-based end tags
Marcell Manfrin
committed
d15d458

Commits on Sep 6, 2026

fix: align end-tag scope model with html5ever foreign-content walk
Marcell Manfrin
committed
fced448
fix: never stack frame elements in the preflight
Marcell Manfrin
committed
da11115
fix: clear foster content above tables on table-family starts
Marcell Manfrin
committed
ea872c6

Commits on Sep 7, 2026

fix: close paragraphs in button scope on every block start tag
Marcell Manfrin
committed
cfd9b80