Summary
moli fetch has two dump paths:
- Document — wrap/serialize as HTML (
html field / default stdout HTML).
- Raw —
html: null + exact body_base64 (and default stdout = original bytes).
After 1.1.3, images / application/octet-stream correctly take the raw path (fixes PNG corruption). But several non-HTML response types still take the Document path, so agents get HTML soup (or lossy markdown) instead of the real bytes. Exit code stays 0 — silent false success for anyone expecting a content-type faithful download.
Verified still broken on moli 1.1.5.
Affected MIME types (1.1.5)
Content-Type |
--dump json |
default (no --dump) |
--dump markdown |
notes |
text/plain |
html = <html>…<body>…</body></html>, no body_base64 |
same HTML wrapper |
newlines collapsed to spaces |
document.contentType → text/html |
text/csv |
same HTML soup |
same |
newlines collapsed |
previously thought “raw”; 1.1.5 soups too |
application/rss+xml |
HTML/XML-viewer soup; feed structure mangled (<link> void, CDATA smashed on live feeds) |
same |
titles smashed together |
Atom likely same class |
application/wasm |
HTML soup; leading \0 dropped (asm\x01…); no body_base64 |
— |
garbage |
same bytes as octet-stream work |
image/png / image/jpeg / application/octet-stream |
html: null + exact body_base64 |
raw bytes |
empty |
correct (post-1.1.3) |
Repro
# local fixture: text/plain, application/rss+xml, application/wasm, application/octet-stream, image/png
moli fetch --dump json http://127.0.0.1:PORT/plain
# html: "<html><head></head><body>hello plain\nline2\n</body></html>"
# body_base64: absent
# headers[].content-type still "text/plain; charset=utf-8"
moli fetch http://127.0.0.1:PORT/plain
# stdout is the HTML wrapper, not "hello plain\nline2\n"
moli fetch --dump markdown http://127.0.0.1:PORT/robots.txt
# "User-agent: * Disallow: /private" (0 newlines)
moli fetch --dump json http://127.0.0.1:PORT/tiny.wasm # Content-Type: application/wasm
# html: "<html>…<body>asm\u0001</body></html>", no body_base64
moli fetch --dump json http://127.0.0.1:PORT/octet # same 8 bytes, application/octet-stream
# html: null, body_base64 decodes to \0asm\x01\x00\x00\x00
moli fetch --eval 'document.contentType' http://127.0.0.1:PORT/plain
# text/html
Help text today: omitted --dump serializes rendered HTML documents and writes PDFs/videos etc. verbatim. The gate that chooses Document vs raw is incomplete / MIME-list based, not “non-HTML → raw”.
Expected
Document a single contract and implement it consistently, e.g.:
- Raw path (
html: null + body_base64; default stdout = exact response body) for non-document payloads at least:
application/wasm (and other binary module types)- preferably also
text/plain, text/csv, text/* that are not HTML
- feed types if not given a real XML document path:
application/rss+xml, application/atom+xml
- Or, if Document wrapping is intentional for plain/RSS:
- preserve bytes losslessly somewhere (
body_base64 or raw field) even when html is set
- wrap plain text in
<pre> (or equivalent) so markdown/innerText keep newlines
- do not claim success while destroying structure (RSS
<link> / CDATA)
document.contentType (and JSON metadata) should reflect the HTTP content-type, or expose both HTTP CT and document CT explicitly — not silently rewrite plain → text/html with no other signal.
Minimum fix that unblocks agents: extend the existing raw/body_base64 gate so application/wasm matches application/octet-stream, and stop treating text/plain default dump as “download” unless the body is actually unmodified.
Why it matters
Agents treat moli fetch as a content-type-faithful downloader. HTML-souping plain/RSS/wasm with rc=0 is a silent contract break: robots.txt rules flatten, feed URLs/CDATA corrupt, WASM modules become unrecoverable. This is the same class of defect as the old PNG-in-html bug (fixed via body_base64), just for MIME types still missing from the raw gate.
Related lab notes
- MOL-205 — PNG in
html UTF-8 corruption → fixed 1.1.3 via body_base64
- MOL-088 —
text/plain markdown newline collapse / HTML wrap
- MOL-085 — RSS/Atom HTML tag soup; original feed bytes lost
- MOL-243 —
application/wasm still souped; no body_base64
Environment
- moli 1.1.5 (also broken on 1.1.3–1.1.4 for wasm/plain/RSS; PNG/octet OK since 1.1.3)
- Linux x86_64
Summary
moli fetchhas two dump paths:htmlfield / default stdout HTML).html: null+ exactbody_base64(and default stdout = original bytes).After 1.1.3, images /
application/octet-streamcorrectly take the raw path (fixes PNG corruption). But several non-HTML response types still take the Document path, so agents get HTML soup (or lossy markdown) instead of the real bytes. Exit code stays 0 — silent false success for anyone expecting a content-type faithful download.Verified still broken on moli 1.1.5.
Affected MIME types (1.1.5)
Content-Type--dump json--dump)--dump markdowntext/plainhtml=<html>…<body>…</body></html>, nobody_base64document.contentType→text/htmltext/csvapplication/rss+xml<link>void, CDATA smashed on live feeds)application/wasm\0dropped (asm\x01…); nobody_base64octet-streamworkimage/png/image/jpeg/application/octet-streamhtml: null+ exactbody_base64Repro
Help text today: omitted
--dumpserializes rendered HTML documents and writes PDFs/videos etc. verbatim. The gate that chooses Document vs raw is incomplete / MIME-list based, not “non-HTML → raw”.Expected
Document a single contract and implement it consistently, e.g.:
html: null+body_base64; default stdout = exact response body) for non-document payloads at least:application/wasm(and other binary module types)text/plain,text/csv,text/*that are not HTMLapplication/rss+xml,application/atom+xmlbody_base64orrawfield) even whenhtmlis set<pre>(or equivalent) so markdown/innerText keep newlines<link>/ CDATA)document.contentType(and JSON metadata) should reflect the HTTP content-type, or expose both HTTP CT and document CT explicitly — not silently rewrite plain →text/htmlwith no other signal.Minimum fix that unblocks agents: extend the existing raw/
body_base64gate soapplication/wasmmatchesapplication/octet-stream, and stop treatingtext/plaindefault dump as “download” unless the body is actually unmodified.Why it matters
Agents treat
moli fetchas a content-type-faithful downloader. HTML-souping plain/RSS/wasm with rc=0 is a silent contract break: robots.txt rules flatten, feed URLs/CDATA corrupt, WASM modules become unrecoverable. This is the same class of defect as the old PNG-in-htmlbug (fixed viabody_base64), just for MIME types still missing from the raw gate.Related lab notes
htmlUTF-8 corruption → fixed 1.1.3 viabody_base64text/plainmarkdown newline collapse / HTML wrapapplication/wasmstill souped; nobody_base64Environment