Skip to content

fix: add charset=utf-8 to .txt responses via _headers - #908

Merged
yusukebe merged 1 commit into
honojs:mainfrom
Debbl:fix/llms-txt-charset
Sep 20, 2026
Merged

yusukebe merged 1 commit into
honojs:mainfrom
Debbl:fix/llms-txt-charset

Conversation

@Debbl

@Debbl Debbl commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Summary

https://hono.dev/llms.txt (and llms-full.txt / llms-small.txt) is served in production with a bare Content-Type: text/plain, while wrangler dev returns text/plain; charset=utf-8 locally.

$ curl -sI https://hono.dev/llms.txt | grep -i content-type
content-type: text/plain

$ curl -sI http://localhost:8787/llms.txt | grep -i content-type
Content-Type: text/plain; charset=utf-8

The difference comes from wrangler: its local asset server appends charset=utf-8 to every text/* type, but the production Static Assets service only maps the extension to text/plain. Since these files contain non-ASCII content (🔥, CJK), clients that fall back to ISO-8859-1 show mojibake.

Fix

Add public/_headers with a rule for *.txt:

/*.txt
  Content-Type: text/plain; charset=utf-8

public/ is copied verbatim into .vitepress/dist, so the file ships with the normal build and is picked up by Workers Static Assets.

Verification

Ran wrangler dev against the built dist with this file included:

[wrangler:info] ✨ Parsed 1 valid header rule.
$ curl -sI http://localhost:8799/llms.txt | grep -i content-type
Content-Type: text/plain; charset=utf-8

Note: after deploy, the Cloudflare edge cache (cf-cache-status: HIT) may need to expire or be purged before the new header shows up.

🤖 Generated with Claude Code

Cloudflare Workers Static Assets serves llms.txt, llms-full.txt and
llms-small.txt with a bare `text/plain` Content-Type in production,
while `wrangler dev` adds `charset=utf-8` locally. The files contain
non-ASCII content (emoji, CJK), so clients that default to ISO-8859-1
render mojibake. Add a `_headers` rule so both match.

@yusukebe yusukebe left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@yusukebe

Copy link
Copy Markdown
Member

@Debbl

This fix is meaningful. Thanks!

@yusukebe
yusukebe merged commit f607a0f into honojs:main Sep 20, 2026
2 checks passed
@Debbl
Debbl deleted the fix/llms-txt-charset branch September 20, 2026 09:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants