Skip to content

Add charset=UTF-8 to Content-Type headers for text-based MIME types #347

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
brianhuster merged 7 commits into from
Jan 2, 2026
Merged

Add charset=UTF-8 to Content-Type headers for text-based MIME types #347

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
5a1b853
Initial plan
Copilot Jan 1, 2026
2283385
Fix unicode/Chinese character rendering by adding charset=UTF-8 to Co…
Copilot Jan 1, 2026
def6d4b
Add charset=UTF-8 to svgz, xhtml, and xht mime types
Copilot Jan 1, 2026
82f27b6
Remove charset from JSON content-type for RFC compliance
Copilot Jan 1, 2026
52b5aa0
Add comprehensive tests and documentation for Unicode fix
Copilot Jan 1, 2026
f88e11d
Remove test documentation files and create minimal index.html with Kanji
Copilot Jan 1, 2026
8924e42
Apply suggestion from @brianhuster
brianhuster Jan 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions lua/livepreview/server/utils/content_type.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ M.mime = {
["csh"] = "application/x-csh",
["csml"] = "chemical/x-csml",
["csp"] = "application/vnd.commonspace",
["css"] = "text/css",
["css"] = "text/css; charset=UTF-8",
["cst"] = "application/x-director",
["csv"] = "text/csv",
["c"] = "text/x-c",
Expand Down Expand Up @@ -344,8 +344,8 @@ M.mime = {
["hqx"] = "application/mac-binhex40",
["h"] = "text/x-c",
["htke"] = "application/vnd.kenameaapp",
["html"] = "text/html",
["htm"] = "text/html",
["html"] = "text/html; charset=UTF-8",
["htm"] = "text/html; charset=UTF-8",
["hvd"] = "application/vnd.yamaha.hv-dic",
["hvp"] = "application/vnd.yamaha.hv-voice",
["hvs"] = "application/vnd.yamaha.hv-script",
Expand Down Expand Up @@ -395,7 +395,7 @@ M.mime = {
["jpm"] = "video/jpm",
["json"] = "application/json",
["jsonml"] = "application/jsonml+json",
["js"] = "text/javascript",
["js"] = "text/javascript; charset=UTF-8",
["kar"] = "audio/midi",
["karbon"] = "application/vnd.kde.karbon",
["kfo"] = "application/vnd.kde.kformula",
Expand Down Expand Up @@ -478,7 +478,7 @@ M.mime = {
["mime"] = "message/rfc822",
["mj2"] = "video/mj2",
["mjp2"] = "video/mj2",
["mjs"] = "text/javascript",
["mjs"] = "text/javascript; charset=UTF-8",

@brianhuster brianhuster Jan 2, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot Is it necessary to have "charset=UTF-8" here? (This is a question, not a request)

Copilot AI Jan 2, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's necessary. Without charset=UTF-8 in the Content-Type header, browsers may default to ISO-8859-1 or other legacy encodings when interpreting text-based files. This causes Unicode characters (Chinese, Japanese, Korean, emoji, etc.) to render as garbled text like Ã,Æ—‡Å instead of 你好世界.

For JavaScript files specifically, this ensures that string literals and comments containing Unicode characters are properly interpreted. While modern browsers often auto-detect UTF-8, the explicit charset declaration guarantees correct rendering across all browsers and prevents encoding issues.

@brianhuster brianhuster Jan 2, 2026

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot Should the encoding be got from the buffer-local option 'encoding'?

Copilot AI Jan 2, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, that wouldn't work in this context. The HTTP server serves files directly from disk using async_read_file, not from Neovim buffers. The content_type.lua module runs in the server's context where buffer-local options like encoding or fileencoding aren't available.

Additionally, UTF-8 is the standard encoding for web content. Files on disk are typically UTF-8 encoded (especially for web development), and hardcoding charset=UTF-8 in the HTTP response ensures consistent browser interpretation regardless of the file's actual encoding on disk.

["mk3d"] = "video/x-matroska",
["mka"] = "audio/x-matroska",
["mks"] = "video/x-matroska",
Expand Down Expand Up @@ -800,8 +800,8 @@ M.mime = {
["sv4crc"] = "application/x-sv4crc",
["svc"] = "application/vnd.dvb.service",
["svd"] = "application/vnd.svd",
["svg"] = "image/svg+xml",
["svgz"] = "image/svg+xml",
["svg"] = "image/svg+xml; charset=UTF-8",
["svgz"] = "image/svg+xml; charset=UTF-8",
["swa"] = "application/x-director",
["swf"] = "application/x-shockwave-flash",
["swi"] = "application/vnd.aristanetworks.swi",
Expand Down Expand Up @@ -848,7 +848,7 @@ M.mime = {
["twds"] = "application/vnd.simtech-mindmapper",
["txd"] = "application/vnd.genomatix.tuxedo",
["txf"] = "application/vnd.mobius.txf",
["txt"] = "text/plain",
["txt"] = "text/plain; charset=UTF-8",
["u32"] = "application/x-authorware-bin",
["udeb"] = "application/x-debian-package",
["ufd"] = "application/vnd.ufdl",
Expand Down Expand Up @@ -972,8 +972,8 @@ M.mime = {
["xer"] = "application/patch-ops-error+xml",
["xfdf"] = "application/vnd.adobe.xfdf",
["xfdl"] = "application/vnd.xfdl",
["xht"] = "application/xhtml+xml",
["xhtml"] = "application/xhtml+xml",
["xht"] = "application/xhtml+xml; charset=UTF-8",
["xhtml"] = "application/xhtml+xml; charset=UTF-8",
["xhvml"] = "application/xv+xml",
["xif"] = "image/vnd.xiff",
["xla"] = "application/vnd.ms-excel",
Expand All @@ -990,7 +990,7 @@ M.mime = {
["xltx"] = "application/vnd.openxmlformats-officedocument.spreadsheetml.template",
["xlw"] = "application/vnd.ms-excel",
["xm"] = "audio/xm",
["xml"] = "application/xml",
["xml"] = "application/xml; charset=UTF-8",
["xo"] = "application/vnd.olpc-sugar",
["xop"] = "application/xop+xml",
["xpi"] = "application/x-xpinstall",
Expand Down
11 changes: 11 additions & 0 deletions tests/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
<!DOCTYPE html>
<html>
<head>
<title>Test - 漢字</title>
</head>
<body>
<h1>你好世界</h1>
<p>これは日本語のテストです。</p>
<p>漢字、ひらがな、カタカナ</p>
</body>
</html>
Loading