Skip to content

fix: escape DEL, C1 controls, and invalid UTF-8 in log output#32

Merged
tobez merged 3 commits into
mainfrom
log-enc-utf8
Jul 8, 2026
Merged

fix: escape DEL, C1 controls, and invalid UTF-8 in log output#32
tobez merged 3 commits into
mainfrom
log-enc-utf8

Conversation

@tobez

@tobez tobez commented Jul 8, 2026

Copy link
Copy Markdown
Owner

log_enc() escaped C0 controls, quotes, and backslashes, but passed bytes
0x7F–0xFF through raw — in bare tokens as well as quoted values. DEL and
single-byte C1 controls (e.g. 0x9B CSI) could therefore reach the terminal
unescaped, and malformed byte sequences could corrupt log-line structure.

The encoder is now UTF-8 aware, in the spirit of Go's log/slog:

  • Non-ASCII input is validated as strict UTF-8 (RFC 3629): overlong
    encodings, surrogates, and code points above U+10FFFF are rejected.
  • Valid sequences encoding U+00A0 and above pass through unchanged, so
    legitimate non-ASCII text is unaffected.
  • DEL (0x7F), C1 controls (U+0080–U+009F), and invalid UTF-8 are escaped
    byte-wise as \xNN.
  • needs_quote() applies the same classification, so bare tokens
    containing such bytes are now quoted and escaped.

Test plan

  • 18 new log_enc() cases in t/test_log.c (DEL, C1 raw and encoded,
    overlongs, surrogates, out-of-range, truncated sequences, resync after a
    bad lead byte, and pass-through of valid 2/3/4-byte UTF-8)
  • full suite passes (367 tests)
  • scan-build: no bugs found

@tobez tobez merged commit d9211c5 into main Jul 8, 2026
3 checks passed
@tobez tobez deleted the log-enc-utf8 branch July 8, 2026 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant