Drafted by Claude (Opus 4.8), an LLM by Anthropic, at andy5995's direction.
Goal
Represent and round-trip message text and header fields that cannot be
expressed in a single-byte character set.
Background
MultiMail translates between two single-byte sets, CP437 and Latin-1. Recent
work lets editable fields (To, Subject, taglines) accept pasted or composed
UTF-8 by folding each character to the single-byte internal encoding, and
sends the header in the area's own charset. That works for characters that fit
Latin-1, such as accented Latin names. Characters outside Latin-1 — Cyrillic,
CJK, emoji — have no single-byte home and are stored as ?.
Why this is large
The "one byte equals one character" assumption runs through the whole program.
Real support needs a UTF-8 / wide internal representation across every layer:
- backend storage and the data model
- the line editor (character entry, cursor movement, and deletion must be
multibyte-aware)
- screen output (wide-curses drawing)
- the packet formats that can carry and label UTF-8 — for example QWKE
HEADERS.DAT can declare UTF-8 on a reply; formats with fixed single-byte
fields, such as plain QWK, cannot, so they stay best-effort
Constraint
MultiMail builds for 16-bit DOS and other old toolchains that have no
wide-character or UTF-8 support. Any design must keep those builds working —
most likely UTF-8 on modern (ncursesw) builds, with the single-byte path
unchanged on DOS.
Reference
Upstream request: wmcbrine/MultiMail#9 ("Modern terminals speak Unicode. So
should MultiMail.").
Not urgent; revisit after a release.
Goal
Represent and round-trip message text and header fields that cannot be
expressed in a single-byte character set.
Background
MultiMail translates between two single-byte sets, CP437 and Latin-1. Recent
work lets editable fields (To, Subject, taglines) accept pasted or composed
UTF-8 by folding each character to the single-byte internal encoding, and
sends the header in the area's own charset. That works for characters that fit
Latin-1, such as accented Latin names. Characters outside Latin-1 — Cyrillic,
CJK, emoji — have no single-byte home and are stored as
?.Why this is large
The "one byte equals one character" assumption runs through the whole program.
Real support needs a UTF-8 / wide internal representation across every layer:
multibyte-aware)
HEADERS.DAT can declare UTF-8 on a reply; formats with fixed single-byte
fields, such as plain QWK, cannot, so they stay best-effort
Constraint
MultiMail builds for 16-bit DOS and other old toolchains that have no
wide-character or UTF-8 support. Any design must keep those builds working —
most likely UTF-8 on modern (ncursesw) builds, with the single-byte path
unchanged on DOS.
Reference
Upstream request: wmcbrine/MultiMail#9 ("Modern terminals speak Unicode. So
should MultiMail.").
Not urgent; revisit after a release.