Releases: mit-d/check-unicode
Releases · mit-d/check-unicode
v0.6.0
Added
- Pipe mode:
check-unicode -reads stdin line-by-line and writes to stdout,
enabling use as a streaming Unix filter for log monitoring, CI pipelines, and
editor buffer filtering --strip [dangerous|all]flag to remove non-ASCII characters;dangerous
strips only invisible/bidi chars,all(default) strips any remaining
non-ASCII after allow-list processing--halt [dangerous|all]flag to stop immediately on first matching character;
dangerous(default) halts on invisible/bidi chars,allhalts on any
non-ASCII--fix,--strip, and--haltare fully composable and work identically
across file and pipe modes
Changed
- En dash and em dash now replace to
-instead of-- - Expanded
--fixreplacement table with: hyphen variants (U+2010-2012, U+2015,
U+FE58), soft hyphen (removed), bullets, dot leaders, arrows (->,<-,^,
v), and math operators (x,/) - Add
pytest-sugarfor improved test output - Replace mypy with ty for type checking
- Move dev dependencies from
optional-dependenciestodependency-groups - Switch CI from pip to uv for faster, reproducible installs; check in
uv.lock - Extract codepoint/range parsing into
check_unicode.parsingmodule for reuse - Codepoint parser now validates the Unicode range (0..U+10FFFF) and rejects
empty/invalid input with clear error messages - Range parser now rejects inverted ranges (lo > hi) and tolerates whitespace
around the dash separator
v0.5.0
Fixed
- Validate
severityvalues from config files and overrides (invalid values
like"warn"now exit 2 instead of silently behaving as warning) - Catch config file errors (missing file, invalid TOML) and exit 2 with a
friendly message instead of a raw traceback - Validate
--allow-categoryand--allow-scriptvalues; unknown names now
exit 2 with a hint to use--list-categoriesor--list-scripts - Warn on unrecognised top-level config keys (e.g. typo
alow-codepoints) - Remove dead
U+FFFDentry fromREPLACEMENT_TABLE(unreachable because
U+FFFD is inDANGEROUS_INVISIBLE, which is checked first) - Exclude
tests/fixtures/from mypy (intentionally malformed Trojan Source
files)
Added
- Grouped output: findings are now grouped per file with a compact line range
header (e.g.file.txt:1,4-80,90:), context lines shown once with multi-caret
markers (^normal,!dangerous,?confusable), deduplicated identical
context lines, and collapsed codepoint listing with(xN)counts
Changed
- Refactor
_apply_replacementsto usestr.translate()for cleaner code and
better performance on large files - Read each file once when
--check-confusablesis enabled (previously
check_fileandcheck_confusableseach read the file independently) - Simplify
_parse_codepointto use prefix-stripping instead of fragile
double-replace chain - Add
slots=TruetoOverridedataclass for consistency withFindingand
AllowConfig
Docs
- Document per-file
[[tool.check-unicode.overrides]]in README and man page - Update man page version to 0.4.0 and fix stale pre-commit
rev - Add man page to
bump-my-versionfiles list
v0.4.0
Added
[[tool.check-unicode.overrides]]per-file config: apply different
allow-lists, severity, and confusable settings per file pattern- Per-file severity: override
severityto"warning"for specific file
patterns so findings don't affect exit code - Per-file confusable toggle: enable or disable
check-confusablesper file
pattern uv.lockadded to.gitignore
v0.3.3
Fixed
--fixmode now fixes all files, not just the first (any()short-circuited
after the first fixable file, skipping the rest)
v0.3.2
Added
CONTRIBUTING.mdguideSECURITY.mdwith private vulnerability reporting instructions- Issue templates for bug reports and feature requests
- Make
bump-my-versionhandle README.md version
v0.3.1
Added
- PR template with changelog and testing checklist
- Coverage threshold (80%) enforced in CI
markdownlintconfig: allow duplicate headings across sibling sections
v0.3.0
Added
- Grouped CLI help with
--helpshowing organized option sections - Usage examples in
--helpoutput - Man page (
docs/check-unicode.1) - Release workflow for automatic GitHub Releases on tag push
bump-my-versionconfig for version management