Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@

### Changed

- En dash and em dash now replace to `-` instead of `--`
- Expanded `--fix` replacement table with: hyphen variants (U+2010-2012, U+2015,
U+FE58), soft hyphen (removed), bullets, dot leaders, arrows (`->`, `<-`, `^`,
`v`), and math operators (`x`, `/`)
- Add `pytest-sugar` for improved test output
- Replace mypy with [ty](https://github.com/astral-sh/ty) for type checking
- Move dev dependencies from `optional-dependencies` to `dependency-groups`
Expand Down
38 changes: 34 additions & 4 deletions docs/check-unicode.1
Original file line number Diff line number Diff line change
Expand Up @@ -317,19 +317,49 @@ Lines already flushed remain in the output.
.B Smart quotes
\(lq\(rq \(oq\(cq and variants \(-> replaced with ASCII quotes
.TP
.B Dashes
Em dash (U+2014), en dash (U+2013), minus sign (U+2212) \(-> replaced with
.B \-\-
or
.B Dashes and hyphens
Em dash, en dash, figure dash, horizontal bar, minus sign, and other
dash\-like characters \(-> replaced with
.BR \- .
.TP
.B Soft hyphen
U+00AD \(-> removed (invisible layout hint, not content).
.TP
.B Fancy spaces
Non\-breaking space, em space, thin space, and 14 other Unicode space characters
\(-> replaced with a regular space.
.TP
.B Ellipsis
Horizontal ellipsis (U+2026) \(-> replaced with
.BR ... .
.TP
.B Bullets
Bullet (U+2022), triangular bullet, hyphen bullet \(-> replaced with
.B *
or
.BR \- .
.TP
.B Dot leaders
One dot leader, two dot leader \(-> replaced with
.B .
or
.BR .. .
.TP
.B Arrows
\(-> and \(<- \(-> replaced with
.B \->
and
.BR <\- ;
\(ua and \(da \(-> replaced with
.B ^
and
.BR v .
.TP
.B Math operators
Multiplication sign (\(mu) \(-> replaced with
.BR x ;
division sign (\(di) and fraction slash \(-> replaced with
.BR / .
.
.SS Dangerous invisible characters (never auto\-fixed)
.TP
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ lint.ignore = [
"ISC001", # single-line implicit string concat -- conflicts with formatter
]
lint.per-file-ignores."tests/**" = [
"PLR0913", # test factory helpers mirror dataclass fields
"PLR2004", # magic values in test assertions are readable
"S101", # assert is fine in tests
]
Expand Down
30 changes: 26 additions & 4 deletions src/check_unicode/categories.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,17 @@
0x201F: '"', # DOUBLE HIGH-REVERSED-9 QUOTATION MARK
0x00AB: '"', # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK
0x00BB: '"', # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK
# Dashes
0x2013: "--", # EN DASH
0x2014: "--", # EM DASH
# Minus
# Dashes and hyphens
0x2010: "-", # HYPHEN
0x2011: "-", # NON-BREAKING HYPHEN
0x2012: "-", # FIGURE DASH
0x2013: "-", # EN DASH
0x2014: "-", # EM DASH
0x2015: "-", # HORIZONTAL BAR
0x2212: "-", # MINUS SIGN
0xFE58: "-", # SMALL EM DASH
# Soft hyphen (invisible layout hint, not content)
0x00AD: "", # SOFT HYPHEN
# Fancy spaces -> regular space
0x00A0: " ", # NO-BREAK SPACE
0x2000: " ", # EN QUAD
Expand All @@ -57,4 +63,20 @@
0x3000: " ", # IDEOGRAPHIC SPACE
# Ellipsis
0x2026: "...", # HORIZONTAL ELLIPSIS
# Bullets
0x2022: "*", # BULLET
0x2023: "*", # TRIANGULAR BULLET
0x2043: "-", # HYPHEN BULLET
# Dot leaders
0x2024: ".", # ONE DOT LEADER
0x2025: "..", # TWO DOT LEADER
# Arrows
0x2190: "<-", # LEFTWARDS ARROW
0x2192: "->", # RIGHTWARDS ARROW
0x2191: "^", # UPWARDS ARROW
0x2193: "v", # DOWNWARDS ARROW
# Math operators
0x00D7: "x", # MULTIPLICATION SIGN
0x00F7: "/", # DIVISION SIGN
0x2044: "/", # FRACTION SLASH
}
Loading
Loading