feat(1d): rebuild OneD font from font-db cubic SVG sources#2
Closed
AmitMY wants to merge 18 commits into
Closed
Conversation
AmitMY
commented
May 14, 2026
Contributor
Author
There was a problem hiding this comment.
the sizes of the circles are now correct, but the locations are not.
3 tasks
AmitMY
commented
May 14, 2026
Contributor
Author
AmitMY
left a comment
AmitMY
left a comment
There was a problem hiding this comment.
we want to generate the full OneD font - and to also generate a report.pdf. the report should include the size of the original 1D font (https://github.com/sutton-signwriting/font-ttf/raw/master/src/font/SuttonSignWritingOneD.ttf) compared to our new one, compared to our new and optimized one.
It should show some comparison cases, like what we currently have in "comparison.png" with the 3 symbols of circles. or another comparison should be rotated symbols - basically every optimization should be shown. render them using the new fonts, on a grid, so that we can tell what the differences are. then we don't need comparison.png, we have the report to look at.
Issue #1: regenerate the SignWriting 1D font using font-db's cubic-Bezier source SVGs as the source of truth, instead of relying on the existing quadratic-approximated TTFs. Adds a four-step pipeline (extract → optimize → build → vtp + volt2ttf): - `font_1d/extract.py` reads `iswa2010.db` (fetched by the Makefile) and writes one SVG per symbol. Drops the `sym-fill` (white-interior) path so the resulting monochrome 1D glyph renders correctly as a ring. - `font_1d/optimize.py` detects circular sub-paths via LSQ circle fit (robust to the one extra "stitch" anchor that font-db's closed loops include) and replaces them with synthetic 4-segment cubic-Bezier ellipses — both shrinks the path data and removes hand-traced wobble. - `font_1d/build_font.py` is a FontForge Python script that imports each SVG, maps the symkey to its SWU plane-4 codepoint, and emits a base TTF. - `font_1d/generate_vtp.py` emits a minimal VTP; `volt2ttf` combines it with the base TTF to produce the final font. Also reorganises into `signwriting_fonts/font_1d/` and `signwriting_fonts/font_2d/` for clarity, and updates the 2D Makefile targets to the new module paths. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…rtions FontForge's importOutlines() ignores the SVG's width/height attributes and fits every glyph to roughly the em-square, so S2ff00 (big circle), S17600 (small ring), and S21e00 (two-dot) all came out at ~294 units wide — the big circle and the dots looked the same size. The upstream OneD font sizes each glyph as roughly `svg_width × 10` in font units (e.g. S2ff00 with svg width=36 → bbox 358; S17600 with svg width=16 → bbox 156). After importOutlines we now read width/height out of the SVG header and apply a uniform `psMat.scale()` so the imported bbox matches `svg_width × 10 by svg_height × 10`, re-anchor to (0, -descent), and set the advance width to `target_width + 20` for a small side-bearing. The comparison image now matches the original OneD's relative sizing: S2ff00 is the largest, S17600 is half that, S21e00 dots are smaller still. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
AmitMY
force-pushed
the
feat/regenerate-1d-from-cubic
branch
from
ec58485 to
dd9a182
Compare
AmitMY
changed the base branch from
main
to
refactor/folder-structure-and-toolchain
PR #2 review feedback: "the sizes of the circles are now correct, but the locations are not." Each glyph was being placed with its bottom at -descent, which lined glyph bottoms up at y=-205 instead of giving them a shared visual centre. The upstream OneD font instead places every glyph so its bbox centre is at y≈166 (read off the original glyf table) and pads the left side by 20 font units, with a matching right side-bearing. After this change the new font's per-glyph coordinates match the upstream OneD's within ±2 units: S10000 orig 20..170 / 15..315 new 20..170 / 16..316 S17600 orig 20..176 / 87..244 new 20..180 / 86..246 S21e00 orig 20..159 / 138..194 new 20..160 / 137..195 S2ff00 orig 20..378 / -8..341 new 20..370 / -9..341 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Addresses the review feedback on #2: 1. **Full font** — `extract.py` now defaults to all ~37k symbols and accepts `--symbols dev` for the hand-picked dev subset. Full pipeline runs in ~25s end-to-end (extract → optimize → FontForge build). 2. **report.pdf** at `assets/regen/report.pdf` replaces the old `comparison.png`. Three pages: - file-size + glyph-count tables (original 8014 KB, new unopt 7227 KB, new ellipse-opt 7115 KB — 11.2 % smaller than upstream), - circles (S21e00, S2ff00, S17600) rendered in all three variants, - rotation family S10000–S10005 (placeholder for future GSUB dedup). New `signwriting_fonts/font_1d/report.py` builds it with matplotlib's PdfPages; new `report.pdf` Makefile target wires it in. 3. **Tests** at `signwriting_fonts/font_1d/test_glyph_render.py`: - per-symbol scale+placement test: render each glyph in the original OneD and our new font, content-bbox-align both renderings, assert IOU above a per-symbol threshold (~0.05 below measured baseline). Catches drift in scale/positioning without needing pixel-perfect equality across rasterizers. - smoke tests: every symbol mapped at the right SWU codepoint; optimized ≤ unoptimized ≤ original in bytes. - skipped placeholder for the rotation-family GSUB dedup test. Refactor: `symkey_to_codepoint` moved into `_symkey.py` so report.py and the tests can use it without dragging in FontForge's Python interpreter. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Pulls SW A/B/L/M/R + SW 250..749 (the FSW box delimiters and coordinate
glyphs that the upstream OneD font carries on plane 1) directly from
Slevinski/signwriting_2010_fonts/source/other_svg.zip and imports them
alongside the font-db symbol SVGs.
- New Makefile targets fetch other_svg.zip and unpack into
fonts/1d/markers/ (gitignored alongside fonts/1d/).
- `build_font.py` gets a `--markers-dir` flag and an `_import_marker`
step that:
* names each glyph "SW A", "SW M", "SW 250", … (matching upstream),
* maps to plane-1 codepoints — letters at 0x1D800..0x1D804, numbers
at 0x1D80C + (n - 250) (the 0x1D805..0x1D80B range is reserved,
same gap the upstream font has),
* keeps the 1:1 SVG-to-font-unit scale these glyphs ship at (markers
don't get the 10× scale that symbol SVGs do),
* applies the upstream's per-class side-bearings (20 for letters,
10 for numbers) and bottom-anchors at y=25.
Per-glyph dimensions now match the upstream OneD byte-for-byte for
SW A/M/250/500/749; total mapped codepoints 38,316 (= 37,811 symbols
+ 505 markers), the same as the upstream font.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replaces each (base, fill) family's cardinal rotation siblings (rot 2/4/6 plus their mirrors 8/a/c/e) with composite glyphs that reference the rot-0 base outline plus a 2×2 transform matrix: rot 2: rotate +90° CCW rot 4: rotate 180° rot 6: rotate 270° CCW rot 8: mirror rot A: mirror + 270° CCW (mirror flips rotation direction) rot C: mirror + 180° rot E: mirror + 90° CCW Verified empirically vs the upstream OneD: each cardinal sibling lands above IOU 0.95 after this transform. Diagonals (rot 1/3/5/7/9/B/D/F) have IOU ~0.45 with any rigid transform — they're hand-redrawn in the source SVGs — so we leave them as independent outlines. Replaced 43,604 glyphs with composites. Size impact (apples-to-apples, full 38,316-codepoint fonts): upstream OneD 8,206,192 B 8014 KB new (no optimisations) 7,537,524 B 7361 KB −8.1 % new (ellipse + rotation dedup) 5,309,584 B 5185 KB −35.3 % So the rotation dedup alone contributes the bulk of the saving (~28 % on top of the 8 % baseline drop, which came from stripping the sym-fill layer and a fresh FontForge build). `build_font.py` gets a `--no-rotation-dedup` flag so the report can isolate the dedup's contribution. New tests `test_cardinal_rotation_dedup_renders_correctly` (IOU ≥ 0.6 vs upstream for 8 cardinal samples spanning two families) and `test_cardinal_rotation_is_actually_a_composite` (sanity check that they're stored as composite glyphs in the glyf table) pin this behaviour. The previously-skipped `test_rotation_family_matches_via_gsub_transform` placeholder is replaced by these. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Building on the cardinal dedup, the diagonal indices (1, 3, 5, 7, 9, B, D, F) turn out to form their own internally-consistent family centred on rot 1: empirical IOU search vs the upstream OneD lands every diagonal sibling above 0.94 with a pure rotate/mirror of rot 1. So each `(base, fill)` family now keeps exactly two outlines — rot 0 for the cardinals, rot 1 for the diagonals — and every other orientation is a composite glyph referencing one of those two bases with a rotate/mirror matrix. Replaced 87,921 glyphs with composites (was 43,604 with cardinals-only). Cumulative size (apples-to-apples, full 38,316-codepoint fonts): upstream OneD 8,206,192 B 8014 KB — new (no optimisations) 7,537,524 B 7361 KB −8.1 % new (ellipse + cardinal+diagonal dedup) 2,668,892 B 2606 KB −67.5 % Adds 7 diagonal samples to `test_cardinal_rotation_dedup_renders_correctly` and `test_cardinal_rotation_is_actually_a_composite` (48 tests pass). Also adds a "Rotation + reflection encoding" page to `report.pdf` that spells out the last-hex-digit → orientation → composite-source mapping for all 16 orientations, and extends the rotation-family comparison page to cover all 16 (was 6). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Three related quality fixes uncovered by a new full-font e2e IOU test:
1. Vertical metrics override
FontForge derived hhea/OS/2 ascent/descent from the actual glyph data,
which is slightly larger here because composite-rotation bboxes can push
outside the per-glyph bbox. The result was that hb-view (and any other
browser/UA) sized our font's line-height ~3% taller than upstream, so the
same glyph rendered at the same point size took up a TALLER PNG and looked
visually smaller side-by-side. Lock hhea / OS/2 metrics to match the
upstream OneD values so all renders match.
2. Ellipse detector
The previous detector replaced any closed sub-path whose anchors fit a
circle within 5 % radius error. For glyphs like S15401 (a hand-traced
"comb" with an outer outline that traces a near-circular arc), that
replaced the outline with a synthetic full-circle ellipse, turning the
comb into a filled disc — and every diagonal-rotation composite of S15401
inherited the wrong base.
Tightened to 2.5 % tolerance and added an angular-coverage check
(anchors must span ≥ 300 ° around the fitted centre). Ring outlines still
qualify (all observed errors well under 1 %); the false-positive comb-arc
sub-path no longer matches. Drops ellipse replacements from 4,421 to 128.
3. Rotation dim-compatibility check
_dedup_rotations now skips siblings whose bbox dimensions don't match the
expected (W, H) or (H, W) of the base — catches families where rot 2/4/6
are independently hand-redrawn at incompatible sizes (e.g. S10b40 is
160×290 but S10b42 is 260×150). Without the check we'd composite-rotate
a 160×290 base into a 290×160 slot, producing IOU ~0.
e2e (RUN_E2E=1) results on 38,316 glyphs vs upstream OneD:
before after
glyphs below 0.5 2948 627 −78 %
p1 IOU 0.050 0.340 +6.8x
p5 IOU 0.192 0.713 +3.7x
p50 IOU 0.878 0.884 —
p95 IOU 0.982 0.983 —
Size: 3127 KB (vs upstream 8014 KB; −61 %). Slightly larger than the
2606 KB peak because the dim-compat check keeps ~6k more original
outlines instead of producing wrong composites.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Known Issues page now samples one example per 0.05 IOU bucket across the full range so a reviewer can scroll low→high and pick a cutoff. Renders are black-on-white (was white-on-black) to match the rest of the report. - New "Threshold zoom" page(s) show up to 20 evenly-spaced examples from the [0.40, 0.50) IOU band, sorted low→high, so the call between "acceptable composite" and "reject" can be confirmed visually. - Footer prints both n_below_0.44 and n_below_0.50 counts. - Motivation page (page 1) prefaces the report: why bother regenerating a font that already exists — cubic→quadratic loss + rotation outline duplication. - Rotation table page: title and intro paragraph repositioned so they no longer overlap; intro restricted to hand symbols (non-hands keep their outlines). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…d-time
Replaces the hardcoded CARDINAL/DIAGONAL_ROTATIONS tables with a search
over the font-db cubic source SVGs. For every (base, fill) family and
every non-base rotation index, tune_dedup.py finds the rigid D4 transform
that best maps the family's base SVG (rot 0 for even, rot 1 for odd)
onto the sibling SVG, and records it in `duplicates.json` regardless of
how good the match is. build_font.py applies an --iou-threshold filter
at build time, so moving the threshold no longer requires re-running
the 7-minute search.
Schema:
"S<sym>": {"duplicate_of": "<base>", "transform": "<name>", "iou": <float>}
8 possible transforms (the dihedral group D4): I, R90, R180, R270, M,
MR90, MR180, MR270. Image-space mirror-then-rotate; build_font maps each
to a font-space psMat.
Coverage on font-db's 37,811 symbols:
candidates recorded: 32,129 (all rotation siblings)
above 0.6 threshold: 30,154 (composite glyphs in the TTF)
below 0.6 threshold: 1,975 (kept as outlines)
e2e (RUN_E2E=1) vs upstream OneD:
before → 7,558 glyphs below IOU 0.5 (sign bug in MR90/MR270)
after → 4 glyphs below IOU 0.5
p1: 0.653 p50: 0.885 p95: 0.982 median composite IOU 0.885
The bug: my initial MR90/MR270 psMat used the "mirror reverses rotation"
intuition (270° and 90° respectively), but searches' "MR90" name means
"image-space CCW 90° after the mirror" — in font-space that composes
as scale(-1, 1) followed by rotate(+90°), not rotate(+270°). Now
mirror + the same angle, matching the search's image-space convention.
Size: 3,072 KB (62.6 % smaller than upstream's 8,206 KB).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
User review of the threshold-tuning page found that the bottom of the 0.60-accepted band (IOU 0.60–0.65) looked wrong — the composite didn't faithfully match upstream. Bumping to 0.70 trades 211 more glyphs as outlines for fewer borderline composites. composites: 30,154 → 29,943 (−211) outlines: 1,975 → 2,186 (+211) file size: 3,072 → 3,130 KB e2e below 0.5: 4 → 2 The build_font and report defaults move to 0.70 and the Makefile passes --iou-threshold 0.7 explicitly. duplicates.json is unchanged (still records every candidate regardless of IOU — threshold is build-time). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… families
Replace the auto-detected primitive system with a declarative rules
pipeline (rules.json → compositions.py → compositions.json → build_font),
plus a symbol-explorer website. The font now collapses ~21,500 hand
rotations (D4 + C8 formulas) and 301 multi-part composite glyphs.
## New pipeline
rules.json (human-authored patterns: head+marker, multiples, …)
│
▼
compositions.py (resolver): expands rules → per-symbol composite
specs with font-unit offsets, derived by canonical sub-path matching
in the source SVGs. Each glyph's own SVG `g` transform is honoured
(per-glyph scale/translate, not a single shared constant).
│
▼
compositions.json (302 entries)
│
▼
build_font.py (_apply_compositions pass) → TT composite refs
## Rule families covered
- **Forehead** S311–S313 (head + crease).
- **Eyebrows** S30a–S310 (6-fill: base+mirror+pair+head versions).
- **Eyes** S314–S31f, S320 (5-fill: inline mirror, no separate-mirror sub-base).
- **Nose + mouth-simple** S331–S334, S33b–S355 (head + marker).
- **Mouth wrinkles** S357, S358 (eyebrow pattern).
- **Mouth corners + teeth simple** S356, S362, S363, S365–S367 (head + fill-3 partial).
- **Air blow/suck** S335, S336 (eyebrow pattern, partial).
- **Movement multiples** S206/S209/S20c/S20f/S212/S218/S219/S21d/S21e — 72 entries, auto-detected N from sub-path ratios.
- **S220 SQUEEZE FLICK ALTERNATING** — 16 entries, `mix` of 3×flick + 2×squeeze (fill 0) / 2×flick + 3×squeeze (fill 1).
- **S231 paired rotation** — `S22a0i + S22a0(i+4 mod 8)`.
- **Face direction** S307 (mirror column), S308, S309 (head + fill-2 marker + rotation duplicates 8..f ≡ 0..7).
- **C8 rotation** S37f, S380 (added to tune_dedup).
## Centering
`_center_axis_font` uses the part's composed-bbox when the part is
itself an already-resolved composite, so symmetric mirror pairs
(eyebrows, eyes) stay perfectly centred even when the source SVG has
hand-drawn asymmetry. The previous source-bbox approach left S31a00
eyes 6px off-centre; now it's 0px.
## Schemas
- `multiples`: `{target_base, single, copies?}` — N copies of one base
per target variant; N auto-detected from sub-path ratio when omitted.
- `multiples.mix`: `[{single, copies}, …]` — different bases per part.
- `rotation_dedup`: `{target_base, rot_offset}` — identity duplicates
where rot R+N = rot R.
- `position_from`: chain `[[target, ref, transform?], …]` — inherit a
part's font offset from earlier-resolved compositions (keeps the
same sub-symbol's position identical across every parent).
- `center: "x"|"y"|"xy"` — geometric centring on the target's bbox.
## Other changes
- Circle replacement (optimize.py) now sizes the synthetic ellipse to
the source sub-path's actual rendered bbox (analytic cubic-Bezier
extrema), not the anchor-circle radius. Removes a ~2.5% inward
shrink that previously made head/face glyphs render visibly smaller
than the source.
- `head.yMax/yMin` post-clamp restricts the font's vertical extent to
encoded glyphs only (unencoded primitives no longer inflate it).
- C8 rotation family (S37f + S380) emits 8 rigid rotations per fill
by formula in tune_dedup.py.
- Symbol-explorer website (site.py): grouped by ISWA 2010 categories,
with hover tooltips showing composition parts, font-toggle persistence,
live-reload polling via version.txt. `make all` rebuilds everything
in dependency order (svg → fits/dedup → font → site).
- 170 regression tests covering: eyebrow overlay invariant, eye overlay
invariant, forehead/head-plus-30 oracle match, movement-multiple ink
ratio, S220 ink match, S307 mirror, S308/S309 rotation duplicates,
glyph-render fidelity vs unopt.
## Files added
- `signwriting_fonts/font_1d/rules.json` — human-authored rules.
- `signwriting_fonts/font_1d/compositions.py` — resolver.
- `signwriting_fonts/font_1d/compositions.json` — pre-resolved output.
- `signwriting_fonts/font_1d/circles.json`, `ellipsed.json` — circle
detector outputs (drive site coloring + reports).
- `signwriting_fonts/font_1d/site.py` — symbol-explorer website
generator.
- `signwriting_fonts/font_1d/test_compositions.py`,
`test_site_stats.py` — regression tests.
- `signwriting_fonts/font_1d/TODO.md` — pending composition rules.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- New svg_path.py replaces near-duplicate parsers in optimize.py and
compositions.py. Verified byte-identical output against the existing
fonts/1d/svg-opt/ and committed compositions.json.
- _apply_compositions: validate all parts before mutating the target
glyph; drop the try/except that could leave a glyph partially
rebuilt on a mid-loop FontForge error.
- _apply_duplicates: missing crossings_match/topology_match keys now
default False (fail closed) so search-based dedup without topology
evidence can't slip through.
- New _warn_dup_comp_overlap pass surfaces symkeys listed in both
duplicates.json and compositions.json (the second wins silently
otherwise).
- _clamp_head_bbox_to_encoded_glyphs subprocess: $BUILD_FONT_PYTHON →
shutil.which("python3") → "python3", instead of bare "python3".
- _TRANSFORM_MATRICES pre-computed values (no lambdas / re-invocation).
- compositions.py: drop dead _load_d, _combined_bbox, _font_translate
duplicate, canonical().
- extract.py sym-fill regex hardened to match the explicit class
attribute rather than "no slash until close".
- _symkey_cp duplicate removed from site.py and test_compositions.py;
both use the canonical symkey_to_codepoint.
- optimize.py: docstring drift fixed (MIN_ANGULAR_COVERAGE_DEG →
MAX_ANGULAR_GAP_DEG); count_circles_in_svg now shares
_fit_circle_lsq with a relaxed max_gap_deg argument.
- test_glyph_render IOU_THRESHOLDS[S21e00] 0.95→0.80: S21e is now a
multiples-composition target, drift from raw-source oracle is
expected; test_compositions.py asserts the structural invariant.
- Cleanups: unused f_xmax in compositions, unused numpy/new_ft in
report, ruff E702/E731 across changed files.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… compositions internals 61 new tests across 5 files. The integration suite (test_glyph_render, test_compositions) was render-IOU based with coarse tolerances; these tests pin the smaller pure-Python invariants so a bug at the matcher or parser layer can't hide inside an IOU margin. - test_svg_path.py: M/L/C/H/V/Z, relative cursor, implicit-L after M, scientific notation, error paths, control vs render bbox, kappa- circle parity. - test_extract.py: in-memory SQLite fixture; sym-fill stripping with reordered attributes and adjacent-element safety; SVG header well- formed. - test_optimize.py: circle LSQ recovery, angular-gap thresholds (strict vs lenient), kappa anchor placement, mixed-content paths, count_circles_in_svg lenient counter, KAPPA constant identity. - test_tune_dedup.py: D4 hand-formula expansion (rot 0/1 bases; even derive from 0, odd from 1), out-of-range bases skipped, C8 family per-fill expansion, missing-rot-0 short-circuit. - test_compositions_unit.py: symkey_to_codepoint formula + malformed- input rejection, _mirror_x reflection + winding reversal, apply_transform identity/M/error, _match_part_in_target single-part offset, rightmost tie-break, size-mismatch rejection, multi-part consistency tolerance. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Delete report.py + report.pdf (draft tooling). - Untrack the build-generated JSONs (duplicates, compositions, circles, ellipsed) — only the human-authored rules.json stays. - Move every regenerable artifact (downloads, intermediate SVGs/TTFs, generated JSON, symbol-explorer site) under fonts/tmp/. Update the Makefile to read/write the new paths and add a `clean` target. - Simplify .gitignore: a single `fonts/tmp/` instead of the per-file list. fonts/ itself is no longer ignored. - Move test_*.py into signwriting_fonts/font_1d/tests/ (with __init__.py); fix REPO_ROOT depth and the fonts/tmp/* paths. - Drop one stray f-string in font_2d/generate_vtp.py. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ork) Explains the SVG-pipeline-into-TT-composite-glyphs approach, the make targets (all/serve/watch/clean), the symbol explorer layout/decoration, and keeps the unchecked items from the old TODO under "Pending work". Completed items dropped — git history preserves the audit trail. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
GitHub renders mermaid natively; the ASCII tree was hard to scan and got out of date easily. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
AmitMY
added a commit
that referenced
this pull request
May 15, 2026
- `font-ttf-scripts` (silnrsi) isn't on PyPI; pip install was failing
with "No matching distribution found". It's only needed for the
full Makefile build (volt2ttf), not for lint/test, so dropping from
the dev extras. A pyproject comment points to the git URL for users
who need it.
- Lint: add `fail-under = 8.5` so the existing baseline (9.08/10)
passes. Warnings still surface; the score floor blocks regressions.
- Test workflow: tolerate exit 5 ("no tests collected"). The font_1d
test suite lands in #2; until then, an empty run is success.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
AmitMY
added a commit
that referenced
this pull request
May 15, 2026
) * refactor: split package into font_1d/ + font_2d/, add toolchain vars Lays the groundwork for separate 1D and 2D build pipelines without introducing any new logic: - Move `modify_ttx.py`, `generate_vtp.py`, and `boxes/` from `signwriting_fonts/` into `signwriting_fonts/font_2d/`, and add an empty `signwriting_fonts/font_1d/` package alongside it (so future 1D scripts have a place to live). - Add `FONTFORGE ?= fontforge` and `VOLT2TTF ?= volt2ttf` variables at the top of the Makefile so they're overridable from the CLI; use them in the 2D `volt2ttf` step. `brew install fontforge` installs a native arm64 CLI on Apple Silicon — no Rosetta needed. - Update the 2D build rules to reference the new module path (`signwriting_fonts.font_2d.{modify_ttx,generate_vtp}`). - Document the build tools in a new `## Development setup` section of the README, plus a short repo-layout overview. - Declare the new sub-packages in `pyproject.toml`. No behavioural change for the 2D pipeline; the 1D package is intentionally empty in this PR. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs: trim README setup; add font-ttf-scripts to dev deps - Tighten the Development setup section to just `brew install harfbuzz` + `pip install .[dev]`; drop the explanatory comments, the repo-layout block, and the duplicate install hints under "Recreating the Font". - Move `font-ttf-scripts` (provides `volt2ttf`) into the pyproject dev extras so a single `pip install .[dev]` is enough. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * makefile: drop FONTFORGE/VOLT2TTF vars; restore fontforge in README setup - Inline `volt2ttf` directly in the 2D recipe rather than via `$(VOLT2TTF)`. - Drop the FONTFORGE variable entirely; nothing in the 2D pipeline calls it yet, and 1D scripts can just invoke `fontforge` directly when added. - Put `fontforge` back into the README's `brew install` line so the development setup covers the build tools the pipelines will need. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(ci): drop font-ttf-scripts dev dep + tolerate empty test suite - `font-ttf-scripts` (silnrsi) isn't on PyPI; pip install was failing with "No matching distribution found". It's only needed for the full Makefile build (volt2ttf), not for lint/test, so dropping from the dev extras. A pyproject comment points to the git URL for users who need it. - Lint: add `fail-under = 8.5` so the existing baseline (9.08/10) passes. Warnings still surface; the score floor blocks regressions. - Test workflow: tolerate exit 5 ("no tests collected"). The font_1d test suite lands in #2; until then, an empty run is success. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
AmitMY
added a commit
that referenced
this pull request
May 15, 2026
* refactor: split package into font_1d/ + font_2d/, add toolchain vars
Lays the groundwork for separate 1D and 2D build pipelines without
introducing any new logic:
- Move `modify_ttx.py`, `generate_vtp.py`, and `boxes/` from
`signwriting_fonts/` into `signwriting_fonts/font_2d/`, and add an
empty `signwriting_fonts/font_1d/` package alongside it (so future
1D scripts have a place to live).
- Add `FONTFORGE ?= fontforge` and `VOLT2TTF ?= volt2ttf` variables
at the top of the Makefile so they're overridable from the CLI;
use them in the 2D `volt2ttf` step. `brew install fontforge`
installs a native arm64 CLI on Apple Silicon — no Rosetta needed.
- Update the 2D build rules to reference the new module path
(`signwriting_fonts.font_2d.{modify_ttx,generate_vtp}`).
- Document the build tools in a new `## Development setup` section of
the README, plus a short repo-layout overview.
- Declare the new sub-packages in `pyproject.toml`.
No behavioural change for the 2D pipeline; the 1D package is intentionally
empty in this PR.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs: trim README setup; add font-ttf-scripts to dev deps
- Tighten the Development setup section to just `brew install harfbuzz`
+ `pip install .[dev]`; drop the explanatory comments, the repo-layout
block, and the duplicate install hints under "Recreating the Font".
- Move `font-ttf-scripts` (provides `volt2ttf`) into the pyproject dev
extras so a single `pip install .[dev]` is enough.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* makefile: drop FONTFORGE/VOLT2TTF vars; restore fontforge in README setup
- Inline `volt2ttf` directly in the 2D recipe rather than via `$(VOLT2TTF)`.
- Drop the FONTFORGE variable entirely; nothing in the 2D pipeline calls
it yet, and 1D scripts can just invoke `fontforge` directly when added.
- Put `fontforge` back into the README's `brew install` line so the
development setup covers the build tools the pipelines will need.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(1d): rebuild OneD font from font-db cubic SVG sources
Issue #1: regenerate the SignWriting 1D font using font-db's cubic-Bezier
source SVGs as the source of truth, instead of relying on the existing
quadratic-approximated TTFs.
Adds a four-step pipeline (extract → optimize → build → vtp + volt2ttf):
- `font_1d/extract.py` reads `iswa2010.db` (fetched by the Makefile) and
writes one SVG per symbol. Drops the `sym-fill` (white-interior) path so
the resulting monochrome 1D glyph renders correctly as a ring.
- `font_1d/optimize.py` detects circular sub-paths via LSQ circle fit
(robust to the one extra "stitch" anchor that font-db's closed loops
include) and replaces them with synthetic 4-segment cubic-Bezier
ellipses — both shrinks the path data and removes hand-traced wobble.
- `font_1d/build_font.py` is a FontForge Python script that imports each
SVG, maps the symkey to its SWU plane-4 codepoint, and emits a base TTF.
- `font_1d/generate_vtp.py` emits a minimal VTP; `volt2ttf` combines it
with the base TTF to produce the final font.
Also reorganises into `signwriting_fonts/font_1d/` and
`signwriting_fonts/font_2d/` for clarity, and updates the 2D Makefile
targets to the new module paths.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(1d): scale imported glyphs by SVG width to restore relative proportions
FontForge's importOutlines() ignores the SVG's width/height attributes and
fits every glyph to roughly the em-square, so S2ff00 (big circle), S17600
(small ring), and S21e00 (two-dot) all came out at ~294 units wide — the
big circle and the dots looked the same size.
The upstream OneD font sizes each glyph as roughly `svg_width × 10` in font
units (e.g. S2ff00 with svg width=36 → bbox 358; S17600 with svg width=16 →
bbox 156). After importOutlines we now read width/height out of the SVG
header and apply a uniform `psMat.scale()` so the imported bbox matches
`svg_width × 10 by svg_height × 10`, re-anchor to (0, -descent), and set
the advance width to `target_width + 20` for a small side-bearing.
The comparison image now matches the original OneD's relative sizing:
S2ff00 is the largest, S17600 is half that, S21e00 dots are smaller still.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(1d): centre glyphs on the OneD baseline (cy=166, lsb=20)
PR #2 review feedback: "the sizes of the circles are now correct, but
the locations are not." Each glyph was being placed with its bottom at
-descent, which lined glyph bottoms up at y=-205 instead of giving them
a shared visual centre. The upstream OneD font instead places every
glyph so its bbox centre is at y≈166 (read off the original glyf table)
and pads the left side by 20 font units, with a matching right
side-bearing.
After this change the new font's per-glyph coordinates match the
upstream OneD's within ±2 units:
S10000 orig 20..170 / 15..315 new 20..170 / 16..316
S17600 orig 20..176 / 87..244 new 20..180 / 86..246
S21e00 orig 20..159 / 138..194 new 20..160 / 137..195
S2ff00 orig 20..378 / -8..341 new 20..370 / -9..341
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(1d): full-font build, report.pdf, IOU regression tests
Addresses the review feedback on #2:
1. **Full font** — `extract.py` now defaults to all ~37k symbols and accepts
`--symbols dev` for the hand-picked dev subset. Full pipeline runs in
~25s end-to-end (extract → optimize → FontForge build).
2. **report.pdf** at `assets/regen/report.pdf` replaces the old
`comparison.png`. Three pages:
- file-size + glyph-count tables (original 8014 KB, new unopt 7227 KB,
new ellipse-opt 7115 KB — 11.2 % smaller than upstream),
- circles (S21e00, S2ff00, S17600) rendered in all three variants,
- rotation family S10000–S10005 (placeholder for future GSUB dedup).
New `signwriting_fonts/font_1d/report.py` builds it with matplotlib's
PdfPages; new `report.pdf` Makefile target wires it in.
3. **Tests** at `signwriting_fonts/font_1d/test_glyph_render.py`:
- per-symbol scale+placement test: render each glyph in the original
OneD and our new font, content-bbox-align both renderings, assert
IOU above a per-symbol threshold (~0.05 below measured baseline).
Catches drift in scale/positioning without needing pixel-perfect
equality across rasterizers.
- smoke tests: every symbol mapped at the right SWU codepoint;
optimized ≤ unoptimized ≤ original in bytes.
- skipped placeholder for the rotation-family GSUB dedup test.
Refactor: `symkey_to_codepoint` moved into `_symkey.py` so report.py and
the tests can use it without dragging in FontForge's Python interpreter.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(1d): add 505 structural markers from upstream source SVGs
Pulls SW A/B/L/M/R + SW 250..749 (the FSW box delimiters and coordinate
glyphs that the upstream OneD font carries on plane 1) directly from
Slevinski/signwriting_2010_fonts/source/other_svg.zip and imports them
alongside the font-db symbol SVGs.
- New Makefile targets fetch other_svg.zip and unpack into
fonts/1d/markers/ (gitignored alongside fonts/1d/).
- `build_font.py` gets a `--markers-dir` flag and an `_import_marker`
step that:
* names each glyph "SW A", "SW M", "SW 250", … (matching upstream),
* maps to plane-1 codepoints — letters at 0x1D800..0x1D804, numbers
at 0x1D80C + (n - 250) (the 0x1D805..0x1D80B range is reserved,
same gap the upstream font has),
* keeps the 1:1 SVG-to-font-unit scale these glyphs ship at (markers
don't get the 10× scale that symbol SVGs do),
* applies the upstream's per-class side-bearings (20 for letters,
10 for numbers) and bottom-anchors at y=25.
Per-glyph dimensions now match the upstream OneD byte-for-byte for
SW A/M/250/500/749; total mapped codepoints 38,316 (= 37,811 symbols
+ 505 markers), the same as the upstream font.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(1d): dedup cardinal rotations via TrueType composite glyphs
Replaces each (base, fill) family's cardinal rotation siblings (rot
2/4/6 plus their mirrors 8/a/c/e) with composite glyphs that reference
the rot-0 base outline plus a 2×2 transform matrix:
rot 2: rotate +90° CCW
rot 4: rotate 180°
rot 6: rotate 270° CCW
rot 8: mirror
rot A: mirror + 270° CCW (mirror flips rotation direction)
rot C: mirror + 180°
rot E: mirror + 90° CCW
Verified empirically vs the upstream OneD: each cardinal sibling lands
above IOU 0.95 after this transform. Diagonals (rot 1/3/5/7/9/B/D/F)
have IOU ~0.45 with any rigid transform — they're hand-redrawn in the
source SVGs — so we leave them as independent outlines.
Replaced 43,604 glyphs with composites.
Size impact (apples-to-apples, full 38,316-codepoint fonts):
upstream OneD 8,206,192 B 8014 KB
new (no optimisations) 7,537,524 B 7361 KB −8.1 %
new (ellipse + rotation dedup) 5,309,584 B 5185 KB −35.3 %
So the rotation dedup alone contributes the bulk of the saving
(~28 % on top of the 8 % baseline drop, which came from stripping
the sym-fill layer and a fresh FontForge build).
`build_font.py` gets a `--no-rotation-dedup` flag so the report can
isolate the dedup's contribution. New tests
`test_cardinal_rotation_dedup_renders_correctly` (IOU ≥ 0.6 vs
upstream for 8 cardinal samples spanning two families) and
`test_cardinal_rotation_is_actually_a_composite` (sanity check that
they're stored as composite glyphs in the glyf table) pin this
behaviour. The previously-skipped `test_rotation_family_matches_via_gsub_transform`
placeholder is replaced by these.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(1d): dedup diagonals too — 2 outlines per family (~67% smaller)
Building on the cardinal dedup, the diagonal indices (1, 3, 5, 7, 9, B, D, F)
turn out to form their own internally-consistent family centred on rot 1:
empirical IOU search vs the upstream OneD lands every diagonal sibling
above 0.94 with a pure rotate/mirror of rot 1.
So each `(base, fill)` family now keeps exactly two outlines — rot 0 for
the cardinals, rot 1 for the diagonals — and every other orientation is
a composite glyph referencing one of those two bases with a rotate/mirror
matrix.
Replaced 87,921 glyphs with composites (was 43,604 with cardinals-only).
Cumulative size (apples-to-apples, full 38,316-codepoint fonts):
upstream OneD 8,206,192 B 8014 KB —
new (no optimisations) 7,537,524 B 7361 KB −8.1 %
new (ellipse + cardinal+diagonal dedup) 2,668,892 B 2606 KB −67.5 %
Adds 7 diagonal samples to `test_cardinal_rotation_dedup_renders_correctly`
and `test_cardinal_rotation_is_actually_a_composite` (48 tests pass).
Also adds a "Rotation + reflection encoding" page to `report.pdf` that
spells out the last-hex-digit → orientation → composite-source mapping
for all 16 orientations, and extends the rotation-family comparison page
to cover all 16 (was 6).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(1d): metrics + ellipse precision + rotation dimension check
Three related quality fixes uncovered by a new full-font e2e IOU test:
1. Vertical metrics override
FontForge derived hhea/OS/2 ascent/descent from the actual glyph data,
which is slightly larger here because composite-rotation bboxes can push
outside the per-glyph bbox. The result was that hb-view (and any other
browser/UA) sized our font's line-height ~3% taller than upstream, so the
same glyph rendered at the same point size took up a TALLER PNG and looked
visually smaller side-by-side. Lock hhea / OS/2 metrics to match the
upstream OneD values so all renders match.
2. Ellipse detector
The previous detector replaced any closed sub-path whose anchors fit a
circle within 5 % radius error. For glyphs like S15401 (a hand-traced
"comb" with an outer outline that traces a near-circular arc), that
replaced the outline with a synthetic full-circle ellipse, turning the
comb into a filled disc — and every diagonal-rotation composite of S15401
inherited the wrong base.
Tightened to 2.5 % tolerance and added an angular-coverage check
(anchors must span ≥ 300 ° around the fitted centre). Ring outlines still
qualify (all observed errors well under 1 %); the false-positive comb-arc
sub-path no longer matches. Drops ellipse replacements from 4,421 to 128.
3. Rotation dim-compatibility check
_dedup_rotations now skips siblings whose bbox dimensions don't match the
expected (W, H) or (H, W) of the base — catches families where rot 2/4/6
are independently hand-redrawn at incompatible sizes (e.g. S10b40 is
160×290 but S10b42 is 260×150). Without the check we'd composite-rotate
a 160×290 base into a 290×160 slot, producing IOU ~0.
e2e (RUN_E2E=1) results on 38,316 glyphs vs upstream OneD:
before after
glyphs below 0.5 2948 627 −78 %
p1 IOU 0.050 0.340 +6.8x
p5 IOU 0.192 0.713 +3.7x
p50 IOU 0.878 0.884 —
p95 IOU 0.982 0.983 —
Size: 3127 KB (vs upstream 8014 KB; −61 %). Slightly larger than the
2606 KB peak because the dim-compat check keeps ~6k more original
outlines instead of producing wrong composites.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(report): bucketed IOU samples + 0.40-0.50 zoom + layout fixes
- Known Issues page now samples one example per 0.05 IOU bucket across
the full range so a reviewer can scroll low→high and pick a cutoff.
Renders are black-on-white (was white-on-black) to match the rest of
the report.
- New "Threshold zoom" page(s) show up to 20 evenly-spaced examples
from the [0.40, 0.50) IOU band, sorted low→high, so the call between
"acceptable composite" and "reject" can be confirmed visually.
- Footer prints both n_below_0.44 and n_below_0.50 counts.
- Motivation page (page 1) prefaces the report: why bother regenerating
a font that already exists — cubic→quadratic loss + rotation outline
duplication.
- Rotation table page: title and intro paragraph repositioned so they
no longer overlap; intro restricted to hand symbols (non-hands keep
their outlines).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(1d): search-based dedup with duplicates.json + threshold at build-time
Replaces the hardcoded CARDINAL/DIAGONAL_ROTATIONS tables with a search
over the font-db cubic source SVGs. For every (base, fill) family and
every non-base rotation index, tune_dedup.py finds the rigid D4 transform
that best maps the family's base SVG (rot 0 for even, rot 1 for odd)
onto the sibling SVG, and records it in `duplicates.json` regardless of
how good the match is. build_font.py applies an --iou-threshold filter
at build time, so moving the threshold no longer requires re-running
the 7-minute search.
Schema:
"S<sym>": {"duplicate_of": "<base>", "transform": "<name>", "iou": <float>}
8 possible transforms (the dihedral group D4): I, R90, R180, R270, M,
MR90, MR180, MR270. Image-space mirror-then-rotate; build_font maps each
to a font-space psMat.
Coverage on font-db's 37,811 symbols:
candidates recorded: 32,129 (all rotation siblings)
above 0.6 threshold: 30,154 (composite glyphs in the TTF)
below 0.6 threshold: 1,975 (kept as outlines)
e2e (RUN_E2E=1) vs upstream OneD:
before → 7,558 glyphs below IOU 0.5 (sign bug in MR90/MR270)
after → 4 glyphs below IOU 0.5
p1: 0.653 p50: 0.885 p95: 0.982 median composite IOU 0.885
The bug: my initial MR90/MR270 psMat used the "mirror reverses rotation"
intuition (270° and 90° respectively), but searches' "MR90" name means
"image-space CCW 90° after the mirror" — in font-space that composes
as scale(-1, 1) followed by rotate(+90°), not rotate(+270°). Now
mirror + the same angle, matching the search's image-space convention.
Size: 3,072 KB (62.6 % smaller than upstream's 8,206 KB).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* chore: remove obsolete exceptions.json (replaced by duplicates.json)
* build(1d): bump composite-IOU threshold 0.60 → 0.70
User review of the threshold-tuning page found that the bottom of the
0.60-accepted band (IOU 0.60–0.65) looked wrong — the composite didn't
faithfully match upstream. Bumping to 0.70 trades 211 more glyphs as
outlines for fewer borderline composites.
composites: 30,154 → 29,943 (−211)
outlines: 1,975 → 2,186 (+211)
file size: 3,072 → 3,130 KB
e2e below 0.5: 4 → 2
The build_font and report defaults move to 0.70 and the Makefile passes
--iou-threshold 0.7 explicitly. duplicates.json is unchanged (still
records every candidate regardless of IOU — threshold is build-time).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(1d): rule-based composition system + 301 compositions across 30+ families
Replace the auto-detected primitive system with a declarative rules
pipeline (rules.json → compositions.py → compositions.json → build_font),
plus a symbol-explorer website. The font now collapses ~21,500 hand
rotations (D4 + C8 formulas) and 301 multi-part composite glyphs.
## New pipeline
rules.json (human-authored patterns: head+marker, multiples, …)
│
▼
compositions.py (resolver): expands rules → per-symbol composite
specs with font-unit offsets, derived by canonical sub-path matching
in the source SVGs. Each glyph's own SVG `g` transform is honoured
(per-glyph scale/translate, not a single shared constant).
│
▼
compositions.json (302 entries)
│
▼
build_font.py (_apply_compositions pass) → TT composite refs
## Rule families covered
- **Forehead** S311–S313 (head + crease).
- **Eyebrows** S30a–S310 (6-fill: base+mirror+pair+head versions).
- **Eyes** S314–S31f, S320 (5-fill: inline mirror, no separate-mirror sub-base).
- **Nose + mouth-simple** S331–S334, S33b–S355 (head + marker).
- **Mouth wrinkles** S357, S358 (eyebrow pattern).
- **Mouth corners + teeth simple** S356, S362, S363, S365–S367 (head + fill-3 partial).
- **Air blow/suck** S335, S336 (eyebrow pattern, partial).
- **Movement multiples** S206/S209/S20c/S20f/S212/S218/S219/S21d/S21e — 72 entries, auto-detected N from sub-path ratios.
- **S220 SQUEEZE FLICK ALTERNATING** — 16 entries, `mix` of 3×flick + 2×squeeze (fill 0) / 2×flick + 3×squeeze (fill 1).
- **S231 paired rotation** — `S22a0i + S22a0(i+4 mod 8)`.
- **Face direction** S307 (mirror column), S308, S309 (head + fill-2 marker + rotation duplicates 8..f ≡ 0..7).
- **C8 rotation** S37f, S380 (added to tune_dedup).
## Centering
`_center_axis_font` uses the part's composed-bbox when the part is
itself an already-resolved composite, so symmetric mirror pairs
(eyebrows, eyes) stay perfectly centred even when the source SVG has
hand-drawn asymmetry. The previous source-bbox approach left S31a00
eyes 6px off-centre; now it's 0px.
## Schemas
- `multiples`: `{target_base, single, copies?}` — N copies of one base
per target variant; N auto-detected from sub-path ratio when omitted.
- `multiples.mix`: `[{single, copies}, …]` — different bases per part.
- `rotation_dedup`: `{target_base, rot_offset}` — identity duplicates
where rot R+N = rot R.
- `position_from`: chain `[[target, ref, transform?], …]` — inherit a
part's font offset from earlier-resolved compositions (keeps the
same sub-symbol's position identical across every parent).
- `center: "x"|"y"|"xy"` — geometric centring on the target's bbox.
## Other changes
- Circle replacement (optimize.py) now sizes the synthetic ellipse to
the source sub-path's actual rendered bbox (analytic cubic-Bezier
extrema), not the anchor-circle radius. Removes a ~2.5% inward
shrink that previously made head/face glyphs render visibly smaller
than the source.
- `head.yMax/yMin` post-clamp restricts the font's vertical extent to
encoded glyphs only (unencoded primitives no longer inflate it).
- C8 rotation family (S37f + S380) emits 8 rigid rotations per fill
by formula in tune_dedup.py.
- Symbol-explorer website (site.py): grouped by ISWA 2010 categories,
with hover tooltips showing composition parts, font-toggle persistence,
live-reload polling via version.txt. `make all` rebuilds everything
in dependency order (svg → fits/dedup → font → site).
- 170 regression tests covering: eyebrow overlay invariant, eye overlay
invariant, forehead/head-plus-30 oracle match, movement-multiple ink
ratio, S220 ink match, S307 mirror, S308/S309 rotation duplicates,
glyph-render fidelity vs unopt.
## Files added
- `signwriting_fonts/font_1d/rules.json` — human-authored rules.
- `signwriting_fonts/font_1d/compositions.py` — resolver.
- `signwriting_fonts/font_1d/compositions.json` — pre-resolved output.
- `signwriting_fonts/font_1d/circles.json`, `ellipsed.json` — circle
detector outputs (drive site coloring + reports).
- `signwriting_fonts/font_1d/site.py` — symbol-explorer website
generator.
- `signwriting_fonts/font_1d/test_compositions.py`,
`test_site_stats.py` — regression tests.
- `signwriting_fonts/font_1d/TODO.md` — pending composition rules.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(1d): consolidate SVG parser + fix build_font correctness
- New svg_path.py replaces near-duplicate parsers in optimize.py and
compositions.py. Verified byte-identical output against the existing
fonts/1d/svg-opt/ and committed compositions.json.
- _apply_compositions: validate all parts before mutating the target
glyph; drop the try/except that could leave a glyph partially
rebuilt on a mid-loop FontForge error.
- _apply_duplicates: missing crossings_match/topology_match keys now
default False (fail closed) so search-based dedup without topology
evidence can't slip through.
- New _warn_dup_comp_overlap pass surfaces symkeys listed in both
duplicates.json and compositions.json (the second wins silently
otherwise).
- _clamp_head_bbox_to_encoded_glyphs subprocess: $BUILD_FONT_PYTHON →
shutil.which("python3") → "python3", instead of bare "python3".
- _TRANSFORM_MATRICES pre-computed values (no lambdas / re-invocation).
- compositions.py: drop dead _load_d, _combined_bbox, _font_translate
duplicate, canonical().
- extract.py sym-fill regex hardened to match the explicit class
attribute rather than "no slash until close".
- _symkey_cp duplicate removed from site.py and test_compositions.py;
both use the canonical symkey_to_codepoint.
- optimize.py: docstring drift fixed (MIN_ANGULAR_COVERAGE_DEG →
MAX_ANGULAR_GAP_DEG); count_circles_in_svg now shares
_fit_circle_lsq with a relaxed max_gap_deg argument.
- test_glyph_render IOU_THRESHOLDS[S21e00] 0.95→0.80: S21e is now a
multiples-composition target, drift from raw-source oracle is
expected; test_compositions.py asserts the structural invariant.
- Cleanups: unused f_xmax in compositions, unused numpy/new_ft in
report, ruff E702/E731 across changed files.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* test(1d): add unit tests for svg_path, extract, optimize, tune_dedup, compositions internals
61 new tests across 5 files. The integration suite (test_glyph_render,
test_compositions) was render-IOU based with coarse tolerances; these
tests pin the smaller pure-Python invariants so a bug at the matcher
or parser layer can't hide inside an IOU margin.
- test_svg_path.py: M/L/C/H/V/Z, relative cursor, implicit-L after M,
scientific notation, error paths, control vs render bbox, kappa-
circle parity.
- test_extract.py: in-memory SQLite fixture; sym-fill stripping with
reordered attributes and adjacent-element safety; SVG header well-
formed.
- test_optimize.py: circle LSQ recovery, angular-gap thresholds
(strict vs lenient), kappa anchor placement, mixed-content paths,
count_circles_in_svg lenient counter, KAPPA constant identity.
- test_tune_dedup.py: D4 hand-formula expansion (rot 0/1 bases; even
derive from 0, odd from 1), out-of-range bases skipped, C8 family
per-fill expansion, missing-rot-0 short-circuit.
- test_compositions_unit.py: symkey_to_codepoint formula + malformed-
input rejection, _mirror_x reflection + winding reversal,
apply_transform identity/M/error, _match_part_in_target single-part
offset, rightmost tie-break, size-mismatch rejection, multi-part
consistency tolerance.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* chore(1d): un-draft the PR — build artifacts live under fonts/tmp/
- Delete report.py + report.pdf (draft tooling).
- Untrack the build-generated JSONs (duplicates, compositions,
circles, ellipsed) — only the human-authored rules.json stays.
- Move every regenerable artifact (downloads, intermediate SVGs/TTFs,
generated JSON, symbol-explorer site) under fonts/tmp/. Update the
Makefile to read/write the new paths and add a `clean` target.
- Simplify .gitignore: a single `fonts/tmp/` instead of the per-file
list. fonts/ itself is no longer ignored.
- Move test_*.py into signwriting_fonts/font_1d/tests/ (with
__init__.py); fix REPO_ROOT depth and the fonts/tmp/* paths.
- Drop one stray f-string in font_2d/generate_vtp.py.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(1d): convert TODO.md to README.md (goal, build, serve, pending work)
Explains the SVG-pipeline-into-TT-composite-glyphs approach,
the make targets (all/serve/watch/clean), the symbol explorer
layout/decoration, and keeps the unchecked items from the old TODO
under "Pending work". Completed items dropped — git history
preserves the audit trail.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(1d): pipeline as a mermaid graph instead of ASCII
GitHub renders mermaid natively; the ASCII tree was hard to scan and
got out of date easily.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(ci): declare numpy + Pillow as dev deps
optimize.py imports numpy at module load (LSQ circle fit), and the
render-IOU tests import both numpy and Pillow. CI's bare `pip install
.[dev]` was succeeding without them, so the test module imports
failed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(ci): module-level skip on test_compositions when hb-view/fonts missing
The render-IOU tests in test_compositions.py raised FileNotFoundError
on CI (no hb-view binary, no built TTFs) instead of skipping cleanly.
test_glyph_render.py was already guarded by per-test _require() calls;
test_compositions.py wasn't.
Adding a pytestmark at module load: skip the whole file if either
hb-view is missing from PATH, or the built fonts don't exist in
fonts/tmp/. Local devs who've run `make all` still get full coverage.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1 (partially — sets up the pipeline; further iteration is follow-up).
Summary
End-to-end pipeline that rebuilds
SuttonSignWritingOneD.ttffrom the cubic-Bezier source SVGs in@sutton-signwriting/font-db, with two dedup layers on top:rules.jsondecompose ~300 face/movement glyphs into multi-part composite references (head + marker, N copies of a single, mirror pair, paired rotation, identity duplicate). The resolver auto-derives each part's offset by canonical sub-path matching in the source SVGs.New pipeline
Composition schemas
Authored in
rules.json— the resolver expands them per-base/per-rotation and writescompositions.json:{target, parts: [{ref, transform?, center?, position_from?}, …]}— each part is matched in target by canonical sub-path, offset derived in font units.{target_base, single, copies?}— every variant oftarget_base= N copies ofsingle. N auto-detected from sub-path ratio when omitted.[{single, copies}, …]— different base symbols per copy (S220 SQUEEZE FLICK ALTERNATING).{target_base, rot_offset}—rot R+N ≡ rot Ridentity duplicates (S308, S309).[[target, ref, transform?], …]— inherit a part's font offset from an earlier-resolved composition (keeps the same sub-symbol at identical positions across siblings; eg. right eyebrow at the same place in head+right and head+both).Families covered (~300 compositions)
S22a0i + S22a0(i+4 mod 8)tune_dedup.pyRender fidelity
head.yMax/yMinpost-clamped to encoded-glyph extents (unencoded internal helper glyphs no longer inflate the font's vertical metric, so hb-view / browsers render text at the same scale as the upstream OneD font).gtransform + import pipeline rather than using a single shared path-to-font scale — accommodates the ~1% per-glyph scale variation in the source SVGs.Symbol-explorer website
assets/regen/symbols/index.html(build withmake all, serve withmake serveon port 8000):version.txt.Test plan
make fonts/SignWritingOneD-base.ttfproduces a valid TTFhb-viewrenders test circles as rings (not filled discs)make allbuilds the explorer site end-to-endS{base}10 + S{base}20 ≈ S{base}00Toolchain
importOutlines+generate(.ttf).pip install font-ttf-scripts.optimize.pyfor circle fits and analytic Bezier bbox.Build totals (after this PR)
Pending follow-ups
Recorded in
signwriting_fonts/font_1d/TODO.md:head + markerwithout a head-shape transform.🤖 Generated with Claude Code