refactor: improve naming, structure, and documentation with regression proof - #253
Merged
Merged
Conversation
Structural refactoring with full regression proof via Regrets. All 10 behavioral clusters GREEN, all 5 chains MATCH, all 40 outputs identical to pre-refactor baseline. ## Naming Improvements - SimpleEpitran._non_deterministic_mappings → _find_ambiguous_mappings (clearer: finds one-to-many grapheme-phoneme mappings) - SimpleEpitran._load_g2p_map → _load_grapheme_to_phoneme_map (self-documenting: no more opaque 'g2p' abbreviation) - SimpleEpitran._construct_regex → _build_greedy_match_regex (explains maximal munch tokenization strategy) - SimpleEpitran.is_korean → contains_korean_syllables (precise: detects Hangul syllable characters, not 'Korean-ness') - ligaturize → convert_affricates_to_ligatures (self-explanatory name; backward-compatible alias preserved) - StripDiacritics.process → strip_specified_diacritics (no more generic 'process'; backward-compatible alias preserved) - Rules._fields_to_function → _compile_replacement_rule (describes what is compiled: a context-sensitive replacement rule) - Rules._fields_to_function_metathesis → _compile_metathesis_rule (same pattern: describes the rule type being compiled) - Rules._sub_symbols → _expand_symbol_references (describes transformation: expanding ::symbol:: references) ## Structural Improvements - Extract AFFRICATE_LIGATURES module-level constant in ligaturize.py (mapping data separated from function logic) - Extract SPECIAL_LANGUAGE_BACKENDS module-level constant in _epitran.py (special backends dict was buried inside class; now discoverable) - Add epitran/adapters/ module for Regrets regression testing (thin wrapper modules exposing instance methods as standalone functions) ## Documentation - Added docstrings to renamed methods explaining their purpose - Added JSDoc-style descriptions for extracted constants ## Backward Compatibility All renamed public methods retain their old names as aliases: - ligaturize = convert_affricates_to_ligatures - StripDiacritics.process = strip_specified_diacritics - SimpleEpitran.is_korean = contains_korean_syllables - Epitran.special = SPECIAL_LANGUAGE_BACKENDS ## Regression Proof (via Regrets) KEBENARAN 1 (pre-refactor raw output, 40 test cases): All outputs IDENTICAL after refactoring. KEBENARAN 2 (pre-refactor fingerprints + chain hashes): All 10 fingerprints MATCH. All 5 chain hashes MATCH. | Cluster | Before | After | Match | |---------|--------|-------|-------| | spa-transliterate | 5x4d98i | 5x4d98i | ✅ | | deu-transliterate | 5j37svh | 5j37svh | ✅ | | fra-transliterate | 3f8syig | 3f8syig | ✅ | | spa-strict-trans | 5x4d98i | 5x4d98i | ✅ | | spa-word-to-tuples | 6724o1o | 6724o1o | ✅ | | ligaturize | 5pxpkk6 | 5pxpkk6 | ✅ | | puncnorm | 1z6dwyb | 1z6dwyb | ✅ | | strip-diacritics | 2daij2n | 2daij2n | ✅ | | ipa-to-xsampa | 5u5264e | 5u5264e | ✅ | | rules-apply | 4mcbm7s | 4mcbm7s | ✅ | Chain hashes (before → after): | Chain | Before | After | Match | |-------|--------|-------|-------| | spanish-to-xsampa | 389p9re | 389p9re | ✅ | | spanish-to-ligatures | 6w1sy7s | 6w1sy7s | ✅ | | french-pipeline | 4ue8mxi | 4ue8mxi | ✅ | | german-pipeline | 9ljbs80 | 9ljbs80 | ✅ | | preprocess-transliterate-postprocess | 2slbp3y | 2slbp3y | ✅ |
Collaborator
|
Why are adapters necessary here? The fingerprints (I'm not sure what this is too) doesn't cover numerous languages that epitran covers too. If not being reasoned more better than this, I prefer to revert this commit, especially |
Merged
Collaborator
|
FYI |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR improves naming clarity, structural organization, and documentation across core epitran modules, with full behavioral regression proof via Regrets.
All 10 behavioral clusters GREEN, all 5 chain tests MATCH, all 40 outputs identical to pre-refactor baseline.
Naming Improvements
_non_deterministic_mappings_find_ambiguous_mappings_load_g2p_map_load_grapheme_to_phoneme_map_construct_regex_build_greedy_match_regexis_koreancontains_korean_syllablesligaturizeconvert_affricates_to_ligaturesStripDiacritics.processstrip_specified_diacritics_fields_to_function_compile_replacement_rule_fields_to_function_metathesis_compile_metathesis_rule_sub_symbols_expand_symbol_referencesStructural Improvements
AFFRICATE_LIGATURESmodule-level constant inligaturize.py— mapping data separated from function logicSPECIAL_LANGUAGE_BACKENDSmodule-level constant in_epitran.py— special backends dict was buried inside class; now discoverableepitran/adapters/module — thin wrapper modules for regression testing of class-based APIsBackward Compatibility
All renamed public methods retain their old names as aliases:
ligaturize = convert_affricates_to_ligaturesStripDiacritics.process = strip_specified_diacriticsSimpleEpitran.is_korean = contains_korean_syllablesEpitran.special = SPECIAL_LANGUAGE_BACKENDSRegression Proof
Tested with Regrets — output-based regression testing:
VERIFICATION 1 — Regrets Fingerprint
All 10 clusters GREEN after refactor.
VERIFICATION 2 — Raw Output vs Pre-Refactor Baseline
All 40 outputs IDENTICAL to pre-refactor baseline.
VERIFICATION 3 — Fingerprint Cross-Check
All 10 fingerprints match pre-refactor truth.
VERIFICATION 4 — Chain Validation
All 5 chains MATCH pre-refactor chain hashes.
Fingerprint Before/After
Chain Hashes Before/After