Skip to content

v0.7.9 detection audit: inflection, domains, and collision fixes - #3

Merged
mianaz merged 1 commit into
mainfrom
claude/jargonslayer-detection-conflicts-lqyyc9
Sep 2, 2026
Merged

v0.7.9 detection audit: inflection, domains, and collision fixes#3
mianaz merged 1 commit into
mainfrom
claude/jargonslayer-detection-conflicts-lqyyc9

Conversation

@mianaz

@mianaz mianaz commented Aug 31, 2026

Copy link
Copy Markdown
Owner

Summary

This PR implements comprehensive improvements to dictionary detection, domain tracking, and collision handling based on the v0.7.9 detection audit. The changes focus on fixing under-detection issues with phrasal idioms, improving domain-based common word suppression, and ensuring deterministic behavior when multiple packs define the same entry.

Key Changes

Expression Inflection (Under-detection Fix)

  • First-word inflection: Expressions now apply inflection tolerance to both the first AND last word, not just the last. This fixes detection of phrasal idioms where the verb is the first word (e.g., "circling back", "pushing back", "reading the room" now match their base forms).
  • E-aware inflection: Words ending in 'e' now use stem-based patterns ("circle" → circl(?:e|es|ed|ing)), while other words use suffix patterns. The bare 'd' suffix is removed from non-e-ending words to eliminate false positives like "wind" matching "win".
  • Middle words stay literal: Only first and last words flex; middle words must match exactly to prevent spurious matches.

Entry-Level Domain Hints

  • Added domains?: DomainTag[] field to DictTermEntry and DictExpressionEntry to support cross-domain packs (like modern-usage) that aren't mapped in PACK_DOMAINS.
  • Common word entries in cross-domain packs now unlock when ANY of their listed domains becomes active, fixing under-detection where AI meeting vocabulary (agent, harness, compute) never activated the "ml" domain.
  • Domain evidence is only emitted on the wire for non-common entries, preserving the invariant that common words don't count toward domain activation.

Term Inflection and Pluralization

  • Added explicit verb variants to "churn" (churning, churned) since terms only get automatic plural tolerance.
  • Implemented automatic plural tolerance for non-acronym terms ≥5 characters ending in lowercase letters (e.g., "epochs", "embeddings").
  • All-caps acronyms now take optional lowercase 's' for written plurals ("KPIs", "PRs").

Cross-Pack Collision Resolution

  • Expression-key collisions: Remote packs now deterministically win over built-in entries when defining the same expression, matching the existing term-key collision behavior (F1 fix).
  • Near-duplicate collapse: Explicit lookups collapse entries with identical Chinese glosses to one card, while genuinely different senses (like SAM in business vs. bioinformatics) are preserved.
  • Regression collision: Tech-terms vs. ml-stats "regression" now resolves deterministically based on pack selection.

Selection Lookup Improvements

  • Personal glossary integration: User's own glossary entries are now searched first and shadow built-in dictionary entries.
  • Offline translation: Translation of selected text now runs for every cross-language lookup, independent of AI detection settings, enabling offline 划词翻译 with local translation providers.
  • Silent translation upgrade: When a dictionary hit occurs, translation runs asynchronously and upgrades the finished entry in place without creating a separate task.
  • Graceful degradation: Translation failures are silently omitted when the provider is unavailable (downloading, no key, etc.).

Domain Tracking

  • Entry-level domains are now counted toward domain activation, allowing cross-domain packs to contribute evidence.
  • Unrecognized domain strings are validated and dropped; domain hints are capped at 4 values.
  • Common word hits never count toward activating a second domain (everyday senses are not evidence).

Notable Implementation Details

  • The inflectedWordPattern() function encapsulates the e-aware inflection logic for reuse across expressions.
  • buildExpressionRegex() now applies inflection to both first and last words via a unified pattern function.
  • shouldIncludeCommonWord() checks entry-level domains before falling back to pack-level PACK_DOMAINS.
  • translateSelection() implements bounded retry for cold-start scenarios (on-device model priming).
  • Remote pack validation now sanitizes and deduplicates domain hints via clampDomains().
  • Tests extensively document the former "REAL BEHAVIOR

https://claude.ai/code/session_013NWYJSE7af6gYR5FciXQ3o

…r-detection, offline 划词翻译

Under-detection (the dictionary was on, the words were in it, and no
card ever appeared):

- Expression inflection now covers the FIRST word as well as the last,
  e-aware ("circle" -> circl(?:e|es|ed|ing)) — phrasal idioms inflect
  their leading verb, so "circling back", "pushed back", "reads the
  room" all used to miss unless a data author had hand-listed the
  variant. The bare "-d" suffix is dropped from non-e endings (it only
  ever manufactured false surfaces like "quick wind").
- Terms get light plural tolerance: lowercase headwords >=5 chars take
  (?:s|es)? ("epochs", "tokens", "harnesses", "premiums"), all-caps
  acronyms take a case-sensitive trailing "s" ("KPIs", "PRs"). Short
  ambiguous headwords stay exact ("mean" never matches "means"). Verby
  terms list their forms as explicit variants (churn, hallucinate).
- Entry-level `domains` hints (DictTermEntry/DictExpressionEntry) for
  cross-domain packs: unambiguous modern-usage AI terms (RAG, MCP,
  context window, ...) now count as domain-tracker evidence toward
  "ml", and the pack's commonWord entries (agent, harness, wrapper,
  compute, checkpoint, ...) unlock once a listed domain is active —
  previously unreachable under the default all-on pack state, even in
  a meeting saturated with AI vocabulary. Unambiguous core business
  metrics (ARR, MRR, GTM, ...) likewise activate sales/finance, which
  also strengthens multi-sense picks (team alignment, CAC, NDA) and
  unlocks finance-consumer common words in funding conversations.
  commonWord hits are never emitted as evidence (an everyday sense must
  not activate a domain). Remote packs can carry `domains` too
  (validated against DOMAIN_TAGS, capped at 4).

Detection conflicts (multiple dictionaries describing the same word):

- The expression loop gets the same F1 conflict handling terms already
  had: remote packs iterate first and the first entry to match claims
  the normalized key — a remote pack redefining "circle back" used to
  produce TWO cards for one occurrence, with the built-in winning the
  merge over the user's explicitly installed pack.
- Explicit lookups (includeAllPackMatches) collapse near-duplicate
  re-authorings of the same fact — same key AND matching zh or
  normalized-en gloss — so "ROI" no longer renders core's and
  business-terms' near-identical cards, while genuine cross-pack sense
  splits (SAM the market metric vs. SAM the alignment format) still
  show every sense.

Offline 划词翻译:

- Selection lookups now translate the selected surface whenever the
  language pair differs, independent of aiDetect and of a dictionary
  hit — the configured provider may be fully local (system/on-device),
  so lookups work with AI off and zero keys. On a hit the translation
  upgrades the already-rendered entry in place; on a miss it registers
  an honest 翻译所选 task. NoKeyError and provider-unavailable quietly
  omit the line; a cold on-device model gets one bounded retry.
- The personal glossary (我的词典) is searched first: a personal entry
  deliberately shadows the built-in dictionary inside scanDictionary,
  so a word the user had saved themselves used to come back as
  "词典未收录". Display-only — the transcript scan still owns emitting
  and counting custom cards.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013NWYJSE7af6gYR5FciXQ3o
@mianaz
mianaz force-pushed the claude/jargonslayer-detection-conflicts-lqyyc9 branch from 67b9d79 to ca4b4af Compare September 1, 2026 18:58
@mianaz
mianaz merged commit 0458454 into main Sep 2, 2026
2 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants