Skip to content

feat(core): typo-tolerant suggestions for missing skills#1

Merged
akl773 merged 2 commits into
akl773:mainfrom
ajatprabha:suggestions-levenshtein-typo-tolerance
May 20, 2026
Merged

feat(core): typo-tolerant suggestions for missing skills#1
akl773 merged 2 commits into
akl773:mainfrom
ajatprabha:suggestions-levenshtein-typo-tolerance

Conversation

@ajatprabha

Copy link
Copy Markdown
Contributor

Summary

closestMatches powers the "did you mean: …" hint shown after /skills when a requested name isn't found. The previous implementation used a bidirectional substring check, so an edit-distance-1 typo returned no suggestion at all — e.g. code-revewer (missing one letter) produced an empty list, defeating the purpose of the prompt.

This PR replaces the matcher with a tiered ranking:

  1. Substring matches (either direction), with exact matches boosted ahead of partial ones. Ties resolved by position in the candidate list, so behaviour is stable when callers pass a sorted catalog.
  2. Levenshtein typo matches within a length-relative threshold of ceil(len/3) (floor 1). Ranked by ascending edit distance.

The threshold scales with query length so short queries don't pull in wildly unrelated candidates (zzz against alpha/beta still returns nothing), while longer queries can absorb a couple of edits.

Why a length-relative threshold

  • len ≤ 3 → threshold 1 (catches single edits, rejects unrelated short strings)
  • len = 4–6 → threshold 2
  • len = 12 → threshold 4

This matches typical autocorrect heuristics — short tokens demand high precision, longer tokens have more slack.

Implementation notes

  • Levenshtein uses a two-row DP table on runes, so multi-byte characters count as a single edit (café vs cafe → distance 1).
  • All ranking goes through sort.SliceStable with explicit tie-breakers, so output is deterministic.
  • No new dependencies — pure stdlib.

Tests

Existing TestClosestMatches table preserved unchanged; new cases added for:

  • Typo tolerance (code-revewercode-reviewer)
  • Case-insensitive typo matching
  • Exact matches outranking partial substring matches
  • Typo matches outranking each other by ascending edit distance
  • Threshold floor on short queries (ab against xy/cd returns nil)
  • Empty query / non-positive n guards

Added TestLevenshtein table covering identity, insertion, deletion, substitution, the canonical kitten/sitting case, empty inputs, and a rune-level (café/cafe) check with symmetry assertions.

Test plan

  • make test — all packages green
  • go vet ./... — clean
  • make build — clean
  • CI: tests / vet / govulncheck / goreleaser snapshot on PR

closestMatches previously used a bidirectional substring check, so any
edit-distance-1 typo (e.g. "code-revewer") returned no suggestions even
when an obvious candidate existed.

Rank suggestions in two tiers: exact and substring matches first
(preserving existing positional ordering), then Levenshtein-close
candidates within a length-relative threshold. Threshold scales as
ceil(len/3) with a floor of 1 so short queries don't pull in unrelated
candidates while longer ones tolerate multiple edits.
@ajatprabha ajatprabha force-pushed the suggestions-levenshtein-typo-tolerance branch from fa6f2c8 to 712397f Compare May 20, 2026 13:28
@akl773 akl773 self-requested a review May 20, 2026 13:47
@akl773 akl773 self-assigned this May 20, 2026
@akl773 akl773 added the enhancement New feature or request label May 20, 2026
@akl773

akl773 commented May 20, 2026

Copy link
Copy Markdown
Owner

Hey, nice change — substring-only was a sharp edge, the tiered approach reads well.

A few thoughts:

  1. minInt3(a, b, c) can just be the builtin min(a, b, c) (Go 1.21+, and we're on 1.25). One less helper to maintain.

  2. Preallocate ranked with make([]scored, 0, len(candidates))? Worst case you realloc as the slice grows each iteration.

  3. The bidirectional strings.Contains(cLower, query) || strings.Contains(query, cLower) is inherited from the old impl so not your problem, but heads up: if a candidate is the empty string the second check always matches. Probably worth either a if cLower == "" { continue } near the top of the loop, or filtering at the caller. Up to you whether you want to deal with it in this PR.

  4. Curious about (len+2)/3 — was that a specific heuristic from somewhere, or just feel? It works (tests back it up), I just want to understand the reasoning before it becomes the canonical answer. A short comment on typoThreshold referencing the choice would help future readers.

  5. Optional perf: once you have n substring-tier hits you could skip Levenshtein for the rest — substring beats typo anyway, so the typo work is throwaway in that case. Probably not worth it at current catalog size, just flagging.

Tests are great — the rune symmetry case is the kind of thing people skip. Once 1 and 2 are in I'm good to merge; 3/4/5 are at your discretion.

- Use builtin min instead of a hand-rolled minInt3 helper. Go 1.21+ has
  variadic min/max for ordered types and the module already targets a
  newer toolchain.
- Preallocate the ranked slice to len(candidates) so the append loop
  doesn't repeatedly grow the backing array.
- Skip empty candidate strings. The bidirectional substring check
  (Contains(query, c)) would otherwise treat every empty candidate as a
  match against every query.
- Document the (len+2)/3 typo threshold as a staircase heuristic so the
  next reader doesn't have to derive the reasoning.
@ajatprabha

Copy link
Copy Markdown
Contributor Author

Thanks for the careful review. Pushed 9363403 addressing 1–4:

  1. minInt3 → builtin min. Gone. The module is already on a recent toolchain so the variadic form works directly inside the Levenshtein inner loop.
  2. Preallocate ranked. Switched to make([]scored, 0, len(candidates)). The upper bound is exact (every candidate could match) so no waste.
  3. Empty candidate guard. Good catch — added if cLower == "" { continue } at the top of the loop and a regression test (empty candidate strings are skipped). The Contains(query, "") path would otherwise have matched every query against every empty entry.
  4. (len+2)/3 rationale. Added a comment with the staircase it produces (1-3 → 1, 4-6 → 2, …) and noted it's a judgment call rather than from a paper. Roughly: one-third edit-distance is the rule-of-thumb upper bound at which a string still "looks like" its target. Tighter loses real typos, looser pulls in unrelated candidates. Happy to adjust the constant if you'd rather be stricter — the test that pins this is typo within length-relative threshold (distance 1 on a 12-char query) and short query does not over-match (rejects distance 2 on a 2-char query).
  5. Skip Levenshtein after n substring hits. Skipped per your suggestion — at current catalog sizes the saving is in the microseconds, and the early-exit would couple two tiers that are cleaner kept independent. Easy to revisit if catalogs grow into the thousands.

CI should be green; let me know if you want anything else tweaked.

@akl773

akl773 commented May 20, 2026

Copy link
Copy Markdown
Owner

Thanks, all looks good. Merging once CI's green — will go out in the next release.

@akl773 akl773 merged commit 81ab5c5 into akl773:main May 20, 2026
4 of 6 checks passed
@ajatprabha ajatprabha deleted the suggestions-levenshtein-typo-tolerance branch May 20, 2026 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants