Local, offline French voice commands (punctuation, corrections, formatting, navigation)

[gutentag266-star]

Problem

When dictating in French (and likely other languages), Whisper transcribes spoken instructions literally instead of acting on them. Saying "bonjour virgule comment allez-vous point" produces the literal words "virgule" and "point" in the output instead of "Bonjour, comment allez-vous."

This is a common pattern in dictation software (Dragon NaturallySpeaking, Windows Speech Recognition, etc.) but Handy doesn't currently support it.

Proposed feature

A local, fully offline post-processing step (no network calls, consistent with Handy's privacy model) that recognizes spoken French command phrases and converts them into their effect:

Punctuation: "point" → ., "virgule" → ,, "deux-points" → :, "point d'interrogation" → ?, quotes, parentheses, etc.
Structure: "à la ligne" → line break, "nouveau paragraphe" → paragraph break
Correction: "annule" (undo), "efface le dernier mot" (delete last word), "remplace X par Y" (replace X with Y), "supprime X" (delete X)
Navigation/Selection: "début"/"fin" (jump to start/end), "sélectionne X" — these operate on a virtual cursor within the transcript being assembled (Handy has no access to the live cursor of the focused external app, so this edits what you just dictated before it's pasted, not a document you had open earlier)
Formatting: "gras"/"italique"/"titre"/"liste" → Markdown-style wrapping (bold, italic, # heading, - list item)
Insertion: date, time, a user-configured email address, and common symbols (@, %, °, €, etc.)

Gated behind an opt-in setting (voice_commands_enabled, off by default) so it never changes behavior for users who don't want it.

What I've already built

I have a working prototype: a self-contained Rust module (audio_toolkit/voice_commands.rs, pure text-in/text-out, no external dependencies beyond chrono which is already a project dependency) with 25 passing unit tests, plus the small wiring needed into the existing post-processing pipeline (post_process_transcription_text) and AppSettings. Everything compiles and tests pass against current main.

Not yet done: the Settings UI toggle and i18n strings, and (deliberately) true live-cursor navigation in external apps, since that would need OS-level accessibility hooks beyond what Handy currently does (paste finished text).

Questions for maintainers/community
Is this something you'd want in Handy itself, or would it be better as a fork / separate opt-in plugin?
Any thoughts on scope — should navigation/selection (which only operate on the in-progress transcript, not a live document) be included, or is that confusing/out of scope?
Preferred language coverage — French only for a first version, or should the command table be designed for easy per-language extension from the start?

Happy to open a PR once there's a green light, and to adjust scope based on feedback.

[tahir-hassan]

This could be an easy addition to make with a simple YAML file:

punctuation:
  - input: "comma"
    output: ','
  - input: "period"
    output: '.'

Though it is a bit more complicated than that because what if a person was to say something like "Oxford comma", then you would need some way of excluding this. Perhaps you would just have input being literal comma being mapped to output comma.

Also, Handy already inserts punctuation, so you might need an option that strips punctuation from the text before processing it:

options:
  strip_punctuation: true

EDIT: Actually someone has actually submitted a PR for this. And another.

[ivangretsky]

This would be really useful indeed. Not only in French!

[tahir-hassan]

@gutentag266-star I have written a little AutoHotkey utility that allows you to insert punctuation marks: caps_up_down.ahk.txt

See this little explainer.