Evaluate and harden anti-hallucination improvements for voice transcription

[Al629176]

Summary

Review external proposal to reduce voice transcription hallucinations and decide on safe integration approach.

Scope

• Evaluate technical impact on transcription quality.
• Perform dependency/security risk review for third-party components.
• Decide: adopt, adapt, or reject with rationale.

Acceptance Criteria

• Written recommendation with risk/benefit analysis.
• Security concerns documented for any external dependency.
• If approved, implementation plan and test strategy included.

[m13v]

voice transcription hallucinations are the worst failure mode because they produce confident-sounding garbage that downstream systems treat as real input. the biggest source in our experience is silence and background noise at the start/end of recordings. whisper in particular loves to hallucinate repeated phrases or common sentences (like 'thank you for watching') when it gets low-signal audio. the most effective mitigation we found was a VAD (voice activity detection) pre-filter that trims silence before sending audio to the model, combined with a confidence threshold on the output. if the average log probability per token drops below a threshold (we use -1.0), we reject the entire transcription instead of passing garbage downstream. also worth checking if any third-party components introduce their own audio preprocessing that might amplify the problem.

[m13v]

our transcription service with VAD pre-filtering, confidence thresholds, and the hallucination rejection logic: https://github.com/m13v/fazm/blob/main/Desktop/Sources/TranscriptionService.swift

fazm is an open source macOS AI agent that uses voice as the primary input, so transcription accuracy is critical for the entire interaction chain.