Russian VOSK speech-to-text output is mojibake / wrong charset, fixed by preserving native bytes and repairing UTF-8

[Sasha12132]

Description

ChatPlus speech-to-text with a Russian VOSK model loads and recognizes speech, but the Cyrillic output is corrupted / mojibake in the chat input.

I was able to reproduce it with vosk-model-ru-0.42 and then test a local source-code fix. After the fix, Russian speech-to-text works correctly with no gibberish.

Environment

• Minecraft: Java Edition 1.21.11
• Loader: Fabric
• ChatPlus: 2.8.1
• OS: Windows 11
• Java: OpenJDK 21.0.7
• VOSK bundled by ChatPlus: com_alphacephei_vosk 0.3.45
• VOSK model: vosk-model-ru-0.42
• Model path:
  config/chatplus/models/vosk-model-ru-0.42

Steps to reproduce

1. Install ChatPlus 2.8.1 on Fabric 1.21.11.
2. Install dependencies, including Architectury and Cloth Config.
3. Download and extract vosk-model-ru-0.42.
4. Put it into:
   config/chatplus/models/vosk-model-ru-0.42
5. Enable Speech To Text.
6. Select vosk-model-ru-0.42.
7. Disable translation so raw speech-to-text output is tested.
8. Speak Russian, for example:
   сейчас я скажу простую фразу для проверки

Expected result

The chat input should contain correct Cyrillic text:

сейчас я скажу простую фразу для проверки

Actual result

The chat input contains mojibake / corrupted Cyrillic.

Examples of tested charset behavior:

• UTF-8: mostly unreadable Ð...Ñ...-style gibberish.
• windows-1252 / Cp1252: partially readable Russian, but letters such as с, э, and я become replacement characters (�) or otherwise break.
• windows-1251, CP1251, ISO-8859-5, KOI8-R, UTF-16: worse / unusable.
• Translation was disabled during testing, so this is not caused by the translator.

Root cause found

The issue appears to be a charset-conversion bug in the speech-to-text path.

In common/src/main/kotlin/com/ebicep/chatplus/features/speechtotext/SpeechToText.kt, the recognized VOSK text is re-encoded like this:

lastSpokenMessage = String(asString.toByteArray(charset(Config.values.speechToTextCharset)), StandardCharsets.UTF_8)

This corrupts non-Latin text. For Russian, the text appears to already be decoded incorrectly by the native/JNA layer or then further damaged by this conversion.

There is also a secondary issue: when model loading fails, the microphone thread calls ChatPlus.sendMessage(...), which can crash with:

java.lang.IllegalStateException: Rendersystem called from wrong thread

Fix tested locally

I tested a patched build locally and Russian speech-to-text now works correctly.

The fix that worked was:

1. Force JNA/native string decoding to preserve raw bytes:

System.setProperty("jna.encoding", "ISO-8859-1")

This must happen early, before VOSK/JNA uses native strings.

2. Replace the broken speech-to-text conversion with a repair function that converts the ISO-8859-1-preserved native bytes back to UTF-8:

private fun repairVoskText(text: String): String {
    return try {
        String(text.toByteArray(Charsets.ISO_8859_1), Charsets.UTF_8)
    } catch (_: Exception) {
        text
    }
}

Then use:

lastSpokenMessage = repairVoskText(asString)

instead of:

lastSpokenMessage = String(asString.toByteArray(charset(Config.values.speechToTextCharset)), StandardCharsets.UTF_8)

3. Fix ChatPlus.sendMessage(...) so it runs on the Minecraft client/render thread when called from the microphone thread. Otherwise, model-load errors can cause:

Rendersystem called from wrong thread

Result after the fix

After rebuilding ChatPlus with the above changes:

• vosk-model-ru-0.42 loads successfully.
• Russian speech-to-text inserts correct Cyrillic text.
• Problem letters like с, э, and я no longer break.
• The ChatPlus charset setting can be left as UTF-8.
• Translation can then be enabled separately.

Suggested implementation notes

The main fix is probably to avoid user-configurable charset conversion for VOSK output, or at least avoid applying it to an already-decoded Java/Kotlin string.

For non-Latin VOSK output on Windows, JNA/native string decoding seems to be the critical part. Preserving native bytes with ISO-8859-1 and then decoding them as UTF-8 fixed the issue in my local build.