Serenade crashes for audio input devices that don't support 16000hz of sample rate

[moorbren]

System Information

Serenade Version: v2.0.2

OS and Version: e.g., Ubuntu 22.10

Application: Everything, crashes when accessing the microphone.

Issue Description

Audio input devices that don't support 16000hz of sample rate will crash the Speech Recorder:

Expression 'paInvalidSampleRate' failed in '/home/tmac/github/speech-recorder/tmp/portaudio/portaudio/src/hostapi/alsa/pa_linux_alsa.c', line: 2050
Expression 'PaAlsaStreamComponent_InitialConfigure( &self->capture, inParams, self->primeBuffers, hwParamsCapture, &realSr )' failed in '/home/tmac/github/speech-recorder/tmp/portaudio/portaudio/src/hostapi/alsa/pa_linux_alsa.c', line: 2721
Expression 'PaAlsaStream_Configure( stream, inputParameters, outputParameters, sampleRate, framesPerBuffer, &inputLatency, &outputLatency, &hostBufferSizeMode )' failed in '/home/tmac/github/speech-recorder/tmp/portaudio/portaudio/src/hostapi/alsa/pa_linux_alsa.c', line: 2845
PortAudio Error: Open Stream
Error number: -9997
Error message: Invalid sample rate

Dug into this very deep, issue comes from the Port Audio OpenStream function (microphone.cpp):

Pa_OpenStream(&stream_, &parameters, 0, sampleRate_, samplesPerFrame_, paClipOff, callback, &callbackData_);

Which is called from microphone.ts in the Serenade Client (client/src/main/stream/microphone.ts):

    this.recorder = new SpeechRecorder({
      device: this.settings.getMicrophone().id,
      sileroVadSilenceThreshold: this.settings.getChunkSilenceThreshold(),
      sileroVadSpeechThreshold: this.settings.getChunkSpeechThreshold(),
      ...
   });

The problem is that there isn't any sample rates being set here, so it defaults to the Speech Recorder library's default of 16000hz:

  ...
  options.sampleRate = options.sampleRate !== undefined ? options.sampleRate : 16000;
  ...

How to Reproduce

Have no audio input devices that support 16000hz. Not sure why none of mine do since the same microphone on Windows doesn't have any issues... Might be a new thing with Ubuntu 22.xx and the audio drivers? Here is the full error log, has some other ALSA errors in it.

Either way, I have a fix for this working locally:

...
type MicrophoneInput = {
  id: number;
  name: string;
  selected?: boolean;
  defaultSampleRate?: number;
};

type SpeechRecorderDevice = {
  id: number,
  name: string,
  apiName: string,
  maxInputChannels: number,
  maxOutputChannels: number,
  defaultSampleRate: number,
  isDefaultInput: boolean,
  isDefaultOutput: boolean
}
...
    const microphoneId = this.settings.getMicrophone().id;
    const selectedMicrophone = this.microphones().find(mic => mic.id === microphoneId);

    this.recorder = new SpeechRecorder({
      device: microphoneId,
      sampleRate: selectedMicrophone?.defaultSampleRate || 16000,
...

  microphones(): MicrophoneInput[] {
    const inputs: [SpeechRecorderDevice] = devices().filter((e: any) => e.maxInputChannels > 0);
    const defaultInputDevice = inputs.find(i => i.isDefaultInput);

    // very important to include the sample rate here
      // the speech processor does not handle default sample rates of devices
      // It defaults to 16000hz for each device, if it's not supported, the program will crash
    const microphones : [MicrophoneInput] = [{
      id: Microphone.systemDefaultMicrophone.id,
      name: Microphone.systemDefaultMicrophone.name,
      defaultSampleRate: defaultInputDevice?.defaultSampleRate,
      selected: Microphone.systemDefaultMicrophone.id == this.settings.getMicrophone().id,
    }];

    inputs.forEach(e => {
      microphones.push({
        id: e.id,
        name: e.name,
        defaultSampleRate: e.defaultSampleRate,
        selected: e.id == this.settings.getMicrophone().id,
      })
    });

    return microphones;
  }
...

Screenshots

Nothing interesting

[notching]

I also see this bug on 20.04, as a result I'm not able to run serenade :-(

[moorbren]

I ended up switching back to Windows to use Serenade. This fixes the crashing issue, but there still appears to be a bug with the Speech Recorder library preventing any input from being picked up.

[notching]

dang, can't switch to windows, all my dev is in Linux.

[shoetten]

Happens exactly as described with Arch & pipewire 0.3.70.

[luandro]

Also had the same on Manjaro Arch, after tweaking mic settings. How to I clear the config? Still haven't been able to get Serenade working.

[dzschille]

Same here with Ubuntu 23.04 and using the app image. After changing the input in the Serenade Settings from "default" to an USB mic the app crashes. Now it crashes every time i click in the settings icon. That's the error message:

Expression 'paInvalidSampleRate' failed in '/home/tmac/github/speech-recorder/tmp/portaudio/portaudio/src/hostapi/alsa/pa_linux_alsa.c', line: 2050
Expression 'PaAlsaStreamComponent_InitialConfigure( &self->capture, inParams, self->primeBuffers, hwParamsCapture, &realSr )' failed in '/home/tmac/github/speech-recorder/tmp/portaudio/portaudio/src/hostapi/alsa/pa_linux_alsa.c', line: 2721
Expression 'PaAlsaStream_Configure( stream, inputParameters, outputParameters, sampleRate, framesPerBuffer, &inputLatency, &outputLatency, &hostBufferSizeMode )' failed in '/home/tmac/github/speech-recorder/tmp/portaudio/portaudio/src/hostapi/alsa/pa_linux_alsa.c', line: 2845
PortAudio Error: Open Stream
Error number: -9997
Error message: Invalid sample rate

After deleting the local config directory rm -r ~/.config/Serenade i was able to open the settings again. If i choose the same USB mic in my Ubuntu audio settings as input and let the microphone "System Default" in the app settings, Serenade works without problems.