Whisper Real Time

Provides an interface to openai-whisper for continuous transcription of the default input device

• System Requirements
• Usage
  • Where to find models
• License

Requirements

NixOS

This repository provides a Nix flake which provides:

• Development Environment via nix develop
• Nix Package as the default flake package output
  • can be built with nix build (binary will be available as ./result/bin/whisper-real-time)

The Development Environment provides all needed libraries to build the project. Note: Runtime Dependencies

Other Linux

Requires the following for building:

• rust
• cmake
• pkg-config (pkgconf on arch-linux)
• shaderc
• clang These are required during linking:
• alsa-lib
• vulkan-headers
• vulkan-loader (vulkan-icd-loader on arch-linux)

List of archlinux packages: rust cmake pkgconf shaderc alsa-lib vulkan-headers vulkan-icd-loader

Then build using cargo build --release (binary will be produced at ./target/release/whisper-real-time)

Note: Runtime Dependencies

Runtime Dependencies

The default mode of operation assumes a Vulkan ready graphics driver is installed and will crash at runtime if none is present.

• NixOS: if you have a graphical user environment enabled and your hardware supports vulkan it will likely just work.
• ArchLinux: Vulkan

Here is a list of Vulkan ready devices. Most modern Graphics drivers should support Vulkan.

If your device does not support vulkan, you could refer to Using special hardware and run the application without using the builtin Vulkan functionality.

Usage

To run with the system default input device

whisper-real-time --model "./path-to-ggml-model.bin"

Where to find models

This project requires models in ggml format. You can download them here or follow these instructions.

Personally i've had the best success with the base.en-q5_1 model. ggml-base.en-q5_1.bin

Suppressing Unwanted output

There is a lot of "debug" output on stderr to suppress it

whisper-real-time --model "./path-to-ggml-model.bin" 2>/dev/null

Running the pipeline on an audio file

• WIP: currently the VAD pipeline is not run when using file mode

If you want to run the VAD pipeline used on a file simply add -f "audiofile.wav"

whisper-real-time --model "./path-to-ggml-model.bin" --file "audiofile.wav" 2>/dev/null

Currently only WAV files are supported. see ffmpeg for file conversion

Using special hardware

If the library included doesn't support your hardware you can build whisper.cpp yourself and supply the binary supporting your hardware like so

whisper-real-time --model "./path-to-ggml-model.bin" --whisper-cpp "./path-to-whisper.cpp-cli-binary"

License

MIT License