PoC: Speech Enhancement using DeepFilterNet

[fkwp]

Content

The default speech enhancement (mainly noise reduction) algorithms of modern web browsers are working speech model based, don't consume a lot of CPU cycles and are quite limited. On the other end of the CPU cycle spectrum deep neural networks have
evolved with impressive performance. On the FOSS side of things RNNoise (https://github.com/xiph/rnnoise) as well as DeepFilterNet (https://github.com/Rikorose/DeepFilterNet) are two prominent examples.

Especially DeepFilterNet is of interest as it's not only carrying out noise reduction but also dereverberation which is very beneficial in open setups.

Recently a livekit plugin has been written leveraging WASM https://github.com/mezonai/mezon-noise-suppression which is used in this PoC

This PR adds a new Options to Settings -> Audio

• You can enable/disable Speech Enhancement at all (defaults to enabled)
• You can modify the amount of Speech Enhancement from 5 - 100% (defaults to 75%)

[bblacher]

Just experimented with this a bit, this works amazingly well - completely eliminating keyboard sound, squeaking chairs, noise from bumping into the mic, etc.

[CrazyNicc]

Just experimented with this a bit, this works amazingly well - completely eliminating keyboard sound, squeaking chairs, noise from bumping into the mic, etc.

How is the CPU usage?

Edit: Tested it myself, it only elevated CPU usage by ~1.5% on my i7-10700F

[bblacher]

I didn't watch my cpu usage while testing, but you can try it yourself by clicking on "View Deployment" above :)