Intelligent voice assistant systems, such as smartphone assistants (e.g., Siri, Cortana, Google Now), Amazon Echo, and Google Home are becoming pervasive in our daily life. These human–machine communication systems are still emerging, mainly due to large researches in Deep Learning. Creating a personal voice assistant system improves the interaction with ALFRED, the xArm robot. This entire project consists of implementing all the voice assistant from the Automatic Speech Recognition (ASR) to Text-to-Speech (TTS) through Wake Word Detection. In this paper, we are focusing on the Wake Word Detection and consists of classifing audio files in a binary way to detect if a specific word is characterized. With the help of Deep Learning (CNN), we are building a Binary Classifier by taking an audio as input and expecting a Boolean as output.
More details can be found on the report.
This project works on python version: 3.6 and more.
Before running the pip installation command for the project, few dependencies need to be installed manually:
librosa and matplotlib packages are only required for plotting audio data.
Command to install all the Python libraries required:
pip3 install -r requirements.txt
After installing the packages, you can run the Demo script.
Command to run the demo:
python3 voice_assistant/core/demo.py
Before starting, there is a structure to follow. A directory, for example named sound, must be created, in which contains resp. 0 and 1 directories. These will serve to classify the new wakeword and the other sounds.
To generate a custom wakeword, it is necessary to create a new dataset of the new hotword. For it, run the collect_audio_data.py file in the data directory to generate audio sample:
python3 collect_audio --seconds 2 --samples_save_path [PATH]
(PATH looks like /sound/0/ or /sound/1/ )
The current model needs json files to load the dataset. Hence, run the following command to create them:
python3 create_wakeword_json.py --zero_label_dir [PATH0] --one_label_dir [PATH1] --save_json_path [PATH]
where PATH0 and PATH1 are the directories containing resp. the zero and one labels. PATH is the directory path for saving the train and test json files.
The train model code have multiple arguments, such as the number of epochs, the batch size or the learning rate. By default, they are put resp. at 100, 32, 1e-3. It can also disactivate cuda:
python3 train_CNN.py --save_path [PATH] --train_data_json [TRAIN] --test_data_json [TEST] --no_cuda
where PATH is the location to save the train model (named model_cnn.pth). TRAIN and TEST are the path of the resp. json files, created just before.