A Deep Learning project using convolutional autoencoder network, that takes in input as an audio (possibly through a mobile app) and produces the corresponding vibrational values for a hand band for deaf people that's embedded with actuators.
We have provided the model checkpoint that can be used as follows.
cd checkpoints_melNow the model can be loaded as follows.
from keras.models import load_model
model = load_model('model.h5')Our model Pipeline :
cd src/pipeline.pyThe pipeline encapsulates our model. You can train your own model on your audio data through this file.
cd src/compression_models/train_melspec_compression_model.pyAdditionally, hyperparamter tuning through RayTune has been done and can be tweaked as per the user requirements through
python main_mel_ray_1.py
python main_mel_raytune.py
Motors can be simulated through PlatformIO or ArduinoIDE as per your convinience. Depending on the quality and quantity of vibrational motor you use, you can adjust the level of vibration being produced after the whole pipeline is replicated.
Python, Keras, PlatformIO, Librosa, Raytune
Some related projects
Contributions are always welcome! To add any additional functionality, or to indicate out errors in the code you may raise an issue.
- Deployment on the android/iOS phones
- User Friendly interface
- Understanding the Learning Curve of the vibrational motors and the actual audio.