We propose a recursive Minus-Plus Network(MP-Net) for visual sound separation task. MP-Net separates sounds recursively in the order of average energy, removing the separated sound from the mixture at the end of each prediction, until the mixture becomes empty or contains only noise.
- Python 3.6 is used. Basic requirements are listed in the 'requirements.txt'
pip install -r requirements.txt
MUSIC can be accessed here. Note that, Zhao. et al further proposed a new and larger dataset MUSIC21 in ICCV 2019.
VEGAS is a 10-category subset of the large-scale dataset AudioSet. VEGAS was cleaned by Zhou et al in CVPR 2018 paper Visual2Sound and can be fetched here.
All the training and testing bash scripts can be found in './scripts'. For the data preparing, we follow the same procedure as PixelPlayer.
The usage of this software is under CC-BY-4.0.
@inproceedings{xu2019recursive,
title={Recursive visual sound separation using minus-plus net},
author={Xu, Xudong and Dai, Bo and Lin, Dahua},
booktitle={Proceedings of the IEEE International Conference on Computer Vision},
pages={882--891},
year={2019}
}
The structure of this codebase is largely borrowed from PixelPlayer.