StepCOVNet is a deep learning project designed to automatically generate StepMania charts from audio files. It utilizes Convolutional Neural Networks (CNNs) and Transformers to detect note onsets and predict arrow patterns, allowing rhythm game enthusiasts to create charts for their favorite songs instantly.
-
Clone the repository
git clone https://github.com/cpuguy96/StepCOVNet.git cd StepCOVNet -
Set up a virtual environment (Recommended)
python -m venv venv source venv/bin/activate # On Windows use `venv\Scripts\activate`
-
Install dependencies
pip install . # For GPU support pip install .[gpu] # For development dependencies pip install .[dev] # For development and GPU support pip install .[gpu-dev]
Generate a StepMania chart (.txt format) from an audio file using pre-trained models. If you do not have onset or arrow models yet, omit their paths and they will be downloaded automatically from Google Drive and cached locally.
Note: The output is currently a
.txtfile. UseSMDataToolsto convert it to a.smfile.
python scripts/generate.py \
--audio_path "path/to/song.mp3" \
--song_title "My Song" \
--onset_model_path "models/onset.keras" \
--arrow_model_path "models/arrow.keras" \
--output_file "output/chart.txt"| Argument | Description |
|---|---|
--audio_path |
Path to the input audio file (.mp3, .wav, etc.) |
--song_title |
Title of the song |
--bpm |
Beats per minute (optional; estimated from audio when omitted) |
--onset_model_path |
Path to the onset detection model (.keras); optional—omit to download and cache |
--arrow_model_path |
Path to the arrow prediction model (.keras); optional—omit to download and cache |
--output_file |
Path where the generated chart text file will be saved |
--use_post_processing |
Refine onset timings with peak-picking (recommended for cleaner charts) |
A simple desktop UI is available to run the generator without the command line: select an input audio file, optionally choose onset and arrow models (.keras)—or leave those fields blank to have default models downloaded from Google Drive and cached—enter the song title and optionally BPM (leave BPM blank to detect it from the audio), pick an output path, and run. Launch it with:
python scripts/generate_ui.pyOnset and arrow model paths are optional; leave them blank to use auto-downloaded default models (same behavior as the CLI). The UI uses the project venv when run from the repo.
Download: A pre-built Windows executable is available: generate_ui.zip (Google Drive). Extract and run; leave the onset/arrow model fields blank to have default models downloaded automatically.
You can build a single executable (e.g. generate_ui.exe on Windows) so others can run the Generator UI without installing Python or stepcovnet:
- Install the project (editable or from wheel) and the build optional dependency:
pip install -e . pip install .[build] - From the project root, run PyInstaller with the provided spec:
Or run the helper script (from project root or anywhere):
pyinstaller scripts/generate_ui.spec
python scripts/build_generate_ui_binary.py
- The executable will be created under
dist/(e.g.dist/generate_ui.exe). The bundle includes TensorFlow/Keras, so the file is large and startup may take a few seconds.
Train your own models using the provided scripts.
Link to training data: Google Drive
- Parse
.smfiles: UseSMDataToolsto convert.smfiles into.txtfiles (training data above already converted.smto.txt). - Organize files: Ensure audio files (
.mp3,.ogg,.wav) and their corresponding.txtchart files (same filename stem) are in the same directory.
Train the model responsible for detecting when a note should occur. Spectrogram normalization is always applied so that training matches the inference pipeline.
python scripts/train_onset.py \
--train_data_dir "data/train" \
--val_data_dir "data/val" \
--model_output_dir "models/onset" \
--epochs 20| Argument | Description | Default |
|---|---|---|
--train_data_dir |
Directory containing training data | Required |
--val_data_dir |
Directory containing validation data | Required |
--model_output_dir |
Directory to save the trained model | Required |
--epochs |
Number of training epochs | 10 |
--callback_root_dir |
Root directory for logs and checkpoints | "" |
--take_count |
Number of batches to use (for debugging) | 1 |
--model_name |
Custom name for the model | Auto-generated |
Train the model responsible for predicting the arrow pattern (Left, Down, Up, Right) for a given onset.
python scripts/train_arrow.py \
--train_data_dir "data/train" \
--val_data_dir "data/val" \
--model_output_dir "models/arrow" \
--epochs 20| Argument | Description | Default |
|---|---|---|
--train_data_dir |
Directory containing training data | Required |
--val_data_dir |
Directory containing validation data | Required |
--model_output_dir |
Directory to save the trained model | Required |
--epochs |
Number of training epochs | 10 |
--callback_root_dir |
Root directory for logs and checkpoints | "" |
--take_count |
Number of batches to use (for debugging) | 1 |
--model_name |
Custom name for the model | Auto-generated |
stepcovnet/
├── scripts/ # Training and generation scripts
│ ├── generate.py
│ ├── train_onset.py
│ └── train_arrow.py
├── src/
│ └── stepcovnet/ # Core package source code
│ ├── datasets.py # Data loading and preprocessing
│ ├── models.py # Model architectures (U-Net, Transformer)
│ ├── trainers.py # Training loops
│ └── ...
├── tests/ # Unit tests
├── pyproject.toml # Project configuration and dependencies
└── README.md # Project documentation
Contributions are welcome! Please follow these steps:
- Fork the repository.
- Create a new branch (
git checkout -b feature/YourFeature). - Commit your changes (
git commit -m 'Add some feature'). - Push to the branch (
git push origin feature/YourFeature). - Open a Pull Request.
Please ensure your code passes existing tests and linting standards.
- Inspiration: Dance Dance Convolution
- Base Code: Derived from musical-onset-efficient
- Collaboration: Special thanks to Jhaco for support and collaboration.
This project is licensed under the Apache 2.0 License - see the LICENSE file for details.