ZEN-OVERLAY is a professional-grade automated pipeline designed to translate and dub videos (specializing in Kannada to English/Hindi) with a "Netflix-style" documentary feel. It features background audio preservation, AI noise reduction, and smart synchronization.
- 🤖 Full AI Automation: Upload a video and get a fully dubbed and subtitled version in minutes.
- 🎙️ Smart-Sync Technology: Automatically time-stretches/compresses generated AI voices to match the original speaker's timing perfectly.
- 🌐 High-Accuracy Translation: Uses Whisper Turbo/Large-v3 for transcription and Google Batch Translation for grammatically correct Hindi.
- 🔊 Premium Audio Overlay:
- Preserves original background audio (lowered to -21 LUFS).
- Applies a mid-frequency EQ cut to the background track so the new voice sits "behind" the mix.
- ✨ Ultra-Accuracy Mode: Includes AI-powered audio denoising and advanced beam-search decoding for maximum precision.
- 🎥 Hardcoded Subtitles: Automatically generates and burns in SRT subtitles for better social media retention.
- Transcription/Translation: Faster-Whisper (Turbo & Large-v3)
- Text-to-Speech: Edge-TTS (Microsoft Neural Voices)
- Audio Processing: FFmpeg & Pydub
- UI Framework: Gradio
- Translation Engine: Deep-Translator
The easiest way to run the application with all dependencies (including FFmpeg) is via Docker.
- Clone the repository:
git clone https://github.com/Mallikarjun63/ZEN-OVERLAY.git cd ZEN-OVERLAY - Build the Docker image:
docker build -t zen-overlay . - Run the container:
docker run -p 7860:7860 zen-overlay
- Open your browser and navigate to
http://localhost:7860.
- Install FFmpeg on your system:
# On Mac brew install ffmpeg - Clone the repository:
git clone https://github.com/Mallikarjun63/ZEN-OVERLAY.git cd ZEN-OVERLAY - Setup Virtual Environment:
python -m venv venv source venv/bin/activate # On Windows: venv\Scripts\activate pip install -r requirements.txt
- Run the Application:
Open your browser and navigate to
python app.py
http://127.0.0.1:7860.
- Auto AI Translator: Drop your Kannada video file, select "Hindi" or "English", and hit Start AI Pipeline.
- Fine-Tuning:
- Quality: Use "Fast" for quick results or "Ultra" for the best possible accuracy.
- Background Volume: Control how loud the original voice stays in the background.
- Voice Speed: Adjust the base speed of the AI narrator.
- Manual Mixer: If you already have a voiceover and an SRT file, use this tab to mix them with professional Netflix-style settings.
Contributions are welcome! Please feel free to submit a Pull Request.
MIT License - created by Mallikarjun63