A real-time iOS application that uses AI-powered person segmentation to replace the camera background with a custom image. The project leverages MediaPipe Selfie Segmentation and AVFoundation to process live camera frames and generate a virtual background effect.
- Real-time camera feed processing
- AI-powered person segmentation
- Dynamic virtual background replacement
- Live preview rendering
- AVFoundation camera integration
- Optimized frame processing for smooth performance
- Swift
- UIKit
- AVFoundation
- MediaPipe
- Core Image
- Xcode
- Captures live camera frames using AVFoundation.
- Uses MediaPipe Selfie Segmentation to generate a segmentation mask.
- Separates the foreground (person) from the background.
- Replaces the original background with a selected image.
- Renders the composited output in real time.
- CameraManager – Handles camera setup and frame capture.
- SegmentationProcessor – Processes frames using MediaPipe.
- ImageCompositor – Combines foreground and replacement background.
- ViewController – Displays the final output.
VirtualBackgroundDemo
├── Camera
├── Segmentation
├── Processing
├── Views
└── Assets
git clone https://github.com/arjunum/VirtualBackgroundDemo.gitopen VirtualBackgroundDemo.xcodeproj- Xcode 15+
- iOS 15.0+
- Swift 5+
- MediaPipe Framework
Build and run the application on a physical iPhone or supported simulator.
- Real-time camera frame processing
- Person segmentation using AI
- Image compositing and masking
- Performance optimization for smooth rendering
- Camera orientation and frame handling
This project provided hands-on experience with:
- AVFoundation
- Computer Vision Concepts
- Real-Time Image Processing
- MediaPipe Integration
- UIKit Development
- Performance Optimization
- Camera-Based iOS Applications
- Multiple background options
- Video recording support
- Background blur mode
- Improved segmentation quality
- AR-based effects
Arjun U M
- Email: arjunum88@gmail.com
- GitHub: https://github.com/arjunum