A collection of specialized ComfyUI workflows optimized for high-end VFX tasks, including camera synchronization, cinematic relighting, 4K upscaling, and production shot work integrating Foundry Nuke, Unreal Engine, and generative AI.
ComfyUI Workflows demo video (Covers March 2026 workflows — production shot work added April/May 2026 not yet included)
- Shot Paint Outs & Replacements — Wan VACE + Foundry Nuke pipeline for production-grade clean plating and set element replacement
- Background Replacement — Unreal Engine grey box driving video + Wan VACE (+ ControlNet) OR + SeeDance 2.0, for environment generation; full composite in Nuke with bluescreen foreground elements
- (WIP) Long Establishing Shots — Iterative generation beyond standard clip length using Unreal grey box driving videos
- (WIP) Interactive Hologram Compositing — Translucent, actor-reactive holographic elements via Nuke + ComfyUI
- Shot Paint Outs & Set Element Replacement A production workflow for video clean plating, paint outs, and replacement of set elements using Wan VACE within ComfyUI, with final compositing in Foundry Nuke.
Developed and successfully tested in active VFX production
- Background Replacement with Unreal Engine Grey Box Driving Plates
End-to-end video background replacement pipeline using Wan VACE or SeeDance 2.0:
- Unreal Engine for a grey box driving video from a camera tracked bluescreen shot
- Direct use of grey box driving video for SeeDance 2.0 and converted to depth ControlNets for Wan VACE (used with generated first frame and reference set concepts, matte paintings, and GenAI reference frames)
- Tested with grey box alone and with a pre-keyed FG plate over the grey box for tighter ControlNet guidance
- Composited in Foundry Nuke with bluescreen FG elements, color correct and fixes
- Tests include occluded geometry — handling views and surfaces not present in the first frame reference image, as revealed by camera movement
Developed in active VFX production, testing ongoing.
-
Long Establishing Shots via Iterative Generation (WIP) A workflow for generating video shots that exceed standard clip generation lengths. Uses Unreal Engine grey box driving plates, breaking a long shot into sequential iterations where each generation references previously generated frames. Final sections are comped together with cinematic and action elements added in post.
-
Interactive Hologram Compositing (WIP) A video compositing workflow for translucent, interactive story elements — for example, in-world holograms that respond to actor performances. Combines Foundry Nuke compositing with ComfyUI generation passes to achieve physically plausible interactive light and translucency.
-
SCAIL Motion Transfer: Image-to-Video generation using Wan 2.1 SCAIL for precise pose and camera matching via driving video.
-
Spatial Re-projection: Image-to-Image multi-angle generation.
- Features a Camera Angle Gizmo for intuitive perspective control.
- Powered by Qwen Image Edit 2511.
-
Lighting Transfer: Source-to-Target lighting plate transfer for consistent shot integration.
-
Dynamic Relighting: Multi-angle lighting control via Lighting Gizmo nodes.
- Utilizes Qwen Image Edit 2509 for high-fidelity shadow and highlight reconstruction.
- Utilizes Qwen Image Edit 2509 for high-fidelity shadow and highlight reconstruction.
-
4K Video Upscaling: High-bandwidth video-to-video upscaling.
- Uses Nvidia RTX Video Super Resolution (VSR). (Note: See documentation for essential post node install fix.)
- Uses Nvidia RTX Video Super Resolution (VSR). (Note: See documentation for essential post node install fix.)
-
Photorealistic Refinement: Advanced Image-to-Image filtering using Z-image Turbo to enhance texture detail and material response.
-
VFX ControlNet Generation: Video-to-video multi-control generation (Masks, Canny, Depth, and Normal).
| Date | Update |
|---|---|
| April/May 2026 | Added Production Shot Work section: paint outs, BG replacement, and two WIP workflows |
| March 2026 | Initial release — camera control, relighting, upscaling, and utility workflows |
Note: These workflows are designed for professional VFX environments and have been tested on 24GB VRAM configurations.
📋 All reference images and footage used in these workflows are original, non-IP material used for testing only.