OpenViCA: Video Continuation for Automotive Driving Scenes by Streamlining and Fine-Tuning Open Source Models with Public Data

Björn Möller, Zhengyang Li, Malte Stelzer, Thomas Graave, Fabian Bettels, Muaaz Ataya and Tim Fingscheidt

This repo contains the official implementation of OpenViCA, an open video continuation system for automotive driving scenes.

🚗 Introduction

OpenViCA encodes input driving-scene frames into discrete latent tokens, predicts future tokens with a world model, and decodes the predicted token sequences into continued video frames.

The pipeline consists of three stages:

1. Image tokenizer (TOK): encodes frames into discrete latent tokens
2. World model (WM): predicts subsequent latent image tokens
3. Video decoder (VDEC): generates output video frames from predicted future tokens

🗂️ Repository Structure

Each component has its own top-level folder.

OpenViCA/
  image_tokenizer/    #TOK: video frames  → tokens
    src/
    configs/
    scripts/
  world_model/        # WM: predict future latent tokens (coming soon)
    src/
    configs/
    scripts/
  video_decoder/      # VDEC: tokens → video frames (coming soon)
    src/
    configs/
    scripts/
  dependencies/       # shared third-party code 
  resources/          # pretrained weights 
  assets/
  README.md

Full implementation and documentation will be available soon.