DreamerV3

Implementation of DreamerV3 — mastering diverse domains through world models. Trains a recurrent world model from pixel observations, then learns behavior entirely within imagined trajectories.

Supported Environments

Env	Config	Backend
CarRacing-v3	configs/car-racing-v3.yml	gymnasium (pixels native)
Walker2d-v5	configs/walker2d-v5.yml	pixels (MuJoCo + render)
Any Gymnasium env	custom	envBackend: gymnasium or pixels

Add new envs by creating a config and setting envBackend — gymnasium for native pixel envs, pixels for state-based envs auto-wrapped with PixelObservationWrapper.

Quick Start

uv sync
python main.py --config configs/car-racing-v3.yml

No GPU? Use the Colab-tuned configs:

python main.py --config configs/car-racing-v3-colab.yml

Config

All hyperparameters live in a single YAML. Key fields:

• gradientSteps — total world model + behavior gradient updates
• replayRatio — env interactions per gradient update (higher = more experience per step)
• batchSize / batchLength — sequence batch dimensions
• resume: True — load latest checkpoint and continue training

See configs/*.yml for full reference.

Project Structure

main.py        # entry point — config → env → Dreamer loop
dreamer.py     # Dreamer agent: world model, behavior training, env interaction, checkpoint I/O
networks.py    # RSSM, CNN encoder/decoder, actor, critic, reward, continue
envs.py        # wrappers, pixel bridge for state-based envs
buffer.py      # circular replay buffer with sequence sampling
utils.py       # config loader, λ-returns, Moments normalizer, CSV/plotly logging
configs/       # one YAML per environment

Paper

DreamerV3: Mastering Diverse Domains through World Models — Hafner et al., 2023.