RL Episode Inspector

Interactive, frame-by-frame analysis and 3D replay of individual reinforcement-learning episodes from NVIDIA Isaac Lab.

TensorBoard tells you the aggregate story — mean reward, losses, curves across thousands of parallel environments. It does not help you answer "why did this one episode fail?" RL Episode Inspector records individual episodes with full reward decomposition, lets you pick the best / worst / median run by return, and replays it in a browser 3D viewer with reward charts that stay synchronized to playback. Click or drag on a chart to jump to the exact frame where something interesting happened.

Key features

• 🎬 3D replay of recorded episodes in the browser (Three.js) — no Isaac Sim required to view.
• 🤖 Full-robot replay — every rigid body's pose is recorded; the viewer shows the whole robot as real 3D meshes exported from the sim's exact USD geometry (default) or lightweight cubes (toggle, for when you don't want to ship/store meshes).
• 📊 Reward decomposition — every reward term recorded raw and weighted, per frame.
• 👥 Multi-agent (MARL) support — per-agent reward terms, returns, and charts (see the Cart-Double-Pendulum demo).
• 🏆 Ranking — select best / worst / median episode by episode_return.
• 🔗 Synchronized timeline — one source of truth; click/drag charts to scrub; keyboard shortcuts.
• 💡 Scene lighting like the task — USD lights are captured at mesh export and replayed.
• 🧱 Generic signal model — the core stores SignalSpecs, not task-specific columns; track any custom variable with zero schema work.
• 🧪 CI-safe — a fake-data generator exercises storage → backend → frontend → E2E with no GPU and no Isaac Lab.

Demo tasks included

Episodes (and exact robot meshes) for three real Isaac Lab tasks are committed, so the app is fully explorable without an Isaac install. A fourth (Cartpole) generates in seconds.

Task	What it shows	Data
Franka Reach (examples/reach/)	Full-arm 3D mesh replay, sparse target_reached reward (staircase return), markers	committed
Cart-Double-Pendulum (examples/cart_double_pendulum/)	Multi-agent (MARL): per-agent reward decomposition & returns	committed
Humanoid (AMP) (examples/humanoid/)	28-body mocap replay (walk/run/dance) with real meshes & scene lights	committed
Cartpole (examples/cartpole/)	Dense six-term reward; also has a fake-data generator (no GPU/Isaac)	generate

Quick start (no Isaac Lab needed)

git clone https://github.com/VShirokun/RL-Episode-Inspector.git
cd RL-Episode-Inspector

# Python backend
make install                       # create .venv and install the package (uses uv)
make backend-dev                   # serve the committed Franka Reach experiment at :8000

# Frontend (in a second terminal)
make frontend-install
make frontend-dev                  # open http://localhost:3000

Then open http://localhost:3000, hit Best, and press Space to play.

No uv? Create the venv manually: python3 -m venv .venv && .venv/bin/pip install -e ".[dev]"

Try the other committed experiments:

make backend-dev SERVE_DIR=sample_data/cart_double_pendulum/episodes   # multi-agent
make backend-dev SERVE_DIR=sample_data/humanoid/episodes               # humanoid mocap

Or generate the lightweight fake Cartpole data (no GPU, seconds):

make generate-fake-cartpole-demo                          # writes sample_data/cartpole/episodes
make backend-dev SERVE_DIR=sample_data/cartpole/episodes  # serve it

Quick start (record real Isaac Lab episodes)

Requires a working Isaac Lab install (see docs/isaac_lab_integration.md). Point ISAACLAB at your checkout (defaults to ~/IsaacLab):

make generate-cartpole-demo ISAACLAB=/path/to/IsaacLab   # real Cartpole (balance)
make generate-reach-demo    ISAACLAB=/path/to/IsaacLab   # Franka Reach (sparse reward)
make generate-cartdp-demo   ISAACLAB=/path/to/IsaacLab   # Cart-Double-Pendulum (MARL)
make generate-humanoid-demo ISAACLAB=/path/to/IsaacLab   # Humanoid AMP mocap replay (offline)

Record your own task

The recorder is task-agnostic — an adapter feeds it values each step and every signal appears in the UI automatically:

from rl_episode_inspector.recorder import EpisodeRecorder

rec = EpisodeRecorder("my_episodes", task_name="MyTask", dt=1 / 60,
                      viewer_type="articulation3d", up_axis="z")
rec.register_bodies(body_names, parent_indices,
                    meshes=[f"myrobot/{n}.glb" for n in body_names])  # optional 3D
rec.start_episode(episode_index=0, seed=42)
for t in range(num_steps):
    rec.record_frame(
        frame_index=t, timestamp=t * rec.dt,
        state={"joint_pos": ...}, action={"torque": ...},
        rewards_raw={"upright": ..., "effort": ...},
        reward_weights={"upright": 1.0, "effort": -0.1},
        observations={"object_speed": ...},           # track ANY extra variable
        poses=read_body_poses(robot, env_origin),     # for 3D replay
        terminated=term, truncated=trunc,
    )
rec.end_episode(reset_reason="goal_reached")

See docs/isaac_lab_integration.md for the full Isaac Lab walkthrough (including automatic mesh export) and docs/3d_replay.md for how meshes, lights, and coordinate frames work.

Keyboard & mouse

Input	Action
Space	Play / pause
→ / ←	Next / previous frame
Shift + → / ←	Jump ± 10 frames
Home / End	First / last frame
Click chart	Seek to that frame
Drag chart	Scrub through the episode

Documentation

Doc	Contents
docs/architecture.md	Components, data flow, generic signal design, extension points
docs/data_format.md	Episode directory, metadata.json, frames.parquet, conventions
docs/3d_replay.md	Mesh export, scene lights, coordinate frames & quaternion conventions
docs/developer_guide.md	Dev setup, running backend/frontend, adding signals/tasks/charts
docs/isaac_lab_integration.md	Supported versions, recorder hooks, running the demo tasks
docs/testing.md	Unit / integration / E2E / Isaac Lab tests, CI
docs/security.md	Local backend assumptions, path-traversal protection
docs/roadmap.md	Shipped since MVP + what's next

Development

make test        # run CI-safe Python tests
make lint        # ruff
make typecheck   # mypy
make ci          # lint + typecheck + tests + frontend build/tests
make e2e         # Playwright end-to-end (boots both servers)

Contributions welcome — see CONTRIBUTING.md.

License

MIT.