FRC Reinforcement Learning (Reefscape 2025)

This repository contains a reinforcement learning project for the 2025 FRC game Reefscape, developed using NVIDIA Isaac Lab and the skrl library.

Project Overview

The goal of this project is to train a swerve-driven FRC robot to autonomously perform key tasks in the Reefscape game, specifically picking up "Coral" game pieces and scoring them at different reef levels.

Key Features

• Isaac Lab Extension: Cleanly integrated as an Isaac Lab extension for high-performance physics simulation.
• Swerve Drive Kinematics: Realistic implementation of swerve drive for omnidirectional movement.
• 2025 Reefscape Assets: Custom USD models for the field, Reef, and Coral.
• Advanced Reward System: A phase-based reward structure (SEEK, INTAKE, CARRY, ALIGN, DOCK, RESET) to improve training stability.
• skrl Integration: Support for various RL algorithms including PPO, IPPO, and MAPPO.

Repository Structure

FRC_RL/
├── logs/           # Training checkpoints and event logs
├── scripts/        # Entry points for training and evaluation
│   ├── skrl/
│   │   ├── train.py
│   │   └── play.py
│   ├── random_agent.py
│   └── zero_agent.py
├── source/         # Main package source code
│   └── FRC_RL/
│       ├── assets/  # USD files for robot and field
│       ├── tasks/   # RL environment definitions
│       │   └── direct/
│       │       └── frc_rl/ # Reefscape specific environment
├── util/           # Swerve kinematics and helper utilities
└── outputs/        # General outputs from runs

Getting Started

Prerequisites

• Isaac Lab (compatible with Isaac Sim 4.5.0+)
• Python 3.10+

Installation

1. Install Isaac Lab following the official documentation.
2. Clone this repository into your Isaac Lab extensions or as a standalone project.
3. Install the package in editable mode:

# From the root of the repo
pip install -e FRC_RL/source/FRC_RL

Usage

Training an Agent

To start training with the default PPO algorithm:

python FRC_RL/scripts/skrl/train.py --task Template-Frc-Rl-Direct-v0 --num_envs 4096 --headless

Playing a Checkpoint

To evaluate a trained model:

python FRC_RL/scripts/skrl/play.py --task Template-Frc-Rl-Direct-v0 --checkpoint path/to/model.pt

Robot Capabilities

The robot (MasterRSCH_Suck_V2) features:

• Swerve Drive: Fully functional kinematics and control.
• Elevator: Multi-stage elevator for reaching Levels 1, 2, 3, and 4 of the Reef.
• Intake: Custom gripper mechanism for picking up and ejecting Coral.

Credits

Developed using Isaac Lab and skrl. Inspired by the First Robotics Competition.