RealROS: A Comprehensive Framework for Real-World Robotic Reinforcement Learning

📚 Full documentation: uniros.readthedocs.io (ecosystem-wide docs hosted via UniROS)

RealROS, is a unified (ROS)-based open-source Python framework to create real-world robotics environments for reinforcement learning (RL) applications.

This framework extends the functionality of ROS by introducing additional Python bindings necessary for automatically launching environments in the real world, eliminating the need for manual configuration of the ROS's low-level features.

This framework provides the following features:

1. A modular architecture that promotes reproducibility and encourages code reuse,
2. Real-time training with any RL simulation framework (Agnostic of any RL simulation framework)
3. Tools to execute concurrent environments and maintain communication.

For more information, please refer to the main UniROS repository which integrates MultiROS and RealROS.

Easiest install: one-shot script

If you're starting fresh on Ubuntu 20.04, run the bootstrap installer that ships in this repo (and identically in every other ecosystem repo: UniROS, MultiROS, sb3_ros_support, rl_environments, rl_training_validation). It installs ROS Noetic, UniROS (with MultiROS

• RealROS as submodules), sb3_ros_support, rl_environments (with all 4 robots' vendor packages + supporting description-extras + cube tracker), and rl_training_validation.

git clone https://github.com/ncbdrck/realros.git /tmp/uniros_bootstrap
bash /tmp/uniros_bootstrap/install_uniros_stack.sh                # interactive
bash /tmp/uniros_bootstrap/install_uniros_stack.sh -y             # unattended
bash /tmp/uniros_bootstrap/install_uniros_stack.sh -p ~/my_ws -y  # custom path

The script asks once whether to install all components or pick per- component; refuses to run on non-Ubuntu-20.04 (Noetic requirement). The manual setup steps below still work for piece-by-piece installs.

Don't have Ubuntu 20.04? A Docker image is shipped under docker/ — see docker/README.md for build / run / GPU / hardware-passthrough instructions. Works on Ubuntu 22.04 / 24.04 hosts, on WSL2, and on machines with GPUs that have no Ubuntu 20.04 driver.

Prerequisites

Before installing RealROS manually, make sure you have the following prerequisites:

ROS Installation

RealROS requires a working installation of ROS. If you haven't installed ROS yet, please follow the official ROS installation guide for your specific operating system. This package has been tested with ROS Noetic version, and the following instructions will guide you through the installation of ROS Noetic on Ubuntu 20.04:

sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
sudo apt install curl
curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | sudo apt-key add -
sudo apt update
sudo apt install ros-noetic-desktop-full

echo "source /opt/ros/noetic/setup.bash" >> ~/.bashrc
source ~/.bashrc

sudo apt install python3-rosdep python3-rosinstall python3-rosinstall-generator python3-wstool build-essential
sudo apt install python3-rosdep
sudo rosdep init
rosdep update

Catkin Tools

RealROS uses Catkin as the build system for ROS packages. Install Catkin Tools by running the following command:

sudo apt-get install python3-catkin-tools

Create ROS Workspace

Before using RealROS, you need to create a ROS workspace to build and run your ROS packages. Follow these steps to create a workspace:

cd ~
source /opt/ros/noetic/setup.bash
mkdir -p ~/catkin_ws/src
cd ~/catkin_ws/
catkin build
echo "source ~/catkin_ws/devel/setup.bash" >> ~/.bashrc
source ~/.bashrc

Other Packages

RealROS also requires the following additional packages:

• XTerm for terminal emulation:

  sudo apt install xterm

• MoveIt for motion planning:

  sudo apt install ros-noetic-moveit

• PyKDL for Kinematics:

  sudo apt install python3-pykdl

• kdl_parser_py for parsing URDF files (for ros_kinematics):

  sudo apt install ros-noetic-kdl-parser-py

• trac_ik_python for inverse kinematics (for Kinematics_pyrobot):

  # Download and install trac_ik_python
  sudo apt-get install ros-noetic-trac-ik
  
  # or
  
  cd ~/catkin_ws/src
  git clone https://bitbucket.org/traclabs/trac_ik.git

• pykdl_utils for kinematics (for Kinematics_pykdl):

  # download the package - modified version of the original package to support ROS Noetic
  cd ~/catkin_ws/src
  git clone https://github.com/ncbdrck/hrl-kdl.git
  
  # install the pykd_utils
  cd ~/catkin_ws/src/hrl-kdl/pykdl_utils
  python3 setup.py build
  sudo python3 setup.py install
  
  # install the hrl_geom
  cd ~/catkin_ws/src/hrl-kdl/hrl_geom
  python3 setup.py build
  sudo python3 setup.py install
  
  # Install urdf_parser and urdfdom-py
  sudo apt-get install ros-noetic-urdf-parser-plugin
  sudo apt-get install ros-noetic-urdfdom-py
  
  # Build the package
  cd ~/catkin_ws
  rosdep install --from-paths src --ignore-src -r -y
  catkin build
  source devel/setup.bash

You are now ready to proceed with the installation and usage of RealROS.

Please note that the instructions assume you are using Ubuntu 20.04 and ROS Noetic. If you are using a different operating system or ROS version, make sure to adapt the commands accordingly.

Installation

Required dependency: Starting with v0.2.0, RealROS shares its multiprocessing proxy and ROS utility modules with the UniROS package (realros.core.RealrosGym is an alias for uniros._proxy.GymProxy, and realros.utils.ros_markers re-exports from uniros.utils.ros_markers). You must clone UniROS into the same catkin workspace. A standalone clone of RealROS will fail at import time with ModuleNotFoundError: No module named 'uniros'.

To get started with RealROS, follow these steps:

1. Clone UniROS (provides the shared uniros runtime) and RealROS into the same catkin workspace. Skip this step if you've already followed the recursive UniROS install — UniROS already vendors RealROS as a submodule under UniROS/realros/, and cloning a second standalone copy alongside it creates a duplicate catkin package that the workspace build will reject:

   cd ~/uniros_ws/src
   git clone -b gymnasium https://github.com/ncbdrck/UniROS.git
   git clone -b gymnasium https://github.com/ncbdrck/realros.git

2. RealROS relies on several Python packages. You can install them by running the following command:

   # Install pip if you haven't already by running this command
   sudo apt-get install python3-pip
   
   # install the required Python packages for RealROS by running
   cd ~/catkin_ws/src/realros/
   pip3 install -r requirements.txt

3. Build the ROS packages and source the environment:

   cd ~/catkin_ws/
   rosdep install --from-paths src --ignore-src -r -y
   catkin build
   source devel/setup.bash

Usage

You can refer to the Docs, templates or the examples to see how to use RealROS to create a real-world environment for RL applications.

It also showcases:

• How to use RealROS to create a real-world environment for RL applications.
• Train robots directly in the real world to perform various tasks.
• Use MultiROS framework to create a simulation environment for the same robot and train it in the simulation environment. Then, transfer the learned policy to the real-world environment.
• Train both environments (sim and real) in real-time to obtain a generalised policy that performs well in both environments.

The installation instructions for the examples are provided in the respective repositories.

Documentation

Full ecosystem documentation — covering installation, ready-made environments, environment creation (sim and real), training with any gymnasium-compatible RL library, and joint sim+real training — lives in the UniROS repository under docs/. RealROS is documented there alongside MultiROS and the rest of the framework.

License

RealROS is released under the MIT License. Please see the LICENSE file for more details.

Acknowledgements

We would like to thank the following projects and communities for their valuable contributions, as well as the authors of relevant libraries and tools used in RealROS.

• ROS (Robot Operating System)
• MoveIt

Cite

If you use RealROS in your research or work and would like to cite it, you can use the following citation:

Articles:

@article{kapukotuwa_uniros_2025,
	title = {{UniROS}: {ROS}-{Based} {Reinforcement} {Learning} Across {Simulated} and {Real}-{World} {Robotics}},
	shorttitle = {{UniROS}},
	doi = {10.3390/s25185679},
	journal = {Sensors},
	author = {Kapukotuwa, Jayasekara and Lee, Brian and Devine, Declan and Qiao, Yuansong},
	volume = {25},
	number = {18},
	month = sep,
	year = {2025},
	pages = {5679},
	publisher = {{MDPI}},
	url = {https://www.mdpi.com/1424-8220/25/18/5679},
}

Repository

@misc{realros,
  author = {Kapukotuwa, Jayasekara},
  booktitle = {GitHub repository},
  publisher = {GitHub},
  title = {RealROS: A Comprehensive Framework for Real-World Robotic Reinforcement Learning},
  url = {https://github.com/ncbdrck/realros},
  year = {2023}
}

Contact

For questions, suggestions, or collaborations, feel free to reach out to the project maintainer at j.kapukotuwa@research.ait.ie.