Open-Source Reinforcement Learning Environments Implemented in MuJoCo with Franka Manipulator

This repository is inspired by panda-gym and Fetch environments and is developed with the Franka Emika Panda arm in MuJoCo Menagerie on the MuJoCo physics engine. Three open-source environments corresponding to three manipulation tasks, FrankaPush, FrankaSlide, and FrankaPickAndPlace, where each task follows the Multi-Goal Reinforcement Learning framework. DDPG, SAC, and TQC with HER are implemented to validate the feasibility of each environment. Benchmark results are obtained with stable-baselines3 and shown below.

There is still a lot of work to be done on this repo, so please feel free to raise an issue and share your idea!

Tasks

FrankaPushSparse-v0	FrankaSlideSparse-v0	FrankaPickAndPlaceSparse-v0

Benchmark Results

FrankaPushSparse-v0	FrankaSlideSparse-v0	FrankaPickAndPlaceSparse-v0

Installation

Create a virtual environment for python 3.10:

conda create --name fm_env python==3.10

If Installing using WSL in Windows:

First, install essential tools you will need to compile code:

sudo apt update && sudo apt install –y build-essential gcc python3-dev

Move to your user folder:

cd

Download Miniconda in WSL:

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

Install Miniconda:

sh Miniconda3-latest-Linux-x86_64.sh

Enter ”yes” where requested.

Refresh your WSL terminal to note the changes by sources (i.e. loading) the new instructions:

source .bashrc

Before we continue with the franka mujoco installation, we will also need a few python packages:

python –m pip install –upgrade pip setuptools wheel

And:

python -m pip install --only-binary=:all: psutil

We are now ready to proceed.

Activate your environment:

conda activate fm_env

And now set python interpreter paths and install dependencies

cd franka_mujoco
pip install -e .
pip install -r requirements.txt

Create an fm_env alias

For convenience, you can create an alias in your .bashrc/.zshrc file to quickly load the environment and cd to a desired folder:

gedit ~/.bashrc # or gedit ~/.zshrc

# Go to the last line and type
alias fm_env="conda activate fm_env; cd ~/code/franka_mujoco"
# Save and exit your file

# Activate new .bashrc file by calling the following command in your terminal
source .bashrc # or source .zshrc

Test

import sys
import time
import gymnasium as gym
import panda_mujoco_gym

if __name__ == "__main__":
    env = gym.make("FrankaPickAndPlaceSparse-v0", render_mode="human")

    observation, info = env.reset()

    for _ in range(30):
        action = env.action_space.sample()
        observation, reward, terminated, truncated, info = env.step(action)

        if terminated or truncated:
            observation, info = env.reset()

        time.sleep(0.2)

    env.close()

Citation

If you use this repo in your work, please cite:

@misc{xu2023opensource,
      title={Open-Source Reinforcement Learning Environments Implemented in MuJoCo with Franka Manipulator}, 
      author={Zichun Xu and Yuntao Li and Xiaohang Yang and Zhiyuan Zhao and Lei Zhuang and Jingdong Zhao},
      year={2023},
      eprint={2312.13788},
      archivePrefix={arXiv},
      primaryClass={cs.RO}
}