Overview

This repository provides solutions to the most popular Reinforcement Learning algorithms.

All code is written in Python 3 and uses RL environments from OpenAI Gym and MADRL. Advanced techniques use Tensorflow and Keras for neural network implementations.

Setup

• Perform a recursive clone of this repository

  git clone --recursive git@github.com:dhavalsalwala/rl-algos.git
  

• Go to modules/MADRL/rllab and run following command. All the dependencies are defined in modules/MADRL/rllab/environment.yml. Please don't change them.

  ./scripts/setup_linux.sh
   or
  ./scripts/setup_osx.sh
  

• Activate the virtual environment created in above step.

• Add directories to PYTHONPATH

  export PYTHONPATH=$(pwd):$(pwd)/modules/MADRL:$(pwd)/modules/MADRL/rltools:$(pwd)/modules/MADRL/rllab:$PYTHONPATH
  

• Install the missing dependencies from modules/MADRL/rllab/environment.yml

Reinforcement Techniques

Multi Agent RL - Go to Repo

• MA Deep Q Network
• MA REINFORCE
• MA Advantage Actor-Critic
  • Pursuit Evasion

Single Agent RL

• Dynamic Programming
  • Policy and Value Iteration- Frozen Lake
• Model Free Learning
  • Sarsa
    • Cliff Walking
    • Frozen Lake
    • Grid World
    • Windy Grid World
  • Q Learning
    • Cliff Walking
    • Frozen Lake
    • Mountain Car
    • Taxi Ride
  • Deep Q Network (Fixed Q values and Double DQN)
    • Cart Pole
    • Lunar Lander
• Policy Gradient
  • Monte Carlo Policy Gradient (REINFORCE)
    • Cart Pole
  • Actor Critic Policy Gradient (A2C)
    • Cart Pole

References

Textbooks:

• Reinforcement Learning: An Introduction (2nd Edition)

Youtube Lecture Series:

• Introduction to Reinforcement Learning by David Silver

MOOC:

• School of AI's Free Course - Move37

Classes:

• David Silver's Reinforcement Learning Course (UCL, 2015)
• CS294 - Deep Reinforcement Learning (Berkeley, Fall 2015)

GitHub Resources:

• I highly recommend School of AI's Free Course Move37 GitHub: https://github.com/colinskow/move37
• Multi-agent deep reinforcement learning library (MADRL). - by Stanford Intelligent Systems Laboratory
• Custom Environments based on OpenAI gym - by Denny Britz

Contact

• If you have a suggestion or you found any bug/issue, shoot a mail at mailto:dhavalsalwala@gmail.com