A Pytorch reimplementation of Andrej Karpathy's blog Deep Reinforcement Learning: Pong from Pixels. The RL agent learns to play Pong via trial and error from pixels, using Policy Gradient RL method implemented in PyTorch.
The training code is located at main.py, and the slightly modified original Andrej's code is stored at karpathys_code.py (it was adapted for Python 3.10).
Requirements:
- Python 3.10+
- CUDA Version: 12 (if CUDA is not available, comment out Nvidia-related packages before installing
requirements.txtto train on CPU) - Install C compiler on your system
sudo apt install build-essential - Install swig
sudo apt-get install swig - Install requirements
pip install -r requirements.txt
To train a new model, run: python3.10 main.py.
Check Andrej Karpathy's Blog for more details of the training, algorithm, etc.
To play, run: python3.10 play.py --model_path=best_reward_model.pth --device="cuda:0"
One episode consists of 21 games, and each point represents an exponentially weighted average across played episodes. The graph shows that the model gradually learns to play the game, and rewards reach positive values after around 4000 episodes.
Learning slows down after ~4000 episodes, since players play the game equally well. This prolongs games and, thus, they are truncated by the Gymnasium library, resulting model receiving a zero reward. This explains why learning slows down after ~4000 episodes.
