andes_gym

Andes environment for OpenAI Gym

• Free software: GPL v3+

Installation

Create an empty conda environment with

conda create -n drl-andes python=3.8

Install ANDES

The detailed instruction of Andes installation can be found here.

conda config --add channels conda-forge
conda config --set channel_priority flexible

Use conda install instead of pip install for andes installation

conda install andes

or in the root directory of andes, run

conda install --file requirements.txt
conda install --file requirements-dev.txt
python3 -m pip install -e .

Test if andes has been successfully installed by

andes selftest

Install GYM

Install mpi4py from Intel with

conda install -c intel mpi4py

Install gym

pip install gym

Install a recent version of ANDES with

Stable-baselines supporting tensorflow

Install tensorflow (cpu version)

pip install andes==1.5.8

Install keras

pip install keras

Install stable-baselines or stable-baselines3 Refer to the stable-baselines website or stable-baselines3 for detailed instruction.

For stable-baselines

Option 1: First Homebrew will be needed. Then, install the dependencies of stable-baselines as

brew install cmake openmpi

Then, open the stable-baselines repo, cd to the directory and

pip install -e .

Option 2:

pip install stable-baselines

Stable-baselines supporting pytorch

For stable-baselines3 （uses Pytorch 1.7）

pip install stable-baselines3

or in the root directory

pip install -e .

Install andes_gym

In the root directory of andes_gym , install andes_gym in the development mode with

pip install -e .

Examples

Try out examples in the folder examples .

Edit the number of time steps in train_freq_ddpg.py to a proper value. The larger the value, the longer it takes to simulate and train.

Run the example with

python train_freq_ddpg.py

Some deprecation warnings can be safely ignored.

When the training is completed, replay the trained model with

python play_freq_ddpg.py

You may need to install additional packages, such as matplotlib, to show the visualization. The matplotlib window might appear frozen, but is actually refreshed after each run.

Performance Tips

While most of the computation time was spent in ANDES, tests show that Linux perform consistently better than Windows, especially with cvxoptklu installed.

Currently, it is difficult to install cvxoptklu on Windows. Therefore, a Linux box is recommended whenever possible.

Some algorithms can take advantage of multi-core processors but most cannot. Please check the algorithm documentation from stable-baselines to verify. A quick way to check if an algorithm is taking advantage of your multi-core processor is to check the utilization of CPU (in Windows Task Manager or in htop of Linux).

Version Control

When working on the source code, please branch from master and work on your own branch.

You can either use GitHub for Desktop or learn the commands. The following are some commands for quick reference.

Branching can be done the collowing command

git checkout -b YOUR_BRANCH_NAME

where YOUR_BRANCH_NAME is the branch name of your choice.

To stage changes, use

git add PATH_TO_FILE

To commit changes, use

git commit

To push to a not-yet-exising branch, use

git push -u origin YOUR_BRANCH_NAME

After the first push, your local git will memorize the tracking branch. Next time, you can simply push with

git push

Features

• TODO