---

Unstable Baselines(USB) is designed to serve as a quick-start guide for Reinforcement Learning beginners and a codebase for agile algorithm development. The algorithms strictly follows the original implementations, and the performance of Unstable Baselines matches those in the original implementations. USB is currently maintained by researchers from lamda-rl.

---

Stable Algorithms (Runnable and has equivalent performance to that of the original implementations):

• Baselines
  1. Deep Q Learning (DQN)
  2. Vanilla Policy Gradient (VPG)
  3. Deep Deterministic Policy Gradient (DDPG)
  4. Trust Region Policy Optimization (TRPO)
  5. Proximal Policy Optimization (PPO)
  6. Soft Actor Critic (SAC)
  7. Twin Delayed Deep Deterministic policy gradient algorithm (TD3)
  8. Randomized Ensembled Double Q-Learning (REDQ)
• Model Based Reinforcement Learning
  1. Model-based Policy Optimization (MBPO)
• Meta Reinforcement Learning
  1. Efficient Off-policy Meta-learning via Probabilistic Context Variables (PEARL)

Unstable Algorithms (Runnable but needs tuning)

Algorithm TODO List

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (MAML)
• The Option-Critic Architecture (OC)

---

Current Performance

Quick Start

Install

git clone --recurse-submodules https://github.com/x35f/unstable_baselines.git
cd unstable_baselines
conda env create -f env.yaml 
conda activate rl_base
pip install -e .

To run an algorithm

python3 /path/to/algorithm/main.py /path/to/algorithm/configs/some-config.json args(optional)

Install environments (optional)

#install metaworld for meta_rl benchmark
cd envs/metaworld
pip install -e .
#install atari
pip install gym[all]

TODO List

• Add comments for algorithms
• Add Documentation