Awesome
Offline Reinforcement Learning with Implicit Q-Learning
This repository contains the official implementation of Offline Reinforcement Learning with Implicit Q-Learning by Ilya Kostrikov, Ashvin Nair, and Sergey Levine.
If you use this code for your research, please consider citing the paper:
@article{kostrikov2021iql,
title={Offline Reinforcement Learning with Implicit Q-Learning},
author={Ilya Kostrikov and Ashvin Nair and Sergey Levine},
year={2021},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
How to run the code
Install dependencies
pip install --upgrade pip
pip install -r requirements.txt
# Installs the wheel compatible with Cuda 11 and cudnn 8.
pip install "jax[cuda111]<=0.21.1" -f https://storage.googleapis.com/jax-releases/jax_releases.html
Also, see other configurations for CUDA here.
Run training
Locomotion
python train_offline.py --env_name=halfcheetah-medium-expert-v2 --config=configs/mujoco_config.py
AntMaze
python train_offline.py --env_name=antmaze-large-play-v0 --config=configs/antmaze_config.py --eval_episodes=100 --eval_interval=100000
Kitchen and Adroit
python train_offline.py --env_name=pen-human-v0 --config=configs/kitchen_config.py
Finetuning on AntMaze tasks
python train_finetune.py --env_name=antmaze-large-play-v0 --config=configs/antmaze_finetune_config.py --eval_episodes=100 --eval_interval=100000 --replay_buffer_size 2000000
Misc
The implementation is based on JAXRL.