Awesome
<!-- Copyright (c) Rutav Shah, Indian Institute of Technlogy Kharagpur --> <!-- Copyright (c) Facebook, Inc. and its affiliates -->Quick Links
RRL: Resnet as representation for Reinforcement Learning
Resnet as representation for Reinforcement Learning (RRL) is a simple yet effective approach for training behaviors directly from visual inputs. We demonstrate that features learned by standard image classification models are general towards different task, robust to visual distractors, and when used in conjunction with standard Imitation Learning or Reinforcement Learning pipelines can efficiently acquire behaviors directly from proprioceptive inputs.
Final Behaviors acquired using RRL on ADROIT benchmark tasks (left to right) (a) Opening a door (b) Hammering a nail (c) Pen-twirling (d)) Object relocation
Setup
RRL
codebase can be installed by cloning this repository. Note that it uses git submodules to resolve dependencies. Please follow the steps as below to install correctly.
-
Clone this repository along with the submodules
git clone --recursive https://github.com/facebookresearch/RRL.git
-
Install the package using
conda
. The dependencies (apart frommujoco_py
) are listed inenv.yml
conda env create -f env.yml conda activate rrl
-
The environment require MuJoCo as a dependency. You may need to obtain a license and follow the setup instructions for mujoco_py. Setting up mujoco_py with GPU support is highly recommended.
-
Install
mj_envs
andmjrl
repositories.cd RRL pip install -e mjrl/. pip install -e mj_envs/. pip install -e .
-
Additionally, it requires the demonstrations published by
hand_dapg
Running Instructions
-
First step is to convert the observations of demonstrations provided by
hand_dapg
to the encoder feature space. An example script is provided here. Note the script saves the demonstrations in a.pickle
format inside therrl/demonstrations
directory.For the
mj_envs
tasks :python convertDemos.py --env_name hammer-v0 --encoder_type resnet34 -c top -d <path-to-the-demo-file>
python convertDemos.py --env_name door-v0 --encoder_type resnet34 -c top -d <path-to-the-demo-file>
python convertDemos.py --env_name pen-v0 --encoder_type resnet34 -c vil_camera -d <path-to-the-demo-file>
python convertDemos.py --env_name relocate-v0 --encoder_type resnet34 -c cam1 -c cam2 -c cam3 -d <path-to-the-demo-file>
-
Launching
RRL
experiments using DAPG.An example launching script is provided
job_script.py
in theexamples/
directory and the configs used are stored in theexamples/config/
directory. Note : Hydra configs are used.python job_script.py demo_file=<path-to-new-demo-file> --config-name hammer_dapg
python job_script.py demo_file=<path-to-new-demo-file> --config-name door_dapg
python job_script.py demo_file=<path-to-new-demo-file> --config-name pen_dapg
python job_script.py demo_file=<path-to-new-demo-file> --config-name relocate_dapg