Home

Awesome

Learning Exploration Policies for Navigation

In ICLR 2019 [Project Website] [Demo Video] [pdf]

<img src='net.jpg' width="400"> <img src="https://thumbs.gfycat.com/ScarceChiefAsianconstablebutterfly-size_restricted.gif" width="400">

If you find this code useful, please consider citing our work:

@inproceedings{chen2018learning,
  author = "Chen, Tao and Gupta, Saurabh and Gupta, Abhinav",
  title = "Learning Exploration Policies for Navigation",
  booktitle = "International Conference on Learning Representations",
  year = "2019",
  url = "https://openreview.net/pdf?id=SyMWn05F7"
}

The code has been tested on Ubuntu 16.04.

Installation

Folder Structure

├── navigation
│   ├── suncg_data
│   ├── SUNCGtoolbox
│   └── exp4nav

  1. Install dependencies

    sudo apt-get install libglfw3-dev libglm-dev libx11-dev libegl1-mesa-dev libpng-dev
sudo apt-get install libpng16-dev libjpeg9 libjpeg-dev build-essential pkg-config
sudo apt-get install git curl wget automake libtool

  2. Install Anaconda

  3. Create a new virtual python environment

    cd ~
mkdir navigation
cd ~/navigation
git clone --recurse-submodules https://github.com/taochenshh/exp4nav.git
cd exp4nav
conda env create -f environment.yml
conda activate exp4nav

  4. Download SUNCG dataset, unzip it in navigation

  5. Clone SUNCGtoolbox:

    cd ~/navigation
git clone  https://github.com/shurans/SUNCGtoolbox.git
cd SUNCGtoolbox/gaps
make clean 
make

  6. Compile the render for House3D

    cd ~/navigation/exp4nav/House3D/renderer
SYSTEM=conda.linux PYTHON_CONFIG=/path/to/anaconda3/envs/exp4nav/bin/python3-config make -j

  7. Add House3D to PYTHONPATH

    echo "export PYTHONPATH=$PYTHONPATH:~/navigation/exp4nav/House3D" >> ~/.bashrc
source ~/.bashrc
conda activate exp4nav

  8. Download trained models with IL and RL, pre-trained models with IL, and house id files

    cd ~/navigation/exp4nav
wget https://www.dropbox.com/s/q2d883k6eb2rvmg/path_data.tar.gz
wget https://www.dropbox.com/s/z2cij8kjttilf7k/pretrain.tar.gz
tar -xvzf path_data.tar.gz
tar -xvzf pretrain.tar.gz

    map_only and map_rgb are models trained with IL and RL. il_map_only and il_map_rgb are models trained with IL.

  9. Preprocess SUNCG houses

    cd ~/navigation/exp4nav/src/utils
python env_remove_components.py

  10. Generate obj+mtl files for houses in EQA

    cd ~/navigation/exp4nav/gutils
python make_houses.py \
    -eqa_path ../path_data/eqa_v1.json \
    -suncg_toolbox_path ../../SUNCGtoolbox \
    -suncg_data_path ../../suncg_data \
    -hf_name house-no-doors
cd ~/navigation/exp4nav/src/utils
python preprocess_house.py

Training:

cd ~/navigation/exp4nav/src

## IL+RL with Map+RGB
python main.py --lr=0.00001 --rnn_hidden_dim=128 --area_reward_scale=0.0005 --gamma=0.999 \
       --collision_penalty=0.006 --ent_coef=0.01  --train_rollout_repeat=2 --max_depth=3 \
       --num_steps=500 --il_pretrain --pretrain_dir=../pretrain/il_map_rgb/model  \
       --num_envs=8 --use_rgb_with_map   --save_dir=data/map_rgb --seed=1

## IL+RL with Map
python main.py --lr=0.00001 --rnn_hidden_dim=128 --area_reward_scale=0.0005 --gamma=0.999 \
       --collision_penalty=0.006 --ent_coef=0.01  --train_rollout_repeat=2 --max_depth=3 \
       --num_steps=500 --il_pretrain --pretrain_dir=../pretrain/il_map_only/model  --num_envs=8  \
       --save_dir=data/map_only --seed=1

## RL with Map+RGB
python main.py --lr=0.00001 --rnn_hidden_dim=128 --area_reward_scale=0.0005 --gamma=0.999 \
       --collision_penalty=0.006 --ent_coef=0.01  --train_rollout_repeat=2 --max_depth=3 \
       --num_steps=500 --num_envs=8   --use_rgb_with_map --save_dir=data/no_pretrain_map_rgb \
       --seed=1

## RL with Map
python main.py --lr=0.00001 --rnn_hidden_dim=128 --area_reward_scale=0.0005 --gamma=0.999 \
       --collision_penalty=0.006 --ent_coef=0.01  --train_rollout_repeat=2 --max_depth=3 \
       --num_steps=500 --num_envs=8   --save_dir=data/no_pretrain_map_only --seed=1

Add --test and change the number of parallel envs to 1 (--num_envs=1) to test the policies. Add --render in the end if you want to visually test the policy.

Note that the policy can be trained with RL only (without imitation learning). However, If you have some demonstration data, it will greatly increase the learning speed.

Testing on the pre-defined testing houses

cd ~/navigation/exp4nav/src

## IL+RL with Map+RGB
python test_policy.py --lr=0.00001 --rnn_hidden_dim=128 --area_reward_scale=0.0005 --gamma=0.999 \
       --collision_penalty=0.006 --ent_coef=0.01  --train_rollout_repeat=2 --max_depth=3 \
       --num_steps=1000 --il_pretrain --pretrain_dir=../pretrain/il_map_rgb/model  \
       --num_envs=8 --use_rgb_with_map   --save_dir=data/map_rgb --seed=1

## IL+RL with Map
python test_policy.py --lr=0.00001 --rnn_hidden_dim=128 --area_reward_scale=0.0005 --gamma=0.999 \
       --collision_penalty=0.006 --ent_coef=0.01  --train_rollout_repeat=2 --max_depth=3 \
       --num_steps=1000 --il_pretrain --pretrain_dir=../pretrain/il_map_only/model  --num_envs=8  \
       --save_dir=data/map_only --seed=1

## RL with Map+RGB
python test_policy.py --lr=0.00001 --rnn_hidden_dim=128 --area_reward_scale=0.0005 --gamma=0.999 \
       --collision_penalty=0.006 --ent_coef=0.01  --train_rollout_repeat=2 --max_depth=3 \
       --num_steps=1000 --num_envs=8   --use_rgb_with_map --save_dir=data/no_pretrain_map_rgb \
       --seed=1

## RL with Map
python test_policy.py --lr=0.00001 --rnn_hidden_dim=128 --area_reward_scale=0.0005 --gamma=0.999 \
       --collision_penalty=0.006 --ent_coef=0.01  --train_rollout_repeat=2 --max_depth=3 \
       --num_steps=1000 --num_envs=8   --save_dir=data/no_pretrain_map_only --seed=1

Plot Performance

cd ~/navigation/exp4nav/src
python performance_plot.py --data_dir=./data