Home

Awesome

Auxiliary Tasks and Exploration Enable ObjectNav

Joel Ye, Dhruv Batra, Abhishek Das, and Erik Wijmans.

Project Site: https://joel99.github.io/objectnav/

This repo is a code supplement for the paper "Auxiliary Tasks and Exploration Enable ObjectNav." The code is a fork of Habitat-Lab and borrows heavily from habitat-pointnav-aux.

Overview

The primary code contributions from the paper are located in:

Requirements

The code is known to be compatible with Python 3.6, Habitat Lab v0.1.5 and Habitat Sim v0.1.6 (headless). We recommend you use conda to initialize the code as follows:

  1. Install habitat-sim. In the habitat-sim repo, run:
conda create -n <ENV_NAME> python=3.6 cmake=3.14.0
conda activate <ENV_NAME>
pip install -r requirements.txt
conda install habitat-sim=0.1.6 headless -c conda-forge -c aihabitat

Then, in this repo, run:

conda env update -f environment.yml --name <ENV_NAME>

Note: If you run into a pip error along the lines of a missing metadata file, it is likely an improperly cleaned package (e.g. numpy-1.19.5), just remove it. Note 2: You may also need to run python setup.py develop --all as in the habitat-lab installation process.

Data

The experiments were done on the Matterport3D dataset. Please refer to habitat-sim for the scene dataset and habitat-lab for the ObjectNav dataset.

Pre-trained Weights

You can download pretrained agent and rednet weights here. We provide the best checkpoint for agents as measured by their performance with RedNet segmentation:

Training

We train our models for around 8 GPU-weeks, on a SLURM-managed cluster. On such a cluster, you can launch your run with:

sbat ./scripts/ddppo_8gpu.sh <args>

With the appropriate configuration file. The code in run.py is set up to parse out the arguments, so to train with the configuration .../objectnav/full/base.yaml, write:

sbat ./scripts/ddppo_8gpu.sh base full

On a machine without SLURM, see, e.g. ./scripts/train_no_ckpt.sh.

Evaluation

You can evaluate a trained checkpoint by configuring the arguments in ./scripts/eval_on.sh. This forwards to habitat_baselines/run.py, which accepts a number of flags. For example:

See run.py for all arguments. Note that an evaluation on the full validation split can take a while (>1 GPU-day).

Citation

If you use this work, you can cite it as

@misc{ye2021auxiliary,
      title={Auxiliary Tasks and Exploration Enable ObjectNav},
      author={Joel Ye and Dhruv Batra and Abhishek Das and Erik Wijmans},
      year={2021},
      eprint={2104.04112},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Additional Notes