Awesome

HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigation

This repository is the official implementation of HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigation.

Prerequisites

# Set up with Anaconda
conda env create -f hop_env.yaml
conda activate hop

Quick Start

1. Download processed data and pretrained models. Please follow the instructions below to prepare the data in directories:

   • MP3D navigability graphs: connectivity
     • Download the connectivity maps .
   • Processed data: data
     • Download the Processed data.

2. Run Pre-training

   bash run/pretrain.bash
   

   The trained model will be saved under result/.

   You can also train model using only the processed PREVALENT data:

   let --prevalent_only = True in pretrain.bash

3. Run finetuning

   • Please check here for experiment setup and HOP application.

Citation

If you use or discuss our HOP, please cite our paper:

@InProceedings{Qiao2022HOP,
    author    = {Qiao, Yanyuan, Qi Yuankai, Hong, Yicong, Yu, Zheng, Wang, Peng and Wu, Qi},
    title     = {HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigation},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2022},
    pages     = {15418-15427}
}