Home

Awesome

PlaTe: Visually-Grounded Planning with Transformers in Procedural Tasks

[ šŸ“ŗ Website | šŸ— Github Repo | šŸŽ“ Paper ]

Requirements

Data Preparation

Follow this folder structure to prepare the dataset:

.
ā””ā”€ā”€ crosstask
    ā”œā”€ā”€ crosstask_features
    ā””ā”€ā”€ crosstask_release
        ā”œā”€ā”€ tasks_primary.txt
        ā”œā”€ā”€ videos.csv
        ā””ā”€ā”€ videos_val.csv

The data root is set here train_gpt.py.

How to Run

conda create -f environment.yml
bash srun.sh

Acknowledgement

We appreciate the following github repos a lot for their valuable code base implementations: joaanna/something_else, karpathy/minGPT.

Citation

@ARTICLE{PlaTe_RAL_2022,  
author={Sun, Jiankai and Huang, De-An and Lu, Bo and Liu, Yun-Hui and Zhou, Bolei and Garg, Animesh},  
journal={IEEE Robotics and Automation Letters},   
title={PlaTe: Visually-Grounded Planning With Transformers in Procedural Tasks},  
year={2022},  volume={7},  number={2},  pages={4924-4930},  
doi={10.1109/LRA.2022.3150855}}