Home

Awesome

HPTR

<p align="center"> <img src="docs/hptr_banner.png" alt="HPTR realizes real-time and on-board motion prediction without sacrificing the performance.", width=750px> <br/>HPTR realizes real-time and on-board motion prediction without sacrificing the performance. <br/>To efficiently predict the multi-modal future of numerous agents (a), HPTR minimizes the computational overhead by: (b) Sharing contexts among target agents. (c) Reusing static contexts during online inference. (d) Avoiding expensive post-processing and ensembling. </p>

Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encoding
Zhejun Zhang, Alexander Liniger, Christos Sakaridis, Fisher Yu and Luc Van Gool.<br/>

NeurIPS 2023<br/> Project Website<br/> arXiv Paper

@inproceedings{zhang2023hptr,
  title = {Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encoding},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  author = {Zhang, Zhejun and Liniger, Alexander and Sakaridis, Christos and Yu, Fisher and Van Gool, Luc},
  year = {2023},
}

Updates

Setup Environment

Prepare Datasets

Training, Validation, Testing and Submission

Please refer to bash/train.sh for the training.

Once the training converges, you can use the saved checkpoints (WandB artifacts) to do validation and testing, please refer to bash/submission.sh for more details.

Once the validation/testing is finished, download the file womd_K6.tar.gz from WandB and submit to the Waymo Motion Prediction Leaderboard. For AV2, download the file av2_K6.parquet from WandB and submit to the Argoverse 2 Motion Forecasting Competition.

Performance

Our submission to the WOMD leaderboard is found here here.

Our submission to the AV2 leaderboard is found here here.

Ablation Models

Please refer to docs/ablation_models.md for the configurations of ablation models.

Specifically you can find the Wayformer and SceneTransformer based on our backbone. You can also try out different hierarchical architectures.

License

This software is made available for non-commercial use under a creative commons license. You can find a summary of the license here.

Acknowledgement

This work is funded by Toyota Motor Europe via the research project TRACE-Zurich (Toyota Research on Automated Cars Europe).