Awesome
GenAD: Generative End-to-End Autonomous Driving
Paper
GenAD: Generative End-to-End Autonomous Driving
Wenzhao Zheng*, Ruiqi Song*, Xianda Guo* $\dagger$, Chenming Zhang, Long Chen$\dagger$
* Equal contributions $\dagger$ Corresponding authors
GenAD casts autonomous driving as a generative modeling problem.
News
- [2024/5/2] Training and evaluation code release.
- [2024/2/18] Paper released on arXiv.
Demo
Overview
Comparisons of the proposed generative end-to-end autonomous driving framework with the conventional pipeline. Most existing methods follow a serial design of perception, prediction, and planning. They usually ignore the high-level interactions between the ego car and other agents and the structural prior of realistic trajectories. We model autonomous driving as a future generation problem and conduct motion prediction and ego planning simultaneously in a structural latent trajectory space.
Results
Code
Dataset
Download nuScenes V1.0 full dataset data and CAN bus expansion data HERE. Prepare nuscenes data as follows.
Download CAN bus expansion
# download 'can_bus.zip'
unzip can_bus.zip
# move can_bus to data dir
Prepare nuScenes data
We genetate custom annotation files which are different from mmdet3d's
Generate the train file and val file:
python tools/data_converter/genad_nuscenes_converter.py nuscenes --root-path ./data/nuscenes --out-dir ./data/nuscenes --extra-tag genad_nuscenes --version v1.0 --canbus ./data
Using the above code will generate genad_nuscenes_infos_temporal_{train,val}.pkl
.
Folder structure
GenAD
├── projects/
├── tools/
├── configs/
├── ckpts/
│ ├── resnet50-19c8e357.pth
├── data/
│ ├── can_bus/
│ ├── nuscenes/
│ │ ├── maps/
│ │ ├── samples/
│ │ ├── sweeps/
│ │ ├── v1.0-test/
| | ├── v1.0-trainval/
| | ├── genad_nuscenes_infos_train.pkl
| | ├── genad_nuscenes_infos_val.pkl
installation
Detailed package versions can be found in requirements.txt.
Getting Started
datasets
https://drive.google.com/drive/folders/1gy7Ux-bk0sge77CsGgeEzPF9ImVn-WgJ?usp=drive_link
Checkpoints
https://drive.google.com/drive/folders/1nlAWJlvSHwqnTjEwlfiE99YJVRFKmqF9?usp=drive_link
Train GenAD with 8 GPUs
cd /path/to/GenAD
conda activate genad
python -m torch.distributed.run --nproc_per_node=8 --master_port=2333 tools/train.py projects/configs/GenAD/GenAD_config.py --launcher pytorch --deterministic --work-dir path/to/save/outputs
Eval GenAD with 1 GPU
cd /path/to/GenAD
conda activate genad
CUDA_VISIBLE_DEVICES=0 python tools/test.py projects/configs/VAD/GenAD_config.py /path/to/ckpt.pth --launcher none --eval bbox --tmpdir outputs
Related Projects
Our code is based on VAD and UniAD.
Citation
If you find this project helpful, please consider citing the following paper:
@article{zheng2024genad,
title={GenAD: Generative End-to-End Autonomous Driving},
author={Zheng, Wenzhao and Song, Ruiqi and Guo, Xianda and Zhang, Chenming and Chen, Long},
journal={arXiv preprint arXiv: 2402.11502},
year={2024}
}