Home

Awesome

This is the official repository of

Rethink Imitation-based Planner for Autonomous Driving, Jie Cheng,Yingbing chen,Xiaodong Mei,Bowen Yang,Bo Li and Ming Liu, arXiv 2023

<p align="left"> <a href="https://jchengai.github.io/planTF"> <img src="https://img.shields.io/badge/Project-Page-blue?style=flat"> </a> <a href='https://arxiv.org/pdf/2309.10443.pdf' style='padding-left: 0.5rem;'> <img src='https://img.shields.io/badge/arXiv-PDF-red?style=flat&logo=arXiv&logoColor=wihte' alt='arXiv PDF'> </a> </p> <image src="https://github.com/jchengai/planTF/assets/86758357/42170613-8759-4359-80ce-0b17c51676b8" height=400 width=400> <image src="https://github.com/jchengai/planTF/assets/86758357/2c06a97e-d543-4b82-8a75-bbf859abe148" height=400 width=400>

Highlight

Get Started

Setup Environment

conda create -n plantf python=3.9
conda activate plantf

# install nuplan-devkit
git clone https://github.com/motional/nuplan-devkit.git && cd nuplan-devkit
pip install -e .
pip install -r ./requirements.txt

# setup planTF
cd ..
git clone https://github.com/jchengai/planTF.git && cd planTF
sh ./script/setup_env.sh

Feature cache

Preprocess the dataset to accelerate training. The following command generates 1M frames of training data from the whole nuPlan training set. You may need:

 export PYTHONPATH=$PYTHONPATH:$(pwd)

 python run_training.py \
    py_func=cache +training=train_planTF \
    scenario_builder=nuplan \
    cache.cache_path=/nuplan/exp/cache_plantf_1M \
    cache.cleanup_cache=true \
    scenario_filter=training_scenarios_1M \
    worker.threads_per_node=40

This process may take some time, be patient (20+hours in my setting).

Training

We modified the training scipt provided by nuplan-devkit a little bit for more flexible training. By default, the training script will use all visible GPUs for training. PlanTF is quite lightweight, which takes about 4~6G GPU memory under the batch size of 32 (each GPU).

CUDA_VISIBLE_DEVICES=0,1,2,3 python run_training.py \
  py_func=train +training=train_planTF \
  worker=single_machine_thread_pool worker.max_workers=32 \
  scenario_builder=nuplan cache.cache_path=/nuplan/exp/cache_plantf_1M cache.use_cache_without_dataset=true \
  data_loader.params.batch_size=32 data_loader.params.num_workers=32 \
  lr=1e-3 epochs=25 warmup_epochs=3 weight_decay=0.0001 \
  lightning.trainer.params.val_check_interval=0.5 \
  wandb.mode=online wandb.project=nuplan wandb.name=plantf

you can remove wandb related configurations if your prefer tensorboard.

Trained models

Place the trained models at planTF/checkpoints/

ModelDocumentDownload
PlanTF (state6+SDE)-OneDrive
RasterModelDocOneDrive
UrbanDriver (openloop)DocOneDrive

Evaluation

Results

Test14-random and Test14-hard benchmarks

<table class="tg"> <thead> <tr> <th class="tg-c3ow" colspan="2">Planners</th> <th class="tg-c3ow" colspan="3">Test14-random</th> <th class="tg-c3ow" colspan="3">Test14-hard</th> <th class="tg-0pky"></th> </tr> </thead> <tbody> <tr> <td class="tg-0pky">Type</td> <td class="tg-0pky">Method</td> <td class="tg-c3ow">OLS↑</td> <td class="tg-c3ow">NR-CLS↑</td> <td class="tg-c3ow">R-CLS↑</td> <td class="tg-c3ow">OLS↑</td> <td class="tg-c3ow">NR-CLS↑</td> <td class="tg-c3ow">R-CLS↑</td> <td class="tg-0pky">Time</td> </tr> <tr> <td class="tg-0pky">Expert</td> <td class="tg-0pky">LogReplay</td> <td class="tg-c3ow">100.0</td> <td class="tg-c3ow">94.03</td> <td class="tg-c3ow">75.86</td> <td class="tg-c3ow">100.0</td> <td class="tg-c3ow">85.96</td> <td class="tg-c3ow">68.80</td> <td class="tg-c3ow">-</td> </tr> <tr> <td class="tg-0pky" rowspan="2">Rule-based</td> <td class="tg-0pky">IDM</td> <td class="tg-c3ow">34.15</td> <td class="tg-c3ow">70.39</td> <td class="tg-c3ow">72.42</td> <td class="tg-c3ow">20.07</td> <td class="tg-c3ow">56.16</td> <td class="tg-c3ow">62.26</td> <td class="tg-c3ow">32</td> </tr> <tr> <td class="tg-0pky">PDM-Closed</td> <td class="tg-c3ow">46.32</td> <td class="tg-c3ow">90.05</td> <td class="tg-7btt">91.64</td> <td class="tg-c3ow">26.43</td> <td class="tg-c3ow">65.07</td> <td class="tg-c3ow">75.18</td> <td class="tg-c3ow">140</td> </tr> <tr> <td class="tg-0pky" rowspan="2">Hybrid</td> <td class="tg-0pky">GameFormer</td> <td class="tg-c3ow">79.35</td> <td class="tg-c3ow">80.80</td> <td class="tg-c3ow">79.31</td> <td class="tg-c3ow">75.27</td> <td class="tg-c3ow">66.59</td> <td class="tg-c3ow">68.83</td> <td class="tg-c3ow">443</td> </tr> <tr> <td class="tg-0pky">PDM-Hybrid</td> <td class="tg-c3ow">82.21</td> <td class="tg-7btt">90.20</td> <td class="tg-c3ow">91.56</td> <td class="tg-c3ow">73.81</td> <td class="tg-c3ow">65.95</td> <td class="tg-7btt">75.79</td> <td class="tg-c3ow">152</td> </tr> <tr> <td class="tg-0pky" rowspan="5">Learning-based<br><br></td> <td class="tg-0pky">PlanCNN</td> <td class="tg-c3ow">62.93</td> <td class="tg-c3ow">69.66</td> <td class="tg-c3ow">67.54</td> <td class="tg-c3ow">52.4</td> <td class="tg-c3ow">49.47</td> <td class="tg-c3ow">52.16</td> <td class="tg-c3ow">82</td> </tr> <tr> <td class="tg-0pky">UrbanDriver <sup><span>&#8224</span></sup></td> <td class="tg-c3ow">82.44</td> <td class="tg-c3ow">63.27</td> <td class="tg-c3ow">61.02</td> <td class="tg-c3ow">76.9</td> <td class="tg-c3ow">51.54</td> <td class="tg-c3ow">49.07</td> <td class="tg-c3ow">124</td> </tr> <tr> <td class="tg-0pky">GC-PGP</td> <td class="tg-c3ow">77.33</td> <td class="tg-c3ow">55.99</td> <td class="tg-c3ow">51.39</td> <td class="tg-c3ow">73.78</td> <td class="tg-c3ow">43.22</td> <td class="tg-c3ow">39.63</td> <td class="tg-c3ow">160</td> </tr> <tr> <td class="tg-0pky">PDM-Open</td> <td class="tg-c3ow">84.14</td> <td class="tg-c3ow">52.80</td> <td class="tg-c3ow">57.23</td> <td class="tg-c3ow">79.06</td> <td class="tg-c3ow">33.51</td> <td class="tg-c3ow">35.83</td> <td class="tg-c3ow">101</td> </tr> <tr> <td class="tg-0pky">PlanTF (Ours)</td> <td class="tg-7btt">87.07</td> <td class="tg-c3ow">86.48</td> <td class="tg-c3ow">80.59</td> <td class="tg-7btt">83.32</td> <td class="tg-7btt">72.68</td> <td class="tg-c3ow">61.7</td> <td class="tg-c3ow">155</td> </tr> </tbody> </table> <p> <sup><span>&#8224</span></sup> open-loop re-implementation </p>

Val14 benchmark

MethodOLSNR-CLSR-CLS
Log-replay1009480
IDM387776
GC-PGP825754
PlanCNN647372
PDM-Hybrid849392
PlanTF (Ours)89.1884.8376.78

Acknowledgements

Many thanks to the open-source community, also checkout these works:

Citation

If you find this repo useful, please consider giving us a star 🌟 and citing our related paper.

@misc{cheng2023plantf,
      title={Rethinking Imitation-based Planner for Autonomous Driving},
      author={Jie Cheng and Yingbing Chen and Xiaodong Mei and Bowen Yang and Bo Li and Ming Liu},
      year={2023},
      eprint={2309.10443},
      archivePrefix={arXiv},
      primaryClass={cs.RO}
}