Awesome
<div align="center">PPGeo: Policy Pre-training for Autonomous Driving via Self-supervised Geometric Modeling
</div>Policy Pre-training for Autonomous Driving via Self-supervised Geometric Modeling
- Penghao Wu, Li Chen, Hongyang Li, Xiaosong Jia, Junchi Yan, Yu Qiao
- arXiv Paper | openreview, ICLR 2023
- video | blog
This repository contains the pytorch implementation for PPGeo in the paper Policy Pre-training for Autonomous Driving via Self-supervised Geometric Modeling. PPGeo is a fully self-supervised driving policy pre-training framework to learn from unlabeled driving videos.
Pre-trained Models
<!--- | [Visual Encoder (ResNet-34)](https://drive.google.com/file/d/1GAeLgT3Bd_koN9bRPDU1ksMpMlWfGXbE/view?usp=sharing) | [DepthNet](https://drive.google.com/file/d/1bzRVs97KbPtfXE-1Iwe60bUD4i0JXxhh/view?usp=sharing) | [PoseNet](https://drive.google.com/file/d/1sDeuJIvfC01NFyuLFyPI3-yihQRsmLY_/view?usp=sharing) | |:--------------:|:--------:|:-------:| --->Model | Google Drive Link | BaiduYun Link |
---|---|---|
Visual Encoder (ResNet-34) | ckpt | ckpt (code: itqi) |
DepthNet | ckpt | ckpt (code: xvof) |
PoseNet | ckpt | ckpt (code: fp2n) |
Get Started
- Clone the repo and build the environment.
git clone https://github.com/OpenDriveLab/PPGeo.git
cd PPGeo
conda env create -f environment.yml --name PPGeo
conda activate PPGeo
-
Download the driving video dataset based on the instructions in ACO.
-
Make a symlink to the dataset root.
ln -s DATA_ROOT data
- Preprocess the data.
python ytb_data_preprocess.py
Training
- First stage training.
python train.py --id ppgeo_stage1_log --stage 1 --epochs 30
- Second stage training.
python train.py --id ppgeo_stage2_log --stage 2 --epochs 20 --ckpt PATH_TO_STAGE1_CKPT
Downstream Tasks
Nuscenes Planning
- Please download the nuScenes dataset first
- Make a symlink to the nuScenes dataset root.
cd nuscenes_planning
cd data
ln -s nuScenes_data_root nuscenes
cd ..
- Training the planning model
python train_planning.py --pretrained_ckpt PATH_TO_STAGE2_CKPT
Navigation & Navigation Dynamic & Reinforcement Learning
We use the DI-drive engine for IL data collection, IL training, IL evaluation, and PPO training following ACO with carla version 0.9.9.4. Some additional details can be found here.
Leaderboard Town05-long
We use the TCP codebase for training and evaluation with default setting.
Citation
If you find our repo or our paper useful, please use the following citation:
@inproceedings{wu2023PPGeo,
title={Policy Pre-training for Autonomous Driving via Self-supervised Geometric Modeling},
author={Penghao Wu and Li Chen and Hongyang Li and Xiaosong Jia and Junchi Yan and Yu Qiao},
booktitle={International Conference on Learning Representations},
year={2023}
}
License
All code within this repository is under Apache License 2.0.
Acknowlegement
Our code is based on monodepth2.