Home

Awesome

Bird's-Eye-View Panoptic Segmentation Using Monocular Frontal View Images

This repository contains the PyTorch implementation of the PanopticBEV model proposed in our RA-L 2021 paper Bird's-Eye-View Panoptic Segmentation Using Monocular Frontal View Images.

Our approach, PanopticBEV, is the state-of-the-art approach for generating panoptic segmentation maps in the bird's eye view using only monocular frontal view images.

PanopticBEV Teaser

If you find this code useful for your research, please consider citing our paper:

@article{gosala21bev,
  author={Gosala, Nikhil and Valada, Abhinav},
  journal={IEEE Robotics and Automation Letters}, 
  title={Bird’s-Eye-View Panoptic Segmentation Using Monocular Frontal View Images}, 
  year={2022},
  volume={7},
  number={2},
  pages={1968-1975},
  doi={10.1109/LRA.2022.3142418}}

Relevant links

System requirements

Installation

a. Create a python virtual environment and activate it.

python3 -m venv panoptic_bev
source panoptic_bev/bin/activate

b. Update pip to the latest version.

python3 -m pip install --upgrade pip

c. Install the required python dependencies using the provided requirements.txt file.

pip3 install -r requirements.txt

d. Install the PanopticBEV code.

python3 setup.py develop

PanopticBEV datasets

KITTI-360

nuScenes

Code execution

Configuration parameters

The configuration parameters of the model such as the learning rate, batch size, and dataloader options are stored in the experiments/config folder. If you intend to modify the model parameters, please do so here.

Training and evaluation

The training and evaluation python codes along with the shell scripts to execute them are provided in the scripts folder. Before running the shell scripts, please fill in the missing parameters with your computer-specific data paths and parameters.

To train the model, execute the following command after replacing * with either kitti or nuscenes.

bash train_panoptic_bev_*.sh

To evaluate the model, execute the following command after replacing * with either kitti or nuscenes.

bash eval_panoptic_bev_*.sh 

Acknowledgements

This work was supported by the Federal Ministry of Education and Research (BMBF) of Germany under ISA 4.0 and by the Eva Mayr-Stihl Stiftung.

This project contains code adapted from other open-source projects. We especially thank the authors of:

License

This code is released under the GPLv3 for academic usage. For commercial usage, please contact Nikhil Gosala.