Awesome
Official code for "Structured Bird’s-Eye-View Traffic Scene Understanding from Onboard Images" (ICCV 2021)
In this work, we study the problem of extracting a directed graph representing the local road network in BEV coordinates, from a single onboard camera image. Moreover, we show that the method can be extended to detect dynamic objects on the BEV plane. The semantics, locations, and orientations of the detected objects together with the road graph facilitates a comprehensive understanding of the scene.
We provide support for Nuscenes and Argoverse datasets.
Check out our extension paper
TPLR (CVPR'22): https://github.com/ybarancan/TopologicalLaneGraph
Monocular video based BEV segmentation:
BEVFeatStitch (ICRA/RAL'22): https://github.com/ybarancan/BEV_feat_stitch
Steps
- Make sure you have installed Nuscenes and/or Argoverse devkits and datasets installed
- In configs/deafults.yml file, set the paths
- Run the make_labels.py file for the dataset you want to use
- If you want to use zoom augmentation (only for Nuscenes currently), run src/data/nuscenes/sampling_grid_maker.py (Set the path to save the .npy file in the sampling_grid_maker.py)
- You can use train_tr.py for training the transformer based model or train_prnn.py to train the Polygon-RNN based model
- We recommend using the Cityscapes pretrained Deeplab model (link provided below) as backbone for training your own model
- Validator files can be used for testing. The link to trained models are given below.
Trained Models
Cityscapes trained Deeplabv3 model is at: https://data.vision.ee.ethz.ch/cany/STSU/deeplab.pth
Nuscenes trained Polygon-RNN based model is at: https://data.vision.ee.ethz.ch/cany/STSU/prnn.pth
Nuscenes trained Transformer based model is at: https://data.vision.ee.ethz.ch/cany/STSU/transformer.pth
Metrics
The implementation of the metrics can be found in src/utils/confusion.py. Please refer to the paper for explanations on the metrics.
<img src="metrics.png" width="350" height="400">Additional Results
The method's results without object supervision:
| Metric/Dataset | Nuscenes | Argoverse |
|---|---|---|
| M-F | 56.7 | 55.6 |
| Detection | 59.9 | 60.1 |
| Assoc C-F | 55.2 | 54.9 |