Home

Awesome

nuScenes devkit

Welcome to the patched devkit of the nuScenes and nuImages datasets. This patched devkit provides lidar points per box in the KITTI format and also supports evaluation on nuScenes front camera.

Overview

Changelog

Citation

If you find this patched nuScenes or nuImages useful in your research, please consider starring the repo and citing:

@inproceedings{nuscenes2019,
  title={{nuScenes: A multimodal dataset for autonomous driving}},
  author={Holger Caesar and Varun Bankiti and Alex H. Lang and Sourabh Vora and 
          Venice Erin Liong and Qiang Xu and Anush Krishnan and Yu Pan and 
          Giancarlo Baldan and Oscar Beijbom},
  booktitle={CVPR},
  year={2019}
}

@inproceedings{kumar2022deviant,
  title={ {DEVIANT: Depth EquiVarIAnt NeTwork for Monocular $3$D Object Detection}},
  author={Kumar, Abhinav and Brazil, Garrick and Corona, Enrique and Parchami, Armin and Liu, Xiaoming},
  booktitle={ECCV},
  year={2022}
}

Devkit setup

We use a common devkit for nuScenes and nuImages. The devkit is tested for Python 3.6 and Python 3.7. To install Python, please check here.

Our devkit is available and can be installed via pip :

pip install nuscenes-devkit

For an advanced installation, see installation for detailed instructions.

nuImages

nuImages is a stand-alone large-scale image dataset. It uses the same sensor setup as the 3d nuScenes dataset. The structure is similar to nuScenes and both use the same devkit, which make the installation process simple.

nuImages setup

To download nuImages you need to go to the Download page, create an account and agree to the nuScenes Terms of Use. For the devkit to work you will need to download at least the metadata and samples, the sweeps are optional. Please unpack the archives to the /data/sets/nuimages folder *without* overwriting folders that occur in multiple archives. Eventually you should have the following folder structure:

/data/sets/nuimages
    samples	-	Sensor data for keyframes (annotated images).
    sweeps  -   Sensor data for intermediate frames (unannotated images).
    v1.0-*	-	JSON tables that include all the meta data and annotations. Each split (train, val, test, mini) is provided in a separate folder.

If you want to use another folder, specify the dataroot parameter of the NuImages class (see tutorial).

Getting started with nuImages

Please follow these steps to make yourself familiar with the nuImages dataset:

jupyter notebook $HOME/nuscenes-devkit/python-sdk/tutorials/nuimages_tutorial.ipynb

nuScenes

nuScenes setup

To download nuScenes you need to go to the Download page, create an account and agree to the nuScenes Terms of Use. After logging in you will see multiple archives. For the devkit to work you will need to download all archives. Please unpack the archives to the /data/sets/nuscenes folder *without* overwriting folders that occur in multiple archives. Eventually you should have the following folder structure:

/data/sets/nuscenes
    samples	-	Sensor data for keyframes.
    sweeps	-	Sensor data for intermediate frames.
    maps	-	Folder for all map files: rasterized .png images and vectorized .json files.
    v1.0-*	-	JSON tables that include all the meta data and annotations. Each split (trainval, test, mini) is provided in a separate folder.

If you want to use another folder, specify the dataroot parameter of the NuScenes class (see tutorial).

nuScenes-panoptic

In August 2021 we published nuScenes-panoptic which contains the panoptic labels of the point clouds for the approximately 40,000 keyframes in nuScenes. To install nuScenes-panoptic, please follow these steps:

nuScenes-lidarseg

In August 2020 we published nuScenes-lidarseg which contains the semantic labels of the point clouds for the approximately 40,000 keyframes in nuScenes. To install nuScenes-lidarseg, please follow these steps:

Prediction challenge

In March 2020 we released code for the nuScenes prediction challenge. To get started:

CAN bus expansion

In February 2020 we published the CAN bus expansion. It contains low-level vehicle data about the vehicle route, IMU, pose, steering angle feedback, battery, brakes, gear position, signals, wheel speeds, throttle, torque, solar sensors, odometry and more. To install this expansion, please follow these steps:

Map expansion

In July 2019 we published a map expansion with 11 semantic layers (crosswalk, sidewalk, traffic lights, stop lines, lanes, etc.). To install this expansion, please follow these steps:

Map versions

Here we give a brief overview of the different map versions:

Getting started with nuScenes

Please follow these steps to make yourself familiar with the nuScenes dataset:

jupyter notebook $HOME/nuscenes-devkit/python-sdk/tutorials/nuscenes_tutorial.ipynb

KITTI conversion and evaluation

See convert_nuscenes_to_kitti_format_and_evaluate.sh

Known issues

Great care has been taken to collate the nuScenes dataset and many users have praised the quality of the data and annotations. However, some minor issues remain:

Maps:

Annotations: