Home

Awesome

<p align="center"> <a href="https://roscenes.github.io#gh-light-mode-only" width="15%"> <img src="https://github.com/roscenes/RoScenes/blob/main/assets/title/light.svg#gh-light-mode-only" alt="RoScenes large-scale multi-view dataset for roadside perception" title="RoScenes" width="80%" /> </a> <a href="https://roscenes.github.io#gh-dark-mode-only" width="15%"> <img src="https://github.com/roscenes/RoScenes/blob/main/assets/title/dark.svg#gh-dark-mode-only" alt="RoScenes large-scale multi-view dataset for roadside perception" title="RoScenes" width="80%" /> </a> </p> <p align="center" width="15%"> <a href="https://arxiv.org/abs/2405.09883" target="_blank" width="15%"> <img src="https://img.shields.io/badge/arXiv-2405.09883-b31b1b?style=for-the-badge&logo=arxiv&labelColor=630b0b" alt="arXiv" height="24em"/> </a> <a href="https://roscenes.github.io" target="_blank" width="15%"> <img src="https://img.shields.io/badge/Page-blue?style=for-the-badge&label=Project&labelColor=505050&color=ff6a00" alt="Project" height="24em"/> </a> <a href="https://modelscope.cn/datasets/Apsara_Lab_Multimodal_Intelligence/RoScenes-release" target="_blank" width="15%"> <img src="https://img.shields.io/badge/Download-blue?style=for-the-badge&label=ModelScope&labelColor=000000&color=ff6a00" alt="Download" height="24em"/> </a> <a href="https://pypi.org/project/roscenes/" target="_blank" width="15%"> <img src="https://img.shields.io/badge/roscenes-blue?style=for-the-badge&label=pypi&labelColor=000000&color=ff6a00" alt="PyPI package" height="24em"/> </a> </p> <br/> <br/>

[!CAUTION] Commercial use of RoScenes is strictly forbidden.

πŸ“° Release Note

[2024-07-14] You can now download the dataset at ModelScope.

[2024-07-13] Devkit for RoScenes released.

[2024-07-01] Paper accepted to ECCV 2024! πŸ₯³

[2024-05-28] Please stay tuned for the updates! We are doing final checks on data privacy.

πŸ™οΈ Features

<p align="center"> <a href="https://github.com/roscenes/RoScenes#-features" width="15%"> <img src="assets/hero.jpg" alt="RoScenes" width="100%"/> </a> </p>

πŸ”– Table of Contents

πŸ”₯ Quick Start

Download

[!NOTE] Please refer to ModelScope for downloading the dataset.

After download and extract, the dataset folder should be organized as follows:

. [DATA_ROOT] # Dataset root folder
β”œβ”€β”€ πŸ“‚train # training set
β”‚   β”œβ”€β”€ πŸ“‚s001_split_train_difficulty_mixed_ambience_day # scene 001's data
β”‚   β”‚   β”œβ”€β”€ πŸ“‚database # annotations, grouped by clip
β”‚   β”‚   β”‚   β”œβ”€β”€ πŸ“‚0076fd69_clip_[0000000000000-0000000029529] # a clip's database, please use our devkit to read
β”‚   β”‚   β”‚   β””   ...
β”‚   β”‚   └── πŸ“‚images # images, grouped by clips
β”‚   β”‚       β”œβ”€β”€ πŸ“‚0076fd69
β”‚   β”‚       β””   ...
β”‚   β”œβ”€β”€ πŸ“‚s002_split_train_difficulty_mixed_ambience_day
β”‚   β”œβ”€β”€ πŸ“‚s003_split_train_difficulty_mixed_ambience_day
β”‚   β”œβ”€β”€ πŸ“‚s004_split_train_difficulty_mixed_ambience_day
β”‚   └── πŸ“‚night_split_train_difficulty_mixed_ambience_night
β”‚
β”‚
β”œβ”€β”€ πŸ“‚validation # validation set
β”‚   β”œβ”€β”€ πŸ“‚s001_split_validation_difficulty_mixed_ambience_day # scene 001's data
β”‚   β”œβ”€β”€ πŸ“‚s002_split_validation_difficulty_mixed_ambience_day
β”‚   β”œβ”€β”€ πŸ“‚s003_split_validation_difficulty_mixed_ambience_day
β”‚   β”œβ”€β”€ πŸ“‚s004_split_validation_difficulty_mixed_ambience_day
β”‚   └── πŸ“‚night_split_validation_difficulty_mixed_ambience_night
β”‚
β”‚
└── πŸ“‚test # test set
    β”œβ”€β”€ πŸ“‚NO_GTs005_split_test_difficulty_mixed_ambience_day # scene 005's data
    β”œβ”€β”€ πŸ“‚NO_GTs006_split_test_difficulty_mixed_ambience_day
    β”œβ”€β”€ πŸ“‚NO_GTs007_split_test_difficulty_mixed_ambience_day
    β”œβ”€β”€ πŸ“‚NO_GTs008_split_test_difficulty_mixed_ambience_day
    β”œβ”€β”€ πŸ“‚NO_GTs009_split_test_difficulty_mixed_ambience_day
    β”œβ”€β”€ πŸ“‚NO_GTs010_split_test_difficulty_mixed_ambience_day
    β”œβ”€β”€ πŸ“‚NO_GTs011_split_test_difficulty_mixed_ambience_day
    β”œβ”€β”€ πŸ“‚NO_GTs012_split_test_difficulty_mixed_ambience_day
    β”œβ”€β”€ πŸ“‚NO_GTs013_split_test_difficulty_mixed_ambience_day
    └── πŸ“‚NO_GTs014_split_test_difficulty_mixed_ambience_day

Install via PyPI

Use PyPI to directly install RoScenes devkit:

pip install roscenes

Install Manually (for dev)

Also, you can clone this repository and install roscenes manually for developing.

git clone https://github.com/roscenes/RoScenes.git

cd RoScenes

pip install -e .

Start Using the Dataset

import roscenes as ro

# load the training set
dataset = ro.load('[DATA_ROOT]/train/*')
# number of total frames
print(len(dataset))

Then, we can iterate over the dataset, in two ways:

You can use indexing:

# use integer indexing
index = 10
# a Frame instance
print(type(dataset[index]))

for i in range(len(dataset)):
    # print num of objects for every frame
    print(len(dataset[index].boxes3D))

OR, you can directly iterate it:

# a frame instance
for frame in dataset:
    print(len(frame.boxes3D))

[!IMPORTANT] Please refer to frame.py, camera.py for the detailed comments on box format, instrinsic and extrinsic definition, etc.

πŸ”Ž Explore the Dataset

python -m roscenes.visualizer [DATA_ROOT]/train/s001_split_train_difficulty_mixed_ambience_day 0 vis_result

πŸ‘©β€πŸ’» Examples

A example in companion with mmdet3d is given in examples.

Please go to that folder README.md for details.

πŸ“ˆ Evaluation

Note that we use nuScenes detection score (NDS) to evaluate performance. Additionally, we provide an optimized implementation of NDS which could significantly speed-up calculation. It should produce identical result with the official NDS under same inputs. Benchmark is shown below:

Input framesBoxes/FrameOriginal nuScenes devkitOurs
3601091697s118s

Unfortunately, since the code is built from scratch, our evaluation suite can not be an in-place replacement to the original one ---- You need to modify the code. To take evaluation, please refer to the example given in examples/mmdet3d/mmdet3d_plugin/datasets/roscenes_dataset.py and the README file in roscenes/evaluation/README.md.

🎯 To-do List