Home

Awesome

TO-Scene: A Large-scale Dataset for Understanding 3D Tabletop Scenes

<img src="./figure/pipeline_new.jpg" width="900"/>

By Mutian Xu*, Pei Chen*, Haolin Liu, and Xiaoguang Han

Introduction

This repository is built for:

TO-Scene: A Large-scale Dataset for Understanding 3D Tabletop Scenes (ECCV2022 Oral) [arXiv] <br>

If you find our work useful in your research, please consider citing:

@inproceedings{xu2022toscene,
  title={TO-Scene: A Large-scale Dataset for Understanding 3D Tabletop Scenes},
  author={Xu, Mutian and Chen, Pei and Liu, Haolin and Han, Xiaoguang},
  booktitle={ECCV},
  year={2022}
}

Dataset

TO-Scene contains 20,740 scenes with three different variants which are TO_Vanilla, TO_Crowd, TO_ScanNet.

1. Full TO-Scene data:

FormatTO_VanillaTO_CrowdTO_ScanNet
ply (point cloud)Download (4.3GB)Download (2.1GB)Download (4.3GB)
npz (xyz, color, semantic_label, instance_label, bbox)Download (6.2GB)Download (2.8GB)Download (6.5GB)
FormatTO_VanillaTO_CrowdTO_ScanNet
ply (point cloud)Download (4.3GB)Download (2.1GB)Download (4.3GB)
npz (xyz, color, semantic_label, instance_label, bbox)Download (6.2GB)Download (2.8GB)Download (6.5GB)

2. Meta data:

You may also need to download meta_data at Google Drive or Baiduyun, including train/val/test split in format of .txt and pre-calculated mean_size_arr (mean size of objects of different classes).

3. Build your own dataset:

We have released the original CAD placement file (json file, describing the position, scale, and orientation of the small objects). You can run your own rendering and reconstruction based on this original annotation file, with parameters (e.g. point density) in your control.

Benchmark on TO-Scene

We have provided the code implementations for running 3D semantic segmentation and 3D object detection on our dataset, with the corresponding instructions.

Note that TO-Scene dataset contains 60,174 tabletop object instances from 52 common classes. For reference, we show the classes of these small tabletop objects below, which can be downloaded here as well.

Big furniture

ClassSemanticClassSemanticClassSemantic
3cabinet9window24refrigerator
4bed10bookshelf28showercurtain
5chair11picture33toilet
6sofa12counter34sink
7table14desk36bathtub
8door16curtain39garbagebin

Small tabletop objects

ClassSemanticClassSemanticClassSemanticClassSemantic
41bag54laptop67chessboard80mirror
42bottle55microphone68coffee_machine81notebook
43bowl56microwave69comb82pencil
44camera57mug70cutting_board83plant
45can58printer71dishes84plate
46cap59remote_control72doll85radio
47clock60phone73eraser86ruler
48keyboard61alarm74eye_glasses87saucepan
49display62book75file_box88spoon
50earphone63cake76fork89tea_pot
51jar64calculator77fruit90toaster
52knife65candle78globe92vase
53lamp66charger79hat93vegetables

Contact

If you have any questions, please contact Mutian Xu (mutianxu@link.cuhk.edu.cn) or Pei Chen (mikecp006@gmail.com).

News

The dataset challenge will be released soon, via a complete website!