Awesome

Dataset

UrbanLF

UrbanLF is a high-quality and challenging urban scene light field dataset, containing real samples as well as synthetic samples. For technical details, please refer to: UrbanLF: A New Light Field Dataset For Semantic Segmentation of Urban Scene. (IEEE TCSVT 2022)

The effective depth does not exceed 100 meters and the value of sky that can not be measured is set to 200.

UrbanLF-Multiscale-Baseline-Dataset

UrbanLF-Multiscale-Baseline-Dataset is a high-quality and challenging urban scene light field dataset, containing sparsely sampled as well as densely sampled synthetic light field samples that share the same scenarios. It is created by Blender on the basis of UrbanLF-Syn. For technical details, please refer to: End-to-End Semantic Segmentation Utilizing Multi-scale Baseline Light Field. (IEEE TCSVT 2024)

Among the 400 scenarios corresponding to 400 samples, 250 of them are from UrbanLF-Syn, and the remaining 150 scenarios are newly built.

Non-Lambertian-LF

Non-Lambertian-LF is a synthetic light field depth estimation dataset focused on non-Lambertian objects. It is available at here.

Benchmark

Our benchmark has been held with the 3rd Workshop on Light Fields for Computer Vision LFNAT: New Applications and Trends in Light Fields in conjunction with CVPR 2023.

Semantic Segmentation

Class Selection

There are 14 classes for evaluation and their definitions are shown in the table.

Dataset Splitting

Data Link

UrbanLF-Real and UrbanLF-Syn are publicly available in Baiduyun (key: 6ew2) and Google. <br/> UrbanLF-Syn-Big is publicly available in Baiduyun (key: ekw1) and Google. <br/> UrbanLF-Syn-Small is publicly available in Baiduyun (key: 0cp1) and Google. <br/>

Note

The metrics (mIoU, mAcc, Acc) are calculated only in full resolution of central view image.

Super Resolution

Data generation

Our benchmark provides ×2 and ×4 light field spatial super resolution (LFSSR). The bicubic interpolation with a factor of 2 and 4 is applied to generate low resolution images of different scales. Due to the limitation of resolution, the real images are preprocessed before generating the test data. For ×2 task, we crop 1 pixel width on the right. For ×4 task, we crop 2 pixel width on the right and 1 pixel width on the left.

Dataset Splitting

Considering that sharing the same test data with other benchmarks will expose the ground truth, we extra collect 80 real and 30 synthetic samples as two new test sets.

Data Link

Data is publicly available in Baiduyun (key: 7cwp) and Google.

Note

The metrics (PSNR, SSIM) are calculated by averaging over all sub-aperture images.

The spatial resolution of the prediction for real images should be 622×432 for ×2 task and 620×432 for ×4 task.

Depth Estimation

Only UrbanLF-Syn has ground truth disparity with range [−0.47,1.55] pixels between adjacent views.

For UrbanLF-Syn-big-dis, it has ground truth disparity with range [−7.09,7.05] pixels between adjacent views.

Dataset Splitting

We create a new test set to avoid the disparity data leakage owing to data sharing among benchmarks and provide maximum and minimum disparity value.

Data Link

Data is publicly available in Baiduyun (key: f54t) and Google.

Note

The metrics (MSE, BP) are calculated only on central view image with cropping 15 pixels at each border.

Benchmark Result of LFNAT 2023 CVPR Workshop

In conjunction with the workshop on Light Fields for Computer Vision LFNAT, we host a light field depth estimation challenge based upon UrbanLF-Syn. For result analysis details, please refer to: LFNAT 2023 Challenge on Light Field Depth Estimation: Methods and Results. (CVPRW 2023)

Citation

@InProceedings{Sheng2023lfnat,
    author    = {Sheng, Hao and Liu, Yebin and Yu, Jingyi and Wu, Gaochang and Xiong, Wei and Cong, Ruixuan and Chen, Rongshan and others},
    title     = {LFNAT 2023 Challenge on Light Field Depth Estimation: Methods and Results},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
    month     = {June},
    year      = {2023},
    pages     = {3472-3484}
}

News & Updates

• Feb 27, 2024: Upload UrbanLF-Multiscale-Baseline-Dataset on the basis of UrbanLF-Syn that contains large disparity as well as small disparity samples.
• Jun 20, 2023: Update Light Field Depth Estimation Challenge Result of 2023 LFNAT CVPR Workshop Based on UrbanLF-Syn dataset.
• Jan 20, 2023: Non-Lambertian-LF dataset for non-Lambertian light field depth estimation is available.
• Jan 17, 2023: Benchmark online available.

Citation

If you find this work is helpful to you, please cite the following paper:

@ARTICLE{sheng2022urbanlf,
  author={Sheng, Hao and Cong, Ruixuan and Yang, Da and Chen, Rongshan and Wang, Sizhe and Cui, Zhenglong},
  journal={IEEE Transactions on Circuits and Systems for Video Technology}, 
  title={UrbanLF: A Comprehensive Light Field Dataset for Semantic Segmentation of Urban Scenes}, 
  year={2022},
  volume={32},
  number={11},
  pages={7880-7893},
  doi={10.1109/TCSVT.2022.3187664}}

@ARTICLE{cong2024lfienetplus,
  author={Cong, Ruixuan and Sheng, Hao and Yang, Dazhi and Yang, Da and Chen, Rongshan and Wang, Sizhe and Cui, Zhenglong},
  journal={IEEE Transactions on Circuits and Systems for Video Technology}, 
  title={End-to-End Semantic Segmentation Utilizing Multi-scale Baseline Light Field}, 
  year={2024},
  doi={10.1109/TCSVT.2024.3367370}}

Statement

Most people and owners of the vehicles captured in the dataset have signed a license to allow the images to be used for scientific research. If the work violates your right to privacy, please send an email to congrx@buaa.edu.cn .