Home

Awesome

LLVIP: A Visible-infrared Paired Dataset for Low-light Vision visitors

Project | Arxiv | Kaggle | PWC | Tweet

News


figure1-LR


Dataset Downloading:

Homepage or DatasetDownload


Citation

If you use this data for your research, please cite our paper LLVIP: A Visible-infrared Paired Dataset for Low-light Vision:

@inproceedings{jia2021llvip,
  title={LLVIP: A visible-infrared paired dataset for low-light vision},
  author={Jia, Xinyu and Zhu, Chuang and Li, Minzhen and Tang, Wenqi and Zhou, Wenli},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={3496--3504},
  year={2021}
}

or

@misc{https://doi.org/10.48550/arxiv.2108.10831,
  doi = {10.48550/ARXIV.2108.10831}, 
  url = {https://arxiv.org/abs/2108.10831},
  author = {Jia, Xinyu and Zhu, Chuang and Li, Minzhen and Tang, Wenqi and Liu, Shengjie and Zhou, Wenli}, 
  keywords = {Computer Vision and Pattern Recognition (cs.CV), Artificial Intelligence (cs.AI), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {LLVIP: A Visible-infrared Paired Dataset for Low-light Vision},
  publisher = {arXiv},
  year = {2021},
  copyright = {arXiv.org perpetual, non-exclusive license}
}
<h2> <p align="center"> Image Fusion </p> </h2>

Baselines

FusionGAN

Preparation

Train

python main.py --epoch 10 --batch_size 32

See more training options in main.py.

Test

python test_one_image.py

Remember to put pretrained model in your checkpoint folder and change corresponding model name in test_one_image.py. To acquire complete LLVIP dataset, please visit https://bupt-ai-cz.github.io/LLVIP/.

Densefuse

Preparation

Train & Test

python main.py 

Check and modify training/testing options in main.py. Before training/testing, you need to rename the images in LLVIP dataset and put them in the designated folder. We have provided a script named rename.py to rename the images and save them in the datasets or test folder. Checkpoints are saved in ./models/densefuse_gray/. To acquire complete LLVIP dataset, please visit https://bupt-ai-cz.github.io/LLVIP/.

IFCNN

Please visit https://github.com/uzeful/IFCNN.

<h2> <p align="center"> Pedestrian Detection </p> </h2>

Baselines

Yolov5

Preparation

Linux and Python>=3.6.0

Train

python train.py --img 1280 --batch 8 --epochs 200 --data LLVIP.yaml --weights yolov5l.pt --name LLVIP_export

See more training options in train.py. The pretrained model yolov5l.pt can be downloaded from here. The trained model will be saved in ./runs/train/LLVIP_export/weights folder.

Test

python val.py --data --img 1280 --weights last.pt --data LLVIP.yaml

Remember to put the trained model in the same folder as val.py.

Our trained model can be downloaded from here: Google-Drive-Yolov5-model or BaiduYun-Yolov5-model (code: qepr)

Results

We retrained and tested Yolov5l and Yolov3 on the updated dataset (30976 images).

<div align="center"> <img src="https://user-images.githubusercontent.com/33684330/138012320-3340bf17-481a-4d69-a8a9-fc7427055cf4.jpg" height="130" width="700"> </div>

Where AP means the average of AP at IoU threshold of 0.5 to 0.95, with an interval of 0.05.

<div align="center"> <img src="https://user-images.githubusercontent.com/33684330/138388453-9953c403-4a5f-4b45-a488-e44fcec7b955.png" height="510" width="700"> </div> The figure above shows the change of AP under different IoU thresholds. When the IoU threshold is higher than 0.7, the AP value drops rapidly. Besides, the infrared image highlights pedestrains and achieves a better effect than the visible image in the detection task, which not only proves the necessity of infrared images but also indicates that the performance of visible-image pedestrian detection algorithm is not good enough under low-light conditions.

We also calculated log average miss rate based on the test results and drew the miss rate-FPPI curve.

<div align="center"> <img src="https://user-images.githubusercontent.com/33684330/138281822-ea7dd310-bb9d-4197-8dfe-f17bb4534986.jpeg" height="110" width="700"> </div> <div align="center"> <img src="https://user-images.githubusercontent.com/33684330/138312218-274810d5-0191-4fbc-abfe-01736dc285bf.png" height="510" width="700"> </div> <h2> <p align="center"> Image-to-Image Translation </p> </h2>

Baseline

pix2pixGAN

Preparation

Train

python train.py --dataroot ./datasets/LLVIP --name LLVIP --model pix2pix --direction AtoB --batch_size 8 --preprocess scale_width_and_crop --load_size 320 --crop_size 256 --gpu_ids 0 --n_epochs 100 --n_epochs_decay 100

Test

python test.py --dataroot ./datasets/LLVIP --name LLVIP --model pix2pix --direction AtoB --gpu_ids 0 --preprocess scale_width_and_crop --load_size 320 --crop_size 256

See ./pix2pixGAN/options for more train and test options.

<img src='imgs/LLVIP.gif' align="right" height=192 width=448> <br>

Results

We retrained and tested pix2pixGAN on the updated dataset(30976 images). The structure of generator is unet256, and the structure of discriminator is the basic PatchGAN as default.

<div align="center"> <img src="https://user-images.githubusercontent.com/33684330/138233570-1440a5bf-7a05-4e96-b8ab-fc32a9c59748.jpeg" height="100" width="700"> </div> <div align="center"> <img src="https://user-images.githubusercontent.com/33684330/135420925-72b9722a-3838-437b-b1a7-5f9e81c91d85.png" height="610" width="700"> </div>

License

This LLVIP Dataset is made freely available to academic and non-academic entities for non-commercial purposes such as academic research, teaching, scientific publications, or personal experimentation. Permission is granted to use the data given that you agree to our license terms.

Call For Contributions

Welcome to point out errors in data annotation. If you want to modify the label, please refer to the annotation tutorial, and email us the corrected label file.

More annotation forms are also welcome (such as segmentation), please contact us.

Acknowledgments

Thanks XueZ-phd for his contribution to LLVIP dataset. He corrected the imperfect annotations in the dataset.

Contact

email: shengjie.Liu@bupt.edu.cn, czhu@bupt.edu.cn, jiaxinyujxy@qq.com, tangwenqi@bupt.edu.cn