Awesome
[ECCV 2022] MuLUT: Cooperating Mulitple Look-Up Tables for Efficient Image Super-Resolution
[T-PAMI 2024] Toward DNN of LUTs: Learning Efficient Image Restoration with Multiple Look-Up Tables
Jiacheng Li, Chang Chen, Zhen Cheng, and Zhiwei Xiong
ECCV Paper | ECCV Paper Supp. | T-PAMI Paper | Poster | Project Page | Intro Video
News
2024.06 Our new work, Diagonal-First Compression for LUT(DFC) has been presented as a highlight paper at CVPR 2024. DFC reduce the storage requirement of restoration LUTs significantly (up to 10%) while preserving their performance.
2024.05 The extended version of MuLUT, DNN-of-LUT, is accepted by IEEE T-PAMI.
2023.03 Extended version of MuLUT is available on arxiv.
2023.02 Our new work, Learning Resampling Function(LeRF), is accepted by CVPR 2023. LeRF makes up for the regrets of MuLUT on replacing interpolation methods via achiving continuous resampling.
2022.10 MuLUT is open sourced.
At A Glance
Please learn more at our project page.
Usage
Code overview
In the sr
directory, we provide the code of training MuLUT networks, transferring MuLUT network into LUts, finetuning LUTs, and testing LUTs, taking the task of single image super-resolution as an example.
In the common/network.py
file, we provide a universal implementation of MuLUT blocks, which can be constructed into MuLUT networks in a lego-like way.
Dataset
Please following the instructions of training and testing.
Step 0: Installation
Clone this repo.
git clone https://github.com/ddlee-cn/MuLUT
Install requirements: torch>=1.5.0
, opencv-python
, scipy
Step 1: Training MuLUT network
First, let us train a MuLUT network.
cd sr
python 1_train_model.py --stages 2 --modes sdy -e ../models/sr_x2sdy \
--trainDir ../data/DIV2K --valDir ../data/SRBenchmark
Our trained model and log are available under the models/sr_x2sdy
directory.
Step 2: Transfer MuLUT blocks into LUTs
Now, we are ready to cache the MuLUT network into multiple LUTs.
python 2_transfer_to_lut.py --stages 2 --modes sdy -e ../models/sr/x2sdy
Step 3: Fine-tuning LUTs
python 3_finetune_lut.py --stages 2 --modes sdy -e ../models/sr_x2sdy \
--trainDir ../data/DIV2K --valDir ../data/SRBenchmark
After fine-tuning, LUTs are saved into .npy
files and can be deployed on other devices. Our fine-tuned LUTs and log are available under the models/sr_x2sdy
directory.
Step 4: Test LUTs
Finally, we provide the following script to execute the LUT retrieval.
python 4_test_lut.py --stages 2 --modes sdy -e ../models/sr_x2sdy
The reference results for the Set5
dataset are provided under the results/sr_x2sdy
directory.
Contact
If you have any questions, feel free to contact me by e-mail jclee [at] mail.ustc.edu.cn
.
Citation
If you find our work helpful, please cite the following papers.
@InProceedings{Li_2022_MuLUT,
author = {Li, Jiacheng and Chen, Chang and Cheng, Zhen and Xiong, Zhiwei},
title = {{MuLUT}: Cooperating Multiple Look-Up Tables for Efficient Image Super-Resolution},
booktitle = {ECCV},
year = {2022},
}
@ARTICLE{10530442,
author = {Li, Jiacheng and Chen, Chang and Cheng, Zhen and Xiong, Zhiwei},
title = {Toward {DNN} of {LUTs}: Learning Efficient Image Restoration with Multiple Look-Up Tables},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2024},
volume = {},
number = {},
pages = {1-18},
doi = {10.1109/TPAMI.2024.3401048}
}
@InProceedings{Li_2023_LeRF,
author = {Li, Jiacheng and Chen, Chang and Huang, Wei and Lang, Zhiqiang and Song, Fenglong and Yan, Youliang and Xiong, Zhiwei},
title = {Learning Steerable Function for Efficient Image Resampling},
booktitle = {CVPR},
year = {2023},
}
License
MIT
Acknowledgement
Our code is build upon SR-LUT.