Awesome
rspy
<div align="center">The core
implementation of Fast Rolling Shutter Correction in the Wild, TPAMI 2023 and Towards Nonlinear-Motion-Aware and Occlusion-Robust Rolling Shutter Correction, ICCV 2023.
3GS | Gpark |
---|---|
🚀 Features
- A lightweight library for rolling shutter correction which is easy to use.
- Support linear, quadratic, and cubic motion models.
- Support sparse feature correction and can be plugged into 3D vision algorithm pipeline, such as SfM, SLAM, etc.
Installation
pip install git+https://github.com/DelinQu/rspy
# pip install rspy is unavailable now
It's recommended to clone the code and run the demo files in the rspy
folder. In our implementation, we use open-mmlab/mmflow to calculate the optical flow. Please refer to the installation for optical flow support, and check the requirements.txt to satisfy the dependency.
Usage
rspy
contains linear
, quadratic
, and cubic
models for faster rolling shutter correction. The solver
is the core of the rspy
, which receives the optical flow fields and returns the correction field. The feats_sampling
function warps the RS image back to GS one.
gamma
: the ratio of the exposure time to the frame interval.tau
: the normalized timestamp warping to.
linear_flow
receives a optical flow field from $I_{0} \to I_{-1}$ and return correction field $u_{0 \to \tau}$.
F0tau = solver(F0n1, gamma, tau) # * (1,h,w,2)
rsc_image = feats_sampling(rs_image, -F0tau)
</p>
</details>
<details>
<summary>2. Quadratic rolling shutter correction</summary>
<p>
quadratic_flow
receives two optical flow fields from $I_{0} \to I_{-1}$ and $I_{0} \to I_{1}$, and return correction field $u_{0 \to \tau}$.
F0tau = solver(F0n1, F01, gamma, tau) # * (1,h,w,2)
rsc_image = feats_sampling(rs_image, -F0tau)
</p>
</details>
<details>
<summary>3. Cubic rolling shutter correction</summary>
<p>
cubic_flow
receives three optical flow fields from $I_{0} \to I_{-2}$, $I_{0} \to I_{-1}$ and $I_{0} \to I_{1}$, and return correction field $u_{0 \to \tau}$.
F0tau = solver(F0n2, F0n1, F01, gamma, tau) # * (1,h,w,2)
rsc_image = feats_sampling(rs_image, -F0tau)
</p>
</details>
🍀 Demo
We provided a demo for rolling shutter correction in rspy
, which read the images from the demo
folder and save the results in the out
folder. Before that, you must download a optical flow model by mim
:
mim download mmflow --config raft_8x2_100k_mixed_368x768
It will be automatically downloaded to ~/.cache/mim
. Then, the demo can be run by the following command.
python rspy/demo.py --model=linear
python rspy/demo.py --model=quadratic
python rspy/demo.py --model=cubic
You can also use your own images with a suitable gamma
and tau
to get a satisfactory result.
📈 Releases
You can see the list of available releases on the GitHub Releases page.
Label | Title in Releases |
---|---|
the foundation of rspy | 🐣 Features |
relase the pypi package | 🚀 Release |
To be continue :) | ⇨ ▶️ |
🛡 License
This project is licensed under the terms of the GNU GPL v3.0
license. See LICENSE for more details.
📃 Citation
If you find this project useful for your research, please use the following BibTeX entry:
@ARTICLE{qu2023fast,
author={Qu, Delin and Liao, Bangyan and Zhang, Huiqing and Ait-Aider, Omar and Lao, Yizhen},
journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
title={Fast Rolling Shutter Correction in the Wild},
year={2023},
volume={},
number={},
pages={1-18},
doi={10.1109/TPAMI.2023.3284847}
}
@article{qu2023towards,
title = {Towards Nonlinear-Motion-Aware and Occlusion-Robust Rolling Shutter Correction},
author = {Delin Qu and Yizhen Lao and Zhigang Wang and Dong Wang and Bin Zhao and Xuelong Li},
year = {2023},
journal = {arXiv preprint arXiv: 2303.18125}
}
Credits
This project was generated with python-package-template