Awesome
AccFlow: Backward Accumulation for Long-Range Optical Flow
ICCV2023 | AccFlow: Backward Accumulation for Long-Range Optical Flow
Guangyang Wu, Xiaohong Liu, Kunming Luo, Xi Liu, Qingqing Zheng, Shuaicheng Liu, Xinyang Jiang, Guangtao Zhai, Wenyi Wang
TODO:
- Add inference and visualization codes.
- Add a demo video for better understanding.
- Add figures and brief introduction of this work.
- Provide google drive link for CVO dataset.
- Add warmstart mode for evaluation.
- Add evaluation using GMFlow models.
Requirements
conda env create -f environment.yml
conda activate accflow
Models
We provide pretrained models. The default path of the models for evaluation is:
├── checkpoints
├── acc+raft-things.pth
├── acc+gma-things.pth
├── acc+raft-cvo.pth
├── acc+gma-cvo.pth
├── raft-cvo.pth
├── gma-cvo.pth
├── raft-things.pth
├── gma-things.pth
Evaluation
Download checkpoints
and put it in the root dir.
Download testing dataset CVO-test, put the files cvo-test.lmdb
and cvo-test.lmdb-lock
in the directory data/datasets
.
To evaluate on the clean and final splits, use '-d' param to specify. To evaluate direct methods (e.g., RAFT, GMA), set '-acc' to 'direct'. To evaluate accumulation methods (i.e., accflow), set '-acc' to 'acc'.
python test_cvo.py -d clean -acc direct -ofe raft --ofe_ckpt checkpoints/raft-things.pth
python test_cvo.py -d clean -acc acc -ofe raft --acc_ckpt checkpoints/acc+raft-things.pth
More samples can be found in test_cvo.sh.
Training
The script will load the config according to the training stage. The trained model will be saved in a directory in logs
and checkpoints
. For example, the following script will load the config configs/***.yml
.
# Fine-tune RAFT and GMA (pretrained on flyingthings) using CVO training set
python fine_tune.py -c configs/RAFT.yml
python fine_tune.py -c configs/GMA.yml
# Train AccFlow based on RAFT and GMA (pretrained on flyingthings) using CVO training set
python train_acc.py -c configs/AccGMA.yml
python train_acc.py -c configs/AccRAFT.yml
# Train AccFlow based on RAFT and GMA (fine-tuned with CVO-train)
python train_acc.py -c configs/AccGMA-CVO.yml
python train_acc.py -c configs/AccRAFT-CVO.yml
License
AccFLow is released under the MIT License
Citation
If you use any part of this code, please kindly cite
@article{wu2023accflow,
title={AccFlow: Backward Accumulation for Long-Range Optical Flow},
author={Guangyang Wu and Xiaohong Liu and Kunming Luo and Xi Liu and Qingqing Zheng and Shuaicheng Liu and Xinyang Jiang and Guangtao Zhai and Wenyi Wang},
journal={arXiv preprint arXiv:2308.13133},
year={2023}
}
Acknowledgement
In this project, we use parts of codes in: