Awesome
TMO
This is the official PyTorch implementation of our paper:
Treating Motion as Option to Reduce Motion Dependency in Unsupervised Video Object Segmentation, WACV 2023
Suhwan Cho, Minhyeok Lee, Seunghoon Lee, Chaewon Park, Donghyeong Kim, Sangyoun Lee
Link: [WACV] [arXiv]
<img src="https://user-images.githubusercontent.com/54178929/208474605-7586894f-11cf-4e38-ac21-75a78216c22d.png" width=800>Treating Motion as Option with Output Selection for Unsupervised Video Object Segmentation, arXiv 2023
Suhwan Cho, Minhyeok Lee, Jungho Lee, MyeongAh Cho, Sangyoun Lee
Link: [arXiv]
You can also find other related papers at awesome-video-object-segmentation.
Abstract
In unsupervised VOS, most state-of-the-art methods leverage motion cues obtained from optical flow maps in addition to appearance cues. However, as they are overly dependent on motion cues, which may be unreliable in some cases, they cannot achieve stable prediction. To overcome this limitation, we design a novel motion-as-option network that is not much dependent on motion cues and a collaborative network learning strategy to fully leverage its unique property. Additionally, an adaptive output selection algorithm is proposed to maximize the efficacy of the motion-as-option network at test time.
Preparation
1. Download DUTS for network training.
2. Download DAVIS for network training and testing.
3. Download FBMS for network testing.
4. Download YouTube-Objects for network testing.
5. Download Long-Videos for network testing.
6. Save optical flow maps of DAVIS, FBMS, YouTube-Objects, and Long-Videos using RAFT.
7. For convenience, I also provide the pre-processed DUTS, DAVIS, FBMS, YouTube-Objects, and Long-Videos.
8. Replace dataset paths in "run.py" file with your dataset paths.
Training
1. Move to "run.py" file.
2. Define model version (ver): 'rn101' or 'mitb1'
3. Check training settings.
4. Run TMO training!!
python run.py --train
Testing
1. Move to "run.py" file.
2. Define model version (ver): 'rn101' or 'mitb1'
3. Use or not adaptive output selection (aos): True or False
4. Select a pre-trained model that accords with the defined model version.
5. Run TMO testing!!
python run.py --test
Attachments
pre-trained model (rn101)
pre-trained model (mitb1)
pre-computed results
Note
Code and models are only available for non-commercial research purposes.
If you have any questions, please feel free to contact me :)
E-mail: suhwanx@gmail.com