Awesome
[CGDenoiser]
Conditional Generative Denoiser for Nighttime UAV Tracking (IROS2024 Oral)
Yucheng Wang, Changhong Fu, Kunhan Lu, Liangliang Yao, and Haobo Zuo
This is the official code for the paper "Conditional Generative Denoiser for Nighttime UAV Tracking".
<img src="./main.png">Abstract
State-of-the-art (SOTA) visual object tracking methods have greatly enhanced the autonomy of unmanned aerial vehicles (UAVs) in robotics applications. However, in low-light conditions, the presence of irregular real noise from the environments severely impacts the performance of these SOTA methods. Moreover, existing SOTA denoising techniques often fail to meet real-time processing requirements due to slow speeds. To design a more practical denoiser for UAV nighttime tracking, this work initiatively breaks free from traditional deterministic paradigm limitation and proposes a conditional generative denoiser (CGDenoiser), which instead generates the noise conditioning on the input and remove it. Specifically, a novel nested residual Transformer conditionalizer is developed to better align the dimension of inputs as well as quicken the rate of inference. Furthermore, an innovative multi-kernel conditional refiner is designed to pertinently refine the denoised output. Extensive experiments show that CGDenoiser promotes the tracking precision of the SOTA tracker by 18.18% on DarkTrack2021 whereas working 5.8 times faster than the second well-performed denoiser. Real-world tests with complex challenges also prove the effectiveness and practicality of CGDenoiser.
Environment Preparing
python 3.7.3
pytorch 1.10.1
Demo
Demo Video
Test
To test CGDenoiser, you should first prepare the test dataset (in our paper we evaluate the method on two authoritative benchmarks: UAVDark135, DarkTrack2021), then you need to meet the environment requirements of base trackers and enhancers, as well as download their snapshots to corresponding folders at first. For a quick demo, this repo has included the snapshots of SiamRPN++(tracker) and DCE++(enhancer). Run the following example script for test:
python test.py \
--dataset UAVDark135 \ # dataset_name
--datasetpath ./test_dataset \ # dataset_path
--trackername SiamRPN++ \ # tracker_name
--e_weights ./checkpoint/DCE/model.pth \ # enhancer_model
--enhancername DCE \ # enhancer_name
--d_weights ./checkpoint/CGD/model.pth \ # denoiser_model
--denoisername CGD \ # denoiser_name
--video girl5 # sequence_name
Train
Before training, you need to prepare the training set of the SIDD dataset. Run the following script for training:
python -m torch.distributed.launch --nproc_per_node=1 --master_port=16888 train.py -opt config/CGD.yml --launcher pytorch
Supplementary proof
Details about the proof of CGDenoiser can be download at proof.