Home

Awesome

Learnable Tree Filter V2

By Lin Song, Yanwei Li, Zhengkai Jiang, Zeming Li, Xiangyu Zhang, Hongbin Sun, Jian Sun, Nanning Zheng.

This repo is an official implementation for "Rethinking Learnable Tree Filter for Generic Feature Transform" (NeurIPS2020) on PyTorch framework.

introduce image

Installation

Requirements

Build from source

Prepare data

See datasets/README.md.

Usage

All the projects are placed in cvpods_playground. You need to select a project and enter the corresponding folder.

# For example
cd cvpods_playground/mask_rcnn.res50.fpn.coco.1x.nn_syncbn.tf-v2

Training

# Running training procedure with specific GPU number
cvpods_train --gpu-nums <GPU_NUM> [optional arguments]

# Please refer to tools/train_net.py for more optional arguments

Inference

# Running inference procedure with specific GPU number
# The program will load the lastest model weights from ./log folder
cvpods_test --gpu-nums <GPU_NUM> [optional arguments]

# Running inference procedure with specific GPU number and model path
cvpods_test --gpu-nums <GPU_NUM> MODEL.WEIGHTS <model_path> [optional arguments]

# Please refer to tools/test_net.py for more optional arguments

Visualize your image [Optional]

# Please refer to demo/demo.py for more optional arguments
python ../../demo/demo.py --config <config_path> --input <input_path> --output <output_path> MODEL.WEIGHTS <model_path> [optional arguments]

Add to your code [Optional]

# Load the learnable tree filter module
from cvpods.layers import TreeFilterV2

# Initialize the module with specific number of channels and groups
tf = TreeFilterV2(<guide_channels>, <in_channels>, <embed_channels>, [num_groups])

# Run the filter procedure with input feature and guided feature
output_feature = tf(<input_feature>, <guided_feature>)

Performance

Instance Segmentation on COCO val set

MethodBackboneScheduleAP<sub>box</sub> (%)AP<sub>mask</sub> (%)ProjectModel
Mask-RCNN + LTF-V2ResNet-501x41.337.0LinkOneDrive
Mask-RCNN + LTF-V2ResNet-1011x42.538.0LinkOneDrive
Mask-RCNN + LTF-V2ResNeXt-1011x44.539.5LinkOneDrive
Cascade Mask-RCNN + LTF-V2ResNeXt-1011x46.940.3LinkOneDrive

Panoptic Segmentation on COCO val set

MethodBackboneSchedulePQ (%)AP<sub>box</sub> (%)AP<sub>mask</sub> (%)ProjectModel
Panoptic-FPNResNet-501x39.237.433.6LinkOneDrive
Panoptic-FPN + LTF-V2ResNet-501x41.739.335.4LinkOneDrive

More Results is Coming

To do

Citation

Please cite the paper in your publications if it helps your research.

@inproceedings{song2020rethinking,
    title = {Rethinking Learnable Tree Filter for Generic Feature Transform},
    author = {Song, Lin and Li, Yanwei and Jiang, Zhengkai and Li, Zeming and Zhang, Xiangyu and Sun, Hongbin and Sun, Jian and Zheng, Nanning},
    booktitle = {Advances in Neural Information Processing Systems},
    year = {2020}
}

Please cite this project in your publications if it helps your research.

@misc{learnabletreefilterv2,
    author = {Song, Lin},
    title = {LearnableTreeFilterV2},
    howpublished = {\url{https://github.com/StevenGrove/LearnableTreeFilterV2}},
    year = {2020}
}