Awesome

Tensorflow Fast-MPNCOV

Introduction

This repository contains the source code under TensorFlow2.0 framework and models trained on ImageNet 2012 dataset for the following paper:<br>

@InProceedings{Li_2018_CVPR,
           author = {Li, Peihua and Xie, Jiangtao and Wang, Qilong and Gao, Zilin},
           title = {Towards Faster Training of Global Covariance Pooling Networks by Iterative Matrix Square Root Normalization},
           booktitle = { IEEE Int. Conf. on Computer Vision and Pattern Recognition (CVPR)},
           month = {June},
           year = {2018}
     }

This paper concerns an iterative matrix square root normalization network (called fast MPN-COV), which is very efficient, fit for large-scale datasets, as opposed to its predecessor (i.e., MPN-COV) published in ICCV17) that performs matrix power normalization by Eigen-decompositon. If you use the code, please cite this fast MPN-COV work and its predecessor (i.e., MPN-COV).

Classification results

Classification results (single crop 224x224, %) on ImageNet 2012 validation set

• We convert the trained fast-MPNCOV-VGG-D model from the PyTorch framework to TensorFlow framework.

Fine-grained classification results (top-1 accuracy rates, %)

• Our method uses neither bounding boxes nor part annotations<br>
• The reproduced results are obtained by simply finetuning our pre-trained fast MPN-COV-ResNet model with a small learning rate, which do not perform SVM as our paper described.<br>

parameter setting
fast-MPNCOV-VGG-D: weightdecay=1e-4, batchsize=10, learningrate=3e-3 for all layers except the FC layer(which is 5×learningrate, and the learning rate is reduced to 3e-4 at epoch 20(FC: 5×3e-4)

Implementation details

We implement our Fast MPN-COV (i.e., iSQRT-COV) meta-layer under Tensorflow2.0 package. We release two versions of code:<br>

• The backpropagation of our meta-layer without using autograd package;<br>
• The backpropagation of our meta-layer with using autograd package(TODO).<br>

For making our Fast MPN-COV meta layer can be added in a network conveniently, we divide any network for three parts: <br>

• features extractor;<br>
• global image representation;<br>
• classifier. <br>

As such, we can arbitrarily combine a network with our Fast MPN-COV or some other global image representation methods (e.g.,Global average pooling, Bilinear pooling, Compact bilinear pooling, etc.)

Installation and Usage

1. Install Tensorflow (2.0.0b0)
2. type git clone https://github.com/XuChunqiao/Tensorflow-Fast-MPNCOV
3. prepare the dataset as follows

.
├── train
│   ├── class1
│   │   ├── class1_001.jpg
│   │   ├── class1_002.jpg
|   |   └── ...
│   ├── class2
│   ├── class3
│   ├── ...
│   ├── ...
│   └── classN
└── val
    ├── class1
    │   ├── class1_001.jpg
    │   ├── class1_002.jpg
    |   └── ...
    ├── class2
    ├── class3
    ├── ...
    ├── ...
    └── classN

train from scratch

1. cp ./trainingFromScratch/imagenet/imagenet_tfrecords.py ./
2. modify the dataset path and run python imagenet_tfrecords.py to create tfrecord files
3. modify the parameters in train.sh sh train.sh

finetune fast-MPNCOV models

1. modify the parameters in finetune.sh
2. sh finetune.sh

Other Implementations

• MatConvNet Implementation
• PyTorch Implementation