Home

Awesome

Refining activation downsampling with SoftPool

supported versions Library GitHub license


Update 10/2021:

We have extended this work with in our paper: AdaPool: Exponential Adaptive Pooling for Information-Retaining Downsampling. Info, code and resources are available at alexandrosstergiou/adaPool

Abstract

Convolutional Neural Networks (CNNs) use pooling to decrease the size of activation maps. This process is crucial to increase the receptive fields and to reduce computational requirements of subsequent convolutions. An important feature of the pooling operation is the minimization of information loss, with respect to the initial activation maps, without a significant impact on the computation and memory overhead. To meet these requirements, we propose SoftPool: a fast and efficient method for exponentially weighted activation downsampling. Through experiments across a range of architectures and pooling methods, we demonstrate that SoftPool can retain more information in the reduced activation maps. This refined downsampling leads to improvements in a CNN's classification accuracy. Experiments with pooling layer substitutions on ImageNet1K show an increase in accuracy over both original architectures and other pooling methods. We also test SoftPool on video datasets for action recognition. Again, through the direct replacement of pooling layers, we observe consistent performance improvements while computational loads and memory requirements remain limited. <p align="center">

<i></i> <br> <i><p align="center"> To appear in <a href="http://iccv2021.thecvf.com/home"> IEEE International Conference on Computer Vision (ICCV) 2021</a></p></i>

<p align="center"> <a href="https://arxiv.org/abs/2101.00440" target="blank" >[arXiv preprint]</a> &nbsp;&nbsp;&nbsp; <a href="https://openaccess.thecvf.com/content/ICCV2021/html/Stergiou_Refining_Activation_Downsampling_With_SoftPool_ICCV_2021_paper.html" target="_blank">[CVF open access]</a> &nbsp;&nbsp;&nbsp; <a href="https://www.youtube.com/watch?v=iqsMoVQSyDw" target="blank" >[video presentation]</a> </p>

Image based pooling. Images are sub-sampled in both height and width by half.

Original<img src="images/buildings.jpg" width="130" /><img src="images/otters.jpg" width="130" /><img src="images/tennis_ball.jpg" width="130" /><img src="images/puffin.jpg" width="130" /><img src="images/tram.jpg" width="130" /><img src="images/tower.jpg" width="130" />
Soft Pool<img src="images/buildings_soft.jpg" width="130" /><img src="images/otters_soft.jpg" width="130" /><img src="images/tennis_ball_soft.jpg" width="130" /><img src="images/puffin_soft.jpg" width="130" /><img src="images/tram_soft.jpg" width="130" /><img src="images/tower_soft.jpg" width="130" />

Video based pooling. Videos are sub-sampled in time, height and width by half.

Original<img src="images/cars.gif" width="130" /><img src="images/basketball.gif" width="130" /><img src="images/parkour.gif" width="130" /><img src="images/bowling.gif" width="130" /><img src="images/pizza_toss.gif" width="130" /><img src="images/pass.gif" width="130" />
Soft Pool<img src="images/cars_soft.gif" width="130" /><img src="images/basketball_soft.gif" width="130" /><img src="images/parkour_soft.gif" width="130" /><img src="images/bowling_soft.gif" width="130" /><img src="images/pizza_toss_soft.gif" width="130" /><img src="images/pass_soft.gif" width="130" />

Dependencies

All parts of the code assume that torch is of version 1.4 or higher. There might be instability issues on previous versions.

! Disclaimer: This repository is heavily structurally influenced on Ziteng Gao's LIP repo https://github.com/sebgao/LIP

Installation

You can build the repo through the following commands:

$ git clone https://github.com/alexandrosstergiou/SoftPool.git
$ cd SoftPool-master/pytorch
$ make install
--- (optional) ---
$ make test

Usage

You can load any of the 1D, 2D or 3D variants after the installation with:

import softpool_cuda
from SoftPool import soft_pool1d, SoftPool1d
from SoftPool import soft_pool2d, SoftPool2d
from SoftPool import soft_pool3d, SoftPool3d

ImageNet models

ImageNet weight can be downloaded from the following links:

Networklink
ResNet-18link
ResNet-34link
ResNet-50link
ResNet-101link
ResNet-152link
DenseNet-121link
DenseNet-161link
DenseNet-169link
ResNeXt-50_32x4dlink
ResNeXt-101_32x4dlink
wide-ResNet50link

Citation

@inproceedings{stergiou2021refining,
  title={Refining activation downsampling with SoftPool},
  author={Stergiou, Alexandros, Poppe, Ronald and Kalliatakis Grigorios},
  booktitle={International Conference on Computer Vision (ICCV)},
  year={2021},
  pages={10357-10366},
  organization={IEEE}
}

Licence

MIT

Additional resources

A great project is Ren Tianhe's pytorh-pooling repo for overviewing different pooling strategies.