Awesome
Refining activation downsampling with SoftPool
Update 10/2021:
We have extended this work with in our paper: AdaPool: Exponential Adaptive Pooling for Information-Retaining Downsampling. Info, code and resources are available at alexandrosstergiou/adaPool
Abstract
Convolutional Neural Networks (CNNs) use pooling to decrease the size of activation maps. This process is crucial to increase the receptive fields and to reduce computational requirements of subsequent convolutions. An important feature of the pooling operation is the minimization of information loss, with respect to the initial activation maps, without a significant impact on the computation and memory overhead. To meet these requirements, we propose SoftPool: a fast and efficient method for exponentially weighted activation downsampling. Through experiments across a range of architectures and pooling methods, we demonstrate that SoftPool can retain more information in the reduced activation maps. This refined downsampling leads to improvements in a CNN's classification accuracy. Experiments with pooling layer substitutions on ImageNet1K show an increase in accuracy over both original architectures and other pooling methods. We also test SoftPool on video datasets for action recognition. Again, through the direct replacement of pooling layers, we observe consistent performance improvements while computational loads and memory requirements remain limited. <p align="center">
<i></i> <br> <i><p align="center"> To appear in <a href="http://iccv2021.thecvf.com/home"> IEEE International Conference on Computer Vision (ICCV) 2021</a></p></i>
<p align="center"> <a href="https://arxiv.org/abs/2101.00440" target="blank" >[arXiv preprint]</a> <a href="https://openaccess.thecvf.com/content/ICCV2021/html/Stergiou_Refining_Activation_Downsampling_With_SoftPool_ICCV_2021_paper.html" target="_blank">[CVF open access]</a> <a href="https://www.youtube.com/watch?v=iqsMoVQSyDw" target="blank" >[video presentation]</a> </p>Image based pooling. Images are sub-sampled in both height and width by half.
Original | <img src="images/buildings.jpg" width="130" /> | <img src="images/otters.jpg" width="130" /> | <img src="images/tennis_ball.jpg" width="130" /> | <img src="images/puffin.jpg" width="130" /> | <img src="images/tram.jpg" width="130" /> | <img src="images/tower.jpg" width="130" /> |
---|---|---|---|---|---|---|
Soft Pool | <img src="images/buildings_soft.jpg" width="130" /> | <img src="images/otters_soft.jpg" width="130" /> | <img src="images/tennis_ball_soft.jpg" width="130" /> | <img src="images/puffin_soft.jpg" width="130" /> | <img src="images/tram_soft.jpg" width="130" /> | <img src="images/tower_soft.jpg" width="130" /> |
Video based pooling. Videos are sub-sampled in time, height and width by half.
Original | <img src="images/cars.gif" width="130" /> | <img src="images/basketball.gif" width="130" /> | <img src="images/parkour.gif" width="130" /> | <img src="images/bowling.gif" width="130" /> | <img src="images/pizza_toss.gif" width="130" /> | <img src="images/pass.gif" width="130" /> |
---|---|---|---|---|---|---|
Soft Pool | <img src="images/cars_soft.gif" width="130" /> | <img src="images/basketball_soft.gif" width="130" /> | <img src="images/parkour_soft.gif" width="130" /> | <img src="images/bowling_soft.gif" width="130" /> | <img src="images/pizza_toss_soft.gif" width="130" /> | <img src="images/pass_soft.gif" width="130" /> |
Dependencies
All parts of the code assume that torch
is of version 1.4 or higher. There might be instability issues on previous versions.
! Disclaimer: This repository is heavily structurally influenced on Ziteng Gao's LIP repo https://github.com/sebgao/LIP
Installation
You can build the repo through the following commands:
$ git clone https://github.com/alexandrosstergiou/SoftPool.git
$ cd SoftPool-master/pytorch
$ make install
--- (optional) ---
$ make test
Usage
You can load any of the 1D, 2D or 3D variants after the installation with:
import softpool_cuda
from SoftPool import soft_pool1d, SoftPool1d
from SoftPool import soft_pool2d, SoftPool2d
from SoftPool import soft_pool3d, SoftPool3d
soft_poolxd
: Is a functional interface for SoftPool.SoftPoolxd
: Is the class-based version which created an object that can be referenced later in the code.
ImageNet models
ImageNet weight can be downloaded from the following links:
Network | link |
---|---|
ResNet-18 | link |
ResNet-34 | link |
ResNet-50 | link |
ResNet-101 | link |
ResNet-152 | link |
DenseNet-121 | link |
DenseNet-161 | link |
DenseNet-169 | link |
ResNeXt-50_32x4d | link |
ResNeXt-101_32x4d | link |
wide-ResNet50 | link |
Citation
@inproceedings{stergiou2021refining,
title={Refining activation downsampling with SoftPool},
author={Stergiou, Alexandros, Poppe, Ronald and Kalliatakis Grigorios},
booktitle={International Conference on Computer Vision (ICCV)},
year={2021},
pages={10357-10366},
organization={IEEE}
}
Licence
MIT
Additional resources
A great project is Ren Tianhe's pytorh-pooling
repo for overviewing different pooling strategies.