Home

Awesome

kEffNet

This repository contains the source code for the paper Grouped Pointwise Convolutions Significantly Reduces Parameters in EfficientNet (PDF) by Joao Paulo Schwarz Schuler, Santiago Romani, Mohamed Abdel-Nasser, Hatem Rashwan and Domenec Puig.

Abstract

EfficientNet is a recent Deep Convolutional Neural Network (DCNN) architecture intended to be proportionally extendible in depth, width and resolution. Through its variants, it can achieve state of the art accuracy on the ImageNet classification task as well as on other classical challenges. Although its name refers to its efficiency with respect to the ratio between outcome (accuracy) and needed resources (number of parameters, flops), we are studying a method to reduce the original number of trainable parameters by more than 84% while keeping a very similar degree of accuracy. Our proposal is to improve the pointwise (1x1) convolutions, whose number of parameters rapidly grows due to the multiplication of the number of filters by the number of input channels that come from the previous layer. Basically, our tweak consists in grouping filters into parallel branches, where each branch processes a fraction of the input channels. However, by doing so, the learning capability of the DCNN is degraded. To avoid this effect, we suggest interleaving the output of filters from different branches at intermediate layers of consecutive pointwise convolutions. Our experiments with the CIFAR-10 dataset show that our optimized EfficientNet has similar learning capacity to the original layout when training from scratch.

Test on Colab

You can test kEffNet on Google Colab:

Prerequisite

All examples in this project require importing the K-CAI Neural API.

Give this Project a Star

This project is an open source project. If you like what you see, please give it a star on github.

Further Parameter and Floating-point Computation Savings

The following papers also deal about parameters and floating-point computation savings:

Citing this Paper

Bibtex:

@inbook{Schuler2021k01,
author = {Schuler, Joao and RomanĂ­, Santiago and Abdel-nasser, Mohamed and Rashwan, Hatem and Puig, Domenec},
year = {2021},
month = {10},
pages = {383-391},
title = {Grouped Pointwise Convolutions Significantly Reduces Parameters in EfficientNet},
booktitle = {Artificial Intelligence Research and Development},
publisher = {IOS Press},
isbn = {9781643682105},
doi = {10.3233/FAIA210158}
}