Home

Awesome

ISDA-Pytorch

The Implicit Semantic Data Augmentation (ISDA) algorithm implemented in Pytorch.

Update on 2021/04/23: Release Code for Visualizing Deep Features on ImageNet!

Update on 2021/01/17: Journal Version of ISDA is Accepted by T-PAMI!

Update on 2020/04/25: Release Pre-trained Models on ImageNet.

Update on 2020/04/24: Release Code for Image Classification on ImageNet and Semantic Segmentation on Cityscapes.

Introduction

We propose a novel implicit semantic data augmentation (ISDA) approach to complement traditional augmentation techniques like flipping, translation or rotation. ISDA consistently improves the generalization performance of popular deep networks on supervised & semi-supervised image classification, semantic segmentation, object detection and instance segmentation.

<p align="center"> <img src="ISDA-overview.png" width= "900"> </p>

Citation

If you find this work valuable or use our code in your own research, please consider citing us with the following bibtex:

@inproceedings{NIPS2019_9426,
        title = {Implicit Semantic Data Augmentation for Deep Networks},
       author = {Wang, Yulin and Pan, Xuran and Song, Shiji and Zhang, Hong and Huang, Gao and Wu, Cheng},
    booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
        pages = {12635--12644},
         year = {2019},
}

@article{wang2021regularizing,
        title = {Regularizing deep networks with semantic data augmentation},
       author = {Wang, Yulin and Huang, Gao and Song, Shiji and Pan, Xuran and Xia, Yitong and Wu, Cheng},
      journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
         year = {2021},
          doi = {10.1109/TPAMI.2021.3052951}
}

Get Started

Please go to the folder Image classification on CIFAR, Image classification on ImageNet, Semantic segmentation on Cityscapes and Visualizing deep features for specific docs.

Pre-trained Models on ImageNet

ModelParamsBaselineISDAModel
ResNet-5025.6M23.021.9Tsinghua Cloud / Google Drive
ResNet-10144.6M21.720.8Tsinghua Cloud / Google Drive
ResNet-15260.3M21.320.3Tsinghua Cloud / Google Drive
DenseNet-BC-1218.0M23.723.2Tsinghua Cloud / Google Drive
DenseNet-BC-26533.3M21.921.2Tsinghua Cloud / Google Drive
ResNeXt50, 32x4d25.0M22.521.3Tsinghua Cloud / Google Drive
ResNeXt101, 32x8d88.8M21.120.1Tsinghua Cloud / Google Drive

Visualization of Augmented Samples

<p align="center"> <img src="Visualization.png" width= "900"> </p>

Results

<p align="center"> <img src="ImageNet.png" width= "900"> </p> <p align="center"> <img src="Complementary results.png" width= "900"> </p> <p align="center"> <img src="Semi supervised learning.png" width= "900"> </p> <p align="center"> <img src="Semantic Segmentation.png" width= "900"> </p> <p align="center"> <img src="Object detection.png" width= "900"> </p> <p align="center"> <img src="Instance Segmentation.png" width= "900"> </p>

Acknowledgment

Our code for semantic segmentation is mainly based on pytorch-segmentation-toolbox.

To Do

Update code for semi-supervised learning.