Awesome

Do Better ImageNet Models Transfer Better?

Hi! We provide simple Tensorflow 2.0 implementations for paper "Do Better Imagnet models Transfer Better?". There are 3 transfer-learning tasks in this implementations.

• As fixed extractors

  L2-regularized Logistic Regression + L-BFGS without data augmentation

• Fine-tuned from ImageNet initialization
• Trained from random initialization

How to use

Enviroment

pip install tensorflow-datasets
conda install tensorflow-gpu

You can install tensorflow-gpu & datasets!

Parameter Setting

You need to change the parameters according to the data set in main.py or Implementation.ipynb. (NUM_TRAIN_SAMPLES, IMG_SHAPE, NUM_CLASSES). You can use 2 models & 6 datasets in this implementation or more.

NUM_GPUS = 4
BS_PER_GPU = 64  # Batchsize = 4x64
NUM_EPOCHS = 200
TASK=2  # Task 1 : Logistic Regression, Task2 : Transfer learning, Task3 : Random Initialization(Scratch Training)
MODEL = "mobilenet_v2"  # mobilenet_v1, mobilenet_v2
DATASET = "cifar10"  # food101, cifar10, cifar100, sun397, oxford_flowers102, caltech101
learning_rate = 0.01

Model Summary

Result

Paper

The higher the accuracy in ImageNet, the higher the performance in transfer learning.

Task 1. Logistic Regression.

Task 2. Fine-Tuned

Task 3. Trained from Random Initialization

My Result.

1. Tensorflow 2.0

Setting

• Preprocessing - (Scale to [-1, 1])
• Dropout
• Data Augmentation
• Optimizer - SGD (Momentum 0.9, Nestrov True, Weight Decay 1e-6)
• Batchsize - 256
• Image Size - 224 x 224 (except cifar10, 100 - 32 x 32)
• Epoch : 200 (~10000 iterations)

Mobilenet v1 vs v2 - acc

Statistical Methods - log odds

2. Pytorch

Setting

• Preprocessing - (Scale to [0, 1])
• Dropout
• Data Augmentation
• Optimizer - SGD (Momentum 0.9, Nestrov True, Weight Decay 1e-6)
• Batchsize - 64
• Image Size - 224 x 224
• Epoch : 100 (~10000 iterations)

Mobilenet v1 vs v2 - acc