Home

Awesome

⛺️ Tent: Fully Test-Time Adaptation by Entropy Minimization

This is the official project repository for Tent: Fully-Test Time Adaptation by Entropy Minimization by Dequan Wang*, Evan Shelhamer*, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell (ICLR 2021, spotlight).

⛺️ Tent equips a model to adapt itself to new and different data during testing ☀️ 🌧 ❄️. Tented models adapt online and batch-by-batch to reduce error on dataset shifts like corruptions, simulation-to-real discrepancies, and other differences between training and testing data. This kind of adaptation is effective and efficient: tent makes just one update per batch to not interrupt inference.

We provide example code in PyTorch to illustrate the tent method and fully test-time adaptation setting.

Please check back soon for reference code to exactly reproduce the ImageNet-C results in the paper.

Installation:

pip install -r requirements.txt

tent depends on

and the example depends on

but feel free to try your own data and model too!

Usage:

import tent

model = TODO_model()

model = tent.configure_model(model)
params, param_names = tent.collect_params(model)
optimizer = TODO_optimizer(params, lr=1e-3)
tented_model = tent.Tent(model, optimizer)

outputs = tented_model(inputs)  # now it infers and adapts!

Example: Adapting to Image Corruptions on CIFAR-10-C

The example adapts a CIFAR-10 classifier to image corruptions on CIFAR-10-C. The purpose of the example is explanation, not reproduction: exact details of the model architecture, optimization settings, etc. may differ from the paper. That said, the results should be representative, so do give it a try and experiment!

This example compares a baseline without adaptation (source), test-time normalization for updating feature statistics during testing (norm), and our method for entropy minimization during testing (tent). The dataset is CIFAR-10-C, with 15 types and 5 levels of corruption.

WRN-28-10

the default model for RobustBench.

Usage:

python cifar10c.py --cfg cfgs/source.yaml
python cifar10c.py --cfg cfgs/norm.yaml
python cifar10c.py --cfg cfgs/tent.yaml

Result: tent reduces the error (%) across corruption types at the most severe level of corruption (level 5).

meangauss_noiseshot_noiseimpulse_noisedefocus_blurglass_blurmotion_blurzoom_blursnowfrostfogbrightnesscontrastelastic_transpixelatejpeg
source code config43.572.365.772.946.954.334.842.025.141.326.09.346.726.658.530.3
norm code config20.428.126.136.312.835.314.212.117.317.415.38.412.623.819.727.3
tent code config18.624.823.533.012.031.813.710.815.916.213.77.912.122.017.324.2

See the full results for this example in the wandb report.

WRN-40-2

WideResNet for AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty.

Usage:

python cifar10c.py --cfg cfgs/source.yaml MODEL.ARCH Hendrycks2020AugMix_WRN
python cifar10c.py --cfg cfgs/norm.yaml MODEL.ARCH Hendrycks2020AugMix_WRN
python cifar10c.py --cfg cfgs/tent.yaml MODEL.ARCH Hendrycks2020AugMix_WRN

Result: tent reduces the error (%) across corruption types at the most severe level of corruption (level 5).

meangauss_noiseshot_noiseimpulse_noisedefocus_blurglass_blurmotion_blurzoom_blursnowfrostfogbrightnesscontrastelastic_transpixelatejpeg
source18.328.823.026.29.520.610.69.314.215.317.57.620.914.741.314.7
norm14.518.516.222.39.021.910.59.712.813.315.07.611.916.315.017.5
tent12.115.713.218.87.918.19.08.010.410.812.46.710.014.011.414.8

Example: Adapting to Adversarial Perturbations on CIFAR-10

See Fighting Gradients with Gradients: Dynamic Defenses against Adversarial Attacks for more details on dent.

Correspondence

Please contact Dequan Wang and Evan Shelhamer at dqwang AT cs.berkeley.edu and shelhamer AT google.com.

Citation

If the tent method or fully test-time adaptation setting are helpful in your research, please consider citing our paper:

@inproceedings{wang2021tent,
  title={Tent: Fully Test-Time Adaptation by Entropy Minimization},
  author={Wang, Dequan and Shelhamer, Evan and Liu, Shaoteng and Olshausen, Bruno and Darrell, Trevor},
  booktitle={International Conference on Learning Representations},
  year={2021},
  url={https://openreview.net/forum?id=uXl3bZLkr3c}
}