Awesome

Revisiting Transferable Adversarial Images

Revisiting Transferable Adversarial Images: Systemization, Evaluation, and New Insights. Zhengyu Zhao*, Hanwei Zhang*, Renjue Li*, Ronan Sicre, Laurent Amsaleg, Michael Backes, Qi Li, Qian Wang, Chao Shen.

We identify two main problems in common evaluation practices: <br>(1) for attack transferability, lack of systematic, one-to-one attack comparisons and fair hyperparameter settings; <br>(2) for attack stealthiness, simply no evaluations.

We address these problems by <br>(1) introducing a complete attack categorization and conducting systematic and fair intra-category analyses on transferability; <br>(2) considering diverse imperceptibility metrics and finer-grained stealthiness characteristics from the perspective of attack traceback.

We draw new insights, e.g., <br>(1) under a fair attack hyperparameter setting, one early attack method, DI, actually outperforms all the follow-up methods; <br>(2) popular diffusion-based defenses give a false sense of security since it is indeed largely bypassed by (black-box) transferable attacks; <br>(3) even when all attacks are bounded by the same Lp norm, they lead to dramatically different stealthiness performance, which negatively correlates with their transferability performance.

We provide the first large-scale evaluation of transferable adversarial examples on ImageNet, involving 23 representative attacks against 9 representative defenses.

We reveal that existing problematic evaluations have indeed caused misleading conclusions and missing points, and as a result, hindered the assessment of the actual progress in this field.

Evaluated Attacks and Defenses

Attack Categorization (Welcome more papers!)

Gradient Stabilization Attacks [Code for 3 representative attacks]

• Boosting Adversarial Attacks with Momentum (CVPR 2018)
• Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks (ICLR 2020)
• Boosting Adversarial Transferability through Enhanced Momentum (BMVC 2021)
• Improving Adversarial Transferability with Spatial Momentum (arXiv 2022)
• Making Adversarial Examples More Transferable and Indistinguishable (AAAI2022)
• Boosting Adversarial Transferability by Achieving Flat Local Maxima (NeurIPS 2023)
• Transferable Adversarial Attack for Both Vision Transformers and Convolutional Networks via Momentum Integrated Gradients (ICCV 2023)
• Boosting Adversarial Transferability via Gradient Relevance Attack (ICCV 2023)
• Enhancing Transferable Adversarial Attacks on Vision Transformers through Gradient Normalization Scaling and High-Frequency Adaptation (ICLR 2024)
• Enhancing Adversarial Transferability Through Neighborhood Conditional Sampling (arxiv 2024)

Input Augmentation Attacks [Code for 5 representative attacks]

• Improving Transferability of Adversarial Examples with Input Diversity (CVPR 2019)
• Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks (CVPR 2019)
• Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks (ICLR 2020)
• Patch-wise Attack for Fooling Deep Neural Network (ECCV 2020)
• Improving the Transferability of Adversarial Examples with Resized-Diverse-Inputs, Diversity-Ensemble and Region Fitting (ECCV 2020)
• Regional Homogeneity: Towards Learning Transferable Universal Adversarial Perturbations Against Defenses (ECCV 2020)
• Enhancing the Transferability of Adversarial Attacks through Variance Tuning (CVPR 2021)
• Admix: Enhancing the Transferability of Adversarial Attacks (ICCV 2021)
• Improving the Transferability of Targeted Adversarial Examples through Object-Based Diverse Input (CVPR 2022)
• Frequency Domain Model Augmentation for Adversarial Attack (ECCV 2022)
• Adaptive Image Transformations for Transfer-based Adversarial Attack (ECCV 2022)
• Boosting the Transferability of Adversarial Attacks with Reverse Adversarial Perturbation (NeurIPS 2022)
• Enhancing the Self-Universality for Transferable Targeted Attacks (CVPR 2023)
• Improving the Transferability of Adversarial Samples by Path-Augmented Method (CVPR 2023)
• The Ultimate Combo: Boosting Adversarial Example Transferability by Composing Data Augmentations (arXiv 2023)
• Structure Invariant Transformation for better Adversarial Transferability (ICCV 2023)
• Boosting Adversarial Transferability across Model Genus by Deformation-Constrained Warping (AAAI 2024)
• Boosting Adversarial Transferability by Block Shuffle and Rotation (CVPR 2024)
• Learning to Transform Dynamically for Better Adversarial Transferability (CVPR 2024)
• Boosting the Transferability of Adversarial Examples via Local Mixup and Adaptive Step Size (arXiv 2024)
• Typography Leads Semantic Diversifying: Amplifying Adversarial Transferability across Multimodal Large Language Models (arXiv 2024)
• Strong Transferable Adversarial Attacks via Ensembled Asymptotically Normal Distribution Learning (CVPR 2024)
• Learning to Transform Dynamically for Better Adversarial Transferability (CVPR2024)

Feature Disruption Attacks [Code for 5 representative attacks]

• Transferable Adversarial Perturbations (ECCV 2018)
• Task-generalizable Adversarial Attack based on Perceptual Metric (arXiv 2018)
• Feature Space Perturbations Yield More Transferable Adversarial Examples (CVPR 2019)
• FDA: Feature Disruptive Attack (ICCV 2019)
• Enhancing Adversarial Example Transferability with an Intermediate Level Attack (ICCV 2019)
• Transferable Perturbations of Deep Feature Distributions (ICLR 2020)
• Boosting the Transferability of Adversarial Samples via Attention (CVPR 2020)
• Towards Transferable Targeted Attack (CVPR 2020)
• Yet Another Intermediate-Level Attack (ECCV 2020)
• Perturbing Across the Feature Hierarchy to Improve Standard and Strict Blackbox Attack Transferability (NeurIPS 2020)
• Feature Importance-aware Transferable Adversarial Attacks (ICCV 2021)
• Improving Adversarial Transferability via Neuron Attribution-Based Attacks (CVPR 2022)
• An Intermediate-level Attack Framework on The Basis of Linear Regression (TPAMI 2022)
• Introducing Competition to Boost the Transferability of Targeted Adversarial Examples through Clean Feature Mixup (CVPR 2023)
• Diversifying the High-level Features for better Adversarial Transferability (BMVC 2023)
• Improving Adversarial Transferability via Intermediate-level Perturbation Decay (NeurIPS 2023)
• Boosting Adversarial Transferability via Fusing Logits of Top-1 Decomposed Feature (arXiv 2023)

Surrogate Refinement Attacks [Code for 5 representative attacks]

• Learning Transferable Adversarial Examples via Ghost Networks (AAAI 2020)
• Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets (ICLR 2020)
• Backpropagating Linearly Improves Transferability of Adversarial Examples (NeurIPS 2020)
• Backpropagating Smoothly Improves Transferability of Adversarial Examples (CVPRw 2021)
• A Little Robustness Goes a Long Way: Leveraging Robust Features for Targeted Transfer Attacks (NeurIPS 2021)
• Early Stop and Adversarial Training Yield Better Surrogate Model: Very Non-Robust Features Harm Adversarial Transferability (OpenReview 2021)
• Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferability (CVPR 2022)
• Training Meta-Surrogate Model for Transferable Adversarial Attack (arXiv 2021)
• On Improving Adversarial Transferability of Vision Transformers (ICLR2022)
• Rethinking Adversarial Transferability from a Data Distribution Perspective (ICLR 2022)
• Boosting the Adversarial Transferability of Surrogate Model with Dark Knowledge (arXiv 2022)
• Towards Transferable Adversarial Attacks on Vision Transformers (AAAI 2022)
• Transferable Adversarial Attacks on Vision Transformers with Token Gradient Regularization (CVPR 2023)
• Minimizing Maximum Model Discrepancy for Transferable Black-box Targeted Attacks (CVPR 2023)
• Towards Transferable Targeted Adversarial Examples (CVPR 2023)
• StyLess: Boosting the Transferability of Adversarial Examples (CVPR 2023)
• How to choose your best allies for a transferable attack? (ICCV 2023)
• An Adaptive Model Ensemble Adversarial Attack for Boosting Adversarial Transferability (ICCV 2023)
• Backpropagation Path Search On Adversarial Transferability (ICCV 2023)
• Going Further: Flatness at the Rescue of Early Stopping for Adversarial Example Transferability (arXiv 2023)
• Making Substitute Models More Bayesian Can Enhance Transferability of Adversarial Examples (ICLR 2023)
• Blurred-Dilated Method for Adversarial Attacks (NeurIPS 2023)
• Rethinking the Backward Propagation for Adversarial Transferability (NeurIPS 2023)
• Rethinking Model Ensemble in Transfer-based Adversarial Attacks (ICLR 2024)
• Why Does Little Robustness Help? Understanding and Improving Adversarial Transferability from Surrogate Training (S&P 2024)
• Enhance Stealthiness and Transferability of Adversarial Attacks with Class Activation Mapping Ensemble Attack (NDSS 2024)
• Improving Transferable Targeted Adversarial Attacks with Model Self-Enhancement (CVPR 2024)
• AGS: Affordable and Generalizable Substitute Training for Transferable Adversarial Attack (AAAI 2024)
• Ensemble Diversity Facilitates Adversarial Transferability (CVPR 2024)

Generative Modeling Attacks

• Generative Adversarial Perturbations (CVPR 2018)
• Cross-Domain Transferability of Adversarial Perturbations (NeurIPS 2019)
• On Generating Transferable Targeted Perturbations (ICCV 2021)
• Learning Transferable Adversarial Perturbations (NeurIPS 2021)
• Beyond ImageNet Attack: Towards Crafting Adversarial Examples for Black-box Domains (ICLR 2022)
• Boosting Transferability of Targeted Adversarial Examples via Hierarchical Generative Networks (ECCV 2022)
• Dynamic Generative Targeted Attacks With Pattern Injection (CVPR 2023)
• Towards Transferable Targeted Adversarial Examples (CVPR 2023)
• Perturbation Towards Easy Samples Improves Targeted Adversarial Transferability (NeurIPS 2023)

Surveys/Evaluations/Explanations

• Delving into Transferable Adversarial Examples and Black-box Attacks (ICLR 2017)
• The Space of Transferable Adversarial Examples (arXiv 2017)
• Why Do Adversarial Attacks Transfer? Explaining Transferability of Evasion and Poisoning Attacks (USENIX Security 2019)
• Selection of Source Images Heavily Influences the Effectiveness of Adversarial Attacks (BMVC 2021)
• On Success and Simplicity: A Second Look at Transferable Targeted Attacks (NeurIPS 2021)
• Evaluating Adversarial Attacks on ImageNet: A Reality Check on Misclassification Classes (NeurIPSw 2021)
• A Unified Approach to Interpreting and Boosting Adversarial Transferability (ICLR 2021)
• Transfer Attacks Revisited: A Large-Scale Empirical Study in Real Computer Vision Settings (IEEE S&P 2022)
• Towards Evaluating Transfer-based Attacks Systematically, Practically, and Fairly (NeurIPS 2023)
• Beyond Boundaries: A Comprehensive Survey of Transferable Attacks on AI Systems (arXiv 2023)
• A Survey on Transferability of Adversarial Examples across Deep Neural Networks (arXiv 2023)
• Bag of Tricks to Boost Adversarial Transferability (arXiv 2024)