Home

Awesome

<font size=6><center><big><b> Awesome AutoDL Awesome </b></big></center></font>

A curated list of automated deep learning related resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers, and awesome-architecture-search.

Please feel free to pull requests or open an issue to add papers.


<font size=5><center><b> Table of Contents </b> </center></font>


Awesome Blogs

Awesome AutoDL Libraies

Awesome Benchmarks

TitleVenueCode
NAS-Bench-101: Towards Reproducible Neural Architecture SearchICML 2019GitHub
NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture SearchICLR 2020Github
NAS-Bench-301 and the Case for Surrogate Benchmarks for Neural Architecture SearcharXiv 2020GitHub
NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture SearchICLR 2020GitHub
NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and SizeTPAMI 2021GitHub
NAS-Bench-ASR: Reproducible Neural Architecture Search for Speech RecognitionICLR 2021GitHub
HW-NAS-Bench: Hardware-Aware Neural Architecture Search BenchmarkICLR 2021GitHub
NAS-Bench-NLP: Neural Architecture Search Benchmark for Natural Language ProcessingarXiv 2020GitHub
NAS-Bench-x11 and the Power of Learning CurvesNeurIPS 2021GitHub

Deep Learning-based NAS and HPO

TypeGRLEAPDOther
Explanationgradient-basedreinforcement learningevolutionary algorithmperformance predictionother types

2021 Venues

TitleVenueTypeCode
CATE: Computation-aware Neural Architecture Encoding with TransformersICMLOGitHub
Searching by Generating: Flexible and Efficient One-Shot NAS with Architecture GeneratorCVPRGGithub
Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image RecognitionICCVEAGithub
AutoFormer: Searching Transformers for Visual RecognitionICCVEAGitHub
LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture SearchCVPREAGitHub
One-Shot Neural Ensemble Architecture Search by Diversity-Guided Search Space ShrinkingCVPREAGitHub
DARTS-: Robustly Stepping out of Performance Collapse Without IndicatorsICLRGGitHub
Zero-Cost Proxies for Lightweight NASICLROGitHub
Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired PerspectiveICLR-GitHub
DrNAS: Dirichlet Neural Architecture SearchICLRGGitHub
Rethinking Architecture Selection in Differentiable NASICLROGitHub
Evolving Reinforcement Learning AlgorithmsICLREAGitHub
AutoHAS: Differentiable Hyper-parameter and Architecture SearchICLR-WG-
FBNetV3: Joint Architecture-Recipe Search using Neural Acquisition FunctionCVPRPDgithub

2020 Venues

TitleVenueTypeCode
Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture SearchNeurIPS-GitHub
PyGlove: Symbolic Programming for Automated Machine LearningNeurIPSlibrary-
Does Unsupervised Architecture Representation Learning Help Neural Architecture SearchNeurIPSPDGitHub
RandAugment: Practical Automated Data Augmentation with a Reduced Search SpaceNeurIPSGitHub
Delta-STN: Efficient Bilevel Optimization for Neural Networks using Structured Response JacobiansNeurIPSGGitHub
A Study on Encodings for Neural Architecture SearchNeurIPSGitHub
AutoBSS: An Efficient Algorithm for Block Stacking Style SearchNeurIPS
Bridging the Gap between Sample-based and One-shot Neural Architecture Search with BONASNeurIPSGGitHub
Interstellar: Searching Recurrent Architecture for Knowledge Graph EmbeddingNeurIPS
Revisiting Parameter Sharing for Automatic Neural Channel Number SearchNeurIPS
Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree SearchNeurIPSMCTSGitHub
Neural Architecture Search using Deep Neural Networks and Monte Carlo Tree SearchAAAIMCTSGitHub
Representation Sharing for Fast Object Detector Search and BeyondECCVGGitHub
Are Labels Necessary for Neural Architecture Search?ECCVG-
Single Path One-Shot Neural Architecture Search with Uniform SamplingECCVEA-
Neural Predictor for Neural Architecture SearchECCVO-
BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage ModelsECCVG-
BATS: Binary ArchitecTure SearchECCV--
AttentionNAS: Spatiotemporal Attention Cell Search for Video ClassificationECCV--
Search What You Want: Barrier Panelty NAS for Mixed Precision QuantizationECCV--
Angle-based Search Space Shrinking for Neural Architecture SearchECCV--
Anti-Bandit Neural Architecture Search for Model DefenseECCV--
TF-NAS: Rethinking Three Search Freedoms of Latency-Constrained Differentiable Neural Architecture SearchECCVGGitHub
Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture SearchECCVGGitHub
Off-Policy Reinforcement Learning for Efficient and Effective GAN Architecture SearchECCVRL-
DA-NAS: Data Adapted Pruning for Efficient Neural Architecture SearchECCVG-
Optimizing Millions of Hyperparameters by Implicit DifferentiationAISTATSG-
Evolving Machine Learning Algorithms From ScratchICMLEA-
Stabilizing Differentiable Architecture Search via Perturbation-based RegularizationICMLGGitHub
NADS: Neural Architecture Distribution Search for Uncertainty AwarenessICML--
Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training DataICML--
Neural Architecture Search in a Proxy Validation Loss LandscapeICML--
Hit-Detector: Hierarchical Trinity Architecture Search for Object DetectionCVPR-GitHub
Designing Network Design SpacesCVPR-GitHub
UNAS: Differentiable Architecture Search Meets Reinforcement LearningCVPRG/RLGitHub
MiLeNAS: Efficient Neural Architecture Search via Mixed-Level ReformulationCVPRGGitHub
A Semi-Supervised Assessor of Neural ArchitecturesCVPRPD-
Binarizing MobileNet via Evolution-based SearchingCVPREA-
Rethinking Performance Estimation in Neural Architecture SearchCVPR-GitHub
APQ: Joint Search for Network Architecture, Pruning and Quantization PolicyCVPRGGitHub
SGAS: Sequential Greedy Architecture SearchCVPRGGithub
Can Weight Sharing Outperform Random Architecture Search? An Investigation With TuNASCVPRRL-
FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel DimensionsCVPRGGithub
AdversarialNAS: Adversarial Neural Architecture Search for GANsCVPRGGithub
When NAS Meets Robustness: In Search of Robust Architectures against Adversarial AttacksCVPRGGithub
Block-wisely Supervised Neural Architecture Search with Knowledge DistillationCVPRGGithub
Overcoming Multi-Model Forgetting in One-Shot NAS with Diversity MaximizationCVPRGGithub
Densely Connected Search Space for More Flexible Neural Architecture SearchCVPRGGithub
EfficientDet: Scalable and Efficient Object DetectionCVPRRL-
NAS-BENCH-201: Extending the Scope of Reproducible Neural Architecture SearchICLR-Github
Understanding Architectures Learnt by Cell-based Neural Architecture SearchICLRGGitHub
Evaluating The Search Phase of Neural Architecture SearchICLR-
AtomNAS: Fine-Grained End-to-End Neural Architecture SearchICLRGitHub
Fast Neural Network Adaptation via Parameter Remapping and Architecture SearchICLR-GitHub
Once for All: Train One Network and Specialize it for Efficient DeploymentICLRGGitHub
Efficient Transformer for Mobile ApplicationsICLR--
PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture SearchICLRGGitHub
Adversarial AutoAugmentICLR--
NAS evaluation is frustratingly hardICLR-GitHub
FasterSeg: Searching for Faster Real-time Semantic SegmentationICLRGGitHub
Computation Reallocation for Object DetectionICLR--
Towards Fast Adaptation of Neural Architectures with Meta LearningICLR-GitHub
AssembleNet: Searching for Multi-Stream Neural Connectivity in Video ArchitecturesICLREA-
How to Own the NAS in Your Spare TimeICLR--
Fast, Accurate and Lightweight Super-Resolution with Neural Architecture SearchICPRGgithub

2019 Venues

TitleVenueTypeCode
Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response FunctionsICLR--
DATA: Differentiable ArchiTecture ApproximationNeurIPS--
Random Search and Reproducibility for Neural Architecture SearchUAIGGitHub
Improved Differentiable Architecture Search for Language Modeling and Named Entity RecognitionEMNLPG-
Continual and Multi-Task Architecture SearchACLRL-
Progressive Differentiable Architecture Search: Bridging the Depth Gap Between Search and EvaluationICCVGGitHub
Multinomial Distribution Learning for Effective Neural Architecture SearchICCV-GitHub
Searching for MobileNetV3ICCVEA-
Multinomial Distribution Learning for Effective Neural Architecture SearchICCV-GitHub
Fast and Practical Neural Architecture SearchICCV
Teacher Guided Architecture SearchICCV-
AutoDispNet: Improving Disparity Estimation With AutoMLICCVG-
Resource Constrained Neural Network Architecture Search: Will a Submodularity Assumption Help?ICCVEA-
One-Shot Neural Architecture Search via Self-Evaluated Template NetworkICCVGGithub
Evolving Space-Time Neural Architectures for VideosICCVEAGitHub
AutoGAN: Neural Architecture Search for Generative Adversarial NetworksICCVRLgithub
Discovering Neural WiringsNeurIPSGGithub
Towards modular and programmable architecture searchNeurIPSOtherGithub
Network Pruning via Transformable Architecture SearchNeurIPSGGithub
Deep Active Learning with a NeuralArchitecture SearchNeurIPS--
DetNAS: Backbone Search for Object DetectionNeurIPSEAGitHub
SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained MicrocontrollersNeurIPS--
Efficient Forward Architecture SearchNeurIPSGGithub
Efficient Neural ArchitectureTransformation Search in Channel-Level for Object DetectionNeurIPSG-
XNAS: Neural Architecture Search with Expert AdviceNeurIPSGGitHub
DARTS: Differentiable Architecture SearchICLRGgithub
ProxylessNAS: Direct Neural Architecture Search on Target Task and HardwareICLRRL/Ggithub
Graph HyperNetworks for Neural Architecture SearchICLRG-
Learnable Embedding Space for Efficient Neural Architecture CompressionICLROthergithub
Efficient Multi-Objective Neural Architecture Search via Lamarckian EvolutionICLREA-
SNAS: stochastic neural architecture searchICLRG-
NetTailor: Tuning the Architecture, Not Just the WeightsCVPRGGithub
Searching for A Robust Neural Architecture in Four GPU HoursCVPRGGithub
ChamNet: Towards Efficient Network Design through Platform-Aware Model AdaptationCVPR--
Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture SearchCVPREAgithub
FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture SearchCVPRG-
RENAS: Reinforced Evolutionary Neural Architecture SearchCVPRG-
Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image SegmentationCVPRGGitHub
MnasNet: Platform-Aware Neural Architecture Search for MobileCVPRRLGithub
MFAS: Multimodal Fusion Architecture SearchCVPREA-
A Neurobiological Evaluation Metric for Neural Network Model SearchCVPROther-
Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary CellsCVPRRL-
Customizable Architecture Search for Semantic SegmentationCVPR--
Regularized Evolution for Image Classifier Architecture SearchAAAIEA-
AutoAugment: Learning Augmentation Policies from DataCVPRRL-
Population Based Augmentation: Efficient Learning of Augmentation Policy SchedulesICMLEA-
The Evolved TransformerICMLEAGithub
EfficientNet: Rethinking Model Scaling for Convolutional Neural NetworksICMLRL-
NAS-Bench-101: Towards Reproducible Neural Architecture SearchICMLOtherGithub
On Network Design Spaces for Visual RecognitionICCVGGithub

2018 Venues

TitleVenueTypeCode
Towards Automatically-Tuned Deep Neural NetworksBOOK-GitHub
NetAdapt: Platform-Aware Neural Network Adaptation for Mobile ApplicationsECCV-github
Efficient Architecture Search by Network TransformationAAAIRLgithub
Learning Transferable Architectures for Scalable Image RecognitionCVPRRLgithub
N2N learning: Network to Network Compression via Policy Gradient Reinforcement LearningICLRRL-
A Flexible Approach to Automated RNN Architecture GenerationICLRRL/PD-
Practical Block-wise Neural Network Architecture GenerationCVPRRL-
Path-Level Network Transformation for Efficient Architecture SearchICMLRLgithub
Hierarchical Representations for Efficient Architecture SearchICLREA-
Understanding and Simplifying One-Shot Architecture SearchICMLG-
SMASH: One-Shot Model Architecture Search through HyperNetworksICLRGgithub
Neural Architecture OptimizationNeurIPSGgithub
Searching for efficient multi-scale architectures for dense image predictionNeurIPSOther-
Progressive Neural Architecture SearchECCVPDgithub
Neural Architecture Search with Bayesian Optimisation and Optimal TransportNeurIPSOthergithub
Differentiable Neural Network Architecture SearchICLR-WG-
Accelerating Neural Architecture Search using Performance PredictionICLR-WPD-

2017 Venues

TitleVenueTypeCode
Neural Architecture Search with Reinforcement LearningICLRRL-
Designing Neural Network Architectures using Reinforcement LearningICLRRL-
Neural Optimizer Search with Reinforcement LearningICMLRL-
Learning Curve Prediction with Bayesian Neural NetworksICLRPD-
Hyperband: A Novel Bandit-Based Approach to Hyperparameter OptimizationICLRPD-
Hyperparameter Optimization: A Spectral ApproachNeurIPS-WOthergithub
Learning to Compose Domain-Specific Transformations for Data AugmentationNeurIPS--

Previous Venues

2012-2016

TitleVenueTypeCode
Speeding up Automatic Hyperparameter Optimization of Deep Neural Networksby Extrapolation of Learning CurvesIJCAIPDgithub

arXiv

TitleDateTypeCode
NSGA-NET: A Multi-Objective Genetic Algorithm for Neural Architecture Search2018.10EA-
Training Frankenstein’s Creature to Stack: HyperTree Architecture Search2018.10G-
Population Based Training of Neural Networks2017.11EAGitHub
EmotionNAS: Two-stream Architecture Search for Speech Emotion Recognition2022.3G-
U-Boost NAS: Utilization-Boosted Differentiable Neural Architecture Search2022.3GGithub

Awesome Surveys

TitleVenueYearCode
A Comprehensive Survey of Neural Architecture Search: Challenges and SolutionsACM Computing Surveys2021-
Automated Machine Learning on Graphs: A SurveyICLR-W2021GitHub
On Hyperparameter Optimization of Machine Learning Algorithms: Theory and PracticeNeurocomputing2020github
AutonoML: Towards an Integrated Framework for Autonomous Machine LearningarXiv2020-
Automated Machine LearningSpringer Book2019-
Neural architecture search: A surveyJMLR2019-
AutoML: A Survey of the State-of-the-ArtarXiv2019GitHub
A Survey on Neural Architecture SearcharXiv2019-
Taking human out of learning applications: A survey on automated machine learningarXiv2018-
IoT Data Analytics in Dynamic Environments: From An Automated Machine Learning PerspectiveEngineering Applications of Artificial Intelligence2022github