Awesome

Awesome Pruning

Visitors

Awesome resources in deep neural network pruning. This collection is inspired by he-y/Awesome-Pruning.

[Note: You are welcome to create pool requests and add more interesting papers.]

Section	Year of Publication
Conference Publications	`2024` `2023` `2022` `2021` `2020` `2019` `2018` `2017`
Journal Publications	`2024` `2023` `2022` `2021` `2020`
Survey Articles	`2020~2023`
Other Publications	`2022~2023`
Pruning Software and Toolbox	`2019~2023`

Symbol	Meaning
`U`	Unstructured or Weight Pruning
`S`	Structured or Filter or Channel Pruning
`A`	Official or Author Implementation
`O`	Unofficial or 3rd Party Implementation

Conference Publications

<h3 align="center">2024</h3>

Venue	Title	Type	Code
`ICLR`	Towards Meta-Pruning via Optimal Transport	`S`	PyTorch[A]
`ICLR`	Towards Energy Efficient Spiking Neural Networks: An Unstructured Pruning Framework	`U`	PyTorch[A]
`ICLR`	Masks, Signs, And Learning Rate Rewinding	`S`	PyTorch[A]
`ICLR`	Scaling Laws for Sparsely-Connected Foundation Models	`S`	PyTorch[A]
`ICLR`	Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging	`S`
`ICLR`	Adaptive Sharpness-Aware Pruning for Robust Sparse Networks	`S`
`ICLR`	What Makes a Good Prune? Maximal Unstructured Pruning for Maximal Cosine Similarity	`U`	PyTorch[A]
`ICLR`	In defense of parameter sharing for model-compression	`S/U`
`ICLR`	ECoFLaP: Efficient Coarse-to-Fine Layer-Wise Pruning for Vision-Language Models	`U`
`ICLR`	Data-independent Module-aware Pruning for Hierarchical Vision Transformers	`S`	PyTorch[A]
`ICLR`	SWAP: Sparse Entropic Wasserstein Regression for Robust Network Pruning	`S`
`ICLR`	Sparse Weight Averaging with Multiple Particles for Iterative Magnitude Pruning	`U`
`ICLR`	Synergistic Patch Pruning for Vision Transformer: Unifying Intra- & Inter-Layer Patch Importance	`S`
`ICLR`	FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity	`S`
`ICLR`	The Need for Speed: Pruning Transformers with One Recipe	`S`	PyTorch[A]
`ICLR`	SAS: Structured Activation Sparsification	`S`	PyTorch[A]
`CVPR`	OrthCaps: An Orthogonal CapsNet with Sparse Attention Routing and Pruning	`S`	PyTorch[A]
`CVPR`	Zero-TPrune: Zero-Shot Token Pruning through Leveraging of the Attention Graph in Pre-Trained Transformers	`S`	PyTorch[A]
`CVPR`	Finding Lottery Tickets in Vision Models via Data-driven Spectral Foresight Pruning	`S`	PyTorch[A]
`CVPR`	BilevelPruning: Unified Dynamic and Static Channel Pruning for Convolutional Neural Networks	`S`
`CVPR`	FedMef: Towards Memory-efficient Federated Dynamic Pruning	`S`
`CVPR`	Resource-Efficient Transformer Pruning for Finetuning of Large Models	`S`
`CVPR`	Device-Wise Federated Network Pruning	`S`
`CVPR`	Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratch	`S`
`CVPR`	Jointly Training and Pruning CNNs via Learnable Agent Guidance and Alignment	`S`
`CVPR`	Diversity-aware Channel Pruning for StyleGAN Compression	`S`	PyTorch[A]
`CVPR`	MADTP: Multimodal Alignment-Guided Dynamic Token Pruning for Accelerating Vision-Language Transformer	`S`	PyTorch[A]
`AAAI`	Dynamic Feature Pruning and Consolidation for Occluded Person Re-Identification	`S`
`AAAI`	REPrune: Channel Pruning via Kernel Representative Selection	`S`
`AAAI`	Revisiting Gradient Pruning: A Dual Realization for Defending against Gradient Attacks	`S`
`AAAI`	IRPruneDet: Efficient Infrared Small Target Detection via Wavelet Structure-Regularized Soft Channel Pruning	`S`
`AAAI`	EPSD: Early Pruning with Self-Distillation for Efficient Model Compression	`S`
`WACV`	Pruning from Scratch via Shared Pruning Module and Nuclear norm-based Regularization	`S`	PyTorch[A]
`WACV`	Towards Better Structured Pruning Saliency by Reorganizing Convolution	`S`	PyTorch[A]
`WACV`	Torque based Structured Pruning for Deep Neural Network	`S`
`WACV`	Revisiting Token Pruning for Object Detection and Instance Segmentation	`S`	PyTorch[A]
`WACV`	Token Fusion: Bridging the Gap Between Token Pruning and Token Merging	`S`
`WACV`	PATROL: Privacy-Oriented Pruning for Collaborative Inference Against Model Inversion Attacks	`S`

<h3 align="center">2023</h3>

Venue	Title	Type	Code
`NIPS`	Diff-Pruning: Structural Pruning for Diffusion Models	`S`	PyTorch[A]
`NIPS`	LLM-Pruner: On the Structural Pruning of Large Language Models	`S`	PyTorch[A]
`ICCV`	Automatic Network Pruning via Hilbert-Schmidt Independence Criterion Lasso under Information Bottleneck Principle	`S`	PyTorch[A]
`ICCV`	Unified Data-Free Compression: Pruning and Quantization without Fine-Tuning	`S`	PyTorch[A]
`ICCV`	Structural Alignment for Network Pruning through Partial Regularization	`S`
`ICCV`	Differentiable Transportation Pruning	`S`
`ICCV`	Dynamic Token Pruning in Plain Vision Transformers for Semantic Segmentation	`S`	PyTorch[A]
`ICCV`	Towards Fairness-aware Adversarial Network Pruning	`S`
`ICCV`	Efficient Joint Optimization of Layer-Adaptive Weight Pruning in Deep Neural Networks	`S`	PyTorch[A]
`CVPR`	DepGraph: Towards Any Structural Pruning	`S`	PyTorch[A]
`CVPR`	X-Pruner: eXplainable Pruning for Vision Transformers	`U/S`
`CVPR`	Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers	`S`	PyTorch[A]
`CVPR`	Global Vision Transformer Pruning with Hessian-Aware Saliency	`S`
`CVPR`	CP3: Channel Pruning Plug-in for Point-based Networks	`S`
`CVPR`	Training Debiased Subnetworks With Contrastive Weight Pruning	`U`
`CVPR`	Pruning Parameterization With Bi-Level Optimization for Efficient Semantic Segmentation on the Edge	`S`
`CVPR`	Structural Alignment for Network Pruning through Partial Regularization	`S`	PyTorch[A]
`ICLR`	JaxPruner: A concise library for sparsity research	`U/S`	PyTorch[A]
`ICLR`	OTOv2: Automatic, Generic, User-Friendly	`S`	PyTorch[A]
`ICLR`	How I Learned to Stop Worrying and Love Retraining	`U`	PyTorch[A]
`ICLR`	Token Merging: Your ViT But Faster	`U/S`	PyTorch[A]
`ICLR`	Revisiting Pruning at Initialization Through the Lens of Ramanujan Graphs	`U`	PyTorch[A] (soon...)
`ICLR`	Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask?	`U`
`ICLR`	NTK-SAP: Improving neural network pruning by aligning training dynamics	`U`
`ICLR`	DFPC: Data flow driven pruning of coupled channels without data	`S`	PyTorch[A]
`ICLR`	TVSPrune - Pruning Non-discriminative filters via Total Variation separability of intermediate representations without fine tuning	`S`	PyTorch[A]
`ICLR`	Pruning Deep Neural Networks from a Sparsity Perspective	`U`	PyTorch[A]
`ICLR`	A Unified Framework of Soft Threshold Pruning	`U`	PyTorch[A]
`WACV`	Calibrating Deep Neural Networks Using Explicit Regularisation and Dynamic Data Pruning	`S`
`WACV`	Attend Who Is Weak: Pruning-Assisted Medical Image Localization Under Sophisticated and Implicit Imbalances	`S`
`ICASSP`	WHC: Weighted Hybrid Criterion for Filter Pruning on Convolutional Neural Networks	`S`	PyTorch[A]

Venue	Title	Type	Code
`CVPR`	Interspace Pruning: Using Adaptive Filter Representations To Improve Training of Sparse CNNs	`U`
`CVPR`	Revisiting Random Channel Pruning for Neural Network Compression	`S`	PyTorch[A] (soon...)
`CVPR`	Fire Together Wire Together: A Dynamic Pruning Approach With Self-Supervised Mask Prediction	`S`	PyTorch[A]
`CVPR`	When to Prune? A Policy towards Early Structural Pruning	`S`
`CVPR`	Dreaming to Prune Image Deraining Networks	`S`
`ICLR`	SOSP: Efficiently Capturing Global Correlations by Second-Order Structured Pruning	`S`
`ICLR`	Learning Pruning-Friendly Networks via Frank-Wolfe: One-Shot, Any-Sparsity, And No Retraining	`U`	PyTorch[A]
`ICLR`	Revisit Kernel Pruning with Lottery Regulated Grouped Convolutions	`S`	PyTorch[A]
`ICLR`	Dual Lottery Ticket Hypothesis	`U`	PyTorch[A]
`NIPS`	SAViT: Structure-Aware Vision Transformer Pruning via Collaborative Optimization	`S`	PyTorch[A](soon...)
`NIPS`	Structural Pruning via Latency-Saliency Knapsack	`S`	PyTorch[A]
`ACCV`	Filter Pruning via Automatic Pruning Rate Search⋆	`S`
`ACCV`	Network Pruning via Feature Shift Minimization	`S`	PyTorch[A]
`ACCV`	Lightweight Alpha Matting Network Using Distillation-Based Channel Pruning	`S`	PyTorch[A]
`ACCV`	Adaptive FSP : Adaptive Architecture Search with Filter Shape Pruning	`S`
`ECCV`	Soft Masking for Cost-Constrained Channel Pruning	`S`	PyTorch[A]
`WACV`	Hessian-Aware Pruning and Optimal Neural Implant	`S`	PyTorch[A]
`WACV`	PPCD-GAN: Progressive Pruning and Class-Aware Distillation for Large-Scale Conditional GANs Compression	`S`
`WACV`	Channel Pruning via Lookahead Search Guided Reinforcement Learning	`S`
`WACV`	EZCrop: Energy-Zoned Channels for Robust Output Pruning	`S`	PyTorch[A]
`ICIP`	One-Cycle Pruning: Pruning Convnets With Tight Training Budget	`U`
`ICIP`	RAPID: A Single Stage Pruning Framework	`U`
`ICIP`	The Rise of the Lottery Heroes: Why Zero-Shot Pruning is Hard	`U`
`ICIP`	Truncated Lottery Ticket for Deep Pruning	`U`
`ICIP`	Which Metrics For Network Pruning: Final Accuracy? or Accuracy Drop?	`S/U`
`ISMSI`	Structured Pruning with Automatic Pruning Rate Derivation for Image Processing Neural Networks	`S`

Venue	Title	Type	Code
`ICLR`	Neural Pruning via Growing Regularization	`S`	PyTorch[A]
`ICLR`	Network Pruning That Matters: A Case Study on Retraining Variants	`S`	PyTorch[A]
`ICLR`	Layer-adaptive Sparsity for the Magnitude-based Pruning	`U`	PyTorch[A]
`NIPS`	Only Train Once: A One-Shot Neural Network Training And Pruning Framework	`S`	PyTorch[A]
`CVPR`	NPAS: A Compiler-Aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration	`S`
`CVPR`	Network Pruning via Performance Maximization	`S`
`CVPR`	Convolutional Neural Network Pruning With Structural Redundancy Reduction*	`S`
`CVPR`	Manifold Regularized Dynamic Network Pruning	`S`	PyTorch[A]
`CVPR`	Joint-DetNAS: Upgrade Your Detector With NAS, Pruning and Dynamic Distillation	`S`
`ICCV`	ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgetting	`S`
`ICCV`	Achieving On-Mobile Real-Time Super-Resolution With Neural Architecture and Pruning Search	`S`
`ICCV`	GDP: Stabilized Neural Network Pruning via Gates With Differentiable Polarization*	`S`
`WACV`	Holistic Filter Pruning for Efficient Deep Neural Networks	`S`
`ICML`	Accelerate CNNs from Three Dimensions: A Comprehensive Pruning Framework	`S`
`ICML`	Group Fisher Pruning for Practical Network Compression	`S`	PyTorch[A]

Venue	Title	Type	Code
`CVPR`	HRank: Filter Pruning using High-Rank Feature Map	`S`	PyTorch[A]
`CVPR`	Towards efficient model compression via learned global ranking	`S`	PyTorch[A]
`CVPR`	Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration	`S`
`CVPR`	Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression	`S`	PyTorch[A]
`CVPR`	APQ: Joint Search for Network Architecture, Pruning and Quantization Policy	`S`	PyTorch[A]
`ICLR`	Budgeted Training: Rethinking Deep Neural Network Training Under Resource Constraints	`U`
`MLSys`	Shrinkbench: What is the State of Neural Network Pruning?		PyTorch[A]
`BMBS`	Similarity Based Filter Pruning for Efficient Super-Resolution Models	`S`

Venue	Title	Type	Code
`CVPR`	Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration	`S`	PyTorch[A]
`CVPR`	Variational Convolutional Neural Network Pruning	`S`
`CVPR`	Towards Optimal Structured CNN Pruning via Generative Adversarial Learning	`S`	PyTorch[A]
`CVPR`	Partial Order Pruning: For Best Speed/Accuracy Trade-Off in Neural Architecture Search	`S`	PyTorch[A]
`CVPR`	Importance Estimation for Neural Network Pruning	`S`	PyTorch[A]
`ICLR`	The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks	`U`	PyTorch[A]
`ICLR`	SNIP: Single-shot Network Pruning based on Connection Sensitivity	`U`	Tensorflow[A]
`ICCV`	MetaPruning: Meta-Learning for Automatic Neural Network Channel Pruning	`S`	PyTorch[A]
`ICCV`	Accelerate CNN via Recursive Bayesian Pruning	`S`

Venue	Title	Type	Code
`CVPR`	PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning	`S`	PyTorch[A]
`CVPR`	NISP: Pruning Networks Using Neuron Importance Score Propagation	`S`
`ICIP`	Online Filter Clustering and Pruning for Efficient Convnets	`S`
`IJCAI`	Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks	`S`	PyTorch[A]

Venue	Title	Type	Code
`CVPR`	Designing Energy-Efficient Convolutional Neural Networks Using Energy-Aware Pruning	`S`
`ICLR`	Pruning Filters for Efficient ConvNets	`S`	PyTorch[O]
`ICCV`	Channel Pruning for Accelerating Very Deep Neural Networks	`S`	PyTorch[A]
`ICCV`	ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression	`S`	Caffe[A]
`ICCV`	Learning Efficient Convolutional Networks Through Network Slimming	`S`	PyTorch[A]

Journal Publications

Journal	Title	Type	Code
Neural Networks	Efficient tensor decomposition-based filter pruning	`S`	PyTorch[A]
IEEE Trans. NNLS	Enhanced Network Compression Through Tensor Decompositions and Pruning	`S`	PyTorch[A]
IEEE Transactions on Artificial Intelligence	Distilled Gradual Pruning with Pruned Fine-tuning	`U`	PyTorch[A]

Journal	Title	Type	Code
IEEE Trans Circuits Syst Video Technol	DCFP: Distribution Calibrated Filter Pruning for Lightweight and Accurate Long-tail Semantic Segmentation	`S`
IEEE Internet Things J.	SNPF: Sensitiveness Based Network Pruning Framework for Efficient Edge Computing	`S`
IEEE Trans. NNLS	Manipulating Identical Filter Redundancy for Efficient Pruning on Deep and Complicated CNN	`S`
IEEE Trans. NNLS	Block-Wise Partner Learning for Model Compression	`S`	PyTorch[A]
IEEE Trans. NNLS	Hierarchical Threshold Pruning Based on Uniform Response Criterion	`S`
IEEE Trans. NNLS	CATRO: Channel Pruning via Class-Aware Trace Ratio Optimization	`S`
IEEE Trans. NNLS	Adaptive Filter Pruning via Sensitivity Feedback	`S`
Neurocomputing	Filter pruning with uniqueness mechanism in the frequency domain for efficient neural networks	`S`
IEEE Trans. PAMI	Compact Neural Network via Stacking Hybrid Units	`S`
IEEE Trans. PAMI	Performance-aware Approximation of Global Channel Pruning for Multitask CNNs	`S`	PyTorch[A]
IEEE Trans. PAMI	Adaptive Search-and-Training for Robust and Efficient Network Pruning	`S`
Image Vis. Comput.	Loss-aware automatic selection of structured pruning criteria for deep neural network acceleration	`S`	PyTorch[A]
Comput. Vis. Image Underst.	Feature independent Filter Pruning by Successive Layers analysis	`S`
IEEE Access	Differentiable Neural Architecture, Mixed Precision and Accelerator Co-Search	`S`

Journal	Title	Type	Code
IEEE Trans. Image Process.	Efficient Layer Compression Without Pruning	`S`
IEEE Trans. PAMI	Learning to Explore Distillability and Sparsability: A Joint Framework for Model Compression	`S`
IEEE Trans. PAMI	1xN Pattern for Pruning Convolutional Neural Networks	`S`	PyTorch[A]
IEEE Trans. NNLS	Filter Pruning by Switching to Neighboring CNNs With Good Attribute	`S`
IEEE Trans. NNLS	Model Pruning Enables Efficient Federated Learning on Edge Devices	`S`
IEEE Trans. NNLS	DAIS: Automatic Channel Pruning via Differentiable Annealing Indicator Search	`S`
IEEE Trans. NNLS	Network Pruning Using Adaptive Exemplar Filters	`S`	PyTorch[A]
IEEE Trans. NNLS	Carrying Out CNN Channel Pruning in a White Box	`S`	PyTorch[A]
IEEE Trans. NNLS	Pruning Networks With Cross-Layer Ranking & k-Reciprocal Nearest Filters	`S`	PyTorch[A]
IEEE Trans. NNLS	Filter Sketch for Network Pruning	`S`	PyTorch[A]
Neurocomputing	FPFS: Filter-level pruning via distance weight measuring filter similarity	`S`
Neurocomputing	RUFP: Reinitializing unimportant filters for soft pruning	`S`
Neural Netw	HRel: Filter pruning based on High Relevance between activation maps and class labels	`S`	PyTorch[A]*
Comput. Intell. Neurosci.	Differentiable Network Pruning via Polarization of Probabilistic Channelwise Soft Masks	`S`
J. Syst. Archit.	Optimizing deep neural networks on intelligent edge accelerators via flexible-rate filter pruning	`S`
Appl. Sci.	Magnitude and Similarity Based Variable Rate Filter Pruning for Efficient Convolution Neural Networks	`S`	PyTorch[A]
Sensors	Filter Pruning via Measuring Feature Map Information	`S`
IEEE Access	Automated Filter Pruning Based on High-Dimensional Bayesian Optimization	`S`
IEEE Signal Process. Lett.	A Low-Complexity Modified ThiNet Algorithm for Pruning Convolutional Neural Networks	`S`

Journal	Title	Type	Code
IEEE Trans. PAMI	Discrimination-Aware Network Pruning for Deep Model Compression	`S`	PyTorch[A]~

Journal	Title	Type
IEEE Trans. NNLS	EDP: An Efficient Decomposition and Pruning Scheme for Convolutional Neural Network Compression	`S`
IEEE Access	Filter Pruning Without Damaging Networks Capacity	`S`
Electronics	Pruning Convolutional Neural Networks with an Attention Mechanism for Remote Sensing Image Classification	`S`

Survey Articles

Year	Venue	Title
`2023`	`Artif. Intell. Rev.`	Deep neural network pruning method based on sensitive layers and reinforcement learning
`2023`	`arVix`	A Survey on Deep Neural Network Pruning: Taxonomy, Comparison, Analysis, and Recommendations
`2023`	`arVix`	Structured Pruning for Deep Convolutional Neural Networks: A survey
`2022`	`Electronics`	A Survey on Efficient Convolutional Neural Networks and Hardware Acceleration
`2022`	`I-SMAC`	A Survey on Filter Pruning Techniques for Optimization of Deep Neural Networks
`2021`	`JMLR`	Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks
`2021`	`Neurocomputing`	Pruning and quantization for deep neural network acceleration: A survey
`2020`	`IEEE Access`	Methods for Pruning Deep Neural Networks

Other Publications

Year	Venue	Title	Code
`2023`	`arVix`	Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning	PyTorch[A](soon...)
`2023`	`arVix`	Ten Lessons We Have Learned in the New "Sparseland": A Short Handbook for Sparse Neural Network Researchers
`2022`	`ICML`	Tutorial -- Sparsity in Deep Learning: Pruning and growth for efficient inference and training

Pruning Software and Toolbox

Year	Title	Type	Code
`2023`	UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers	`S`	PyTorch[A]
`2023`	DepGraph: Towards Any Structural Pruning	`S`	PyTorch[A]
`2023`	Torch-Pruning	`S`	PyTorch[A]
`2023`	JaxPruner: JaxPruner: A concise library for sparsity research	`U/S`	PyTorch[A]
`2022`	FasterAI: Prune and Distill your models with FastAI and PyTorch	`U`	PyTorch[A]
`2022`	Simplify: A Python library for optimizing pruned neural networks		PyTorch[A]
`2021`	PyTorchViz [A small package to create visualizations of PyTorch execution graphs]		PyTorch[A]
`2020`	What is the State of Neural Network Pruning?	`S/U`	PyTorch[A]
`2019`	Official PyTorch Pruning Tool	`S/U`	PyTorch[A]