Skip to content
/ Awesome-Pruning Public
forked from he-y/Awesome-Pruning

A curated list of neural network pruning resources.

0 stars 330 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

XG-zheng/Awesome-Pruning

BranchesTags
 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

58 Commits
README.md
README.md
 
 

Repository files navigation

Awesome Pruning Awesome

A curated list of neural network pruning and related resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers and Awesome-NAS.

Please feel free to pull requests or open an issue to add papers.

Table of Contents

Type of Pruning

Type F W Other
Explanation Filter pruning Weight pruning other types

2022

Title Venue Type Code
FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Server IJCAI F -
On the Channel Pruning using Graph Convolution Network for Convolutional Neural Network Acceleration IJCAI F -
Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterization IJCAI F -
Neural Network Pruning by Cooperative Coevolution IJCAI F -
Recent Advances on Neural Network Pruning at Initialization IJCAI W PyTorch(Author)
Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness ICML F PyTorch(Author)
SPDY: Accurate Pruning with Speedup Guarantees ICML W PyTorch(Author)
Sparse Double Descent: Where Network Pruning Aggravates Overfitting ICML W PyTorch(Author)
PAC-Net: A Model Pruning Approach to Inductive Transfer Learning ICML Other -
Neural Network Pruning Denoises the Features and Makes Local Connectivity Emerge in Visual Tasks ICML Other PyTorch(Author)
Winning the Lottery Ahead of Time: Efficient Early Network Pruning ICML F -
Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning ICML F PyTorch(Author)
The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networks ICML W PyTorch(Author)
DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks ICML Other PyTorch(Author)
Fast Lossless Neural Compression with Integer-Only Discrete Flows ICML F PyTorch(Author)
Fire Together Wire Together: A Dynamic Pruning Approach With Self-Supervised Mask PredictionFire Together Wire Together: A Dynamic Pruning Approach With Self-Supervised Mask Prediction CVPR F -
Revisiting Random Channel Pruning for Neural Network Compression CVPR F PyTorch(Author)(Releasing)
Interspace Pruning: Using Adaptive Filter Representations To Improve Training of Sparse CNNs CVPR W -
When To Prune? A Policy Towards Early Structural Pruning CVPR F -
DiSparse: Disentangled Sparsification for Multitask Model Compression CVPR Other PyTorch(Author)(Releasing)
Contrastive Dual Gating: Learning Sparse Features With Contrastive Learning CVPR WF -
Learning Bayesian Sparse Networks With Full Experience Replay for Continual Learning CVPR F -
Masking Adversarial Damage: Finding Adversarial Saliency for Robust and Sparse Network CVPR W -
DECORE: Deep Compression With Reinforcement Learning CVPR F -
CHEX: CHannel EXploration for CNN Model Compression CVPR F -
Compressing Models With Few Samples: Mimicking Then Replacing CVPR F PyTorch(Author)(Releasing)
Learning Pruning-Friendly Networks via Frank-Wolfe: One-Shot, Any-Sparsity, And No Retraining ICLR (Spotlight) W PyTorch(Author)
On Lottery Tickets and Minimal Task Representations in Deep Reinforcement Learning ICLR (Spotlight) W -
SOSP: Efficiently Capturing Global Correlations by Second-Order Structured Pruning ICLR (Spotlight) F PyTorch(Author)(Releasing)
Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models ICLR (Spotlight) F PyTorch(Author)
Prospect Pruning: Finding Trainable Weights at Initialization using Meta-Gradients ICLR WF PyTorch(Author)
An Operator Theoretic View On Pruning Deep Neural Networks ICLR W -
Revisit Kernel Pruning with Lottery Regulated Grouped Convolutions ICLR F PyTorch(Author)
Effective Model Sparsification by Scheduled Grow-and-Prune Methods ICLR W PyTorch(Author)
The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training ICLR Other PyTorch(Author)
Learning Efficient Image Super-Resolution Networks via Structure-Regularized Pruning ICLR F PyTorch(Author)
Signing the Supermask: Keep, Hide, Invert ICLR W -
Plant 'n' Seek: Can You Find the Winning Ticket? ICLR F PyTorch(Author)
Proving the Lottery Ticket Hypothesis for Convolutional Neural Networks ICLR F PyTorch(Author)
On the Existence of Universal Lottery Tickets ICLR F PyTorch(Author)
Training Structured Neural Networks Through Manifold Identification and Variance Reduction ICLR F PyTorch(Author)
How many degrees of freedom do we need to train deep networks: a loss landscape perspective ICLR W PyTorch(Author)
Dual Lottery Ticket Hypothesis ICLR W PyTorch(Author)
Peek-a-Boo: What (More) is Disguised in a Randomly Weighted Neural Network, and How to Find It Efficiently ICLR W PyTorch(Author)
Sparsity Winning Twice: Better Robust Generalization from More Efficient Training ICLR W PyTorch(Author)
Prior Gradient Mask Guided Pruning-Aware Fine-Tuning AAAI F -
Convolutional Neural Network Compression through Generalized Kronecker Product Decomposition AAAI Other -
Prune and Tune Ensembles: Low-Cost Ensemble Learning with Sparse Independent Subnetworks AAAI W -

2021

Title Venue Type Code
Validating the Lottery Ticket Hypothesis with Inertial Manifold Theory NeurIPS W -
The Elastic Lottery Ticket Hypothesis NeurIPS W PyTorch(Author)
Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot? NeurIPS W PyTorch(Author)
Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Sparse Neural Networks NeurIPS W -
You are caught stealing my winning lottery ticket! Making a lottery ticket claim its ownership NeurIPS W PyTorch(Author)
Pruning Randomly Initialized Neural Networks with Iterative Randomization NeurIPS W PyTorch(Author)
Sparse Training via Boosting Pruning Plasticity with Neuroregeneration NeurIPS W PyTorch(Author)
Rethinking the Pruning Criteria for Convolutional Neural Network NeurIPS F -
Only Train Once: A One-Shot Neural Network Training And Pruning Framework NeurIPS F PyTorch(Author)
CHIP: CHannel Independence-based Pruning for Compact Neural Networks NeurIPS F PyTorch(Author)
Sparse Flows: Pruning Continuous-depth Models NeurIPS WF PyTorch(Author)
A Winning Hand: Compressing Deep Networks Can Improve Out-of-Distribution Robustness NeurIPS W PyTorch(Author)
Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition NeurIPS F PyTorch(Author)
AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks NeurIPS W PyTorch(Author)
RED : Looking for Redundancies for Data-FreeStructured Compression of Deep Neural Networks NeurIPS F -
Scaling Up Exact Neural Network Compression by ReLU Stability NeurIPS F PyTorch(Author)
Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme NeurIPS F PyTorch(Author)
Heavy Tails in SGD and Compressibility of Overparametrized Neural Networks NeurIPS Other PyTorch(Author)
ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgetting ICCV F PyTorch(Author)
Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search ICCV F -
GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarization ICCV F -
Auto Graph Encoder-Decoder for Neural Network Pruning ICCV F -
Sub-Bit Neural Networks: Learning To Compress and Accelerate Binary Neural Networks ICCV Other PyTorch(Author)
Exploration and Estimation for Model Compression ICCV F -
A Probabilistic Approach to Neural Network Pruning ICML F -
Accelerate CNNs from Three Dimensions: A Comprehensive Pruning Framework ICML F -
Group Fisher Pruning for Practical Network Compression ICML F PyTorch(Author)
On the Predictability of Pruning Across Scales ICML W -
Towards Compact CNNs via Collaborative Compression CVPR F PyTorch(Author)
Content-Aware GAN Compression CVPR F PyTorch(Author)
Permute, Quantize, and Fine-tune: Efficient Compression of Neural Networks CVPR F PyTorch(Author)
NPAS: A Compiler-aware Framework of Unified Network Pruning andArchitecture Search for Beyond Real-Time Mobile Acceleration CVPR F -
Network Pruning via Performance Maximization CVPR F -
Convolutional Neural Network Pruning with Structural Redundancy Reduction CVPR F -
Manifold Regularized Dynamic Network Pruning CVPR F -
Joint-DetNAS: Upgrade Your Detector with NAS, Pruning and Dynamic Distillation CVPR FO -
A Gradient Flow Framework For Analyzing Network Pruning ICLR F PyTorch(Author)
Neural Pruning via Growing Regularization ICLR F PyTorch(Author)
ChipNet: Budget-Aware Pruning with Heaviside Continuous Approximations ICLR F PyTorch(Author)
Network Pruning That Matters: A Case Study on Retraining Variants ICLR F PyTorch(Author)
Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Network ICLR W PyTorch(Author)
Layer-adaptive Sparsity for the Magnitude-based Pruning ICLR W PyTorch(Author)
Pruning Neural Networks at Initialization: Why Are We Missing the Mark? ICLR W -
Robust Pruning at Initialization ICLR W -

2020

Title Venue Type Code
HYDRA: Pruning Adversarially Robust Neural Networks NeurIPS W PyTorch(Author)
Logarithmic Pruning is All You Need NeurIPS W -
Directional Pruning of Deep Neural Networks NeurIPS W -
Movement Pruning: Adaptive Sparsity by Fine-Tuning NeurIPS W PyTorch(Author)
Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot NeurIPS W PyTorch(Author)
Neuron Merging: Compensating for Pruned Neurons NeurIPS F PyTorch(Author)
Neuron-level Structured Pruning using Polarization Regularizer NeurIPS F PyTorch(Author)
SCOP: Scientific Control for Reliable Neural Network Pruning NeurIPS F PyTorch(Author)
Storage Efficient and Dynamic Flexible Runtime Channel Pruning via Deep Reinforcement Learning NeurIPS F -
The Generalization-Stability Tradeoff In Neural Network Pruning NeurIPS F PyTorch(Author)
Pruning Filter in Filter NeurIPS Other PyTorch(Author)
Position-based Scaled Gradient for Model Quantization and Pruning NeurIPS Other PyTorch(Author)
Bayesian Bits: Unifying Quantization and Pruning NeurIPS Other -
Pruning neural networks without any data by iteratively conserving synaptic flow NeurIPS Other PyTorch(Author)
EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning ECCV (Oral) F PyTorch(Author)
DSA: More Efficient Budgeted Pruning via Differentiable Sparsity Allocation ECCV F -
DHP: Differentiable Meta Pruning via HyperNetworks ECCV F PyTorch(Author)
Meta-Learning with Network Pruning ECCV W -
Accelerating CNN Training by Pruning Activation Gradients ECCV W -
DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search ECCV Other -
Differentiable Joint Pruning and Quantization for Hardware Efficiency ECCV Other -
Channel Pruning via Automatic Structure Search IJCAI F PyTorch(Author)
Adversarial Neural Pruning with Latent Vulnerability Suppression ICML W -
Proving the Lottery Ticket Hypothesis: Pruning is All You Need ICML W -
Soft Threshold Weight Reparameterization for Learnable Sparsity ICML WF Pytorch(Author)
Network Pruning by Greedy Subnetwork Selection ICML F -
Operation-Aware Soft Channel Pruning using Differentiable Masks ICML F -
DropNet: Reducing Neural Network Complexity via Iterative Pruning ICML F -
Towards Efficient Model Compression via Learned Global Ranking CVPR (Oral) F Pytorch(Author)
HRank: Filter Pruning using High-Rank Feature Map CVPR (Oral) F Pytorch(Author)
Neural Network Pruning with Residual-Connections and Limited-Data CVPR (Oral) F -
Multi-Dimensional Pruning: A Unified Framework for Model Compression CVPR (Oral) WF -
DMCP: Differentiable Markov Channel Pruning for Neural Networks CVPR (Oral) F TensorFlow(Author)
Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression CVPR F PyTorch(Author)
Few Sample Knowledge Distillation for Efficient Network Compression CVPR F -
Discrete Model Compression With Resource Constraint for Deep Neural Networks CVPR F -
Structured Compression by Weight Encryption for Unstructured Pruning and Quantization CVPR W -
Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration CVPR F -
APQ: Joint Search for Network Architecture, Pruning and Quantization Policy CVPR F -
Comparing Rewinding and Fine-tuning in Neural Network Pruning ICLR (Oral) WF TensorFlow(Author)
A Signal Propagation Perspective for Pruning Neural Networks at Initialization ICLR (Spotlight) W -
ProxSGD: Training Structured Neural Networks under Regularization and Constraints ICLR W TF PT(Author)
One-Shot Pruning of Recurrent Neural Networks by Jacobian Spectrum Evaluation ICLR W -
Lookahead: A Far-sighted Alternative of Magnitude-based Pruning ICLR W PyTorch(Author)
Dynamic Model Pruning with Feedback ICLR WF -
Provable Filter Pruning for Efficient Neural Networks ICLR F -
Data-Independent Neural Pruning via Coresets ICLR W -
AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates AAAI F -
DARB: A Density-Aware Regular-Block Pruning for Deep Neural Networks AAAI Other -
Pruning from Scratch AAAI Other -
Reborn filters: Pruning convolutional neural networks with limited data AAAI F -

2019

Title Venue Type Code
Network Pruning via Transformable Architecture Search NeurIPS F PyTorch(Author)
Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks NeurIPS F PyTorch(Author)
Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask NeurIPS W TensorFlow(Author)
One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers NeurIPS W -
Global Sparse Momentum SGD for Pruning Very Deep Neural Networks NeurIPS W PyTorch(Author)
AutoPrune: Automatic Network Pruning by Regularizing Auxiliary Parameters NeurIPS W -
Model Compression with Adversarial Robustness: A Unified Optimization Framework NeurIPS Other PyTorch(Author)
MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning ICCV F PyTorch(Author)
Accelerate CNN via Recursive Bayesian Pruning ICCV F -
Adversarial Robustness vs Model Compression, or Both? ICCV W PyTorch(Author)
Learning Filter Basis for Convolutional Neural Network Compression ICCV Other -
Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration CVPR (Oral) F PyTorch(Author)
Towards Optimal Structured CNN Pruning via Generative Adversarial Learning CVPR F PyTorch(Author)
Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structure CVPR F PyTorch(Author)
On Implicit Filter Level Sparsity in Convolutional Neural Networks, Extension1, Extension2 CVPR F PyTorch(Author)
Structured Pruning of Neural Networks with Budget-Aware Regularization CVPR F -
Importance Estimation for Neural Network Pruning CVPR F PyTorch(Author)
OICSR: Out-In-Channel Sparsity Regularization for Compact Deep Neural Networks CVPR F -
Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search CVPR Other TensorFlow(Author)
Variational Convolutional Neural Network Pruning CVPR - -
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks ICLR (Best) W TensorFlow(Author)
Rethinking the Value of Network Pruning ICLR F PyTorch(Author)
Dynamic Channel Pruning: Feature Boosting and Suppression ICLR F TensorFlow(Author)
SNIP: Single-shot Network Pruning based on Connection Sensitivity ICLR W TensorFLow(Author)
Dynamic Sparse Graph for Efficient Deep Learning ICLR F CUDA(3rd)
Collaborative Channel Pruning for Deep Networks ICML F -
Approximated Oracle Filter Pruning for Destructive CNN Width Optimization github ICML F -
EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasis ICML F PyTorch(Author)
COP: Customized Deep Model Compression via Regularized Correlation-Based Filter-Level Pruning IJCAI F Tensorflow(Author)

2018

Title Venue Type Code
Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers ICLR F TensorFlow(Author), PyTorch(3rd)
To prune, or not to prune: exploring the efficacy of pruning for model compression ICLR W -
Discrimination-aware Channel Pruning for Deep Neural Networks NeurIPS F TensorFlow(Author)
Frequency-Domain Dynamic Pruning for Convolutional Neural Networks NeurIPS W -
Learning Sparse Neural Networks via Sensitivity-Driven Regularization NeurIPS WF -
Amc: Automl for model compression and acceleration on mobile devices ECCV F TensorFlow(3rd)
Data-Driven Sparse Structure Selection for Deep Neural Networks ECCV F MXNet(Author)
Coreset-Based Neural Network Compression ECCV F PyTorch(Author)
Constraint-Aware Deep Neural Network Compression ECCV W SkimCaffe(Author)
A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers ECCV W Caffe(Author)
PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning CVPR F PyTorch(Author)
NISP: Pruning Networks using Neuron Importance Score Propagation CVPR F -
CLIP-Q: Deep Network Compression Learning by In-Parallel Pruning-Quantization CVPR W -
“Learning-Compression” Algorithms for Neural Net Pruning CVPR W -
Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks IJCAI F PyTorch(Author)
Accelerating Convolutional Networks via Global & Dynamic Filter Pruning IJCAI F -

2017

Title Venue Type Code
Pruning Filters for Efficient ConvNets ICLR F PyTorch(3rd)
Pruning Convolutional Neural Networks for Resource Efficient Inference ICLR F TensorFlow(3rd)
Net-Trim: Convex Pruning of Deep Neural Networks with Performance Guarantee NeurIPS W TensorFlow(Author)
Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon NeurIPS W PyTorch(Author)
Runtime Neural Pruning NeurIPS F -
Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruning CVPR W -
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression ICCV F Caffe(Author), PyTorch(3rd)
Channel pruning for accelerating very deep neural networks ICCV F Caffe(Author)
Learning Efficient Convolutional Networks Through Network Slimming ICCV F PyTorch(Author)

2016

Title Venue Type Code
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding ICLR (Best) W Caffe(Author)
Dynamic Network Surgery for Efficient DNNs NeurIPS W Caffe(Author)

2015

Title Venue Type Code
Learning both Weights and Connections for Efficient Neural Networks NeurIPS W PyTorch(3rd)

Related Repo

Awesome-model-compression-and-acceleration

EfficientDNNs

Embedded-Neural-Network

awesome-AutoML-and-Lightweight-Models

Model-Compression-Papers

knowledge-distillation-papers

Network-Speed-and-Compression

About

A curated list of neural network pruning resources.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published