Awesome

awesome-deep-gnn

Papers about developing deep Graph Neural Networks (GNNs). Investigations about over-smoothing and over-squashing problem in GNNs are also included here.

Please feel free to submit a pull request if you want to add good papers.

Most Influential Papers

Selected by CogDL

1. [ICML 2018] Representation Learning on Graphs with Jumping Knowledge Networks [Paper]
2. [AAAI 2018] Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning [Paper]
3. [ICLR 2019] Predict then Propagate: Graph Neural Networks meet Personalized PageRank [Paper][Code]
4. [ICCV 2019] DeepGCNs: Can GCNs Go as Deep as CNNs? [Paper][Code(Pytorch)][Code(TensorFlow)]
5. [NeurIPS 2019] Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks. [Paper][Code]
6. [arXiv 2020] DeeperGCN: All You Need to Train Deeper GCNs [Paper][Code]
7. [ICLR 2020] PairNorm: Tackling Oversmoothing in GNNs [Paper][Code]
8. [ICLR 2020] DropEdge: Towards Deep Graph Convolutional Networks on Node Classification [Paper][Code]
9. [ICML 2020] Simple and Deep Graph Convolutional Networks [Paper][Code]
10. [KDD 2020] Towards Deeper Graph Neural Networks [Paper][Code]

2022

• [NeurIPS 2022] Not too little, not too much: a theoretical analysis of graph (over)smoothing [Paper][Code]
• [NeurIPS 2022] Neural Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNs [Paper][Code]
• [ICML 2022] Graph-Coupled Oscillator Networks [Paper][Code]
• [KDD 2022] Feature Overcorrelation in Deep Graph Neural Networks: A New Perspective [Paper] [code]
• [KDD 2022] Model Degradation Hinders Deep Graph Neural Networks [Paper]
• [ICDM 2022] Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks [Paper][Code]
• [ICLR 2022] Understanding over-squashing and bottlenecks on graphs via curvature [Paper][Code]
• [ICLR 2022] Towards Deepening Graph Neural Networks: A GNTK-based Optimization Perspective [Paper]
• [ICLR 2022] Simple GNN Regularisation for 3D Molecular Property Prediction & Beyond [Paper]
• [ICLR 2022] Revisiting Over-smoothing in BERT from the Perspective of Graph [Paper]
• [AAAI 2022] Orthogonal Graph Neural Networks [Paper]
• [TPAMI] Bag of Tricks for Training Deeper Graph Neural Networks: A Comprehensive Benchmark Study [Paper][Code]

2021

• [ICML 2021] Lipschitz Normalization for Self-Attention Layers with Application to Graph Neural Networks [Paper]
• [ICML 2021] Improving Breadth-Wise Backpropagation in Graph Neural Networks Helps Learning Long-Range Dependencies [Paper]
• [ICML 2021] GRAND: Graph Neural Diffusion [Paper][Code]
• [ICML 2021] Graph Neural Networks Inspired by Classical Iterative Algorithms [Paper][Code]
• [ICML 2021] Optimization of Graph Neural Networks: Implicit Acceleration by Skip Connections and More Depth [Paper]
• [ICML 2021] Training Graph Neural Networks with 1000 Layers [Paper][Code]
• [ICML 2021] Directional Graph Networks [Paper][Code]
• [ICLR 2021] AdaGCN: Adaboosting Graph Convolutional Networks into Deep Models [Paper][Code]
• [ICLR 2021] On the Bottleneck of Graph Neural Networks and its Practical Implications [Paper][Code]
• [ICLR 2021] Adaptive Universal Generalized PageRank Graph Neural Network [Paper][Code]
• [ICLR 2021] Simple Spectral Graph Convolution [Paper]
• [CIKM 2021] Understanding and Resolving Performance Degradation in Graph Convolutional Networks [Paper][Code]

2020

• [arXiv 2020] Deep Graph Neural Networks with Shallow Subgraph Samplers [Paper]
• [arXiv 2020] Revisiting Graph Convolutional Network on Semi-Supervised Node Classification from an Optimization Perspective [Paper]
• [arXiv 2020] Tackling Over-Smoothing for General Graph Convolutional Networks [Paper]
• [arXiv 2020] DeeperGCN: All You Need to Train Deeper GCNs [Paper][Code]
• [arXiv 2020] Revisiting Over-smoothing in Deep GCNs [paper] <br/><br/>
• [NeurIPS 2020] Graph Random Neural Networks for Semi-Supervised Learning on Graphs [Paper][Code]
• [NeurIPS 2020] Scattering GCN: Overcoming Oversmoothness in Graph Convolutional Networks [Paper][Code]
• [NeurIPS 2020] Optimization and Generalization Analysis of Transduction through Gradient Boosting and Application to Multi-scale Graph Neural Networks [Paper][Code]
• [NeurIPS 2020] Towards Deeper Graph Neural Networks with Differentiable Group Normalization [Paper]
• [ICML 2020 Workshop GRL+] A Note on Over-Smoothing for Graph Neural Networks [Paper]
• [ICML 2020] Bayesian Graph Neural Networks with Adaptive Connection Sampling [Paper]
• [ICML 2020] Continuous Graph Neural Networks [Paper]
• [ICML 2020] Simple and Deep Graph Convolutional Networks [Paper][Code]
• [KDD 2020] Towards Deeper Graph Neural Networks [Paper][Code]
• [ICLR 2020] Graph Neural Networks Exponentially Lose Expressive Power for Node Classification [Paper][Code]
• [ICLR 2020] DropEdge: Towards Deep Graph Convolutional Networks on Node Classification [Paper][Code]
• [ICLR 2020] PairNorm: Tackling Oversmoothing in GNNs [Paper][Code]
• [ICLR 2020] Measuring and Improving the Use of Graph Information in Graph Neural Networks [Paper][Code]
• [AAAI 2020] Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View [Paper]
• [CVPR 2020] Geometrically Principled Connections in Graph Neural Networks [Paper]

Before 2020

• [arXiv 2019] Revisiting Graph Neural Networks: All We Have is Low-Pass Filters [Paper] <br/><br/>
• [NeurIPS 2019] Break the Ceiling: Stronger Multi-scale Deep Graph Convolutional Networks [Paper]
• [NeurIPS 2019] Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks. [Paper][Code]
• [ICLR 2019] Predict then Propagate: Graph Neural Networks meet Personalized PageRank [Paper][Code]
• [ICCV 2019] DeepGCNs: Can GCNs Go as Deep as CNNs? [Paper][Code(Pytorch)][Code(TensorFlow)]
• [ICML 2018] Representation Learning on Graphs with Jumping Knowledge Networks [Paper]
• [AAAI 2018] Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning [Paper]