Awesome

Awesome Self-Supervised Learning for Graphs

A curated list for awesome self-supervised graph representation learning resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers, awesome-architecture-search, and awesome-self-supervised-learning.

Background

Self-supervised learning is the future! — Yann LeCun

Recently self-supervised learning (SSL) techniques have gained success in many domains, e.g., visual, natural language processing, and robotics, where SSL methods even outperform their supervised counterparts. However, the development of SSL in the graph domain is still at a nascent stage. Can SSL graph representation achieve similar or even better performance than its supervised opponents? This repository provides you with a curated list of awesome self-supervised graph representation learning resources. Following [Ankesh Anand 2020], we roughly divide papers into two lines: generative/predictive (i.e. optimizing in the output space) and contrastive methods (i.e. optimizing in the latent space). Along with papers, we also list several must-read blog posts and talks.

Contribution

Feel free to send pull requests to add more links!

Table of Contents

• Papers
  • Surveys
  • Generative/Predictive Methods
  • Contrastive Methods
• Blog Posts
• Talks

Papers

Surveys

• Self-supervised Learning: Generative or Contrastive, arXiv 2020 [PDF]
• Self-Supervised Learning of Graph Neural Networks: A Unified Review, arXiv 2021 [PDF]
• Graph Self-Supervised Learning: A Survey, arXiv 2021 [PDF]
• Self-supervised on Graphs: Contrastive, Generative, or Predictive, arXiv 2021 [PDF]

Generative/Predictive Methods

Year 2020

• Multi-Stage Self-Supervised Learning for Graph Convolutional Networks on Graphs with Few Labeled Nodes, AAAI 2020 [PDF, Code]

  ▷Node representation learning

• Strategies for Pre-training Graph Neural Networks, ICLR 2020 [PDF, Code]

  ▷Pretraining graphs

• When Does Self-Supervision Help Graph Convolutional Networks?, ICML 2020 [PDF, Code]

  ▷Node representation learning

• GPT-GNN: Generative Pre-Training of Graph Neural Networks, KDD 2020 [PDF, Code]

  ▷Pretraining graphs

• Self-supervised Auxiliary Learning with Meta-paths for Heterogeneous Graphs, NeurIPS 2020 [PDF]

  ▷Heterogeneous graphs

• CAGNN: Cluster-Aware Graph Neural Networks for Unsupervised Graph Representation Learning, arXiv 2020 [PDF]

  ▷Node representation learning

• Self-supervised Learning on Graphs: Deep Insights and New Direction, arXiv 2020 [PDF, Code]

  ▷Node representation learning

• Self-Supervised Graph Representation Learning via Global Context Prediction, arXiv 2020 [PDF]

  ▷Node representation learning

Contrastive Methods

Year 2021

• Bipartite Graph Embedding via Mutual Information Maximization, WSDM 2021 [PDF, Code]

  ▷Bipartite graph representation learning

• Contrastive Self-supervised Learning for Graph Classification, AAAI 2021 [PDF]

  ▷Graph representation learning

• Contrastive and Generative Graph Convolutional Networks for Graph-Based Semi-Supervised Learning, AAAI 2021 [PDF]

  ▷Semi-supervised node representation learning

• Graph Contrastive Learning with Adaptive Augmentation, WWW 2021 [PDF, Code]

  ▷Node representation learning

• SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanism, WWW 2021 [PDF]

  ▷Graph representation learning

• HDMI: High-order Deep Multiplex Infomax, WWW 2021 [PDF]

  ▷Multiplex graph representation learning

• Self-Supervised Graph Neural Networks Without Explicit Negative Sampling, SSL@WWW 2021 [PDF]

  ▷Node representation learning

• Motif-Driven Contrastive Learning of Graph Representations, SSL@WWW 2021 [PDF]

  ▷Pretraining graphs

• Iterative Graph Self-Distillation, SSL@WWW 2021 [PDF]

  ▷Graph representation learning

• Towards Robust Graph Contrastive Learning, SSL@WWW 2021 [PDF]

  ▷Node representation learning

• Contrastive Learning with Hard Negative Samples, ICLR 2021 [PDF]

  ▷Graph representation learning

• Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning, IJCAI 2021 [PDF]

  ▷Node representation learning

• Graph InfoClust: Maximizing Coarse-Grain Mutual Information in Graphs, PAKDD 2021 [PDF]

  ▷Node representation learning

• Self-supervised Graph-level Representation Learning with Local and Global Structure, CML 2021 [PDF]

  ▷Pretraining graphs

• Graph Contrastive Learning Automated, ICML 2021 [PDF, Code]

  ▷Graph representation learning

• Pairwise Half-graph Discrimination: A Simple Graph-level Self-supervised Strategy for Pre-training Graph Neural Networks, IJCAI 2021 [PDF]

  ▷Graph pretraining, molecule classification

• Graph Debiased Contrastive Learning with Joint Representation Clustering, IJCAI 2021 [PDF]

  ▷Node representation learning

• Multi-View Self-Supervised Heterogeneous Graph Embedding, ECML PKDD 2021 [PDF]

  ▷Heterogeneous graph representation learning

• Adversarial Graph Augmentation to Improve Graph Contrastive Learning, NeurIPS 2021 [PDF]

  ▷Graph representation learning

• InfoGCL: Information-Aware Graph Contrastive Learning, NeurIPS 2021 [PDF]

  ▷Node/graph representation learning

• Directed Graph Contrastive Learning, NeurIPS 2021 [PDF]

  ▷Directed graph representation learning

• GCCAD: Graph Contrastive Coding for Anomaly Detection, arXiv 2021 [PDF]

  ▷Anomaly detection

• Bootstrapped Representation Learning on Graphs, arXiv 2021 [PDF]

  ▷Node representation learning

• Improving Graph Representation Learning by Contrastive Regularization, arXiv 2021 [PDF]

  ▷Graph/node representation learning

• Automated Self-Supervised Learning for Graphs, arXiv 2021 [PDF]

  ▷Node representation learning

• Self-Supervised Graph Learning with Proximity-based Views and Channel Contrast, arXiv 2021 [PDF]

  ▷Node representation learning

• Fairness-Aware Node Representation Learning, arXiv 2021 [PDF]

  ▷Node representation learning

• Group Contrastive Self-Supervised Learning on Graphs, arXiv 2021 [PDF]

  ▷Graph/node representation learning

• AutoGCL: Automated Graph Contrastive Learning via Learnable View Generators, arXiv 2021 [PDF]

  ▷Graph representation learning

• Debiased Graph Contrastive Learning, arXiv 2021 [PDF]

  ▷Node representation learning

• Adaptive Multi-layer Contrastive Graph Neural Networks, arXiv 2021 [PDF]

  ▷Node representation learning

• Jointly Learnable Data Augmentations for Self-Supervised GNNs, arXiv 2021 [PDF]

  ▷Node representation learning

• Spatio-Temporal Graph Contrastive Learning, arXiv 2021 [PDF]

  ▷Spatio-temporal graph representation learning

Year 2020

• Unsupervised Attributed Multiplex Network Embedding, AAAI 2020 [PDF]

  ▷Multiplex graph representation learning

• Graph Representation Learning via Graphical Mutual Information Maximization, WWW 2020 [PDF, Code]

  ▷Node representation learning

• Contrastive Learning of Structured World Models, ICLR 2020 [PDF, Code]

  ▷Relational inference

• Contrastive Multi-View Representation Learning on Graphs, ICML 2020 [PDF, Code]

  ▷Node/graph representation learning

• Deep Graph Contrastive Representation Learning, GRL+@ICML 2020 [PDF, Code]

  ▷Node representation learning

• GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training, KDD 2020 [PDF, Code]

  ▷Pretraining graphs

• Graph Contrastive Learning with Augmentations, NeurIPS 2020 [PDF, Code]

  ▷Node representation learning, pretraining graphs

• Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learning, ICDM 2020 [PDF]

  ▷Sub-graph representation learning

• Self-supervised Smoothing Graph Neural Networks, arXiv 2020 [PDF]

  ▷Node representation learning

• Towards Domain-Agnostic Contrastive Learning, arXiv 2020 [PDF]

  ▷Graph representation learning

Year 2019

• Deep Graph Infomax, ICLR 2019 [PDF, Code]

  ▷Node representation learning

• Spatio-Temporal Deep Graph Infomax, RLGM@ICLR 2019 [PDF]

  ▷Node representation learning

Applications

• Self-Supervised Hypergraph Convolutional Networks for Session-based Recommendation, AAAI 2021 [PDF, Code]

  ▷Session-based recommendation

• Multi-view Graph Contrastive Representation Learning for Drug-Drug Interaction Prediction, WWW 2021 [PDF]

  ▷Drug-drug interaction prediction

• Self-Supervised Multi-Channel Hypergraph Convolutional Network for Social Recommendation, WWW 2021 [PDF]

  ▷Social recommendation

• Self-supervised Graph Learning for Recommendation, SIGIR 2021 [PDF]

  ▷Collaborative filtering

• Social Recommendation with Self-Supervised Metagraph Informax Network, CIKM 2021 [PDF]

  ▷Social recommendation

• CCGL: Contrastive Cascade Graph Learning, arXiv 2021 [PDF]

  ▷Information cascade prediction

Blog Posts

• Contrastive Self-Supervised Learning, [URL]
• Graph Contrastive Learning, [URL]

Talks

• Unsupervised Learning with Graph Neural Networks, ACDL 2019 Satellite Workshop on Graph Neural Networks [URL]