Awesome
Towards Deeper Graph Neural Networks with Differentiable Group Normalization
This is an authors' implementation of "Towards Deeper Graph Neural Networks with Differentiable Group Normalization" in Pytorch.
Authors: Kaixiong Zhou, Xiao Huang, Yuening Li, Daochen Zha, Rui Chen and Xia Hu
Paper: https://arxiv.org/abs/2006.06972
Accepted by NeurIPS 2020.
Introduction
This work presents two metrics to quantify the over-smoothing in GNNs: (1) Group distance ratio and (2) Instance information gain.
Based on these two metrics, we provide the differentiable group normalization (DGN), a general module applied between the graph convolutional layers, to relieve the over-smoothing issue. DGN softly clusters nodes and normalizes each group independently, which prevents the distinct groups from having the close node representations.
Detailed information about the metrics and DGN is provided in (https://arxiv.org/abs/2006.06972).
Requirements
python == 3.6
torch == 1.3.1
torch-geometric==1.3.2
Train over GCN, GAT or SGCN backbone networks
To train GNN model, measure the group distance ratio as well as instance information gain, run:
python main.py --cuda_num=0 --type_norm='group' --type_model='GCN' --dataset='Cora' --miss_rate=0.
Hyperparameter explanations:
--type_norm: the type of normalization layer. We include ['None', 'batch', 'pair', 'group'] for none normalization, batch normalization, pair normalization and differentiable group normalization, respectively.
--type_model: the type of GNN model. We include ['GCN', 'GAT', 'simpleGCN']
--dataset: we include ['Cora', 'Citeseer', 'Pubmed', 'CoauthorCS']
--miss_rate: the missing rate of input features. The value of 0. corresponds to the original dataset. The value of 1. means removing the features in validation and testing sets
Train over GCNII backbone networks
We directly use the provided implementation of GCNII. After the model configuration, run:
python GCNII_Cora.py --type_norm='group'
Citation
If using this code, please cite our paper.
@inproceedings{zhou2020towards,
title={Towards Deeper Graph Neural Networks with Differentiable Group Normalization},
author={Zhou, Kaixiong and Huang, Xiao and Li, Yuening and Zha, Daochen and Chen, Rui and Hu, Xia},
booktitle={Advances in neural information processing systems},
year={2020}
}