Home

Awesome

MagNet

MagNet: A Neural Network for Directed Graphs

@article{zhang2021magnet,
  title={Magnet: A neural network for directed graphs},
  author={Zhang, Xitong and He, Yixuan and Brugnone, Nathan and Perlmutter, Michael and Hirn, Matthew},
  journal={Advances in Neural Information Processing Systems},
  volume={34},
  pages={27003--27015},
  year={2021}
}

MagNet is included in the package: PyTorch Geometric Signed Directed

@article{he2022pytorch,
        title={{PyTorch Geometric Signed Directed: A Software Package on Graph Neural Networks for Signed and Directed Graphs}},
        author={He, Yixuan and Zhang, Xitong and Huang, Junjie and Rozemberczki, Benedek and Cucuringu, Mihai and Reinert, Gesine},
        journal={arXiv preprint arXiv:2202.10793},
        year={2022}
        }

Environment Setup

Overview

The project has been tested on the following environment specification:

  1. Ubuntu 18.04.5 LTS (Other x86_64 based Linux distributions should also be fine, such as Fedora 32)
  2. Nvidia Graphic Card (NVIDIA GeForce RTX 2080 with driver version 440.36, and NVIDIA RTX 8000) and CPU (Intel Core i7-10700 CPU @ 2.90GHz)
  3. Python 3.6.13 (and Python 3.6.12)
  4. CUDA 10.2 (and CUDA 9.2)
  5. Pytorch 1.8.0 (built against CUDA 10.2) and Python 1.6.0 (built against CUDA 9.2)
  6. Other libraries and python packages (See below)

The implementation of MagNet without baselines: SimpleMagNet

Installation

There are two installation methods listed below. Please try to install the packages manually if the first method fails.

Installation method 1 (.yml files)

You should handle (1),(2) yourself. For (3), (4), (5) and (6), we provide a list of steps to install them.

We provide two examples of envionmental setup, one with CUDA 10.2 and GPU, the other with CPU.

Following steps assume you've done with (1) and (2).

  1. Install conda. Both Miniconda and Anaconda are OK.

  2. Create an environment and install python packages (GPU):

conda env create -f environment_GPU.yml
  1. Create an environment and install python packages (CPU):
conda env create -f environment_CPU.yml

Installation method 2 (manual installation)

The codebase is implemented in Python 3.6.12. package versions used for development are below.

networkx           2.5
numpy              1.19.2
pandas             1.1.4
scipy              1.5.4
argparse           1.1.0
sklearn            0.23.2
stellargraph       1.2.1 (for link direction prediction: conda install -c stellargraph stellargraph)
torch              1.8.0
torch-scatter      2.0.5
torch-geometric    1.6.3 (follow https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.html)
matplotlib         3.3.4 (for generating plots and results)

Execution checks

When installation is done, you could check you enviroment via:

cd execution
bash setup_test.sh

Folder structure

wget http://git.savannah.gnu.org/cgit/parallel.git/plain/src/parallel
chmod 755 ./parallel
mkdir joblog
mkdir Output

Options

<p align="justify"> MagNet provides the following command line arguments, which can be viewed in the ./src/param_parser.py. </p>

Synthetic data options for node classification task:

See file ./src/sparse_Magnet.py for example.

  --p_q                   FLOAT         Direction strength, from 0.5 to 1.                      Default is 0.95. 
  --p_inter               FLOAT         Inter-cluster edge probabilities.                       Default is 0.1.
  --seed                  INT           Random seed for training testing 
                                            split/random graph generation.                      Default is 0.

Major model options for node classification:

See file ./src/sparse_Magnet.py for example.

  --epochs                INT         Number of (maximum) training epochs.                      Default is 3000. 
  --q                     FLOAT       q value for the phase matrix.                             Default is 0. 
  --new_setting, -NS      BOOL        Whether not to load best settings.                        Default is False.
  --num_filter            INT         Numer of filters.                                         Default is 1.
  --layer                 INT         Numer of layers.                                          Default is 2.
  --lr                    FLOAT       Learning rate.                                            Default is 0.005.  
  --l2                    FLOAT       Weight decay (L2 loss on parameters).                     Default is 5^-4. 
  --dropout               FLOAT       Dropout rate (1 - keep probability).                      Default is 0.0.
  --debug, -D             BOOL        Debug with minimal training setting, not to get results.  Default is False.
  --dataset               STR         Data set to consider.                                     Default is 'WebKB/Cornell'.

Major model options for link prediction:

See file ./src/Edge_sparseMagnet.py for example.

  --epochs                INT         Number of (maximum) training epochs.                      Default is 1500. 
  --num_class_link        INT         Number of classes for link direction prediction(2 or 3).  Default is 2.
  --task                  INT         Task to conduct: 1 is existence prediction while 2 is
                                        link (direction) prediction, could be 2 or 3 classes.   Default is 2.

Reproduce results

First, get into the ./execution/ folder:

cd execution

To reproduce synthetic results on node classification on cyclic meta-graph structure.

bash cyclic.sh

To reproduce results on link prediction.

bash link_prediction.sh

Other execution files are similar to run.

Note that if you are operating on CPU, you may delete the commands ``CUDA_VISIBLE_DEVICES=xx". You can also set you own number of parallel jobs, not necessarily following the j numbers in the .sh files.

Direct execution with training files

First, get into the ./src/ folder:

cd src

Then, below are various options to try:

Creating a MagNet model of the default setting.

python ./sparse_Magnet.py

Creating an APPNP model for cyclic meta-graph synthetic data with seed 10 and do not load "best setting".

python ./APPNP.py --dataset syn/cyclic --seed 10 -NS

Creating a model for Telegram data set on DGCN with some custom learning rate and epoch number.

python ./Sym_DiGCN.py --dataset telegram --lr 0.001 --epochs 300

Creating a model on Digraph with inception block for link prediction on a binary classification problem of direction prediction.

python ./Digraph.py --method_name DiG_ib --task 2 --num_class_link 2

Generate Synthetic Graphs

cd src/utils

To generate cyclic DSBM graphs:

python generate_cyclic.py

To generate ordered DSBM graphs:

python generate_complete.py
python generate_syn2_3.py

To generate noisy cyclic DSBM graphs:

python generate_fill.py

Notes