Home

Awesome

<h1 align='center'>AnyGraph: Graph Foundation Model in the Wild</h1> <div align='center'> <a href='https://arxiv.org/pdf/2408.10700'><img src='https://img.shields.io/badge/Paper-PDF-green'></a> <!-- <a href=''><img src='https://img.shields.io/badge/公众号-blue' /></a> --> <!-- <a href=''><img src='https://img.shields.io/badge/CSDN-orange' /></a> --> <img src="https://badges.pufler.dev/visits/hkuds/anygraph?style=flat-square&logo=github"> <img src='https://img.shields.io/github/stars/hkuds/anygraph?color=green&style=social' />

<a href='https://akaxlh.github.io/'>Lianghao Xia</a> and <a href='https://sites.google.com/view/chaoh/group-join-us'>Chao Huang</a>

Introducing AnyGraph, a graph foundation model designed for zero-shot predictions across domains.

<img src='imgs/article cover.png' /> </div>

Objectives of AnyGraph:

<br>

Key Features of AnyGraph:

<img src='imgs/framework_final.jpeg' />

News

Environment Setup

Download the data files at <a href='https://huggingface.co/datasets/hkuds/AnyGraph_datasets'>this link</a>. And fill in your own directories for data storage at function get_data_files(self) of class DataHandler in the file data_handler.py.

Download the pre-trained AnyGraph models at <a href='https://huggingface.co/hkuds/AnyGraph/'>hugging face</a> or <a href='https://hkuhk-my.sharepoint.com/:u:/g/personal/lhaoxia_hku_hk/Efmm5TJm0B5EnmYzTqg8GWEB1loKzeIR5tcr3hPIOJDXXA?e=2wMgZC'>one drive</a>, and put it into Models/.

Packages: Our experiments were conducted with the following package versions:

Device Requirements: The training and testing of AnyGraph requires only one GPU with 24G memory (e.g. 3090, 4090). Using larger input graphs may require devices with larger memory.

Code Structure

Here is a brief overview of the code structures. The explanations for each directory are enclosed in quotes (##...##).

./
│   ├── .gitignore
│   ├── README.md
│   ├── data_handler.py ## load and process data
│   ├── model.py ## implementation for the model
│   ├── params.py ## hyperparameters
│   └── main.py ## main file for pretraining and link prediction
│   ├── History/ ## training and testing logs
│   │   ├── pretrain_link1.his
│   │   └── pretrain_link2.his
│   ├── Models/ ## pre-trained models
│   │   └── README.md
│   ├── Utils/ ## utility function
│   │   └── TimeLogger.py
│   ├── imgs/ ## images used in readme
│   │   ├── ablation.png
│   │   ├── article cover.png
│   │   ├── datasets.png
│   │   ├── framework_final.jpeg
│   │   ├── framework_final.pdf
│   │   ├── framework_final.png
│   │   ├── overall_performance2.png
│   │   ├── routing.png
│   │   ├── scaling_law.png
│   │   ├── training_time.png
│   │   ├── tuning_steps.png
│   │   └── overall_performance1.png
│   ├── node_classification/ ## test code for node classification
│   │   ├── data_handler.py
│   │   ├── model.py
│   │   ├── params.py
│   │   └── main.py
│   │   ├── Utils/
│   │   │   └── TimeLogger.py

Usage

To reproduce the test performance reported in the paper, run the following command lines:

# Test on Link2 and Link1 data, respectively
python main.py --load pretrain_link1 --epoch 0 --dataset link2 
python main.py --load pretrain_link2 --epoch 0 --dataset link1

# Test on the Ecommerce datasets in the Link2 and Link1 group, respectively.
# Testing on Academic and Others datasets are conducted similarily.
python main.py --load pretrain_link1 --epoch 0 --dataset ecommerce_in_link2
python main.py --load pretrain_link2 --epoch 0 --dataset ecommerce_in_link1

# Test the performance for node classification datasets
cd ./node_classification
python main.py --load pretrain_link2 --epoch 0 --dataset node

To re-train the two models by yourself, run:

python main.py --dataset link2+link1 --save pretrain_link2
python main.py --dataset link1+link2 --save pretrain_link1

Datasets

<img src='imgs/datasets.png' />

The statistics for the experimental datasets are presented in the table above. We categorize them into distinct groups as below. Note that Link1 and Link2 include datasets from different sources, and the datasets do not share the same feature spaces. This separation ensures a robust evaluation of true zero-shot performance in graph prediction tasks.

GroupIncluded Datasets
Link1Products-tech, Yelp2018, Yelp-textfeat, Products-home, Steam-text, Amazon-text, Amazon-book, Citation-2019, Citation-20Century, Pubmed-link, Citeseer, OGB-PPA, P2P-Gnutella06, Soc-Epinions1, Email-Enron
Link2Photo, Goodreads, Fitness, Movielens-1M, Movielens10M, Gowalla, Arxiv, Arxiv-t, Cora, CS, OGB-Collab, Proteins-0, Proteins- 1, Proteins-2, Proteins-3, OGB-DDI, Web-Stanford, RoadNet-PA
EcommerceProducts-tech, Yelp2018, Yelp-textfeat, Products-home, Steam-text, Amazon-text, Amazon-book, Photo, Goodreads, Fitness, Movielens-1M, Movielens10M, Gowalla
AcademicCitation-2019, Citation-20Century, Pubmed-link, Citeseer, OGB-PPA, Arxiv, Arxiv-t, Cora, CS, OGB-Collab
OthersP2P-Gnutella06, Soc-Epinions1, Email-Enron, Proteins-0, Proteins- 1, Proteins-2, Proteins-3, OGB-DDI, Web-Stanford, RoadNet-PA
NodeCora, Arxiv, Pubmed, Home, Tech

Experiments

Model Pre-Training Curves

We present the training logs with respect to epochs below. Each figure contains two curves, each corresponding to two instances of repeated pre-training.

<img src='imgs/link1_loss_curve.png' width=32%/>  <img src='imgs/link1_fullshot_ndcg_curve.png' width=32%/>  <img src='imgs/link1_zeroshot_ndcg_curve.png' width=32%/>

<img src='imgs/link2_loss_curve.png' width=32%/>  <img src='imgs/link2_fullshot_ndcg_curve.png' width=32%/>  <img src='imgs/link2_zeroshot_ndcg_curve.png' width=32%/>

Overall Performance Comparison

Scaling Law of AnyGraph

We explore the scaling law of AnyGraph by evaluating 1) model performance v.s. the number of model parameters, and 2) model performance v.s. the number of training samples.

Below shows the evaluation results on

In each subfigure, we show

The outcome outlines the following key observations: (see Sec. 4.3 for details)

Ablation Study

The ablation study investigates the impact of the following modules:

<img src='imgs/ablation.png' width=60% />

Expert Routing Mechanism

We visualize the competence scores between datasets and experts, given by the routing algorithm of AnyGraph.

The resulting scores below demonstrates the underlying relatedness between different datasets, thus demonstrating the intuitive effectiveness of the routing mechanism. (see Sec. 4.5 for details)

<img src='imgs/routing.png' width=60% />

Fast Adaptation of AnyGraph

We study the fast adaptation abilities of AnyGraph from two aspects:

<img src='imgs/tuning_steps.png' width=60% /> <img src='imgs/training_time.png' width=60% />

Citation

If you find our work useful, please consider citing our paper:

@article{xia2024anygraph,
  title={AnyGraph: Graph Foundation Model in the Wild},
  author={Xia, Lianghao and Huang, Chao},
  journal={arXiv preprint arXiv:2408.10700},
  year={2024}
}