Awesome

This repository contains a paper list related to Large Graph Models. Similar to Large Language Models (LLMs) for natural languages, we believe large graph models will revolutionaize graph machine learning with exciting opportunities for both researchers and practioners! For more details, please refer to our perspective paper: Graph Meets LLMs: Towards Large Graph Models

We will try our best to make this paper list updated. If you notice some related papers missing or have any suggestion, do not hesitate to contact us via pull requests at our repo.

Papers

Perspective and Survey

Overall

• [NeurIPS 2023 GLFrontiers Workshop] Graph Meets LLMs: Towards Large Graph Models [paper]
• [arXiv 2024.02] Future Directions in Foundations of Graph Machine Learning [paper]
• [arXiv 2024.02] Graph Foundation Models [paper]
• [arXiv 2023.12] Large Language Models on Graphs: A Comprehensive Survey [paper]
• [arXiv 2023.11] A Survey of Graph Meets Large Language Model: Progress and Future Directions [paper]
• [arXiv 2023.10] Towards Graph Foundation Models: A Survey and Beyond [paper]
• [arXiv 2023.10] Integrating Graphs with Large Language Models: Methods and Prospects [paper]

Prompt

• [arXiv 2024.02] Few-Shot Learning on Graphs: from Meta-learning to Pre-training and Prompting [paper]
• [arXiv 2023.11] Graph Prompt Learning: A Comprehensive Survey and Beyond [paper]
• [arXiv 2023.03] A Survey of Graph Prompting Methods: Techniques, Applications, and Challenges [paper]

Model

LLMs as Graph Models

2024

• [arXiv 2024.02] Rendering Graphs for Graph Reasoning in Multimodal Large Language Models [paper]
• [arXiv 2024.02] Exploring the Limitations of Graph Reasoning in Large Language Models [paper]
• [arXiv 2024.01] Efficient Tuning and Inference for Large Language Models on Textual Graphs [paper]
• [arXiv 2024.01] Scalable Link Prediction on Large-Scale Heterogeneous Graphs with Large Language Models [paper]
• [arXiv 2024.01] ChatGraph: Chat with Your Graphs [paper]

2023

• [NeurIPS 2023] Can Language Models Solve Graph Problems in Natural Language? [paper] [code]
• [NeurIPS 2023] WalkLM: A Uniform Language Model Fine-tuning Framework for Attributed Graph Embedding [paper]
• [EMNLP 2023] Unleashing the Power of Language Models in Text-Attributed Graph [paper]
• [ICML workshop 2023] Pretrained Language Models to Solve Graph Tasks in Natural Language [paper]
• [arXiv 2023.12] Efficient Large Language Models Fine-Tuning On Graphs [paper]
• [arXiv 2023.11] Large Language Models as Topological Structure Enhancers for Text-Attributed Graphs [paper]
• [arXiv 2023.10] LLM4DyG: Can Large Language Models Solve Problems on Dynamic Graphs? [paper]
• [arXiv 2023.10] Disentangled Representation Learning with Large Language Models for Text-Attributed Graphs [paper]
• [arXiv 2023.10] Empower Text-Attributed Graphs Learning with Large Language Models (LLMs) [paper]
• [arXiv 2023.10] GraphLLM: Boosting Graph Reasoning Ability of Large Language Model [paper] [code]
• [arXiv 2023.10] Beyond Text: A Deep Dive into Large Language Models’ Ability on Understanding Graph Data [paper]
• [arXiv 2023.10] Label-free Node Classification on Graphs with Large Language Models (LLMS) [paper] [code]
• [arXiv 2023.10] Talk like a Graph: Encoding Graphs for Large Language Models [paper]
• [arXiv 2023.10] GraphText Graph Reasoning in Text Space [paper]
• [arXiv 2023.10] One for All Towards Training One Graph Model for All Classification Tasks [paper] [code]
• [arXiv 2023.09] Can LLMs Effectively Leverage Structural Information for Graph Learning: When and Why [paper] [code]
• [arXiv 2023.08] Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysis [paper] [code]
• [arXiv 2023.08] Natural Language is All a Graph Needs [paper]
• [arXiv 2023.08] SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning [paper] [code]
• [arXiv 2023.07] Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs [paper] [code]
• [arXiv 2023.05] Explanations as Features LLM-Based Features for Text-Attributed Graphs [paper] [code]
• [arXiv 2023.05] GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarking [paper]

Graph Prompts

2024

• [WWW 2024] MultiGPrompt for Multi-Task Pre-Training and Prompting on Graphs [paper]
• [AAAI 2024] HGPROMPT: Bridging Homogeneous and Heterogeneous Graphs for Few-shot Prompt Learning [paper]
• [AAAI 2024] Measuring task similarity and its implication in fine-tuning graph neural networks [paper] [code]
• [KBS 2024] GPL-GNN: Graph prompt learning for graph neural network [paper] [code]
• [IPM 2024] G-Prompt: Graphon-based Prompt Tuning for graph classification [paper]

2023

• [NeurIPS 2023] PRODIGY: Enabling In-context Learning Over Graphs [paper] [code]
• [NeurIPS 2023] Universal Prompt Tuning for Graph Neural Networks [paper]
• [MM 2023] Chain of Propagation Prompting for Node Classification [paper]
• [CVPR 2023] Deep Graph Reprogramming [paper]
• [KDD 2023] All in One: Multi-Task Prompting for Graph Neural Networks [paper] [code]
• [WWW 2023] GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networks [paper] [code]
• [WWW 2023] Structure Pre-training and Prompt Tuning for Knowledge Graph Transfer [paper] [code]
• [KDD 2022] GPPT: Graph Pre-training and Prompt Tuning to Generalize Graph Neural Networks [paper]
• [arXiv 2023.11] Generalized Graph Prompt: Toward a Unification of Pre-Training and Downstream Tasks on Graphs [paper] [code]
• [arXiv 2023.10] Enhancing Graph Neural Networks with Structure-Based Prompt [paper]
• [arXiv 2023.10] HetGPT: Harnessing the Power of Prompt Tuning in Pre-Trained Heterogeneous Graph Neural Networks [paper]
• [arXiv 2023.10] ULTRA-DP: Unifying Graph Pre-training with Multi-task Graph Dual Prompt [paper]
• [arXiv 2023.10] Prompt Tuning for Multi-View Graph Contrastive Learning [paper]
• [arXiv 2023.09] Prompt-based Node Feature Extractor for Few-shot Learning on Text-Attributed Graphs [paper]
• [arXiv 2023.09] Deep Prompt Tuning for Graph Transformers [paper]
• [arXiv 2023.07] Prompt-Based Zero- and Few-Shot Node Classification: A Multimodal Approach [paper]
• [arXiv 2023.02] SGL-PT: A Strong Graph Learner with Graph Prompt Tuning [paper]

Graph Parameter-efficient Fine-tuning

• [AAAI 2024] G-Adapter: Towards Structure-Aware Parameter-Efficient Transfer Learning for Graph Transformer Networks [paper]
• [AAAI 2024] AdapterGNN: Efficient Delta Tuning Improves Generalization Ability in Graph Neural Networks [paper]
• [arXiv 2023.08] Search to Fine-tune Pre-trained Graph Neural Networks for Graph-level Tasks [paper]

Applications

Knowledge Graph

• [arXiv 2023.08] Large Language Models and Knowledge Graphs: Opportunities and Challenges [paper]
• [arXiv 2023.06] Unifying Large Language Models and Knowledge Graphs: A Roadmap [paper]

Molecules

• [NeurIPS 2023] GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learning [paper] [code]
• [arXiv 2023.09] DrugChat: Towards Enabling ChatGPT-Like Capabilities on Drug Molecule Graphs [paper] [code]
• [arXiv 2023.08] GIT-Mol A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Text [paper]
• [arXiv 2023.07] Can Large Language Models Empower Molecular Property Prediction? [paper] [code]

Neural Architecture Search

• [arXiv 2023.12] Heterogeneous Graph Neural Architecture Search with GPT-4 [paper]
• [arXiv 2023.10] Graph Neural Architecture Search with GPT-4 [paper]
• [arXiv 2023.09] Unleashing the Power of Graph Learning through LLM-based Autonomous Agents [paper]
• [arXiv 2023.02] EvoPrompting: Language Models for Code-Level Neural Architecture Search [paper] [code]

Miscellaneous

• [arXiv 2023.10] AUTOPARLLM: GNN-Guided Automatic Code Parallelization using Large Language Models [paper]
• [arXiv 2023.09] VulnSense: Efficient Vulnerability Detection in Ethereum Smart Contracts by Multimodal Learning with Graph Neural Network and Language Model [paper]
• [arXiv 2023.08] FoodGPT A Large Language Model in Food Testing Domain with Incremental Pre-training and Knowledge Graph Prompt [paper]
• [arXiv 2023.06] ChatGPT Informed Graph Neural Network for Stock Movement Prediction [paper]
• [arXiv 2023.05] Graph Meets LLM A Novel Approach to Collaborative Filtering for Robust Conversational Understanding [paper]

Graphs for LLMs

Graph of Thoughts

• [arXiv 2023.08] MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Models [paper]
• [arXiv 2023.08] Graph of Thoughts: Solving Elaborate Problems with Large Language Models [paper] [code]
• [arXiv 2023.08] Enhancing Reasoning Capabilities of Large Language Models: A Graph-Based Verification Approach [paper]
• [arXiv 2023.08] Boosting Logical Reasoning in Large Language Models through a New Framework: The Graph of Thought [paper]
• [arXiv 2023.08] Thinking Like an Expert: Multimodal Hypergraph-of-Thought (HoT) Reasoning to boost Foundation Modals [paper]
• [arXiv 2023.07] Think-on-Graph: Deep and Responsible Reasoning of Large Language Model with Knowledge Graph [paper]
• [arXiv 2023.05] Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Large Language Model [paper]

Graph as Tools

• [arXiv 2023.05] StructGPT: A General Framework for Large Language Model to Reason over Structured Data [paper] [code]
• [arXiv 2023.04] Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPT [paper] [code]

Cite

Please consider citing our perspective paper if you find this repository helpful:

@article{zhang2023large,
  title={Graph Meets LLMs: Towards Large Graph Models},
  author={Zhang, Ziwei and Li, Haoyang and Zhang, Zeyang and Qin, Yijian and Wang, Xin and Zhu, Wenwu},
  journal={NeurIPS 2023 GLFrontiers Workshop},
  year={2023}
}