Awesome

Awesome-Multimodal-Learning-on-Graphs

A curated list of papers and resources about multimodal learning on graphs. The models we focus here are mainly large multimodal models and diffusion models.

An awesome repo on large language models (LLMs) on graphs an be found here.

This repo will be continuously updated. Don't forget to star <img src="./fig/star.svg" width="15" height="15" /> it and keep tuned!

Please cite the paper in Citations if you find the resource helpful for your research. Thanks!

Benchmark

1. Multimodal Graph Benchmark. preprint

   Jing Zhu, Yuhang Zhou, Shengyi Qian, Zhongmou He, Tong Zhao, Neil Shah, Danai Koutra [PDF] [Code] [PDF] 2024.6

2. When Graph meets Multimodal: Benchmarking on Multimodal Attributed Graphs Learning. preprint

   Hao Yan, Chaozhuo Li, Zhigang Yu, Jun Yin, Ruochen Liu, Peiyan Zhang, Weihao Han, Mingzheng Li, Zhengxin Zeng, Hao Sun, Weiwei Deng, Feng Sun, Qi Zhang, Senzhang Wang [PDF] [Code] 2024.10

3. Benchmarking and Improving Large Vision-Language Models for Fundamental Visual Graph Understanding and Reasoning. preprint

   Yingjie Zhu, Xuefeng Bai, Kehai Chen, Yang Xiang, Min Zhang [PDF] [Code] 2024.12

Reasoning

1. VisionGraph: Leveraging Large Multimodal Models for Graph Theory Problems in Visual Context. ICML2024

   Yunxin Li, Baotian Hu, Haoyuan Shi, Wei Wang, Longyue Wang, Min Zhang [PDF], 2024.5

2. Multimodal LLMs Struggle with Basic Visual Network Analysis: a VNA Benchmarkt. preprint

   Evan M. Williams, Kathleen M. Carley [PDF], 2024.5

Image Generation

1. SG-Adapter: Enhancing Text-to-Image Generation with Scene Graph Guidance. ICML2024

   Guibao Shen, Luozhou Wang, Jiantao Lin, Wenhang Ge, Chaozhe Zhang, Xin Tao, Yuan Zhang, Pengfei Wan, Zhongyuan Wang, Guangyong Chen, Yijun Li, Ying-Cong Chen [PDF], 2024.5

2. InstructG2I: Synthesizing Images from Multimodal Attributed Graphs. NeurIPs2024

   Bowen Jin, Ziqi Pang, Bingjun Guo, Yu-Xiong Wang, Jiaxuan You, Jiawei Han [PDF], 2024.10

Citations

Please cite the following paper if you find the resource helpful for your research.

@article{jin@llmgraph,
  title={Large Language Models on Graphs: A Comprehensive Survey},
  author={Jin, Bowen and Liu, Gang and Han, Chi and Jiang, Meng and Ji, Heng and Han, Jiawei},
  journal={arXiv preprint arXiv:2312.02783},
  year={2023}
}