Home

Awesome

<div align="center"> <img src="https://github.com/wangxiao5791509/MultiModal_BigModels/blob/main/figures/MM_PTMs.png" width="1000px"> </div>

This github will be continuously updated for the survey paper:

<div align="center">

Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey, Xiao Wang, Guangyao Chen, Guangwu Qian, Pengcheng Gao, Xiao-Yong Wei, Yaowei Wang, Yonghong Tian, Wen Gao. [arXiv] [MIR] [极市平台公众号] [机器智能研究MIR(MIR编辑部)] [Machine Intelligence Research (Youtube)]

</div>

News

Framework of this survey

<img src="https://github.com/wangxiao5791509/MultiModal_BigModels/blob/main/figures/framework.png" width="1000px"> <img src="https://github.com/wangxiao5791509/MultiModal_BigModels/blob/main/figures/milestone.jpg" width="1000px">

Review and Surveys

Please check this file [Surveys.md]

Datasets

Please check this file [Datasets.md]

Publications

Please check this file [paperList.md]

Experimental Analysis

<img src="https://github.com/wangxiao5791509/MultiModal_BigModels/blob/main/figures/experimentResults.png" width="1000px"> <img src="https://github.com/wangxiao5791509/MultiModal_BigModels/blob/main/figures/modelsGPUsParmas.png" width="1000px">

Other Useful Materials

:page_with_curl: BibTex:

If you find this survey useful for your research, please cite the following papers:

@article{wang2022MMPTMSurvey,
  title={Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey},
  author={Wang, Xiao and Chen, Guangyao and Qian, Guangwu and Gao, Pengcheng and Wei, Xiao-Yong and Wang, Yaowei and Tian, Yonghong and Gao, Wen},
  url={https://github.com/wangxiao5791509/MultiModal_BigModels_Survey},
  year={2022}
}

If you have any questions about this survey, please email me via: xiaowang@ahu.edu.cn or wangxiaocvpr@foxmail.com