Awesome

Mirror Gradient

By Shanshan Zhong and Zhongzhan Huang and Daifeng Li and Wushao Wen and Jinghui Qin and Liang Lin

This repository is the implementation of "Mirror Gradient: Towards Robust Multimodal Recommender Systems via Exploring Flat Local Minima" [paper]. Our paper has been accepted at the 2024 ACM Web Conference (WWW 2024).

🌻 Introduction

Multimodal recommender systems utilize various types of information, such as texts and images, to model user preferences and item features, helping users discover items aligned with their interests. However, it simultaneously magnifies certain risks from multimodal information inputs, such as inherent noise risk and information adjustment risk. To this end, we propose a concise gradient strategy Mirror Gradient (MG) that inverses the gradient signs appropriately during training to make the multimodal recommendation models approach flat local minima easier compared to models with normal training.

🌻 Quick Training

As a general gradient method of recommender systems, MG can be applied to the training of various recommendation models. The code of this repository is based on MMRec, a modern multimodal recommendation toolbox, and the core code of MG is in src/common/trainer.py. We also integrate MG to the modern multimodal recommendation toolbox MMRec

(1) Clone the code.

git clone https://github.com/Qrange-group/Mirror-Gradient
cd Mirror-Gradient

(2) Prepare the enviroment.

conda env create -f environment.yaml
conda activate mg
pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/cu118
pip install --no-index torch_cluster -f https://pytorch-geometric.com/whl/torch-2.0.1+cu118.html
pip install --no-index torch-scatter -f https://pytorch-geometric.com/whl/torch-2.0.1+cu118.html
pip install --no-index torch_sparse -f https://pytorch-geometric.com/whl/torch-2.0.1+cu118.html
pip install --no-index torch_spline_conv -f https://pytorch-geometric.com/whl/torch-2.0.1+cu118.html
pip install torch_geometric

(3) Prepare datasets.

See data. If you want to train models on other Amazon datasets, please see the processing tutorial.

(4) Train the model with MG.

cd src 
python main.py

You can change the model and dataset by the code,

python main.py --model DRAGON --dataset sports
python main.py --model LayerGCN --dataset sports

If you don't want to use MG during training, please use --not_mg. For example,

python main.py --not_mg

🌻 Supported Models and Datasets

• Models: BM3, DRAGON, DualGNN, FREEDOM, GRCN, ItemKNNCBF, LATTICE, LayerGCN, MGCN, MMGCN, MVGAE, SELFCFED_LGN, SLMRec, VBPR. See models' details in src/models. You also can add others recommendation models to src/models refering to the format of itemknncbf.py.

• Datasets: Amazon datasets. You can download the processed datasets baby, sports, clothing, and elec from GoogleDriver. If you want to train models on other Amazon datasets, please see the processing tutorial.

🌻 Checkpoints

For the evaluation of recommendation performance, we pay attention to top-5 accuracy as recommendations in the top positions of rank lists are more important, and adopt four widely used metrics including recall (REC), precision (PREC), mean average precision (MAP), and normalized discounted cumulative gain (NDCG).

You can resume the checkpoint by,

python main.py --model DRAGON --dataset sports --resume DRAGON-sports.pth

Sports

Elec

🌸 Citation

@inproceedings{zhong2024mirror,
  title={Mirror Gradient: Towards Robust Multimodal Recommender Systems via Exploring Flat Local Minima},
  author={Zhong, Shanshan and Huang, Zhongzhan and Li, Daifeng and Wen, Wushao and Qin, Jinghui and Lin, Liang},
  booktitle={Proceedings of the ACM on Web Conference 2024},
  pages={3700--3711},
  year={2024}
}

💖 Acknowledgments

Many thanks to enoche for their MMRec for multimodal recommendation task.