Home

Awesome

MVSS-Net

Code and models for ICCV 2021 paper: Image Manipulation Detection by Multi-View Multi-Scale Supervision

Image text

Update

We now have an improved version of MVSS-Net, denoted as MVSS-Net++. Check here.

Environment

Requirements

Usage

Dataset

An example of the dataset index file is given as data/CASIAv1plus.txt, where each line contains:

img_path mask_path label
Training sets
Test sets

<span id = "jump">Trained Models</span>

We offer FCNs and MVSS-Nets trained on CASIAv2 and DEFACTO_84k, respectively. Please download the models and place them in the ckpt directory:

The performance of these models for image-level manipulation detection (metric: AUC and image-level F1) is as follows. More details are reported in the paper.

Performance metric: AUC
ModelTraining dataCASIAv1plusColumbiaCOVERDEFACTO-12k
MVSS_NetCASIAv20.9320.9800.7310.573
MVSS_NetDEFACTO-84k0.7710.5630.5250.886
FCNCASIAv20.7690.7620.5410.551
FCNDEFACTO-84k0.6290.5350.5430.840
Performance metric: Image-level F1 (threshold=0.5)
ModelTraining dataCASIAv1plusColumbiaCOVERDEFACTO-12k
MVSS_NetCASIAv20.7590.8020.2440.404
MVSS_NetDEFACTO-84k0.6850.3530.3600.799
FCNCASIAv20.6840.4810.1800.458
FCNDEFACTO-84k0.5610.4920.5110.709

Inference & Evaluation

You can specify which pre-trained model to use by setting model_path in do_pred_and_eval.sh. Given a test_collection (e.g. CASIAv1plus or DEFACTO12k-test), the prediction maps and evaluation results will be saved under save_dir. The default threshold is set as 0.5.

bash do_pred_and_eval.sh $test_collection
#e.g. bash do_pred_and_eval.sh CASIAv1plus

For inference only, use following command to skip evaluation:

bash do_pred.sh $test_collection
#e.g. bash do_pred.sh CASIAv1plus

Demo

Citation

If you find this work useful in your research, please consider citing:

@InProceedings{MVSS_2021ICCV,  
  author = {Chen, Xinru and Dong, Chengbo and Ji, Jiaqi and Cao, juan and Li, Xirong},  
  title = {Image Manipulation Detection by Multi-View Multi-Scale Supervision},  
  booktitle = {The IEEE International Conference on Computer Vision (ICCV)},  
  year = {2021}  
}

@ARTICLE{MVSS_2022TPAMI,
  author={Dong, Chengbo and Chen, Xinru and Hu, Ruohan and Cao, Juan and Li, Xirong},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence}, 
  title={MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection}, 
  year={2022},
  volume={},
  number={},
  pages={1-14},
  doi={10.1109/TPAMI.2022.3180556}
}

Acknowledgments

Contact

If you enounter any issue when running the code, please feel free to reach us either by creating a new issue in the github or by emailing