Home

Awesome

Migrating Face Swap to Mobile Devices: A lightweight Framework and A Supervised Training Solution

Introduction

This is the repository for the source code of the papar
Migrating Face Swap to Mobile Devices: A lightweight Framework and A Supervised Training Solution

Accepted to IEEE International Conference on Multimedia and Expo 2022

[ICME 2022] [arXiv]

Framework overview

Selected results

Preferable configurations

torch==1.6.0

torchvision==0.7.0

cuda version: 10.2

How to use the code

Firstly, I strongly recommend implementing your own dataset class for loading training data, since my implemenation is based the structure of my data folder.

class FaceShifterDataset(torch.utils.data.TensorDataset):
    def __init__(self):
        super(FaceShifterDataset, self).__init__()
        # TODO: add your code

    def __getitem__(self, item):
        # TODO: add your code
        return Xs, Xt, GT, with_gt, src_as_true

    def __len__(self):
        # TODO: add your code
        return len(self.data_list)

Basically, the __getitem__ function returns five elements:

Note that images are with shape being (3, 256, 256) and values in (-1, 1). Images are in format torch.tensor with dtype being torch.float32.

After that, reset the arugments inside main.py and refer to run_training.sh. Note that my implementation uses DistributedDataParallel on 4 GPUs within one machine. Please adjust the settings based on your own hardware.

Performance

Qualitative performance

Comparisons of model size
ModelModel Size (MB)
FaceShifter (PyTorch)669.0 + 64.0 + 168.0
MegaFS (PyTorch)3529.0 + 364.0
SimSwap (PyTorch)766.9 + 220.2
Ours (Pytorch)10.2 + 4.9
Ours (CoreML)15.0
Ours (TNN)14.8

Citation

If you find my work useful, please cite:

@article{mobilefsgan,
author = {Haiming Yu and Hao Zhu and Xiangju Lu and Junhui Liu},
title = {Migrating Face Swap to Mobile Devices: A lightweight Framework and A Supervised Training Solution},
journal = {in ICME}, 
year = 2022
}