Home

Awesome

ReFit

Code repository for the paper:
ReFit: Recurrent Fitting Network for 3D Human Recovery
Yufu Wang, Kostas Daniilidis
ICCV 2023
[Project Page]

<img src="data/method.png" width="700">

Updates

2024-01-05: Added an example for multiview refit. <br> 2023-12-20: Added support for GART with a preprocessing script.<br>

Installation instructions

python3 -m venv refit
source refit/bin/activate
pip install -U pip
pip install -r requirements.txt

Prepare data

There are some few key data you need for the demo, training and evaluation.

  1. SMPL: Please download <strong>SMPL_NEUTRAL/FEMALE/MALE.pkl</strong> and put them under data/smpl
  2. ReFit weights: download each checkpoint folders and put them under data/pretrain
  3. Yolov7: for detection, download this Yolov7 checkpoint and put it under data/pretrain. Be sure to also clone the yolov7 submodule with git clone --recurse-submodules this-repo.

Additionally for training and evaluation, please follow the dataset preparation guide here.

Demo

We provide a few examples in data/examples. Running the following demo will detect each person in each image and then run ReFit for 3D reconstruction. The resulting rendering will be saved.

python demo.py

For multiview refit, run the following demo

python scripts/multiview_demo.py

To run on a folder of images (eg. frames from a video), and save the SMPL and camera pose for GART, run the following.

python scripts/process_folder.py

Training and evaluation

Training: <strong>config.yaml</strong>, <strong>config_bedlam.yaml</strong> and <strong>config_all.yaml</strong> control the dataset composition.

python train.py --cfg configs/config.yaml

Evaluation:

python scripts/eval.py

Acknowledgements

We benefit greatly the following repos, from which we adapted parts of our code.

Citing

If you find the model and code useful, please consider citing the following paper:

@Inproceedings{wang23refit,
  Title          = {ReFit: Recurrent Fitting Network for 3D Human Recovery},
  Author         = {Yufu Wang and Kostas Daniilidis},
  Booktitle      = {International Conference on Computer Vision},
  Year           = {2023}
}