Awesome
pytorch-deep-image-matting
This repository includes the non-official pytorch implementation of deep image matting.
Performance
| model | SAD ↓ | MSE ↓ | Grad ↓ | Conn ↓ | link |
|---|---|---|---|---|---|
| stage0-paper | 59.6 | 0.019 | 40.5 | 59.3 | |
| stage1-paper | 54.6 | 0.017 | 36.7 | 55.3 | |
| stage0-our | 56.01 | 0.0173 | 33.71 | 57.57 | |
| stage1-our | 54.42 | 0.0175 | 35.01 | 54.85 | download |
| stage1-our-skip | 52.99 | 0.0171 | 31.56 | 53.24 | download |
- Lower metrics show better performance.
- Training batch=1, images=43100, epochs=12 for stage1-our-skip model, it takes about 1 days.
- Test maxSize=1600.
- GPU memory >= 10GB
Updates
- 2020.09.09: run the demo.py with the latest model (stage1-skip-sad-52.9.pth), and update the visualization results.
- 2020.09.09: adopt VGG-16 backbone with skip connection, and we get better performance with less training costs (12 epochs), Get Stage1-Skip-SAD=52.9
- 2019.10.29: conduct stage0 experiment using current code. Get Stage0-SAD=56.0.
- 2019.09.09: conv6 kernel size from 1x1 to 3x3. Get Stage1-SAD=54.4. The performance of stage1 is as good as paper. While using model released before this day, please change the kernel_size=1 and padding=0 of conv6 in file core/net.py.
- 2019.08.24: Fix cv2.dilate and cv2.erode iterations is set default = 1 and set triamp dilate and erode as the test 1k tirmap (k_size:2-5, iterations:5-15). Get Stage1-SAD=57.1.
- 2019.07.05: Training with refine stage, fixed encoder-decoder. Get Stage2-SAD=57.7.
- 2019.06.23: Training with alpha loss and composite loss. Get Stage1-SAD=58.7.
- 2019.06.17: Training trimap generated by erode as well as dialte to balance the 0 and 1 value. Get Stage0-SAD=62.0.
- 2019.04.22: Input image is normalized by mean=[0.485, 0.456, 0.406] and std=[0.229, 0.224, 0.225] and fix crop error. Get Stage0-SAD=69.1.
- 2018.12.14: Initial. Get Stage0-SAD=72.9.
Installation
- Python 2.7.12 or 3.6.5
- Pytorch 0.4.0 or 1.0.0
- OpenCV 3.4.3
Demo
Download our model to the ./model and run the following command. Then the predict alpha mattes will locate in the folder ./result/example/pred.
python core/demo.py
Training
Adobe-Deep-Image-Matting-Dataset
Please concat author for available.
MSCOCO-2017-Train-Dataset
PASCAL-VOC-2012
Composite-Dataset
Run the following command and the composite training and test dataset will locate in Combined_Dataset/Training_set/comp and Combined_Dataset/Test_set/comp, Combined_Dataset is the extracted folder of Adobe-Deep-Image-Matting-Dataset
python tools/composite.py
Pretrained-Model
Run the following command and the pretrained model will locate in ./model/vgg_state_dict.pth
python tools/chg_model.py
Start Training
Run the following command and start the training
bash train.sh
Test
Run the following command and start the test of Adobe-1k-Composite-Dataset
bash deploy.sh
Evaluation
Please eval with official Matlab Code. and get the SAD, MSE, Grad Conn.
Visualization
Running model is Stage1-Skip-SAD=52.9, please click to view whole images.
| Image | Trimap | Pred-Alpha | GT-Alpha |
|---|---|---|---|
 |  |  |  |
 |  |  |  |
 |  |  |  |
 |  |  |  |
 |  |  |  |
Disclaimer
As covered by the ADOBE IMAGE DATASET LICENSE AGREEMENT, the pre-trained models included in this repository can only be used and distributed for non-commercial purposes.