Awesome
DeepLab V3
Lei Mao, Shengjie Lin
University of Chicago
Toyota Technological Institute at Chicago
Introduction
DeepLab is a series of image semantic segmentation models, whose latest version, i.e. v3+, proves to be the state-of-art. Its major contribution is the use of atrous spatial pyramid pooling (ASPP) operation at the end of the encoder. While the model works extremely well, its open source code is hard to read (at least from my personal perspective). Here we re-implemented DeepLab V3, the earlier version of v3+ (which only additionally employs the decoder architecture), in a much simpler and more understandable way.
Dependencies
- Python 3.5
- TensorFlow 1.8
- Tqdm 4.26.0
- Numpy 1.14
- OpenCV 3.4.3
- Pillow 5.3.0
Files
.
├── archieved
├── download.py
├── feature_extractor.py
├── LICENSE.md
├── model.py
├── modules.py
├── nets
├── README.md
├── test_demo.py
├── test_any_image.py
├── train.py
└── utils.py
The nets
directory contains network definition files that are directly copied from tensorflow/models/research/slim/nets
Usages
Install Dependencies
To install dependencies, please run the following command to install everything required automatically:
$ chmod +x install_dependencies.sh
$ pip install -r requirements.txt
$ ./install_dependencies.sh
If found permission problems, please run the following command instead:
$ chmod +x install_dependencies.sh
$ pip install -r requirements.txt
$ sudo ./install_dependencies.sh
Download Dataset
Download and extract VOC2012 dataset, SBD dataset, and pretrained models to designated directories.
$ python download.py --help
usage: download.py [-h] [--downloads_dir DOWNLOADS_DIR] [--data_dir DATA_DIR]
[--pretrained_models_dir PRETRAINED_MODELS_DIR]
[--pretrained_models PRETRAINED_MODELS [PRETRAINED_MODELS ...]]
Download DeepLab semantic segmentation datasets and pretrained backbone
models.
optional arguments:
-h, --help show this help message and exit
--downloads_dir DOWNLOADS_DIR
Downloads directory
--data_dir DATA_DIR Data directory
--pretrained_models_dir PRETRAINED_MODELS_DIR
Pretrained models directory
--pretrained_models PRETRAINED_MODELS [PRETRAINED_MODELS ...]
Pretrained models to download: resnet_50, resnet_101,
mobilenet_1.0_224
For example, to download and extract datasets and models into directories specified:
$ python download.py --downloads_dir ./downloads --data_dir ./data --pretrained_models_dir ./models/pretrained --pretrained_models resnet_50 resnet_101 mobilenet_1.0_224
For simplicity, please just run the following command in terminal:
$ python download.py
Train Model
$ python train.py --help
usage: train.py [-h] [--network_backbone NETWORK_BACKBONE]
[--pre_trained_model PRE_TRAINED_MODEL]
[--trainset_filename TRAINSET_FILENAME]
[--valset_filename VALSET_FILENAME] [--images_dir IMAGES_DIR]
[--labels_dir LABELS_DIR]
[--trainset_augmented_filename TRAINSET_AUGMENTED_FILENAME]
[--images_augmented_dir IMAGES_AUGMENTED_DIR]
[--labels_augmented_dir LABELS_AUGMENTED_DIR]
[--model_dir MODEL_DIR] [--log_dir LOG_DIR]
[--random_seed RANDOM_SEED]
Train DeepLab V3 for image semantic segmantation.
optional arguments:
-h, --help show this help message and exit
--network_backbone NETWORK_BACKBONE
Network backbones: resnet_50, resnet_101,
mobilenet_1.0_224. Default: resnet_101
--pre_trained_model PRE_TRAINED_MODEL
Pretrained model directory
--trainset_filename TRAINSET_FILENAME
Train dataset filename
--valset_filename VALSET_FILENAME
Validation dataset filename
--images_dir IMAGES_DIR
Images directory
--labels_dir LABELS_DIR
Labels directory
--trainset_augmented_filename TRAINSET_AUGMENTED_FILENAME
Train augmented dataset filename
--images_augmented_dir IMAGES_AUGMENTED_DIR
Images augmented directory
--labels_augmented_dir LABELS_AUGMENTED_DIR
Labels augmented directory
--model_dir MODEL_DIR
Trained model saving directory
--log_dir LOG_DIR TensorBoard log directory
--random_seed RANDOM_SEED
Random seed for model training.
For simplicity, please run the following command in terminal:
$ python train.py
With learning rate of 1e-5
, the mIOU could be greater 0.7 after 20 epochs, which is comparable to the test statistics of DeepLab V3 in the publication.
Demos
To show some demos, please run the following command in terminal:
$ python test_demo.py
Image | Label | Prediction |
---|---|---|
Image | Label | Prediction |
---|---|---|
Image | Label | Prediction |
---|---|---|
Image | Label | Prediction |
---|---|---|
Custom Demo
Just put some JPG-format images into demo_dir
and run the following command in the terminal.
$ python test_any_image.py
Results will be written into same folder. Make sure that proper model trained and a checkpoint is saved in models_dir
. See the script for details.
Contributed by pinaxe1. Will modify to accept arguments and multiple image formats.
References
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Deeplab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. TPAMI, 2017.
L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam. Rethinking Atrous Convolution for Semantic Image Segmentation. arXiv:1706.05587, 2017.
L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, H. Adam. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. arXiv:1802.02611, 2018.
To-Do List
- Test script for new arbitrary test images.