Awesome
Panoptic Feature Pyramid Networks
This is an unofficial implementation of Panoptic-FPN in a gluon-cv style, we implemented this framework in a fully Gluon API, please stay tuned!
Main Results
Cityscapes
- panoptic_fpn_resnet50_v1b_citys
- | PQ | SQ | RQ | N |
---|---|---|---|---|
All | 55.4 | 77.9 | 69.3 | 19 |
Things | 52.4 | 78.1 | 66.6 | 8 |
Stuff | 57.6 | 77.7 | 71.2 | 11 |
Installation
- Install cuda
10.0
and mxnet1.4.0
.
sudo pip3 install mxnet-cu100==1.4.0.post0
- Clone the code, and install gluoncv with
setup.py
.
cd panoptic-fpn-gluon
sudo python3 setup.py build
sudo python3 setup.py install
Preparation
Cityscapes
- Download
Cityscapes
datasets follow the official tutorials and create a soft link.
ln -s $DOWNLOAD_PATH ~/.mxnet/datasets/citys
You can also download from Cityscapes and execute the command above.
- Create Panoptic images for training and Inference, the code can be found here. Then execute the command below:
python3 createPanopticImgs.py --dataset-folder ~/.mxnet/datasets/citys/gtFine/ --output-folder ~/.mxnet/datasets/citys/gtFine/
Note that the correct data structure is shown below:
$ ls ~/.mxnet/datasets/citys
├── gtFine
│ ├── train/
│ ├── val/
│ ├── test/
│ ├── cityscapes_panoptic_train/
│ ├── cityscapes_panoptic_val/
│ ├── cityscapes_panoptic_test/
│ ├── cityscapes_panoptic_train.json
│ └── cityscapes_panoptic_val.json
├── leftImg8bit
│ ├── train/
│ ├── val/
│ └── test/
-
More preparations can also refer to GluonCV.
-
All experiments are performed on
8 * 2080ti
GPU withPython3.5
,cuda10.0
andcudnn7.5.0
.
COCO
- TODO
Structure
* Model : $ROOT/gluoncv/model_zoo/panoptic/
* Train & valid scripts : $ROOT/scripts/panoptic/
* Metric : $ROOT/gluoncv/utils/metric/
Training & Inference
Cityscapes
- Clone the training scripts here, then train
panoptic_fpn_resnet50_v1b_citys
with:
python3 train_panoptic_fpn.py --network resnet50_v1b --use-fpn --gpus 0,1,2,3,4,5,6,7 --num-workers 32 --log-interval 10 --save-interval 20 --val-interval 10 --epochs 700 --lr-decay-epoch 430,590 --lr-warmup 1600
Note that we follow the training settings described in original paper.
- Clone the validation scripts here, then validate
panoptic_fpn_resnet50_v1b_citys
with:
python3 eval_panoptic_fpn.py --network resnet50_v1b --gpus 0,1,2,3,4,5,6,7 --pretrained ./XXX.params
COCO
- TODO
Reference
- Panoptic FPN: Alexander Kirillov, Ross Girshick, Kaiming He, Piotr Dollár.<br />"Panoptic Feature Pyramid Networks." CVPR (2019 oral). [paper]