Awesome
[CVPR 2022] Eigencontours: Novel Contour Descriptors Based on Low-Rank Approximation (Oral)
Wonhui Park, Dongkwon Jin, and Chang-Su Kim
<img src="https://github.com/dnjs3594/Eigencontours/blob/master/overview.png" alt="overview" width="90%" height="90%" border="10"/>Official implementation for "Eigencontours: Novel Contour Descriptors Based on Low-Rank Approximation" [paper] [arxiv] [video]
Related work
We wil also present another paper, "Eigenlanes: Data-Driven Lane Descriptors for Structurally Diverse Lanes", accepted to CVPR 2022 [paper] [arxiv] [video] [github]. Congratulations to my eigenbrother!
Requirements
- PyTorch >= 1.4.0
- CUDA >= 10.0
- CuDNN >= 7.6.5
- python >= 3.6
Installation
- Download repository. We call this directory as
ROOT
:
$ git clone https://github.com/dnjs3594/Eigencontours.git
- Download preprocessed-data in
ROOT/Preprocessing/code_v1_COCO
:
$ cd ROOT/Preprocessing/code_v1_COCO
$ unzip data.zip
- Create conda environment:
$ conda create -n eigencontours python=3.7 anaconda
$ conda activate eigencontours
- Install dependencies:
$ conda install pytorch==1.4.0 torchvision==0.5.0 cudatoolkit=10.1 -c pytorch
$ pip install -r requirements.txt
-
Optional install dependencies: Our framework for instance segmentation is based on mmdetection. Please check mmdetection.md for installation instructions.
-
Download instance segmentation checkpoint
$ mkdir checkpoints
$ mv [download_root] checkpoints
Directory structure
. # ROOT
├── Preprocessing # directory for data preprocessing
│ ├── code_v1_COCO # COCO dataset preprocessing
│ ├── code_v1_SBD # SBD dataset preprocessing (TBD)
│ ├── data # dataset storage
│ │ ├── COCO # COCO dataset
│ │ │ ├──train2017
│ │ │ ├──val2017
│ │ │ └──annotations
│ └── ...
├── Instance_segmentation
│ ├── code_v1_COCO
│ │ ├── configs # configuration for instance segmentation
│ │ ├── mmdet # models
│ │ ├── mmdetection # installed folder
│ │ ├── tools # execution
└── └── ...
COCO dataset
Download COCO dataset in `ROOT/Preprocessing/data as directory structure:
Preprocessing (Construct contour descriptors called "Eigencontours".)
Data preprocessing is divided into three steps, which are "encoding", "svd", "convert". Below we describe each step in detail.
- In "encoding", star-convex contours are generated and saved to pickle format. you can set the dimensionality of the contours, N = node_num)
$ cd ROOT/Preprocessing/code_v1_COCO/
$ python main.py --mode encoding --node_num N (ex:360) --display False
- In "svd", a contour matrix is constucted and the eigencontours are generated by SVD (Singular Value Decomposition). you can get contour descriptors, matrix "U".
$ cd ROOT/Preprocessing/code_v1_COCO/
$ python main.py --mode svd --display False
- In "convert", coefficient vectors are generated by calculating between the star-convex contours and the eigencontours. Moreover, the F-scores of descriptors are also saved. you can set the dimensionality of coefficient vectors, M = rank-M approximation)
$ cd ROOT/Preprocessing/code_v1_COCO/
$ python main.py --mode encoding --dim M (ex:36) --display False
Optionally, you can adjust image size and threshold IoU (in "encoding") in ROOT/Preprocessing/code_v1_COCO/options/config.py
.
In paper, objects are cropped and centerally allgned for evaluating. If you want to preprocess data in crop version, you replace "datasets.dataset_coco_not_crop" with "dataset.dataset_coco" in ROOT/Preprocessing/code_v1_COCO/libs/prepare.py
.
Instance segmentation (PolarMask-based method using "Eigencontours".)[arxiv]
We only train models for ResNet50 backbone and single-scale training and single-gpu environment.
- Train
$ cd ROOT/Instance_segmentation/code_v1_COCO/tools
$ python train.py --work_dir <work_dir> --config <config_dir> --data_root <COCO_dir> --resume_from False
You can adjust more details in a config file. And, <work_dir> is a storage directory for checkpoint.
- Test
$ cd ROOT/Instance_segmentation/code_v1_COCO/tools
$ python test.py --work_dir <work_dir> --checkpoint <ckpt_dir> --config <config_dir> --data_root <COCO_dir> --show False
Reference
@Inproceedings{
park2022eigencontours,
title={Eigencontours: Novel Contour Descriptors Based on Low-Rank Approximation},
author={Park, Wonhui and Jin, Dongkwon and Kim, Chang-Su},
booktitle={CVPR},
year={2022}
}