Awesome

Holistically-Attracted Wireframe Parsing: From Supervised Learning to Self-Supervised Learning

This is the official implementation of our paper.

[News] Our journal version has been accepted in PAMI!

[News] The upgraded HAWPv2 and HAWPv3 are available now!

Highlights

• HAT Fields: A General and Robust Representation of Line Segments for Wireframe Parsing
• HAWPv2: A state-of-the-art fully-supervised wireframe parser. Please checkout HAWPv2.md for its details.
• HAWPv3: A state-of-the-art self-supervised wireframe parser. Please checkout HAWPv3.md for its details and HAWPv3.train.md for the training recipe.
• HAWPv3 can be used as a good wireframe parser for the out-of-distribution images.

• We provide a running example on the images of DTU dataset (scene24) as below.

  python -m hawp.ssl.predict --ckpt checkpoints/hawpv3-imagenet-03a84.pth  \
    --threshold 0.05  \
    --img ~/datasets/DTU/scan24/image/*.png \ 
    --saveto docs/figures/dtu-24 --ext png \
  

  <p align="center"> <img src="docs/figures/dtu-24/000000.png" width="30%" height=auto> <img src="docs/figures/dtu-24/000001.png" width="30%" height=auto> <img src="docs/figures/dtu-24/000002.png" width="30%" height=auto> </p> <p align="center"> <img src="docs/figures/dtu-24/000003.png" width="30%" height=auto> <img src="docs/figures/dtu-24/000004.png" width="30%" height=auto> <img src="docs/figures/dtu-24/000005.png" width="30%" height=auto> </p> <p align="center"> <img src="docs/figures/dtu-24/000009.png" width="30%" height=auto> <img src="docs/figures/dtu-24/000015.png" width="30%" height=auto> <img src="docs/figures/dtu-24/000045.png" width="30%" height=auto> </p>

Data Downloading

• The training and testing data (including Wireframe dataset and YorkUrban dataset) for HAWPv2 can be downloaded via Google Drive. Many thanks to authors of these two excellent datasets!

• You can also use the gdown to download the data in the terminal by

  gdown 134L-u9pgGtnzw0auPv8ykHqMjjZ2claO
  unzip data.zip
  

Installation

• Clone the code repo: git clone https://github.com/cherubicXN/hawp.git.
• Install ninja-build by sudo apt install ninja-build.
• Create a conda environment by

conda create -n hawp python==3.9
conda activate hawp
pip install -e .

• Run the following command lines to install the dependencies of HAWP

# Install pytorch, please be careful for the version of CUDA on your machine
pip install torch==1.12.0+cu116 torchvision==0.13.0+cu116 torchaudio==0.12.0 --extra-index-url https://download.pytorch.org/whl/cu116 
# Install other dependencies
pip install -r requirement.txt

• Verify the installation.

python -c "import torch; print(torch.cuda.is_available())" # Check if the installed pytorch supports CUDA.

• Downloading the offically-trained checkpoints of both HAWPv2 and HAWPv3.

sh downloads.sh

We also provide a Dockerfile. You could build the docker image by running the following command lines.

sudo docker build - < Dockerfile --tag hawp:latest

Citations

If you find our work useful in your research, please consider citing:

@article{HAWP-journal,
title = "Holistically-Attracted Wireframe Parsing: From Supervised to Self-Supervised Learning",
author = "Nan Xue and Tianfu Wu and Song Bai and Fu-Dong Wang and Gui-Song Xia and Liangpei Zhang and Philip H.S. Torr
journal = "IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI)",
year = {2023}
}

and

@inproceedings{HAWP,
title = "Holistically-Attracted Wireframe Parsing",
author = "Nan Xue and Tianfu Wu and Song Bai and Fu-Dong Wang and Gui-Song Xia and Liangpei Zhang and Philip H.S. Torr
",
booktitle = "IEEE Conference on Computer Vision and Pattern Recognition (CVPR)",
year = {2020},
}

Acknowledgment

We acknowledge the effort from the authors of the Wireframe dataset and the YorkUrban dataset. These datasets make accurate line segment detection and wireframe parsing possible. We also thank Rémi Pautrat for helpful discussions.

TODO

• Documentations
• Google Colab Notebook