Home

Awesome

The Surprisingly Straightforward Scene Text Removal Method With Gated Attention and Region of Interest Generation: A Comprehensive Prominent Model Analysis

The Surprisingly Straightforward Scene Text Removal Method With Gated Attention and Region of Interest Generation: A Comprehensive Prominent Model Analysis | Paper and Supplementary
Hyeonsu Lee, Chankyu Choi
Naver Corp.
In ECCV 2022.

This repository is official Pytorch implementation of GaRNet

Sample results

The sample image was taken from SCUT-EnsText. <img width="100%" src=ASSETS/sample1.png> <img width="100%" src=ASSETS/sample2.gif>

Getting started

Requirements

The requirements can be installed by running

  pip install -r requirements.txt

Dataset

To compare the performance of previously proposed methods with the same evaluation dataset. we combined Oxford-synthetic data and SCUT-EnsText data. We provide subset information of combined dataset.

Pre-trained models

We provide the weight of EnsNet, MTRNet, MTRNet++, EraseNet and Our's trained on our combined dataset.

Methodpre-trained model
EnsNetsaved_mode.pth
MTRNetsaved_mode.pth
MTRNet++saved_mode.pth
EraseNetsaved_mode.pth
GaRNetlink Generic badge

Note that all models are pre-trained with Synthetic datasets and SCUT-EnsText, and the SCUT-EnsText dataset can only be used for non-commercial research purposes.

Instructions for Comparison between Related works (Table 2 and 3 in the paper)

ImageEval

For MTRNet++ and EraseNet, you need to get official model (or network) codes of them.

DetectionEval (GPU is required to run CRAFT)

  bash eval_DetectionEval.sh [REAL or SYN] [Model type (ex. EnsNet, GaRNet ...)]

For Detection Eval, you need to get CRAFT and DetEVAL script.

Instructions for proposed method

For Eval

  cd ./CODE
  python ./eval.py --model_path=[pretrained model path] default: ../WEIGHTS/GaRNet/saved_model.pth \\
        --test_path=[the path of Test json file] default: ../DATA/JSON/REAL/test.json \\
        --batch=[batch size] default: 10 \\
        --gpu (use gpu for evaluation) defulat: False \\

For Inference

  cd ./CODE
  python ./inference.py --result_path=[the path for save output image] default: ./result \\
        --image_path=[the path of input image] default: ../DATA/IMG \\
        --box_path=[the path of text files which have box information] default: ../DATA/TXT \\
        --input_size=[inference size] default: 512 \\
        --model_path=[the path of trained model] default: ../WEIGHTS/GaRNet/saved_model.pth \\
        --attention_vis (visualize attention) default: False \\
        --gpu (use gpu for inference) default: False \\

Citation

Please cite our paper if this work is useful for your research.

@inproceedings{lee2022surprisingly,
  title={The Surprisingly Straightforward Scene Text Removal Method with Gated Attention and Region of Interest Generation: A Comprehensive Prominent Model Analysis},
  author={Lee, Hyeonsu and Choi, Chankyu},
  booktitle={European Conference on Computer Vision},
  pages={457--472},
  year={2022},
  organization={Springer}
}

License

GaRNet is licensed under Apache-2.0. See LICENSE for the full license text.

Copyright 2022-present NAVER Corp.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Acknowledgments

Our model codes starts from EnsNet.
Our research is benefit a lot from MTRNet++ and EraseNet, we wish to thank their work for providing model codes and real-dataset.
We acknowledge the official code and pre-trained weight CRAFT
We use ssim calculation code in piq.