Awesome
TPS++: Attention-Enhanced Thin-Plate Spline for Scene Text Recognition
The official code of TPS_PP (IJCAI 2023) Paper Link
TPS++, an attention-enhanced TPS transformation that incorporates the attention mechanism to text rectification for the first time. TPS++ builds a more flexible content-aware rectifier, generating a natural text correction that is easier to read by the subsequent recognizer. This code is based on MMOCR 0.4.0 ( Documentation ) with PyTorch 1.6+.
Code List
- NRTR + TPS_PP
- CRNN + TPS_PP
- ABINet-LV + TPS_PP
Installation
Please refer to Install Guide.
Get Started
Please see Getting Started for the basic usage of MMOCR 0.4.0.
Datasets
The specific configuration of the dataset for training and testing can be found here Dataset Document
testing
├── mixture
│ ├── icdar_2013
│ ├── icdar_2015
│ ├── III5K
│ ├── ct80
│ ├── svt
│ ├── svtp
training
├── mixture
│ ├── Syn90k
│ ├── SynthText
Pretrained Models
Get the pretrained models from BaiduNetdisk(passwd:cd9r), GoogleDrive.
checkpoint model in model/xxx/latest.pth
, pre-train model in pre_train/xxx/latest.pth
Methods | IIIT5K | SVT | IC13 | IC15 | SVTP | CUTE | AVG |
---|---|---|---|---|---|---|---|
NRTR + TPS_PP | 96.3 | 94.6 | 96.6 | 85.7 | 89.0 | 92.4 | 92.4 |
NRTR + TPS_PP * | 95.6 | 95.1 | 97.2 | 85.9 | 89.8 | 90.3 | 92.3 |
First, the model needs to be pre-trained using without TPS_PP (pre-train), and then trained end-to-end with a network that incorporates TPS_PP (checkpoint). * denotes the performance of the implemented code. checkpoint model in model/xxx/latest.pth
, pre-train model in pre_train/xxx/latest.pth
.
Train
Please refer to the training configuration Training Doc
NRTR+TPS++
Setp 1 : Download NRTR pre_train/nrtr/latest.pth
in mmocr_ijcai/nrtr/latest.pth
#Step 2
PORT=1234 ./tools/dist_train.sh configs/textrecog/nrtr/nrtr_tps++.py ./ckpt/ijcai_nrtr_tps_pp 4
--seed=123456 --load-from=mmocr_ijcai/nrtr/nrtr_latest.pth
<!-- ### CRNN+TPS++
Step 1 : Download [CRNN](https://pan.baidu.com/s/1qdFBhC-6Ahb6EID5UGyMiQ?pwd=cd9r) `pre_train/crnn/latest.pth` in `mmocr_ijcai/crnn/latest.pth`
Step 2
```
PORT=1234 ./tools/dist_train.sh configs/textrecog/crnn/crnn_tps++.py ./ckpt/ijcai_crnn_tps_pp 4
--seed=123456 --load-from=mmocr_ijcai/crnn/latest.pth
``` -->
Testing
Please refer to the testing configuration Testing Doc
Acknowledgement
This code is based on MMOCR
Citation
If you find our method useful for your reserach, please cite
@article{zheng2023tps++,
title={TPS++: Attention-Enhanced Thin-Plate Spline for Scene Text Recognition},
author={Zheng, Tianlun and Chen, Zhineng and Bai, Jinfeng and Xie, Hongtao and Jiang, Yu-Gang},
journal={IJCAI},
year={2023}
}
License
This project is released under the Apache 2.0 license.