Awesome
Perception and Semantic Aware Regularization for Sequential Confidence Calibration
<!-- <p align="center"> <a href="https://arxiv.org/pdf/2404.04624.pdf"><img src="https://img.shields.io/badge/arXiv-Paper-<color>"></a> </p> -->This is the official PyTorch implementation for the paper: "Perception and Semantic Aware Regularization for Sequential Confidence Calibration". For more details, please refer to our paper.<br>
Runtime Environment
- This work was tested with PyTorch 1.9.0, CUDA 11.1, python 3.8 <br>
- Requirements : fastai==1.0.60 lmdb pillow opencv-python tensorboardX editdistance nltk natsort numpy
Datasets
We use lmdb dataset for training and evaluation. The training datasets can be downloaded in clovaai and the ensemble of evalutaion datasets can be downloaded here.
- Training datasets
- Evaluation datasets
- Regular datasets
- Irregular dataset
- Tree structure of
data_lmdb_release
directorydata_lmdb_release ├── charset_36.txt ├── charset_94.txt ├── test └── training ├── MJ │ ├── MJ_test │ ├── MJ_train │ └── MJ_valid └── ST
Calibration
1. Get Correlated Sequence
Perception Similarity Sequence
bash get_perc_sim_seq.sh
Semantic Correlation Sequence
CUDA_VISIBLE_DEVICES=0 python abinet/get_sem_cor_seq.py --checkpoint /path/to/checkpoint
--checkpoint
: path to pretrained model of BCN in ABINet.
2. Calibrate Model With PSSR
CUDA_VISIBLE_DEVICES=0 python train.py \
--train_data data_lmdb_release/training --valid_data data_lmdb_release/test \
--select_data MJ-ST --batch_ratio 0.5-0.5 --lr 0.1 \
--Transformation TPS --FeatureExtraction ResNet --SequenceModeling BiLSTM --Prediction Attn \
--saved_model /path/to/checkpoint --calibrator PSSR --alpha 0.05 --with_vis
--saved_model
: path to model to be calibrated.--calibrator
: select calibration method.--alpha
: weight factor for balancing in the Eq. (2) in our paper. <br> Note: Detailed argument settings can be found in clovaai.
Evaluation
CUDA_VISIBLE_DEVICES=0 python test.py \
--eval_data data_lmdb_release/test \
--Transformation TPS --FeatureExtraction ResNet --SequenceModeling BiLSTM --Prediction Attn \
--saved_model pretrained_models/TPS-ResNet-BiLSTM-Attn.pth
--saved_model
: path to calibrated model.
Calibrated Models Download
Download calibrated model by PSSR from this link.
<!-- Performances of the calibrated model are: | Model | ECE(%) | ACE(%) | MCE(%) | | ------- | :---: | :---: | :---: | | TRBA | 0.36 | 0.28 | 3.99 | | TRBC | 0.47 | 0.25 | 6.22 | | CRNN | 0.97 | 0.49 | 8.52 | -->Acknowledgements
This implementation has been based on clovaai and ABINet.
Citation
If you find our work useful in your research, please cite the following:
@inproceedings{peng2023perception,
title={Perception and Semantic Aware Regularization for Sequential Confidence Calibration},
author={Peng, Zhenghua and Luo, Yu and Chen, Tianshui and Xu, Keke and Huang, Shuangping},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={10658--10668},
year={2023}
}