Awesome

Predicting the Original Appearance of Damaged Historical Documents

</div>

</div> <a href="#🖼️-Gallery">🖼️ Gallery </a> • <a href="#📊-HDR28K">📊 HDR28K </a> • <a href="#🔥-Model-Zoo">🔥 Model Zoo</a> • <a href="#🚧-Installation">🚧 Installation</a> • <a href="#📺-Inference">📺 Inference</a> • <a href="#📏-Evaluation">📏 Evaluation</a>

🌟 Highlight

Vis_1 Vis_2

We introduce a Historical Document Repair (HDR) task, which endeavors to predict the original appearance of damaged historical document images.
We build a large-scale historical document repair dataset, termed HDR28K, which includes 28,552 damaged-repaired image pairs with character-level annotations and multi-style degradation.
🔥🔥🔥 We propose a Diffusion-based Historical Document Repair method (DiffHDR), which augments the DDPM framework with semantic and spatial information

📰 News

2024.12.17: Release inference code.
2024.12.10: 🎉🎉 Our paper is accepted by AAAI2025.

🏗️ TODO List

Inference Code.
HDR28K Dataset Release.
Repair Demo.
Traning Code. (Maybe release, due to the copyright)

🔥 Model Zoo

Model	chekcpoint	status
DiffHDR	GoogleDrive / BaiduYun:x62f	Released

🚧 Installation

Prerequisites (Recommended)

Linux
Python 3.9
Pytorch 1.13.1
CUDA 11.7

Environment Setup

Clone this repo:

git clone https://github.com/yeungchenwa/HDR.git

Step 0: Download and install Miniconda from the official website.

Step 1: Create a conda environment and activate it.

conda create -n diffhdr python=3.9 -y
conda activate diffhdr

Step 2: Install related version Pytorch following here.

# Suggested
pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu117

Step 3: Install the required packages.

pip install -r requirements.txt

📺 Inference

Using DiffHDR for damaged historical documents repair (Some examples including damaged images, mask images, and content images are provided in /examples):

sh scripts/inference.sh

device: CUDA or CPU used for inference,
image_path: The damaged image path.
mask_image_path: The masked image path.
content_image_path: The content image path.
save_dir: The directory for saving repaired image.
content_mask_guidance_scale: The guidance scale of content image and masked image.
degraded_guidance_scale: The guidance scale of damaged image.
ckpt_path: The unet checkpoint path.
num_inference_steps: The number of inference steps.

📊 HDR28K

HDR28K

Coming soon ...

📏 Evaluation

Coming soon ...

💙 Acknowledgement

diffusers

⛔️ Copyright

This repository can only be used for non-commercial research purposes.
For commercial use, please contact Prof. Lianwen Jin (eelwjin@scut.edu.cn).
Copyright 2024, Deep Learning and Vision Computing Lab (DLVC-Lab), South China University of Technology.

📇 Citation

@inproceedings{yang2024fontdiffuser,
  title={Predicting the Original Appearance of Damaged Historical Documents},
  author={Yang, Zhenhua and Peng, Dezhi and Shi, Yongxin and Zhang, Yuyi and Liu, Chongyu and Jin, Lianwen},
  booktitle={Proceedings of the AAAI conference on artificial intelligence},
  year={2025}
}