Awesome
R2Gen-Mamba
This is the implementation of [R2Gen-Mamba: A Selective State Space Model for Radiology Report Generation]
Requirements
torch==1.12.1torchvision==0.8.2opencv-python==4.4.0.42
R2Gen-Mamba Models
The pre-trained R2Gen-Mamba models on different datasets. You can download the models and run them on the corresponding datasets to replicate our results.
| Section | BaiduNetDisk | GoogleDrive | Description |
|---|---|---|---|
| IU X-Ray | download (Password: 9e8m) | download | R2Gen-Mamba model trained on IU X-Ray |
| MIMIC-CXR | download (Password: auii) | download | R2Gen-Mamba model trained on MIMIC-CXR |
Datasets
We use two datasets (IU X-Ray and MIMIC-CXR) in our paper.
Train
Run bash train_iu_xray.sh to train a model on the IU X-Ray data.
Run bash train_mimic_cxr.sh to train a model on the MIMIC-CXR data.
Test
Run bash test_iu_xray.sh to test a model on the IU X-Ray data.
Run bash test_mimic_cxr.sh to test a model on the MIMIC-CXR data.
Follow or CheXbert to extract the labels and then run python compute_ce.py. Note that there are several steps that might accumulate the errors for the computation, e.g., the labelling error and the label conversion. We refer the readers to those new metrics, e.g., RadGraph and RadCliQ.
Visualization
Run python help.py to visualize the attention maps on the MIMIC-CXR data.
Acknowledgements
This project is developed based on R2Gen, and we appreciate their original work and open-source contribution.