Awesome
<p align="center"> <h1 align="center">A Systematic Review of Deep Learning-based Research on Radiology Report Generation</h1>The official GitHub repository of the survey paper titled "A Systematic Review of Deep Learning-based Research on Radiology Report Generation".
We maintain this repository to summarize papers and resources related to the radiology report generation (RRG) task.
In reference.bib, we summarize the bibtex references of existing RRG papers, widely used datasets, and related toolkits.
If you have any suggestions about papers, code implementations, and other resources, please feel free to start a new issue or pull requests.
Papers involving large language models (LLMs) are in boldface.
News 🔥
[2024 Jan. 9th] We have updated all accepted papers by EMNLP 2023.
Table of Contents
Papers
2024
[TMI 2024] Multi-Grained Radiology Report Generation With Sentence-Level Image-Language Contrastive Learning [paper]
[TMI 2024] Attribute Prototype-guided Iterative Scene Graph for Explainable Radiology Report Generation [paper] [code]
[TMI 2024] An Organ-aware Diagnosis Framework for Radiology Report Generation [paper]
[Academic Radiology 2024] Practical Evaluation of ChatGPT Performance for Radiology Report Generation [paper]
[arXiv 2024] ReXplain: Translating Radiology into Patient-Friendly Video Reports [paper]
[arXiv 2024] ONCOPILOT: A Promptable CT Foundation Model For Solid Tumor Evaluation [paper]
[arXiv 2024] MedVisionLlama: Leveraging Pre-Trained Large Language Model Layers to Enhance Medical Image Segmentation [paper]
[arXiv 2024] Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Models [paper] [code]
[arXiv 2024] CanvOI, an Oncology Intelligence Foundation Model: Scaling FLOPS Differently [paper]
[nature biomedical engineering 2024] A vision–language foundation model for the generation of realistic chest X-ray images [paper] [code]
[arXiv 2024] M4CXR: Exploring Multi-task Potentials of Multi-modal Large Language Models for Chest X-ray Interpretation [paper]
[arXiv 2024] ICON: Improving Inter-Report Consistency for Radiology Report Generation via Lesion-aware Mix-up Augmentation [paper] [code]
[ACL 2024] Two-Pronged Human Evaluation of ChatGPT Self-Correction in Radiology Report Simplification [paper]
[ACL 2024] Fine-Grained Image-Text Alignment in Medical Imaging Enables Explainable Cyclic Image-Report Generation [paper] [code]
[AAAI 2024] Bootstrapping Large Language Models for Radiology Report Generation [paper] [code]
[AAAI 2024] PromptMRG: Diagnosis-Driven Prompts for Medical Report Generation [paper]
[BSPC 2024] An Efficient but Effective Writer: Diffusion-based Semi-autoregressive Transformer for Automated Radiology Report Generation [paper]
[BSPC 2024] Unsupervised Disease Tags for Automatic Radiology Report Generation [paper]
[CVPR 2024] Instance-level Expert Knowledge and Aggregate Discriminative Attention for Radiology Report Generation [paper]
[ICASSP 2024] Improving Radiology Report Generation with D^2-Net: When Diffusion Meets Discriminator [paper] [code]
[Neurocomputing 2024] Improving Radiology Report Generation with Multi-grained Abnormality Prediction [paper]
[Neurocomputing 2024] Trust It or Not: Confidence-guided Automatic Radiology Report Generation [paper]
[TMI 2024] An Organ-aware Diagnosis Framework for Radiology Report Generation [paper]
[TMI 2024] Multi-Grained Radiology Report Generation With Sentence-Level Image-Language Contrastive Learning [paper]
[TMI 2024] PhraseAug: An Augmented Medical Report Generation Model with Phrasebook [paper]
[TMM 2024] From Observation to Concept: A Flexible Multi-View Paradigm for Medical Report Generation [paper]
[WACV 2024] Complex Organ Mask Guided Radiology Report Generation [paper] [code]
2023
[arXiv 2023] LLM-CXR: Instruction-Finetuned LLM for CXR Image Understanding and Generation [paper] [code] [code]
[arXiv 2023] MAIRA-1: A Specialised Large Multimodal Model for Radiology Report Generation [paper] [project]
[arXiv 2023] MedXChat: Bridging CXR Modalities with a Unified Multimodal Large Model [paper]
[arXiv 2023] Pragmatic Radiology Report Generation [paper]
[arXiv 2023] RadLLM: A Comprehensive Healthcare Benchmark of Large Language Models for Radiology [paper] [project]
[arXiv 2023] RaDialog: A Large Vision-Language Model for Radiology Report Generation and Conversational Assistance [paper] [code]
[arXiv 2023] Towards Generalist Biomedical AI [paper] [reproduced code]
[arXiv 2023] Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Data [paper] [project] [code]
[arXiv 2023] Cross-Modal Causal Intervention for Medical Report Generation [paper] [code]
[arXiv 2023] Medical Report Generation based on Segment-Enhanced Contrastive Representation Learning [paper]
[arXiv 2023] Radiology Report Generation Using Transformers Conditioned with Non-imaging Data [paper]
[arXiv 2023] Can Prompt Learning Benefit Radiology Report Generation? [paper]
[ACL 2023] ORGAN: Observation-Guided Radiology Report Generation via Tree Reasoning [paper] [code]
[ACL 2023] Replace and Report: NLP Assisted Radiology Report Generation [paper]
[AMI 2023] Improving Chest X-ray Report Generation by Leveraging Warm Starting [paper] [code]
[CBM 2023] Visual Prior-based Cross-modal Alignment Network for Radiology Report Generation [paper]
[CBM 2023] Visual Prior-based Cross-modal Alignment Network for Radiology Report Generation [paper]
[CVPR 2023] Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generation [paper] [code]
[CVPR 2023] Interactive and Explainable Region-guided Radiology Report Generation [paper] [code]
[CVPR 2023] KiUT: Knowledge-injected U-Transformer for Radiology Report Generation [paper]
[CVPR 2023] METransformer: Radiology Report Generation by Transformer with Multiple Learnable Expert Tokens [paper]
[EACL 2023] KGVL-BART: Knowledge Graph Augmented Visual Language BART for Radiology Report Generation [paper] [code]
[EMNLP 2023] Style-Aware Radiology Report Generation with RadGraph and Few-Shot Prompting [paper]
[EMNLP 2023] PhenotypeCLIP: Phenotype-based Contrastive Learning for Medical Imaging Report Generation [paper]
[EMNLP 2023] RECAP: Towards Precise Radiology Report Generation via Dynamic Disease Progression Reasoning [paper] [code]
[ICASSP 2023] MvCo-DoT: Multi-View Contrastive Domain Transfer Network for Medical Report Generation [paper]
[ICCV 2023] Unify, Align and Refine: Multi-Level Semantic Alignment for Radiology Report Generation [paper]
[ICIP 2023] Self Adaptive Global-Local Feature Enhancement for Radiology Report Generation [paper]
[TMI 2023] Attributed Abnormality Graph Embedding for Clinically Accurate X-Ray Report Generation [paper]
[TMI 2023] SGT++: Improved Scene Graph-guided Transformer for Surgical Report Generation [paper]
[TMM 2024] From Observation to Concept: A Flexible Multi-view Paradigm for Medical Report Generation [paper]
[TMM 2023] Joint Embedding of Deep Visual and Semantic Features for Medical Image Report Generation [paper]
[TMM 2023] Semi-supervised Medical Report Generation via Graph-guided Hybrid Feature Consistency [paper]
[WWW 2023] Auxiliary Signal‑guided Knowledge Encoder‑decoder for Medical Report Generation [paper] [code]
2022
[arXiv 2022] CAMANet: Class Activation Map Guided Attention Network for Radiology Report Generation [paper]
[arXiv 2022] Hybrid Reinforced Medical Report Generation with M-Linear Attention and Repetition Penalty [paper]
[arXiv 2022] DeltaNet: Conditional Medical Report Generation for COVID-19 Diagnosis [paper]
[AACL 2022] Multimodal Generation of Radiology Reports using Knowledge-Grounded Extraction of Entities and Relations [paper]
[ACL 2022] Reinforced Cross-modal Alignment for Radiology Report Generation [paper] [code]
[BSPC 2022] Clinically Coherent Radiology Report Generation with Imbalanced Chest X-rays [paper]
[BMVC 2022] On the Importance of Image Encoding in Automated Chest X-Ray Report Generation [paper] [code]
[COLING] Cross-modal Contrastive Attention Model for Medical Report Generation [paper]
[CVPR 2022] Cross-modal Clinical Graph Transformer for Ophthalmic Report Generation [paper]
[ECCV 2022] Cross-Modal Prototype Driven Network for Radiology Report Generation [paper]
[EMNLP 2022] Improving the Factual Correctness of Radiology Report Generation with Semantic Rewards [paper] [code]
[EMNLP 2022] Factual Accuracy is not Enough: Planning Consistent Description Order for Radiology Report Generation [paper]
[ICBB 2022] Clinically Coherent Radiology Report Generation with Imbalanced Chest X-rays [paper]
[MIA 2022] Knowledge Matters: Chest Radiology Report Generation with General and Specific Knowledge [paper] [code]
[MICCAI 2023] SGT: Scene Graph-Guided Transformer for Surgical Report Generation [paper] [code]
[MICCAI 2022] RepsNet: Combining Vision with Language for Automated Medical Reports [paper]
[MICCAI 2022] A Self-guided Framework for Radiology Report Generation [paper] [code]
[MICCAI 2022] A Medical Semantic-Assisted Transformer for Radiographic Report Generation [paper]
[MICCAI 2022] An Inclusive Task-Aware Framework for Radiology Report Generation [paper]
[MICCAI 2022] Lesion Guided Explainable Few Weak-Shot Medical Report Generation [paper]
[MICCAI 2022] TranSQ: Transformer-Based Semantic Query for Medical Report Generation [paper] [code]
[TMI 2022] Automated Radiographic Report Generation Purely on Transformer: A Multicriteria Supervised Approach [paper]
2021
[ACL 2021] Cross-modal Memory Networks for Radiology Report Generation [paper] [code]
[COLING 2021] JPG - Jointly Learn to Align: Automated Disease Prediction and Radiology Report Generation [paper]
[CVPR 2021] Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generation [paper]
[CVPR 2021] A Self-boosting Framework for Automated Radiographic Report Generation [paper]
[EMNLP 2021] Automated Generation of Accurate & Fluent Medical X-ray Reports [paper] [code]
[EMNLP 2021] Progressive Transformer-Based Generation of Radiology Reports [paper] [code]
[EMNLP 2021] Weakly Supervised Contrastive Learning for Chest X-Ray Report Generation [paper] [code]
[ICCV 2021] Visual-Textual Attentive Semantic Consistency for Medical Report Generation [paper]
[MICCAI 2021] RATCHET: Medical Transformer for Chest X-ray Diagnosis and Reporting [paper] [code]
[MICCAI 2021] Trust It or Not: Confidence-Guided Automatic Radiology Report Generation [paper]
[MICCAI 2021] AlignTransformer: Hierarchical Alignment of Visual Regions and Disease Tags for Medical Report Generation [paper]
[NAACL 2021] Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation [paper] [code]
[NeurIPS 2021] Auto-encoding Knowledge Graph for Unsupervised Medical Report Generation [paper]
[PMLR 2021] Retrieval-Based Chest X-Ray Report Generation Using a Pre-trained Contrastive Language-Image Model [paper]
2020
[AAAI 2020] When Radiology Report Generation Meets Knowledge Graph [paper]
[ACCV 2020] Hierarchical X-Ray Report Generation via Pathology tags and Multi Head Attention [paper]
[EMNLP 2020] Generating Radiology Reports via Memory-driven Transformer [paper] [code]
[EMNLP 2020] Reinforcement Learning with Imbalanced Dataset for Data-to-Text Medical Report Generation [paper]
2019
[AAAI 2019] Knowledge-Driven Encode, Retrieve, Paraphrase for Medical Image Report Generation [paper]
[ACL 2019] Show, Describe and Conclude: On Exploiting the Structure Information of Chest X-Ray Reports [paper]
[MICCAI 2019] Automatic Radiology Report Generation based on Multi-view Image Fusion and Medical Concept Enrichment [paper]
[BMVC 2019] Addressing Data Bias Problems for Chest X-ray Image Report Generation [paper]
2018
[ACL 2018] On the Automatic Generation of Medical Imaging Reports [paper] [code]
[MICCAI 2018] Multimodal Recurrent Model with Attention for Automated Radiology Report Generation [paper] [code]
[NeurIPS 2018] Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation [paper]
Datasets
[J. Am. Medical Informatics Assoc.] Preparing A Collection of Radiology Examinations for Distribution and Retrieval [paper] [dataset (the second QA)]
[Scientific Data] MIMIC-CXR, A De-identified Publicly Available Database of Chest Radiographs with Free-text Reports [paper] [dataset]
[arXiv] MIMIC-CXR-JPG, A Large Publicly Available Database of Labeled Chest Radiographs [paper] [dataset]
[arXiv] Detailed Annotations of Chest X-Rays via CT Projection for Report Understanding [paper] [dataset]
[arXiv] VinDr-CXR: An open dataset of chest X-rays with radiologist’s annotations [paper] [dataset]
[CVPR] ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases [paper] [dataset] or [dataset]
[NeurIPS 2021 Datasets & Benchmark] FFA-IR: Towards an Explainable and Reliable Medical Report Generation Benchmark [paper] [dataset] [GitHub]
[NeurIPS 2021 Datasets & Benchmark] RadGraph: Extracting Clinical Entities and Relations from Radiology Reports [paper] [dataset]
Toolkits
[NeurIPS 2021 Datasets & Benchmark] RadGraph [paper] [project] [code]
[AAAI 2019] CheXpert [paper] [code]