Awesome
Deep Learning Papers on Medical Image Analysis
Background
To the best of our knowledge, this is the first list of deep learning papers on medical applications. There are couple of lists for deep learning papers in general, or computer vision, for example Awesome Deep Learning Papers. In this list, I try to classify the papers based on their deep learning techniques and learning methodology. I believe this list could be a good starting point for DL researchers on Medical Applications.
Criteria
- A list of top deep learning papers published since 2015.
- Papers are collected from peer-reviewed journals and high reputed conferences. However, it may have recent papers on arXiv.
- A meta-data is required along with the paper, i.e. Deep Learning technique, Imaging Modality, Area of Interest, Clinical Database (DB).
List of Journals / Conferences (J/C):
- Medical Image Analysis (MedIA)
- IEEE Transaction on Medical Imaging (IEEE-TMI)
- IEEE Transaction on Biomedical Engineering (IEEE-TBME)
- IEEE Journal of Biomedical and Health Informatics (IEEE-JBHI)
- International Journal on Computer Assisted Radiology and Surgery (IJCARS)
- International Conference on Information Processing in Medical Imaging (IPMI)
- International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI)
- International Conference on Information Processing in Computer-Assisted Interventions (IPCAI)
- IEEE International Symposium on Biomedical Imaging (ISBI)
Shortcuts
Deep Learning Techniques:
- NN: Neural Networks
- MLP: Multilayer Perceptron
- RBM: Restricted Boltzmann Machine
- SAE: Stacked Auto-Encoders
- CAE: Convolutional Auto-Encoders
- CNN: Convolutional Neural Networks
- RNN: Recurrent Neural Networks
- LSTM: Long Short Term Memory
- M-CNN: Multi-Scale/View/Stream CNN
- MIL-CNN: Multi-instance Learning CNN
- FCN: Fully Convolutional Networks
Imaging Modality:
- US: Ultrasound
- MR/MRI: Magnetic Resonance Imaging
- PET: Positron Emission Tomography
- MG: Mammography
- CT: Computed Tompgraphy
- H&E: Hematoxylin & Eosin Histology Images
- RGB: Optical Images
Table of Contents
Deep Learning Techniques
- AutoEncoders/ Stacked AutoEncoders
- Convolutional Neural Networks
- Recurrent Neural Networks
- Generative Adversarial Networks
Medical Applications
- Annotation
- Classification
- Detection/ Localization
- Segmentation
- Registration
- Regression
- Image Reconstruction and Post-Processing
- Other tasks
Deep Learning Techniques
Auto-Encoders/ Stacked Auto-Encoders
Convolutional Neural Networks
- AggNet: Deep Learning From Crowds for Mitosis Detection in Breast Cancer Histology Images
- Fast Convolutional Neural Network Training Using Selective Data Sampling: Application to Hemorrhage Detection in Color Fundus Images
Recurrent Neural Networks
Generative Adversarial Networks
Medical Applications
Annotation
| Technique | Modality | Area | Paper Title | DB | J/C | Year |
|---|---|---|---|---|---|---|
| NN | H&E | N/A | Deep learning of feature representation with multiple instance learning for medical image analysis [pdf] | ICASSP | 2014 | |
| M-CNN | H&E | Breast | AggNet: Deep Learning From Crowds for Mitosis Detection in Breast Cancer Histology Images [pdf] | AMIDA | IEEE-TMI | 2016 |
| FCN | H&E | N/A | Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentation pdf | MICCAI | 2017 |
Classification
| Technique | Modality | Area | Paper Title | DB | J/C | Year |
|---|---|---|---|---|---|---|
| M-CNN | CT | Lung | Multi-scale Convolutional Neural Networks for Lung Nodule Classification [pdf] | LIDC-IDRI | IPMI | 2015 |
| 3D-CNN | MRI | Brain | Predicting Alzheimer's disease: a neuroimaging study with 3D convolutional neural networks [pdf] | ADNI | arXiv | 2015 |
| CNN+RNN | RGB | Eye | Automatic Feature Learning to Grade Nuclear Cataracts Based on Deep Learning [pdf] | IEEE-TBME | 2015 | |
| CNN | X-ray | Knee | Quantifying Radiographic Knee Osteoarthritis Severity using Deep Convolutional Neural Networks [pdf] | O.E.1 | arXiv | 2016 |
| CNN | H&E | Thyroid | A Deep Semantic Mobile Application for Thyroid Cytopathology [pdf] | SPIE | 2016 | |
| 3D-CNN, 3D-CAE | MRI | Brain | Alzheimer's Disease Diagnostics by a Deeply Supervised Adaptable 3D Convolutional Network [pdf] | ADNI | arXiv | 2016 |
| M-CNN | RGB | Skin | Multi-resolution-tract CNN with hybrid pretrained and skin-lesion trained layers [pdf] | Dermofit | MLMI | 2016 |
| CNN | RGB | Skin, Eye | Towards Automated Melanoma Screening: Exploring Transfer Learning Schemes [pdf] | EDRA, DRD | arXiv | 2016 |
| M-CNN | CT | Lung | Pulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks [pdf] | LIDC-IDRI, ANODE09, DLCST | IEEE-TMI | 2016 |
| 3D-CNN | CT | Lung | DeepLung: Deep 3D Dual Path Nets for Automated Pulmonary Nodule Detection and Classification [pdf] | LIDC-IDRI, LUNA16 | IEEE-WACV | 2018 |
| 3D-CNN | MRI | Brain | 3D Deep Learning for Multi-modal Imaging-Guided Survival Time Prediction of Brain Tumor Patients [pdf] | MICCAI | 2016 | |
| SAE | US, CT | Breast, Lung | Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans [pdf] | LIDC-IDRI | Nature | 2016 |
| CAE | MG | Breast | Unsupervised deep learning applied to breast density segmentation and mammographic risk scoring [pdf] | IEEE-TMI | 2016 | |
| MIL-CNN | MG | Breast | Deep multi-instance networks with sparse label assignment for whole mammogram classification [pdf] | INbreast | MICCAI | 2017 |
| GCN | MRI | Brain | Spectral Graph Convolutions for Population-based Disease Prediction [pdf] | ADNI, ABIDE | arXiv | 2017 |
| CNN | RGB | Skin | Dermatologist-level classification of skin cancer with deep neural networks | Nature | 2017 | |
| FCN + CNN | MRI | Liver-Liver Tumor | SurvivalNet: Predicting patient survival from diffusion weighted magnetic resonance images using cascaded fully convolutional and 3D convolutional neural networks [pdf] | ISBI | 2017 |
Detection / Localization
| Technique | Modality | Area | Paper Title | DB | J/C | Year |
|---|---|---|---|---|---|---|
| MLP | CT | Head-Neck | 3D Deep Learning for Efficient and Robust Landmark Detection in Volumetric Data [pdf] | MICCAI | 2015 | |
| CNN | US | Fetal | Standard Plane Localization in Fetal Ultrasound via Domain Transferred Deep Neural Networks [pdf] | IEEE-JBHI | 2015 | |
| 2.5D-CNN | MRI | Femur | Automated anatomical landmark detection ondistal femur surface using convolutional neural network [pdf] | OAI | ISBI | 2015 |
| LSTM | US | Fetal | Automatic Fetal Ultrasound Standard Plane Detection Using Knowledge Transferred Recurrent Neural Networks [pdf] | MICCAI | 2015 | |
| CNN | X-ray, MRI | Hand | Regressing Heatmaps for Multiple Landmark Localization using CNNs [pdf] | DHADS | MICCAI | 2016 |
| CNN | MRI, US, CT | - | An artificial agent for anatomical landmark detection in medical images [pdf] | SATCOM | MICCAI | 2016 |
| FCN | US | Fetal | Real-time Standard Scan Plane Detection and Localisation in Fetal Ultrasound using Fully Convolutional Neural Networks [pdf] | MICCAI | 2016 | |
| CNN+LSTM | MRI | Heart | Recognizing end-diastole and end-systole frames via deep temporal regression network [pdf] | MICCAI | 2016 | |
| M-CNN | MRI | Heart | Improving Computer-Aided Detection Using Convolutional Neural Networks and Random View Aggregation Neural Networks [pdf] | IEEE-TMI | 2016 | |
| CNN | PET/CT | Heart | Automated detection of pulmonary nodules in PET/CT images: Ensemble false-positive reduction using a convolutional neural network technique Neural Networks [pdf] | MP | 2016 | |
| 3D-CNN | MRI | Brain | Automatic Detection of Cerebral Microbleeds From MR Images via 3D Convolutional Neural Networks [pdf] | IEEE-TMI | 2016 | |
| CNN | X-ray, MG | - | Self-Transfer Learning for Fully Weakly Supervised Lesion Localization [pdf] | NIH,China, DDSM,MIAS | MICCAI | 2016 |
| CNN | RGB | Eye | Fast Convolutional Neural Network Training Using Selective Data Sampling: Application to Hemorrhage Detection in Color Fundus Images [pdf] | DRD, MESSIDOR | MICCAI | 2016 |
| GAN | - | - | Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery | IPMI | 2017 | |
| FCN | X-ray | Cardiac | CathNets: Detection and Single-View Depth Prediction of Catheter Electrodes | MIAR | 2016 | |
| 3D-CNN | CT | Lung | DeepLung: Deep 3D Dual Path Nets for Automated Pulmonary Nodule Detection and Classification [pdf] | LIDC-IDRI, LUNA16 | IEEE-WACV | 2018 |
| 3D-CNN | CT | Lung | DeepEM: Deep 3D ConvNets with EM for weakly supervised pulmonary nodule detection [pdf] | LIDC-IDRI, LUNA16 | MICCAI | 2018 |
Segmentation
| Technique | Modality | Area | Paper Title | DB | J/C | Year |
|---|---|---|---|---|---|---|
| U-Net | - | - | U-net: Convolutional networks for biomedical image segmentation | MICCAI | 2015 | |
| FCN | MRI | Head-Neck | Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation [pdf] | arXiv | 2016 | |
| U-Net | CT | Head-Neck | AnatomyNet: Deep learning for fast and fully automated whole‐volume segmentation of head and neck anatomy [pdf] | Medical Physics | 2018 | |
| FCN | CT | Liver-Liver Tumor | Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields [pdf] | MICCAI | 2016 | |
| 3D-CNN | MRI | Spine | Model-Based Segmentation of Vertebral Bodies from MR Images with 3D CNNs | MICCAI | 2016 | |
| FCN | CT | Liver-Liver Tumor | Automatic Liver and Tumor Segmentation of CT and MRI Volumes using Cascaded Fully Convolutional Neural Networks [pdf] | arXiv | 2017 | |
| FCN | MRI | Liver-Liver Tumor | SurvivalNet: Predicting patient survival from diffusion weighted magnetic resonance images using cascaded fully convolutional and 3D convolutional neural networks [pdf] | ISBI | 2017 | |
| 3D-CNN | Diffusion MRI | Brain | q-Space Deep Learning: Twelve-Fold Shorter and Model-Free Diffusion MRI [pdf] (Section II.B.2) | IEEE-TMI | 2016 | |
| GAN | MG | Breast Mass | Adversarial Deep Structured Nets for Mass Segmentation from Mammograms [pdf] | INbreast, DDSM-BCRP | ISBI | 2018 |
| 3D-CNN | CT | Liver | 3D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumes pdf | MICCAI | 2017 | |
| 3D-CNN | MRI | Brain | Unsupervised domain adaptation in brain lesion segmentation with adversarial networks pdf | IPMI | 2017 | |
| FCN | FUNDUS | Retina | A Fully Convolutional Neural Network based Structured Prediction Approach Towards the Retinal Vessel Segmentation pdf | ISBI | 2017 |
Registration
| Technique | Modality | Area | Paper Title | DB | J/C | Year |
|---|---|---|---|---|---|---|
| 3D-CNN | CT | Spine | An Artificial Agent for Robust Image Registration [pdf] | 2016 |
Regression
| Technique | Modality | Area | Paper Title | DB | J/C | Year |
|---|---|---|---|---|---|---|
| 2.5D-CNN | MRI | Automated anatomical landmark detection ondistal femur surface using convolutional neural network [pdf] | OAI | ISBI | 2015 | |
| 3D-CNN | Diffusion MRI | Brain | q-Space Deep Learning: Twelve-Fold Shorter and Model-Free Diffusion MRI [pdf] (Section II.B.1) | [HCP] and other | IEEE-TMI | 2016 |
Image Reconstruction and Post Processing
| Technique | Modality | Area | Paper Title | DB | J/C | Year |
|---|---|---|---|---|---|---|
| CNN | CS-MRI | A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction pdf | IEEE-TMI | 2017 | ||
| GAN | CS-MRI | Deep Generative Adversarial Networks for Compressed Sensing Automates MRI pdf | NIPS | 2017 |
Image synthesis for data augmentation
| Technique | Modality | Area | Paper Title | DB | J/C | Year |
|---|---|---|---|---|---|---|
| GAN | RGB (Microscopy) | Red Blood Cells | Red blood cell image generation for data augmentation using Conditional Generative Adversarial Networks [pdf] | arXiv | 2019 | |
| GAN | MRI | Brain | Learning Data Augmentation for Brain Tumor Segmentation with Coarse-to-Fine Generative Adversarial Networks [pdf] | arXiv | 2018 | |
| GAN | MRI | Brain | Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks [pdf] | arXiv | 2018 | |
| GAN | CT, MRI | Brain | GAN Augmentation: Augmenting Training Data using Generative Adversarial Networks [pdf] | arXiv | 2018 | |
| GAN | CT | Liver | GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification [pdf] | arXiv | 2018 |