Awesome

A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics

• 🔔News
• 🔔Table of Contents
• 🔔Important Tables and Figures
  • 🔔LLM Information
  • 🔔PLM Information
  • 🔔Availble Training Data
• 🔔Citation

News

• 2023-05-31 update.new paper "Polaris: A Safety-focused LLM Constellation Architecture for Healthcare"

• 2023-05-31 update.new paper "Medical mT5: an open-source multilingual text-to-text LLM for the medical domain"

• 2023-05-31 update.new paper "Apollo: An Lightweight Multilingual Medical LLM towards Democratizing Medical AI to 6B People"

• 2023-05-31 update.new paper "LLM-CXR: INSTRUCTION-FINETUNED LLM FOR CXR IMAGE UNDERSTANDING AND GENERATION"

• 2023-05-31 update.new paper "Me LLaMA: Foundation large language models for medical applications"

• 2023-05-31 update.new paper "BioMistral: A collection of open-source pretrained large language models for medical domains"

• 2023-05-31 update.new paper "OncoGPT: A medical conversational model tailored with oncology domain expertise on a large language model Meta-AI (LLaMA)"

• 2023-03-17 update.new paper "Health-LLM: Personalized Retrieval-Augmented Disease Prediction System"

• 2023-03-17 update.new paper "HealAI: A Healthcare LLM for Effective Medical Documentation"

• 2023-03-17 update.new paper "BiMediX: Bilingual Medical Mixture of Experts LLM"

• 2023-03-17 update.new paper "JMLR: Joint Medical LLM and Retrieval Training for Enhancing Reasoning and Professional Question Answering Capability"

• 2023-03-17 update.new paper "MedChatZH: A tuning LLM for traditional Chinese medicine consultation"

• 2023-10-18 added new paper "Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-world Multi-turn Dialogue".

• 2023-10-18 added new paper "Qilin-Med: Multi-stage Knowledge Injection Advanced Medical Large Language Model".

• 2023-10-9 We release the version 1 of the survey (https://arxiv.org/abs/2310.05694).

Table of Contents

• Introduction

• What LLMs Can Do for Healthcare? From Fundamental Tasks to Advanced Applications

  • NER and RE for Healthcare Alpacare
  • Text Classification for Healthcare
  • Semantic Textual Similarity for Healthcare
  • Question Answering for Healthcare
  • Dialogue System for Healthcare
  • Generation of Medical Reports from Images
  • Summary

• From LMs to LLMs for Healthcare

  • LMs for Healthcare
  • LLMs for Healthcare

• Train and Use LLM for Healthcare

  • Pre-training Methods
  • Masked Language Modeling
  • Next Word Prediction
  • Sequence-to-sequence MLM
  • Replaced Token Detection
  • Sentence Boundary Detection
  • Next Sentence Prediction
  • Sentence Order Prediction

• Post-training Methods

  • From predicting tokens to follow instructions: Instruction Fine-Tuning and Supervised Fine-tuning
  • Reinforced Learning from Human Feedback
  • From Human Feedback to AI Feedback
  • Summary

• Usage

  • From Fine-tuning to In-context Learning
  • From System 1 Deep Learning To System 2 Deep Learning: Chain-of-Thought
  • AI Agents
  • Summary

• Parameters-, Memory-, and Compute-efficient Methods

  • Parameters-efficient Methods
  • Compute-efficient and Memory-efficient Methods

• Useful Resources

  • OpenBMB
  • DeepSpeed Chat
  • Training Data
  • Summary

• Evaluation Method

  • General NLP tasks Evaluation
  • Healthcare Evaluation
  • Evaluation of Robustness, Bias, and Ethics
  • Future Directions for Health Evaluation
  • Summary

• Improving Fairness, Accountability, Transparency, and Ethics

  • Fairness
  • Accountability
  • Transparency
  • Ethics

• Future work and Conclusion

  • Future Work
  • Medical knowledge enhancement
  • Integration with Healthcare process
  • Effective Interaction with Patients and Doctors
  • Hallucinations, Misunderstandings and Prompt Brittleness

• Conclusion

Important Tables and Figures

Fig. 2. The organizational framework for the content. Section III, Section IV, Section V are technology details, while Section II, Section VI and Section VI are more valued for Healthcare professionals

<br> <br>

LLM Information

<br> <br>

PLM Information

TABLE I BRIEF SUMMARIZATION OF EXISTING PLMS FOR HEALTHCARE.

<br> <br>

TABLE II SUMMARIZATION OF TRAINING DATA AND EVALUATION TASKS FOR EXISTING PLMS FOR HEALTHCARE.

<br> <br>

Availble Training Data

The Statistics of Computation Cost

TABLE VIII THE STATISTICS OF COMPUTATION COST FOR EXISTING HEALTHCARE LLM.

<br> <br>

TABLE IX ESTIMATED FLOPS AND TRAINING TOKENS FOR DIFFERENT MODEL SIZES.

Citation

@misc{he2023survey,
      title={A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics}, 
      author={Kai He and Rui Mao and Qika Lin and Yucheng Ruan and Xiang Lan and Mengling Feng and Erik Cambria},
      year={2023},
      eprint={2310.05694},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}