Home

Awesome

Knowledge of Knowledge

Original Repository for the paper Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models

Abstract: This paper investigates the capabilities of Large Language Models (LLMs) in the context of understanding their knowledge and uncertainty over questions. Specifically, we focus on addressing known-unknown questions, characterized by high uncertainty due to the absence of definitive answers. To facilitate our study, we collect a new dataset with Known-Unknown Questions (KUQ) and establish a categorization framework to clarify the origins of uncertainty in such queries. Subsequently, we examine the performance of open-source LLMs, fine-tuned using this dataset, in distinguishing between known and unknown queries within open-ended question-answering scenarios. The fine-tuned models demonstrated a significant improvement, achieving a considerable increase in F1-score relative to their pre-fine-tuning state. Through a comprehensive analysis, we reveal insights into the models' improved uncertainty articulation and their consequent efficacy in multi-agent debates. These findings help us understand how LLMs can be trained to identify and express uncertainty, improving our knowledge of how they understand and express complex or unclear information.

Known KnownsKnown Unknowns
Things we are aware of and understandThings we are aware of but do not understand
e.g. What’s the boiling temperature of water?e.g How many planets are there in the universe?
Unknown KnownsUnknown Unknowns
Things we understand but are not aware ofThings we are neither aware of nor understand
e.g. How to tell the stomach to digest?e.g How does gravity work (before it was discovered)

Data

<p align="left"> <img width="65%" src="images/Dataset-Generation.png" style="display: inline;" /> </p>

:hugs: HF Dataset: Dataset Known-Unknown Questions (KUQ) is available in HuggingFace: link

The dataset is also available in the following folder in GDrive: link

The folder contains the following files:

We also include the split train/dev generated for finetuning in folder KUQ-Known-vs-Unknown and KUQ-Known-vs-Unknown-Categories

Fields contained in the dataset (knowns_unknowns.jsonl):

  1. "question": The question
  2. "answer": Knowns: Correct Answers (list), Unknowns: Source of uncertainty (list)
  3. "unknown": True/False True if unknown
  4. source: Data source
  5. category: One of the categories defined in the paper

Install

You will need to install

(some more packages might be missing but these should be the main ones)

Training

Finetuning is done with llama_finetune/train.sh

The following arguments need to be considered

Run LLama

Used for generating answers from models: run_model.py. It should be used throught scripts: run_llama.sh and run_lora.sh.

Main arguments to consider:

Evaluation

Templated-based

evaluate template.py: Evaluates the generated answers based on the templates they have been trained on.

Arguments:

Citation

@article{amayuelas2023knowledge,
  title={Knowledge of knowledge: Exploring known-unknowns uncertainty with large language models},
  author={Amayuelas, Alfonso and Pan, Liangming and Chen, Wenhu and Wang, William},
  journal={arXiv preprint arXiv:2305.13712},
  year={2023}
}