Awesome
Knowledge of Knowledge
Original Repository for the paper Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models
Abstract: This paper investigates the capabilities of Large Language Models (LLMs) in the context of understanding their knowledge and uncertainty over questions. Specifically, we focus on addressing known-unknown questions, characterized by high uncertainty due to the absence of definitive answers. To facilitate our study, we collect a new dataset with Known-Unknown Questions (KUQ) and establish a categorization framework to clarify the origins of uncertainty in such queries. Subsequently, we examine the performance of open-source LLMs, fine-tuned using this dataset, in distinguishing between known and unknown queries within open-ended question-answering scenarios. The fine-tuned models demonstrated a significant improvement, achieving a considerable increase in F1-score relative to their pre-fine-tuning state. Through a comprehensive analysis, we reveal insights into the models' improved uncertainty articulation and their consequent efficacy in multi-agent debates. These findings help us understand how LLMs can be trained to identify and express uncertainty, improving our knowledge of how they understand and express complex or unclear information.
Known Knowns | Known Unknowns |
---|---|
Things we are aware of and understand | Things we are aware of but do not understand |
e.g. What’s the boiling temperature of water? | e.g How many planets are there in the universe? |
Unknown Knowns | Unknown Unknowns |
Things we understand but are not aware of | Things we are neither aware of nor understand |
e.g. How to tell the stomach to digest? | e.g How does gravity work (before it was discovered) |
Data
<p align="left"> <img width="65%" src="images/Dataset-Generation.png" style="display: inline;" /> </p>:hugs: HF Dataset: Dataset Known-Unknown Questions (KUQ) is available in HuggingFace: link
The dataset is also available in the following folder in GDrive: link
The folder contains the following files:
knowns_unknowns.jsonl
: [Main dataset file] It contains the unknown questions and the paired known questionsunknowns.jsonl
: It contains the original unknown questions generating through crowd-sourcing
We also include the split train/dev generated for finetuning in folder KUQ-Known-vs-Unknown
and KUQ-Known-vs-Unknown-Categories
Fields contained in the dataset (knowns_unknowns.jsonl
):
"question"
: The question"answer"
: Knowns: Correct Answers (list), Unknowns: Source of uncertainty (list)"unknown"
:True/False
True if unknownsource
: Data sourcecategory
: One of the categories defined in the paper
Install
You will need to install
- pytorch
- tranformers
- datasets
- accelerate
- tqdm
- pandas
- scikit-learn
- rouge_score
- openai
- peft
- wandb
- trl
- backoff
(some more packages might be missing but these should be the main ones)
Training
Finetuning is done with llama_finetune/train.sh
The following arguments need to be considered
--model_name
: HuggingFace model to be finetuned--dataset_path
: Dataset used for training (it needs to contain variable 'text')--output_dir
: Dir where the trained model with be stored
Run LLama
Used for generating answers from models: run_model.py
. It should be used throught scripts: run_llama.sh
and run_lora.sh
.
Main arguments to consider:
--dataset
: dataset name as defined indataset_dir
variable--output_dir
: folder to store the generated answers--model_name
: model name as defined in corresponding dicts inrun_model.py
(for finetuned models, the follwing arguments are also needed)based_model_name
: based model name in huggingfacecheckpoint_path
: checkpoint dir of the trained model
Evaluation
Templated-based
evaluate template.py
: Evaluates the generated answers based on the templates they have been trained on.
Arguments:
--input-file
: File with answers generated from the previous step.--dataset
: Dataset name to retrieve the answers from.--sim_threshold
: Similarity Threshold
Citation
@article{amayuelas2023knowledge,
title={Knowledge of knowledge: Exploring known-unknowns uncertainty with large language models},
author={Amayuelas, Alfonso and Pan, Liangming and Chen, Wenhu and Wang, William},
journal={arXiv preprint arXiv:2305.13712},
year={2023}
}