Awesome
LLM Hallucinantions
Code for reproducing the experiments in Unfamiliar Finetuning Examples Control How Language Models Hallucinate.
Setup
pip install torch --extra-index-url https://download.pytorch.org/whl/cu118
pip install -e .
Install FActScore according to instructions here.
Replace “InstructGPT”
with “ChatGPT”
in line 26 of factscore/atomic_facts.py
in your FActScore installation, since text-davinci-003 is no longer supported.
Download the training data and our finetuned checkpoints here.
Place the downloaded folder inside llm_hallucinations/examples
.
Usage
See llm_hallucinations/examples
for training, evaluation, and plotting code.
Acknowledgements
Our codebase was built on top of trlx.