Home

Awesome

<div align="center"> <div> <h1>Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors</h1> </div> <div> Thomas Hartvigsen&emsp; Swami Sankaranarayanan&emsp; Hamid Palangi&emsp; Yoon Kim&emsp; Marzyeh Ghassemi </div> <br/> </div>

Official implementation of Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors (NeurIPS 2023).

<img width="1866" alt="image" src="https://github.com/Thartvigsen/GRACE/assets/26936677/8f28ab99-2411-4fd8-949b-8373ebfff3b5">

Please feel free to email Tom or raise an issue with this repository and we'll get back to you as soon as possible.

Setup

  1. Create a virtual environment (we use conda)
    conda env create --name grace_env --file environment.yml
    
  2. Activate the virtual environment
    conda activate grace_env
    
  3. Install the repository
    pip install -e .
    

Data

The QA experiments use data linked by the MEND repository. Per their instructions, you can download the data for NQ and zsRE from their Google Drive link and unzip each sub-directory into grace/data. SCOTUS and Hallucination data are handled through huggingface.

Running experiments

Experiments are run using main.py. Experiment settings and hyperparameters are chosen using hydra. While more examples are available in ./scripts/main.sh, three representative experiments can be run as follows:

Editing GPT2-XL on Hallucination with GRACE

python grace/main.py experiment=hallucination model=gpt2xl editor=grace

Editing BERT on SCOTUS with GRACE

python grace/main.py experiment=scotus model=bert editor=grace

Editing T5 on zsRE with GRACE

python grace/main.py experiment=qa model=t5small editor=grace

Repository Roadmap

Citation

Please use the following to cite this work:

@inproceedings{hartvigsen2023aging,
  title={Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors},
  author={Hartvigsen, Thomas and Sankaranarayanan, Swami and Palangi, Hamid and Kim, Yoon and Ghassemi, Marzyeh},
  booktitle={Advances in Neural Information Processing Systems},
  year={2023}
}