Home

Awesome

Grade-School Math with Irrelevant Context (GSM-IC)

This repository contains the dataset Grade-School Math with Irrelevant Context (GSM-IC) used in this paper: Large Language Models Can Be Easily Distracted by Irrelevant Context.

Data Format

Field nameValue
questionInput question.
answerThe ground truth answer.
n_stepsThe number of intermediate steps to calculate the answer.
Field nameValue
original_questionOriginal question from the GSM8K development set.
new_questionThe new question with irrelevant context added to the original question.
answerThe ground truth answer.
n_stepsThe number of intermediate steps to calculate the answer.
role_label, number_label, sentence_labelCategories of the added irrelevant context. Needed for result analysis, not needed for model prediction.
role, number, sentence_templateAdded irrelevant context. Not needed for experiments.

Citation

If you use the data released through this repository, please cite the following paper:

@article{shi2023large,
  title={Large Language Models Can Be Easily Distracted by Irrelevant Context},
  author={Shi, Freda and Chen, Xinyun and Misra, Kanishka and Scales, Nathan and Dohan, David and Chi, Ed and Schärli, Nathanael and Zhou, Denny},
  journal={arXiv preprint arXiv:2302.00093},
  year={2023}
}