Awesome
CLUTRR-Baselines
Codebase for experiments on the CLUTRR benchmark suite.
Usage
- We use Comet.ml to monitor and log experiments / accuracies
- After creating an account in Comet.ml, (Academic accounts are free) put the key and workspace info in
config.log
- To debug, use
config.log.disabled=True
to not pollute the Comet dashboard space. - To run an experiment, first make sure that the codebase is in the Pythonpath:
export PYTHONPATH="${PYTHONPATH}:/home/.../experiment_base
- Then, cd into
codes/app
and runpython main.py --config_id <experiment_name>
experiment_name
is the name of the config file (without .yaml)
Dependencies
- Pytorch, 1.0
- Comet.ml
- Addict
- NLTK
- Pandas
- Pyyaml
- tqdm
Citation
If our work is useful for your research, consider citing it using the following bibtex:
@article{sinha2019clutrr,
Author = {Koustuv Sinha and Shagun Sodhani and Jin Dong and Joelle Pineau and William L. Hamilton},
Title = {CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text},
Year = {2019},
journal = {Empirical Methods of Natural Language Processing (EMNLP)},
arxiv = {1908.06177}
}
Join the CLUTRR community
- Main Repo: https://github.com/facebookresearch/clutrr
- Website: https://www.cs.mcgill.ca/~ksinha4/clutrr/
License
CLUTRR-Baselines is CC-BY-NC 4.0 (Attr Non-Commercial Inter.) licensed, as found in the LICENSE file.