Awesome
Conditional Theorem Provers
Reproducing Results on CLUTRR
./bin/clutrr-cli.py --train data/clutrr-emnlp/data_db9b8f04/1.2,1.3,1.4_train.csv --test data/clutrr-emnlp/data_db9b8f04/1.10_test.csv data/clutrr-emnlp/data_db9b8f04/1.2_test.csv data/clutrr-emnlp/data_db9b8f04/1.3_test.csv data/clutrr-emnlp/data_db9b8f04/1.4_test.csv data/clutrr-emnlp/data_db9b8f04/1.5_test.csv data/clutrr-emnlp/data_db9b8f04/1.6_test.csv data/clutrr-emnlp/data_db9b8f04/1.7_test.csv data/clutrr-emnlp/data_db9b8f04/1.8_test.csv data/clutrr-emnlp/data_db9b8f04/1.9_test.csv -S 1 -s concat -V 128 -b 32 -d 2 --test-max-depth 4 --hops 2 2 2 -e 5 -o adagrad -l 0.1 --init random --ref-init random -m 5 -t min -k 20 -i 1.0 -r linear -R 0 --seed 1
[..]
INFO:clutrr-cli.py:Test Accuracy on data/clutrr-emnlp/data_db9b8f04/1.10_test.csv: 89.344262
INFO:clutrr-cli.py:Test Accuracy on data/clutrr-emnlp/data_db9b8f04/1.2_test.csv: 100.000000
INFO:clutrr-cli.py:Test Accuracy on data/clutrr-emnlp/data_db9b8f04/1.3_test.csv: 71.962617
INFO:clutrr-cli.py:Test Accuracy on data/clutrr-emnlp/data_db9b8f04/1.4_test.csv: 74.025974
INFO:clutrr-cli.py:Test Accuracy on data/clutrr-emnlp/data_db9b8f04/1.5_test.csv: 97.837838
INFO:clutrr-cli.py:Test Accuracy on data/clutrr-emnlp/data_db9b8f04/1.6_test.csv: 96.190476
INFO:clutrr-cli.py:Test Accuracy on data/clutrr-emnlp/data_db9b8f04/1.7_test.csv: 98.064516
INFO:clutrr-cli.py:Test Accuracy on data/clutrr-emnlp/data_db9b8f04/1.8_test.csv: 97.777778
INFO:clutrr-cli.py:Test Accuracy on data/clutrr-emnlp/data_db9b8f04/1.9_test.csv: 94.354839
[..]
Reproducing Link Prediction Results
UMLS
./bin/hoppy-cli.py --train data/umls/train.tsv --dev data/umls/dev.tsv --test data/umls/test.tsv -k 50 -K 4 -b 8 -e 100 -f 0 -l 0.1 -o adagrad -I standard -V 3 -q --hops 0 2 --nb-negatives 3 -r linear -R 0 --init uniform --ref-init random --refresh 100 --index nms
Kinship
./bin/hoppy-cli.py --train data/kinship/train.tsv --dev data/kinship/dev.tsv --test data/kinship/test.tsv -k 50 -K 4 -b 8 -e 100 -f 0 -l 0.1 -o adagrad -I standard -V 3 -q --hops 0 1 1R --nb-negatives 3 -r attentive -R 0 --init uniform --ref-init random --refresh 100 --index nms
Nations
./bin/hoppy-cli.py --train data/nations/train.tsv --dev data/nations/dev.tsv --test data/nations/test.tsv -k 50 -K 4 -b 16 -e 100 -f 0 -l 0.1 -o adagrad -I standard -V 3 -q --hops 0 2 1R --nb-negatives 3 -r linear -R 0 --init uniform --ref-init random --refresh 100 --index nms
Note that the reasoning depth here is always 1, and the rule templates are given by the --hops
parameter - e.g. 0 2 1R
means there are three reformulation modules, one for rules in the form p(X,Y) :- q(X,Z), r(Z,Y)
(2 hops from the subject to the object) and one for rules in the form p(X,Y) :- q(Y,X)
(1 hop from the object to the subject). Here 0 means no hops is also a legal choice (just match the query with the facts in the KB). For matching with facts in the KB, we use a nearest neighbour search engine for finding the most similar facts to the query (otherwise we should have to compute the Gaussian kernel between each sub-goal and all facts in the KB all the time, and backpropagate through all these comparisons - https://arxiv.org/abs/1912.10824)
Citing
@inproceedings{minervini20icml,
author = {Pasquale Minervini and
Sebastian Riedel and
Pontus Stenetorp and
Edward Grefenstette and
Tim Rockt{\"{a}}schel},
title = {Learning Reasoning Strategies in End-to-End Differentiable Proving},
booktitle = {{ICML}},
series = {Proceedings of Machine Learning Research},
volume = {119},
pages = {6938--6949},
publisher = {{PMLR}},
year = {2020}
}