Awesome

ExpressivE

This repository contains the official source code for the ExpressivE model, presented at ICLR 2023 in our paper "ExpressivE: A Spatio-Functional Embedding For Knowledge Graph Completion". The repository includes the following:

the implementation of ExpressivE
the code for training and testing ExpressivE on WN18RR and FB15k-237 to reproduce the results presented in our paper
an environment.yml to set up a conda environment with all dependencies automatically

Requirements

Python >= 3.9
PyKEEN 1.7.0
Pytorch 1.10.2
Tensorboard 2.8.0

Installation

We have provided an environment.yml file that can be used to create a conda environment with all required dependencies. To install ExpressivE, simply run conda env create -f environment.yml to create the conda environment Env_ExpressivE. Afterward, use conda activate Env_ExpressivE to activate the environment before rerunning our experiments.

Running ExpressivE

Training and evaluation of ExpressivE are done by running the run_experiments.py file. In particular, a configuration file must be specified for an ExpressivE model, containing all model, training, and evaluation parameters. The best configuration files for WN18RR and FB15k-237 are provided in the Best_Configurations directory and can be adapted to try out different parameter configurations. To run an experiment, the following parameters need to be specified:

config contains the path to the model configuration (e.g., config=Best_Configurations/ExpressivE/Base_ExpressivE/WN18RR.json)
train contains true if the model shall be trained and false otherwise.
test contains true if the model shall be evaluated on the test set and false otherwise.
expName contains the name of the experiment (e.g., expName=Base_ExpressivE_WN18RR)
gpu contains the id of the gpu that shall be used (e.g., gpu=0)
seeds contains the seeds for repeated runs of the experiment (e.g., seeds=1,2,3)

Finally, you can run an experiment with python run_experiments.py config=<config> train=<true|false> test=<true|false> expName=<expName> gpu=<gpuID> seeds=<seeds>, where angle brackets represent a parameter value. If the test parameter is set to true, then the selected model will be evaluated on the test set, and the evaluation results will be stored in Benchmarking/final_result/<expName>.

Reproducing the Results

In the following, we provide the commands to reproduce ExpressivE's benchmark results (listed in Table 3 of our paper):

Benchmarks

WN18RR

Train & Test: python run_experiments.py gpu=0 train=true test=true seeds=1,2,3 config=Best_Configurations/ExpressivE/Base_ExpressivE/WN18RR.json expName=Base_ExpressivE_WN18RR

Fb15k-237

Train & Test: python run_experiments.py gpu=0 train=true test=true seeds=1,2,3 config=Best_Configurations/ExpressivE/Functional_ExpressivE/FB15k-237.json expName=Functional_ExpressivE_FB15k-237

Per-Relation Evaluation

To evaluate the performance of the Base ExpressivE model per relation on WN18RR, first, train a Base ExpressivE model as described above. After the training has finished, run the following commands (which evaluate the trained models of each seed):

Seed 1: python run_per_relation_evaluation.py model_dir=Benchmarking sub_dir=Base_ExpressivE_WN18RR_seed_1 config_path=Best_Configurations/ExpressivE/Base_ExpressivE/WN18RR.json

Seed 2: python run_per_relation_evaluation.py model_dir=Benchmarking sub_dir=Base_ExpressivE_WN18RR_seed_2 config_path=Best_Configurations/ExpressivE/Base_ExpressivE/WN18RR.json

Seed 3: python run_per_relation_evaluation.py model_dir=Benchmarking sub_dir=Base_ExpressivE_WN18RR_seed_3 config_path=Best_Configurations/ExpressivE/Base_ExpressivE/WN18RR.json

The relation-wise performance results will be stored at Benchmarking/per_relation_result/Base_ExpressivE_WN18RR_seed_<s>, where <s> is 1, 2, or 3.

Citation

If you use this code or its corresponding paper, please cite our work as follows:

@inproceedings{
pavlovic2023expressive,
title={ExpressivE: A Spatio-Functional Embedding For Knowledge Graph Completion},
author={Aleksandar Pavlovi{\'c} and Emanuel Sallinger},
booktitle={International Conference on Learning Representations},
year={2023},
url={https://openreview.net/forum?id=xkev3_np08z}
}

Contact

Aleksandar Pavlović

Research Unit of Databases and Artificial Intelligence

Vienna University of Technology (TU Wien)

Vienna, Austria

aleksandar.pavlovic@tuwien.ac.at

Licenses

The benchmark datasets WN18RR and FB15k-237 are already included in the PyKEEN library. PyKEEN uses the MIT license. FB15k-237 is a subset of FB15k which uses the CC BY 2.5 license. The license of FB15k-237 and WN18RR is unknown. This project runs under the MIT license.