Awesome
GNN-TabFact
This is the state-of-the-art model on TabFact Dataset dataset, we leverage the idea proposed in NumGNN into the encoding of tabular data.
Requirements:
- HuggingFace Transformers 2.6.0
- tensorboardX
- pandas 0.25.1
- AllenNLP 0.9.0
Architecture
- Cross-Attention Between Table and Statement to obtain the representation
- Construct the greater/less mask for the table numeric columns
- Use the the dense greater/less connection to propagate the information in each cell
- Obtain the graph repsentation obtained by the NumGNN
- Finally use it to do the two-way classification.
Performance
We demonstrate our results as follows:
Model | Dev | Test |
---|---|---|
TableBERT | 66.1 | 65.1 |
GNN | 72.1 | 72.2 |
Training and Evaluating
Creating a folder for saving the model
mkdir models
Downloading the pre-trained model from Amazon S3, also link the folder of all_csv from TabFact dataset.
ln -s TABFACT/data/all_csv .
cd models
wget https://gnntabfact.s3-us-west-2.amazonaws.com/gnn_fp16_numeric.zip
unzip gnn_fp16_numeric.zip
Loading the trained GNN Model and reproduce our results:
CUDA_VISIBLE_DEVICES=0 python gnn.py --model bert-base-multilingual-uncased --do_test --encoding gnn --load_from models/gnn_fp16_numeric/model_ep4.pt
Retrain your own GNN Model on TabFact:
CUDA_VISIBLE_DEVICES=0 python gnn.py --model bert-base-multilingual-uncased --do_train --encoding gnn --output_dir models/gnn_fp16_numeric_test --attention cross --lr_default 5e-6