Awesome
Semantic Evaluation for Text-to-SQL with Distilled Test Suites, EMNLP 2020
Ruiqi Zhong, Tao Yu, Dan Klein
https://arxiv.org/abs/2010.02840
demo.ipynb contains examples on how to run the core functions defined in our paper.
Test suite release as of EMNLP 2020 is in the "unofficial" folder - we do not plan to update this github when we update the test suite in the future. The official metrics are maintained here: https://github.com/taoyds/test-suite-sql-eval .