Graphix-T5: Mixing Pre-trained Transformers with Graph-Aware Layers for Text-to-SQL Parsing
Authors: Jinyang Li, Binyuan Hui, Reynold Cheng, Bowen Qin, Chenhao Ma, Nan Huo, Fei Huang, Wenyu Du, Luo Si, Yongbin Li
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments and analysis demonstrate the effectiveness of GRAPHIX-T5 across four text-to-SQL benchmarks: SPIDER, SYN, REALISTIC and DK. GRAPHIX-T5 surpass all other T5-based parsers with a significant margin, achieving new state-of-the-art performance. |
| Researcher Affiliation | Collaboration | 1The University of Hong Kong 2DAMO Academy, Alibaba Group 3Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences 4The Chinese University of Hong Kong (Shenzhen) 5Guangdong Hong Kong-Macau Joint Laboratory |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks (clearly labeled algorithm sections or code-like formatted procedures). |
| Open Source Code | Yes | We implement our codes 2 mainly based on hugging-face transformers library (Wolf et al. 2020) 3. 2https://github.com/Alibaba Research/DAMO-Conv AI/tree/ main/graphix |
| Open Datasets | Yes | We conduct extensive experiments on four challenging benchmarks for cross-domain text-to SQLs and two different training settings. (1) SPIDER (Yu et al. 2018b) is a large-scale cross-domain text-to-SQL benchmark. ... (2) SYN (Gan et al. 2021a) replaces the simple string-matched question tokens or schema names with their synonyms. (3) DK (Gan, Chen, and Purver 2021) requires the text-to-SQL parsers to equip with the capability of domain knowledge reasoning. (4) REALISTIC removes and switches the obvious mentions of schema items in questions... |
| Dataset Splits | Yes | SPIDER (Yu et al. 2018b) is a large-scale cross-domain text-to-SQL benchmark. It contains 8659 training examples and 1034 development examples, which covers 200 complex databases across 138 domains. |
| Hardware Specification | Yes | All experiments are conducted on one NVIDIA Tesla A100, which is available for most research centers. |
| Software Dependencies | No | The paper mentions using the "hugging-face transformers library (Wolf et al. 2020)" but does not specify a version number for this or any other software dependency. |
| Experiment Setup | Yes | We set the max input length as 1024, generation max length as 128, and batch size as 32. We also adopt Adafactor (Shazeer and Stern 2018) as our primary optimizer with a linear decayed learning rate of 5e-5. |