RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL
Authors: Haoyang Li, Jing Zhang, Cuiping Li, Hong Chen
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our proposed framework on Spider and its three robustness variants: Spider-DK, Spider-Syn, and Spider-Realistic. The experimental results show that our framework delivers promising performance and robustness. |
| Researcher Affiliation | Academia | 1 Key Laboratory of Data Engineering and Knowledge Engineering of Ministry of Education, Renmin University of China 2Engineering Research Center of Ministry of Education on Database and BI 3 Information School, Renmin University of China {lihaoyang.cs, zhang-jing, licuiping, chong}@ruc.edu.cn |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available at https://github.com/RUCKBReasoning/RESDSQL. |
| Open Datasets | Yes | We evaluate our proposed framework on Spider and its three robustness variants: Spider-DK, Spider-Syn, and Spider-Realistic. Spider (Yu et al. 2018c) is the most challenging benchmark for the cross-domain and multi-table Text-to-SQL task. Spider contains a training set with 7,000 samples1, a dev set with 1,034 samples, and a hidden test set with 2,147 samples. |
| Dataset Splits | Yes | Spider contains a training set with 7,000 samples1, a dev set with 1,034 samples, and a hidden test set with 2,147 samples. |
| Hardware Specification | Yes | We conduct all experiments on a server with one NVIDIA A100 (80G) GPU, one Intel(R) Xeon(R) Silver 4316 CPU, 256 GB memory and Ubuntu 20.04.2 LTS operating system. |
| Software Dependencies | No | The paper mentions software components like RoBERTa, T5, AdamW, and Adafactor, but does not provide specific version numbers for these software libraries or frameworks. It mentions the operating system version (Ubuntu 20.04.2 LTS), but not specific programming language or library versions relevant for replication. |
| Experiment Setup | Yes | We use Adam W (Loshchilov and Hutter 2019) with batch size 32 and learning rate 1e-5 for optimization. In the focal loss, the focusing parameter γ and the weighted factor α are set to 2 and 0.75 respectively. Then, k1 and k2 are set to 4 and 5 according to the statistics of the datasets. For training the seq2seq model in the second stage, we consider three scales of T5: Base, Large, and 3B. We fine-tune them with Adafactor (Shazeer and Stern 2018) using different batch size (bs) and learning rate (lr), resulting in RESDSQL-Base (bs = 32, lr = 1e-4), RESDSQL-Large (bs = 32, lr = 5e-5), and RESDSQL-3B (bs = 96, lr = 5e-5). For both stages of training, we adopt linear warm-up (the first 10% training steps) and cosine decay to adjust the learning rate. We set the beam size to 8 during decoding. |