Data Continuity Matters: Improving Sequence Modeling with Lipschitz Regularizer
Authors: Eric Qu, Xufang Luo, Dongsheng Li
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on various tasks demonstrate that altering data continuity via Lipschitz Regularizer can largely improve the performance of many deep models for sequence modeling.1 |
| Researcher Affiliation | Collaboration | Eric Qu Duke Kunshan University Kunshan, Jiangsu 215316, China zhonghang.qu@duke.edu Xufang Luo & Dongsheng Li Microsoft Research Asia Shanghai 200232, China {xufluo, dongsheng.li}@microsoft.com |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | 1Code is available at https://Eric Qu.site/Lip Reg/ |
| Open Datasets | Yes | We use 5 datasets in this experiment and their descriptions are in Appendix C.1. Evaluation metrics are Mean Square Error (MSE) and Mean Absolute Error (MAE). |
| Dataset Splits | Yes | Hyperparameter λ is chosen from {1, 2, 3, 4, 5} when the model performs best on the validation set. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types with speeds, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details (e.g., library or solver names with version numbers like Python 3.8, CPLEX 12.4) needed to replicate the experiment. |
| Experiment Setup | Yes | Hyperparameter λ is chosen from {1, 2, 3, 4, 5} when the model performs best on the validation set. Hyperparameter λ is chosen from {1, 2, 3, 4, 5, 6, 7, 8} when the model performs best on the validation set. |