Neural Spline Search for Quantile Probabilistic Modeling
Authors: Ruoxi Sun, Chun-Liang Li, Sercan Ö. Arik, Michael W. Dusenberry, Chen-Yu Lee, Tomas Pfister
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments Comparison Methods... Results To demonstrate the effectiveness of proposed methods, we conduct experiments on synthetic, real-world tabular regression, and time series forecasting datasets. |
| Researcher Affiliation | Industry | Ruoxi Sun1 , Chun-Liang Li1 , Sercan Ö. Arık1, Michael W. Dusenberry2, Chen-Yu Lee1, Tomas Pfister1 1Google Cloud AI 2Google Research, Brain Team {ruoxis, chunliang, soarik, dusenberrymw, chenyulee, tpfister}@google.com |
| Pseudocode | Yes | Algorithm 1: Neural Spline Search |
| Open Source Code | No | The paper does not include any statement or link indicating that the source code for the methodology is openly available. It only contains a copyright statement. |
| Open Datasets | Yes | We use UCI benchmarks (Dua and Graff 2017) We evaluate the accuracy for both point predictions and quantiles. As the point predictions, we use the 50th quantile estimator as our estimates. Table 1 shows that the proposed NSS methods outperform the other existing methods on most datasets in mean absolute error (MAE). We observe that the NSS-sum performs better than NSS-chain. For quantile metrics, we use the pinball loss (Eq. 2) over 100 quantile levels α = {0.01, 0.02, ...0.99} in Table 2. The results indicate that NSS consistently outperforms other alternatives across different UCI benchmarks. In pinball loss, NSS-sum performs better than NSS-chain. We attribute the superiority of NSS-sum for regression to make balance between different transformation, which is helpful in explaining the variance in the data. Retail Demand Forecasting For time series forecasting, we focus on the M5 dataset, which contains time-varying sales data for retail goods, along with other relevant covariates like price, promotions, day of the week, special events etc. It represents an important real-world scenario, where the accurate estimation of the output distribution is crucial, as retailers use them to optimize prices or promotions. |
| Dataset Splits | No | For synthetic data, the paper states: 'We construct the validation and test sets to come from the same distribution.' However, for the UCI benchmarks and M5 dataset, it does not explicitly provide the specific percentages or sample counts for the training, validation, and test splits, nor does it refer to specific predefined splits with citations for these datasets. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU models, CPU types, or cloud computing instances) used to run the experiments. |
| Software Dependencies | No | The paper does not list any specific software dependencies with version numbers, such as programming languages, libraries, or frameworks used for implementation. |
| Experiment Setup | Yes | We use a batch size=128 and a learning rate of 0.005 for 100 epochs. NSS-sum is tuned with λ in the range of [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]. |