Drift-Resilient TabPFN: In-Context Learning Temporal Distribution Shifts on Tabular Data
Authors: Kai Helli, David Schnurr, Noah Hollmann, Samuel Müller, Frank Hutter
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Comprehensive evaluations across 18 synthetic and realworld datasets demonstrate large performance improvements over a wide range of baselines, such as XGB, Cat Boost, Tab PFN, and applicable methods featured in the Wild-Time benchmark. |
| Researcher Affiliation | Academia | 1 University of Freiburg, 2 Technical University of Munich, 3 ETH Zurich, 4 Charité University Medicine Berlin, 5 ELLIS Institute Tübingen |
| Pseudocode | Yes | Algorithm 1 This algorithm provides a high-level overview for generating a synthetic dataset in our prior. Although steps are depicted sequentially for clarity, many can be parallelized in actual implementation. |
| Open Source Code | Yes | In an effort to ensure reproducibility, we release code, version specification of our baselines, our pre-trained Drift-Resilient Tab PFN and an interactive Colab notebook, that lets you interact with our scikit-learn interface, at https://github.com/automl/Drift-Resilient_ Tab PFN. |
| Open Datasets | Yes | All real-world datasets used in our experiments are freely available at Open ML.org [53] or via the original sources referenced in Section A.7, with downloading scripts or instructions included in the submission code. Code to generate our synthetic datasets can be found in our code repository, see above. |
| Dataset Splits | Yes | Here, each dataset D is split into three subsets: Dtrain, DID, and DOOD. Splits are based on a randomly sampled temporal domain ck that serves as the boundary between the train and test (OOD) portion. We only use such splits, where Dtrain comprises between 30% and 80% of the total domains and samples. |
| Hardware Specification | Yes | The experiments were conducted on an internal SLURM cluster equipped with RTX 2080 TI GPUs and CPUs of type AMD EPYC 7502, 32C/64T, @ 2.50-3.35GHz. |
| Software Dependencies | No | The paper states it releases 'version specification of our baselines' with the code, but the paper text itself does not explicitly list specific software dependencies with version numbers. |
| Experiment Setup | Yes | An overview of the hyperparameters used for running Drift-Resilient Tab PFN and our baselines can be found in Tables 8, 9 and 10 respectively. |