Detecting and Adapting to Irregular Distribution Shifts in Bayesian Online Learning
Authors: Aodong Li, Alex Boyd, Padhraic Smyth, Stephan Mandt
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our approach leads to a more compact and interpretable latent structure and significantly improved performance in the supervised experiments. Furthermore, it is highly scalable; we demonstrate it on models with hundreds of thousands of parameters and tens of thousands of feature dimensions. Our paper is structured as follows: we review related work in Section 2, introduce our methods in Section 3, report our experiments in Section 4, and draw conclusions in Section 6. We test our framework on a variety of real-world datasets that show concept drift, including basketball player trajectories, malware characteristics, sensor data, and electricity prices. We also study sequential versions of SVHN and CIFAR-10 with covariate drift, where we simulate the shifts in terms of image rotations. Finally, we study word embedding dynamics in an unsupervised learning approach. |
| Researcher Affiliation | Academia | Aodong Li1 Alex Boyd2 Padhraic Smyth1,2 Stephan Mandt1,2 1Department of Computer Science 2Department of Statistics University of California, Irvine {aodongl1, alexjb, mandt}@uci.edu smyth@ics.uci.edu |
| Pseudocode | No | The paper describes algorithms like Greedy Search and Beam Search in prose, but does not present them in a structured pseudocode block or a clearly labeled Algorithm section. |
| Open Source Code | No | The paper does not provide a general statement about releasing source code for the methodology described, nor does it include a link to a repository containing their implementation. The link 'https://github.com/linouk23/NBA-Player-Movements' is for a dataset used, not their code. |
| Open Datasets | Yes | We test our framework on a variety of real-world datasets that show concept drift, including basketball player trajectories, malware characteristics, sensor data, and electricity prices. We also study sequential versions of SVHN and CIFAR-10 with covariate drift, where we simulate the shifts in terms of image rotations. Our larger-scale experiments involve Bayesian convolutional neural networks trained on sequential batches for image classification using CIFAR-10 [Krizhevsky et al., 2009] and SVHN [Netzer et al., 2011]. Malware This dataset is a collection of 100K malignous and benign computer programs, collected over 44 months [Huynh et al., 2017]. Sensor Drift A collection of chemical sensor readings [Vergara et al., 2012]. Elec2 The dataset contains the electricity price over three years of two Australian states [Harries and Wales, 1999]. Our first dataset is the Google Books corpus [Michel et al., 2011] in n-grams form. Second, we used the Congressional Records dataset [Gentzkow et al., 2018]. Third, we used the UN General Debates corpus [Jankin Mikhaylov et al., 2017]. |
| Dataset Splits | No | The paper mentions evaluating on a 'reserved test set' and using 'batches' of data, but does not specify clear train/validation/test splits by percentages, sample counts, or refer to predefined splits with citations for reproducibility. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used for running the experiments (e.g., GPU models, CPU types, memory). |
| Software Dependencies | No | The paper refers to software and frameworks implicitly (e.g., Bayesian deep learning suggests PyTorch or TensorFlow), but does not list any specific software components with version numbers required for reproducibility (e.g., 'Python 3.x', 'PyTorch 1.x'). |
| Experiment Setup | Yes | We set the temperature β =2/3 and set the CELBO temperature T =20, 000 (in Eq. 6) for all supervised experiments. |