Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Reducing sequential change detection to sequential estimation
Authors: Shubhanshu Shekhar, Aaditya Ramdas
ICML 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we empirically compare the performance of the our RCS-Detector with the BCS-Detector scheme of Shekhar & Ramdas (2023) on the problem of detecting changes in mean of bounded observations. |
| Researcher Affiliation | Academia | 1Department of Statistics and Data Science, Carnegie Mellon University 2Machine Learning Department, Carnegie Mellon University. |
| Pseudocode | Yes | Definition 2.1 (RCS-Detector). Suppose we are given a stream of observations X1, X2, . . ., and a functional θ associated with the source. For Problem 1.1 (non-partitioned), define the C(0) n = Θ for all n ≥ 1, while for Problem 1.2 (partitioned) set C(0) n = Θ0, for all n ≥ 1. Proceed as follows for n = 1, 2, . . . : |
| Open Source Code | No | The paper does not contain any explicit statements or links indicating that source code for the described methodology is publicly available. |
| Open Datasets | No | The paper describes generating synthetic data for its experiments: 'We consider the problem of detecting changes in the mean of bounded observations, where X1, X2, . . . , are independent observations, supported on X = [0, 1]. For t ≤ T, each Xt is drawn according to a Beta distribution... For t > T, each Xt is drawn from a Beta distribution...' It does not refer to or provide access information for a publicly available, pre-existing dataset. |
| Dataset Splits | No | The paper describes generating sequential data streams for simulations and experiments ('For t ≤ T, each Xt is drawn according to a Beta distribution... For t > T, each Xt is drawn from a Beta distribution...'). It does not define or use standard train, validation, or test splits typical for pre-existing datasets. |
| Hardware Specification | No | The paper does not provide any specific details regarding the hardware (e.g., CPU, GPU models, memory, or cloud instance types) used to run the experiments. |
| Software Dependencies | No | The paper mentions using 'Hoeffding CS' and 'betting CS' from Waudby-Smith & Ramdas (2023), but it does not specify any software dependencies with version numbers (e.g., 'Python 3.x', 'PyTorch 1.x'). |
| Experiment Setup | Yes | For this, we set T = 500, and consider five values of α in the set {10−i : 1 ≤ i ≤ 5}, and estimate the ARL as the average of the stopping times over 50 trials (capped at 50000). |