Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Dynamic Structure Estimation from Bandit Feedback using Nonvanishing Exponential Sums
Authors: Motoya Ohnishi, Isao Ishikawa, Yuko Kuroki, Masahiro Ikeda
TMLR 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We provide sample complexity bounds for our algorithms, and we experimentally validate our theoretical claims on simulations of toy examples, including Cellular Automata. |
| Researcher Affiliation | Academia | Motoya Ohnishi* EMAIL Paul G. Allen School of Computer Science & Engineering University of Washington Isao Ishikawa* EMAIL Ehime University RIKEN Center for Advanced Intelligence Project Yuko Kuroki EMAIL CENTAI Institute Masahiro Ikeda EMAIL RIKEN Center for Advanced Intelligence Project Keio University |
| Pseudocode | Yes | Algorithm 1 Period estimation (DFT) Algorithm 2 Eigenvalue estimation |
| Open Source Code | No | The paper mentions using Julia and Lyceum but does not provide specific access to the authors' implementation code for the methodology described in this paper. It states: "We also used some tools and functionalities of Lyceum Summers et al. (2020). The licenses of Julia and Lyceum are [The MIT License; Copyright (c) 2009-2021: Jeff Bezanson, Stefan Karpinski, Viral B. Shah, and other contributors: https://github.com/Julia Lang/julia/contributors], and [The MIT License; Copyright (c) 2019 Colin Summers, The Contributors of Lyceum], respectively." |
| Open Datasets | Yes | We will treat Life Game Conway et al. (1970), a special cellular automata, in our simulation experiment (see Section 6). Finally, we present an attempt on applications to more realistic situation, which will demonstrate some of the important aspects that need to be taken into account when naively applying our algorithms to the real world problems. In particular, we use Open Worm (https://openworm.org/) (Szigeti et al., 2014) to simulate a worm motion (we ran more than around 10 hours for simulations); example images from this simulation is shown in Figure 6 (left). |
| Dataset Splits | No | The paper describes generating data through simulations (Life Game, µ-nearly periodic system, eigenvalue estimation, Open Worm) and evaluating performance, often using multiple random seeds for repeated runs. However, it does not specify traditional machine learning dataset splits such as training, validation, and test sets with proportions or sample counts for model training or evaluation. For example, for the Life Game experiment, it states: "We tested 50 different random seeds (i.e., 1, 51, 101, 151, ... , 2451), and computed the error rate (the number of runs producing a wrong estimate other than the fundamental period eight, which is divided by 50); and it was zero." |
| Hardware Specification | Yes | Julia Version 1.6.3 Platform Info: OS: Linux (x86_64-pc-linux-gnu) CPU: Intel(R) Core(TM) i7-6850K CPU @ 3.60GHz WORD_SIZE: 64 LIBM: libopenlibm LLVM: lib LLVM-11.0.1 (ORCJIT, broadwell) Environment: JULIA_NUM_THREADS = 12 |
| Software Dependencies | Yes | Throughout, we used the following version of Julia Bezanson et al. (2017); for each experiment, the running time was less than a few minutes. Julia Version 1.6.3 |
| Experiment Setup | Yes | The hyperparameters of Life Game environment and the algorithm are summarized in Table 3. Table 3: Hyperparameters used for period estimation of Life Game. Table 4: Hyperparameters used for µ-nearly periodic system. Table 5: Hyperparameters used for eigenvalue estimation. |