Dichotomous Optimistic Search to Quantify Human Perception
Authors: Julien Audiffren
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We also empirically evaluate DOS and show that it significantly outperforms these methods, both in experiments that mimics the conduct of a psychometric experiment, and in tests with large pulls budgets that illustrate the faster convergence rate. We now evaluate the performance of DOS in three different settings, with multiple psychometric functions. First, we use a small time budget (T = 200) this aims at reproducing the constraints of real psychometric experiments, where only a few hundred stimuli can be presented to the observer before the fatigue and learning effects significantly interfere with the experiment (Wichmann & Hill, 2001a). |
| Researcher Affiliation | Academia | 1Departement of Neuroscience, University of Fribourg, Fribourg, Switzerland. Correspondence to: Julien Audiffren <julien.audiffren@unifr.ch>. |
| Pseudocode | Yes | Algorithm 1 DOS Parameters µ (objective), T (time horizon) Initialization i 1 (current arm), s1 1/2 (current stimulus ), N1 0 (number of pulls of s1), ˆµ1 0 (empirical average of s1), t 0 (total pulls), S = NULL the latest promising arm, N as in (8) and BT ( ) as in (7). Main Loop While t T : |
| Open Source Code | No | The paper mentions 'All experiments were performed with custom script using Python 3.7' but does not provide any explicit statement about making the source code for DOS publicly available or a link to a repository. |
| Open Datasets | No | The paper describes six different 'psychometric functions' (e.g., 'Nflat', 'Nsteep', 'βflat', 'βsteep', 'Ψm flat', 'Ψm steep') which are mathematical definitions, not publicly accessible datasets. No external datasets are mentioned as being used for training. |
| Dataset Splits | No | The paper describes the 'time horizon (i.e. the maximum number of stimulus presented during the experiment)' as T, but does not specify any explicit training, validation, or test dataset splits in terms of percentages or sample counts. |
| Hardware Specification | No | The paper states 'All experiments were performed with custom script using Python 3.7' but does not specify any hardware details such as CPU, GPU models, or memory. |
| Software Dependencies | No | The paper mentions 'All experiments were performed with custom script using Python 3.7'. While it refers to 'PsychoPy2' for baselines, it does not list specific version numbers for any other software dependencies or libraries used for its own method, DOS. |
| Experiment Setup | No | The paper states 'DOS is completely model-free, in the sense that it does not uses any parameter or prior knowledge regarding Ψ, including (ν, ρ), the parameters of Ψ local smoothness.' While it defines internal parameters like 'N' and 'BT' within Algorithm 1, it does not provide concrete hyperparameter values or detailed system-level training settings common in experimental setups. |