Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Analytically Tractable Hidden-States Inference in Bayesian Neural Networks
Authors: Luong-Ha Nguyen, James-A. Goulet
JMLR 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We compare the performance of TAGI with EADL1 (Chen et al., 2018), PGDL2 (Madry et al., 2017), and CWL2 (Carlini and Wagner, 2017) on the MNIST (Le Cun et al., 1998) and CIFAR10 (Krizhevsky et al., 2009) data sets. ... Figure 5 shows the average reward over 100 episodes with respect to the number of steps for both environments. Table 3 presents the average reward over the last 100 episodes for both environments. |
| Researcher Affiliation | Academia | Luong-Ha Nguyen EMAIL James-A. Goulet EMAIL Department of Civil Engineering, Polytechnique Montréal, Montréal, Canada |
| Pseudocode | Yes | Algorithm 1: Optimization of a function using TAGI ... Algorithm 2: Continuous-action reinforcement learning with TAGI |
| Open Source Code | No | The paper does not provide an unambiguous statement or a direct link to open-source code for the methodology described. It references 'Open AI baselines' which is a third-party tool. |
| Open Datasets | Yes | We compare the performance of TAGI with EADL1 (Chen et al., 2018), PGDL2 (Madry et al., 2017), and CWL2 (Carlini and Wagner, 2017) on the MNIST (Le Cun et al., 1998) and CIFAR10 (Krizhevsky et al., 2009) data sets. ... We perform this comparison on the half-cheetah and inverted pendulum problems from the Mujoco environment (Todorov et al., 2012) implemented in Open AI Gym (Brockman et al., 2016). |
| Dataset Splits | Yes | We compare the performance of TAGI with EADL1 (Chen et al., 2018), PGDL2 (Madry et al., 2017), and CWL2 (Carlini and Wagner, 2017) on the MNIST (Le Cun et al., 1998) and CIFAR10 (Krizhevsky et al., 2009) data sets. ... Figure 5 shows the average reward over 100 episodes with respect to the number of steps for both environments. Table 3 presents the average reward over the last 100 episodes for both environments. |
| Hardware Specification | No | The paper mentions 'computational time of 18 seconds' but does not specify any hardware details like CPU, GPU models, or memory. |
| Software Dependencies | No | The paper mentions 'Open AI baselines (Dhariwal et al., 2017)' but does not provide specific version numbers for any software dependencies used in their own implementation. |
| Experiment Setup | Yes | For TAGI, we set σX = 0.03 with a maximal number of epochs E = 100. ... For the classification tasks trained with backpropagation, we employ the same training setup for both data sets in which the learning rate is 0.003, the number of epochs is 50, the batch size is 64, and the optimizer is Adam. ... The standard deviation σV in Equation 8 and 9 is initialized at 2 and is decayed each 1024 steps with a decaying factor of 0.9999. The minimal standard deviation is σmin V = 0.3. ... Table 10: Hyper-parameters for half-cheetah and inverted pendulum problems: Horizon 1024, Initial standard deviation for the value function (σV ) 2, Decay factor (η) 0.9999, Minimal standard deviation for the value function (σmin V ) 0.3, Batch size 16, Number of epochs 1, Discount (γ) 0.99. |