Inference Networks for Sequential Monte Carlo in Graphical Models
Authors: Brooks Paige, Frank Wood
ICML 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In testing we found that a hybrid training procedure, which samples new synthetic datasets based on performance on a held-out set of synthetic validation data, appeared more efficient than resampling a new synthetic dataset for each new gradient update. We perform mini-batch gradient updates on η using synthetic training data, while evaluating on the validation set. If the validation error increases, or after a set maximum number of steps, we draw new sets of both synthetic training and validation data from p(x, y). In all experiments we use Adam (Kingma and Ba, 2015) with the suggested default parameters to update learning rates online, and use rectified linear activation functions. 4. Examples 4.1. Inverting a single factor (...) The trained network used here 200 hidden units in each of two hidden layers, and a mixture of 3 Gaussians as each output. 4.2. A hierarchical Bayesian model (...) We test our proposals on the actual power pump failure data analyzed in George et al. (1993). The relative convergence speeds of marginal likelihood estimators from importance sampling from prior and neural network proposals, and SMC with neural network proposals, are shown in Figure 5. (...) 4.3. Factorial hidden Markov model (...) The effect of the learned proposals on the overall number of surviving particles is shown in Figure 6. |
| Researcher Affiliation | Academia | Brooks Paige BROOKS@ROBOTS.OX.AC.UK Frank Wood FWOOD@ROBOTS.OX.AC.UK Department of Engineering Science, University of Oxford |
| Pseudocode | No | The paper describes its methods in text and mathematical formulas but does not include any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any explicit statements about releasing source code or providing links to a code repository. |
| Open Datasets | Yes | We test our proposals on the actual power pump failure data analyzed in George et al. (1993). |
| Dataset Splits | No | The paper mentions using 'synthetic validation data' and that new training and validation sets are drawn if validation error increases, but it does not specify fixed percentages, sample counts, or a predefined split for reproducibility. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper mentions using 'Adam' for optimization but does not list any specific software packages or libraries with their version numbers that are required for replication. |
| Experiment Setup | No | The paper mentions using 'Adam with the suggested default parameters' but does not provide specific hyperparameter values (e.g., learning rate, batch size, number of epochs) or explicit system-level training settings for the experiments. |