Modeling Uncertainty with Hedged Instance Embeddings
Authors: Seong Joon Oh, Kevin P. Murphy, Jiyan Pan, Joseph Roth, Florian Schroff, Andrew C. Gallagher
ICLR 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical results on our new N-digit MNIST dataset show that our method leads to the desired behavior of hedging its bets across the embedding space upon encountering ambiguous inputs. This results in improved performance for image matching and classification tasks, more structure in the learned embedding space, and an ability to compute a per-exemplar uncertainty measure which is correlated with downstream performance. |
| Researcher Affiliation | Industry | Seong Joon Oh Kevin Murphy Jiyan Pan LINE Corporation Google Research Google Research coallaoh@linecorp.com kpmurphy@google.com jiyanpan@google.com Joseph Roth Florian Schroff Andrew Gallagher Google Research Google Research Google Research josephroth@google.com fschroff@google.com agallagher@google.com |
| Pseudocode | No | The paper does not contain any sections or figures explicitly labeled 'Pseudocode' or 'Algorithm'. |
| Open Source Code | No | The paper states, 'To evaluate our method, we propose a novel dataset, N-digit MNIST, which we will open source.' This refers to the dataset, not the source code for the methodology or experiments. |
| Open Datasets | Yes | We conduct all our experiments on a new dataset we created called N-digit MNIST, which consists of images composed of N adjacent MNIST digits, which may be randomly occluded (partially or fully). See appendix A for details. ... We will open source the data to ensure reproducibility. |
| Dataset Splits | No | The paper mentions 'training' and 'test' sets for N-digit MNIST in Table 3, and a 'held out test set' for the pet dataset. It does not explicitly define or provide details for a separate 'validation' set split. |
| Hardware Specification | No | The paper does not specify any hardware details such as CPU, GPU models, or cloud computing resources used for the experiments. |
| Software Dependencies | No | Our networks are built with Tensor Flow (Abadi et al., 2015). (No specific version number for TensorFlow is provided, nor for any other libraries or dependencies). |
| Experiment Setup | Yes | We use a batch size of 128 and 500k training iterations. Each model is trained from scratch with random weight initialization. The KL-divergence hyperparameter β is set to 10 4 throughout the experiments. ... The CNN portion of the model is a Mobile Net (Howard et al., 2017) with a width multiplier of 0.25. |