Inverse-Reference Priors for Fisher Regularization of Bayesian Neural Networks
Authors: Keunseo Kim, Eun-Yeol Ma, Jeongman Choi, Heeyoung Kim
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental Evaluation In this section, we evaluate the performance of the BNNs with the IR prior. Specifically, we analyze the results of experiments conducted using various benchmark datasets to demonstrate that the IR prior effectively reduces the trace of the FIM during training and improves the validation accuracy. |
| Researcher Affiliation | Collaboration | Keunseo Kim1*, Eun-Yeol Ma2, Jeongman Choi2, Heeyoung Kim2 1 Samsung Advanced Institute of Technology, Suwon, Republic of Korea 2 Department of Industrial and Systems Engineering, KAIST, Daejeon, Republic of Korea |
| Pseudocode | Yes | Algorithm 1: Algorithm for training the BNN with the IR prior |
| Open Source Code | No | The paper does not provide any explicit statements or links regarding the public availability of source code for the described methodology. |
| Open Datasets | Yes | We used three benchmark image datasets, CIFAR-10, CIFAR-100, and SVHN, for the experiments. |
| Dataset Splits | Yes | We evaluated the validation accuracy of the BNNs with the four considered priors using the CIFAR-10, SVHN, and CIFAR100 datasets. We compared the average validation accuracy over five repeated experiments using different initial values of the BNNs in Table 1 with standard errors in parentheses. We employed the validation accuracy as a metric of the generalization ability because it measures the performance on unseen data points. The validation accuracy reported in Table 1 represents the maximum validation accuracy recorded during 100 training epochs. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used for running the experiments, such as GPU or CPU models, or memory specifications. |
| Software Dependencies | No | The paper mentions optimizers and specific methods, but it does not specify any software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | We used a radial distribution (Farquhar, Osborne, and Gal 2020) as the variational posterior distribution and used the Adam optimizer (Kingma and Ba 2014) with learning rate set to 0.001. The validation accuracy reported in Table 1 represents the maximum validation accuracy recorded during 100 training epochs. |