Bayesian Neural Networks Avoid Encoding Complex and Perturbation-Sensitive Concepts
Authors: Qihan Ren, Huiqi Deng, Yunuo Chen, Siyu Lou, Quanshi Zhang
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments verify our theoretical proofs. ... We conducted experiments to verify the above conclusions. ... Figure 3 shows that the average variance V (s) noise and V (s) BNN increased exponentially along with the order s, while the relative stability K(s) noise and K(s) BNN both decreased along with the order. This demonstrated that high-order interactive concepts were much more sensitive to input perturbations and the weight uncertainty in the BNN, thereby verifying Theorem 2.2 and Theorem 2.4. |
| Researcher Affiliation | Academia | 1Shanghai Jiao Tong University, Shanghai, China 2Quanshi Zhang is the corresponding author. He is with the Department of Computer Science and Engineering, the John Hopcroft Center, at the Shanghai Jiao Tong University, China. |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. It describes mathematical proofs and experimental procedures in narrative text. |
| Open Source Code | Yes | The code is available at https://github.com/sjtu-xai-lab/BNN-concepts. |
| Open Datasets | Yes | We trained BNNs on image datasets and tabular datasets to verify the quality of using the surrogate DNN model to approximate the feature distribution of the BNN. For image datasets, we tested BNNs with two architectures. For the MNIST dataset (Le Cun et al., 1998), we constructed a BNN with the architecture of a 5-layer MLP. We also tested a BNN with the Le Net architecture (Le Cun et al., 1998), which was trained on the CIFAR-10 dataset (Krizhevsky et al., 2009). We used two tabular datasets, including the UCI TV news dataset (termed TV news) and the UCI census income dataset (termed Census) (Dua & Graff, 2017). |
| Dataset Splits | No | The paper mentions training, but does not specify explicit training/validation/test dataset splits with percentages or sample counts. It refers to standard datasets but doesn't detail the splits used for them, nor does it specify splits for the tabular datasets. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper mentions using the 'Adam optimizer (Kingma & Ba, 2015)' but does not specify version numbers for any software, libraries, or programming languages used in the implementation. |
| Experiment Setup | Yes | For the training of BNNs, the prior distribution of network weights was set to N(W ; 0, I), and the number of Monte Carlo sampling of network weights was set to 1. All standard DNNs and BNNs were trained using the Adam optimizer (Kingma & Ba, 2015) with learning rate 0.001. The 5-layer MLPs (standard DNN and BNN) on the MNIST dataset was trained for 50 epochs. The Le Net (standard DNN and BNN) on the CIFAR-10 dataset was trained for 300 epochs. The 8-layer MLPs (standard DNN and BNN) on tabular datasets were trained for 200 epochs. |