Full Bayesian Significance Testing for Neural Networks
Authors: Zehua Liu, Zimeng Li, Jingyuan Wang, Yue He
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | A range of experiments on both simulated and real data are conducted to show the advantages of our method.The main contributions can be summarized as follows:We conduct extensive experiments to verify the advantage of our method on better testing results. |
| Researcher Affiliation | Academia | Zehua Liu1, Zimeng Li1, Jingyuan Wang1,2,3*, Yue He4 1School of Computer Science and Engineering, Beihang University, Beijing, China 2School of Economics and Management, Beihang University, Beijing, China 3Key Laboratory of Data Intelligence and Management (Beihang University), Ministry of Industry and Information Technology, Beijing, China 4Department of Computer Science and Technology, Tsinghua University, Beijing, China |
| Pseudocode | No | The paper describes procedures and steps in prose but does not include any formally structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any information or links regarding the availability of open-source code for the described methodology. |
| Open Datasets | Yes | The energy efficiency dataset comprises 768 samples and 8 features. It aims to predict the dependent target y (HL, heating load), which determines the specifications of the heating equipment needed to maintain comfortable indoor air conditions. The descriptions of the features and target are shown in the Appendix. From figure 4, we find that testing results are more concentrated around one when x8 equals zero, while others are not. It indicates that the instance-wise significance of x8 is different under different values, insignificant if its value is zero. The research in (Tsanas and Xifara 2012) confirms our findings.The testing problem for MNIST is defined as testing each pixel of a digit image and distinguishing significant pixels from insignificant pixels for the target. |
| Dataset Splits | No | The paper describes the generation of simulated datasets (Dataset 1, 2, 3) and mentions using real-world datasets (Energy Efficiency, MNIST), but it does not specify any training, validation, or test splits for these datasets. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used to run the experiments, such as GPU or CPU models. |
| Software Dependencies | No | The paper does not specify any software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions, or specific libraries). |
| Experiment Setup | No | The paper describes the data generation process for simulated data and mentions the use of Bayesian neural networks and methods like VI and KDE. However, it lacks specific experimental setup details such as neural network architectures (beyond hidden layer count), hyperparameter values (learning rate, batch size, epochs), or optimization settings for the models used in experiments. |