Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Explaining Generalization Power of a DNN Using Interactive Concepts
Authors: Huilin Zhou, Hao Zhang, Huiqi Deng, Dongrui Liu, Wen Shen, Shih-Han Chan, Quanshi Zhang
AAAI 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Specifically, we trained a seven-layer MLP (MLP-7-census) on the census dataset (Asuncion and Newman 2007) and a seven-layer MLP (MLP-7-TV) on the TV news dataset (Asuncion and Newman 2007), respectively. We also trained Alex Net (Krizhevsky, Sutskever, and Hinton 2017), Res Net-20 (He et al. 2016), and VGG-11 (Simonyan and Zisserman 2014) on the MNIST dataset (Le Cun et al. 1998) (Alex Net-MNIST, Res Net-20-MNIST, VGG-11-MNIST) and the CIFAR-10 dataset (Krizhevsky, Hinton et al. 2009) (Alex Net-CIFAR10, Res Net-20-CIFAR-10, VGG-11-CIFAR-10). ... Various experiments have verified our findings. |
| Researcher Affiliation | Academia | Huilin Zhou1, Hao Zhang1, Huiqi Deng1, Dongrui Liu1, Wen Shen1, Shih-Han Chan1, 2, Quanshi Zhang1* 1Shanghai Jiao Tong University 2University of California San Diego EMAIL EMAIL, EMAIL |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The code will be released when the paper is accepted. |
| Open Datasets | Yes | Specifically, we trained a seven-layer MLP (MLP-7-census) on the census dataset (Asuncion and Newman 2007) and a seven-layer MLP (MLP-7-TV) on the TV news dataset (Asuncion and Newman 2007), respectively. We also trained Alex Net (Krizhevsky, Sutskever, and Hinton 2017), Res Net-20 (He et al. 2016), and VGG-11 (Simonyan and Zisserman 2014) on the MNIST dataset (Le Cun et al. 1998) (Alex Net-MNIST, Res Net-20-MNIST, VGG-11-MNIST) and the CIFAR-10 dataset (Krizhevsky, Hinton et al. 2009) (Alex Net-CIFAR10, Res Net-20-CIFAR-10, VGG-11-CIFAR-10). |
| Dataset Splits | No | The paper mentions 'training samples' and 'testing samples' but does not provide explicit details about train/validation/test dataset splits (e.g., percentages, sample counts, or specific split files) needed for reproduction. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., exact GPU/CPU models, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details, such as library names with version numbers, needed to replicate the experiment. |
| Experiment Setup | Yes | Specifically, we trained a seven-layer MLP (MLP-7-census) on the census dataset... Each layer of the MLPs contained 100 neurons. ... We trained Alex Net, Res Net20 and VGG-11 with ρ = 0, 0.05, 0.1, 0.2, 0.3 noise on the MNIST dataset and the CIFAR-10 dataset. ... We trained a five-layer MLP on the dataset X = {0, 1}n... Here, n = 10. |