Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Local Intrinsic Dimensional Entropy
Authors: Rohan Ghosh, Mehul Motani
AAAI 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments and Discussions Here we showcase two experiments, where we compute the ID-Entropy of the feature layers of classifier and autoencoder architectures on MNIST and CIFAR-10, and contrast it with generalization performance. |
| Researcher Affiliation | Academia | Rohan Ghosh1, Mehul Motani1,2 1 College of Design and Engineering, Department of Electrical and Computer Engineering, National University of Singapore 2 N.1 Institute for Health, Institute for Digital Medicine (Wis DM), Institute of Data Science, National University of Singapore |
| Pseudocode | Yes | Algorithm 1: Estimation of ID-Entropy Input: S = {X1, .., Xm} (i.i.d samples of RV X), a global ID estimator f ID(S ) of points in S , & parameters (k, n). Output: IDX (Estimate of ID-Entropy of X) 1: IDsum = 0 2: for j = 1, 2m/n , 3m/n , .. . . . , m do 3: Let S = k-nearest neighbors of Xj in S 4: IDsum = IDsum + f ID(S ) 5: IDX = IDsum/n |
| Open Source Code | Yes | code is available at: https://github.com/kentridgeai/ID-Entropy. |
| Open Datasets | Yes | Experiments and Discussions Here we showcase two experiments, where we compute the ID-Entropy of the feature layers of classifier and autoencoder architectures on MNIST and CIFAR-10 |
| Dataset Splits | No | The paper mentions 'training data size' and 'test accuracy' for MNIST and CIFAR-10 but does not specify explicit training, validation, or test dataset splits (e.g., percentages or sample counts). |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU models, CPU types, or memory specifications used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers (e.g., Python, TensorFlow, PyTorch versions). |
| Experiment Setup | Yes | For all experiments we estimated the ID-entropy using Algorithm 1, with n = 2000 and k = 100. We used Fisher’s intrinsic dimensionality estimator in (Bac and Zinovyev 2020) as the global ID estimator f ID, as we found it to be a robust choice. With a fixed 4-layer CNN architecture for MNIST and a Res Net-44 for CIFAR-10, we repeat the training routine with different choices of the training data size and random network initializations. For both MNIST (4-layer CNN) and CIFAR-10 (Res Net-44), we add label noise by randomly changing the label of a training data point with a certain probability. |