Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Do Invariances in Deep Neural Networks Align with Human Perception?
Authors: Vedant Nanda, Ayan Majumdar, Camila Kolling, John P. Dickerson, Krishna P. Gummadi, Bradley C. Love, Adrian Weller
AAAI 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We then conduct an in-depth investigation of how different components (e.g. architectures, training losses, data augmentations) of the deep learning pipeline contribute to learning models that have good alignment with humans. Code: github.com/nvedant07/Human-NN-Alignment. We strongly recommend reading the arxiv version of this paper: https://arxiv.org/abs/2111.14726. |
| Researcher Affiliation | Academia | Vedant Nanda1,2, Ayan Majumdar2, Camila Kolling2, John P. Dickerson1, Krishna P. Gummadi2, Bradley C. Love3,4, Adrian Weller3,5 1University of Maryland, College Park, USA 2Max Planck Institute for Software Systems (MPI-SWS), Germany 3The Alan Turing Institute, London, England 4University College London, London, England 5University of Cambridge, Cambridge, England |
| Pseudocode | No | No explicit pseudocode or algorithm blocks labeled as 'Pseudocode' or 'Algorithm' were found. |
| Open Source Code | Yes | Code: github.com/nvedant07/Human-NN-Alignment. |
| Open Datasets | Yes | Feather et al. studied representational invariance for different layers of DNNs trained over Image Net data (using the standard cross-entropy loss). |
| Dataset Splits | No | No specific details on train/validation/test splits, percentages, or cross-validation setup are explicitly provided in the main text. |
| Hardware Specification | No | No specific hardware details (e.g., GPU models, CPU types, or cloud instance specifications) used for running experiments are provided in the paper. |
| Software Dependencies | No | The paper mentions software like 'Py Torch Lightning', 'Pytorch', 'Num Py', 'Matplotlib', and 'robustness (Python Library)' but does not provide specific version numbers for these dependencies. |
| Experiment Setup | Yes | This is achieved by performing gradient descent on x0 (in our experiments we use SGD with a learning rate of 0.1) to minimize a loss of the following general form: |