On 1/n neural representation and robustness
Authors: Josue Nassar, Piotr Sokol, Sueyeon Chung, Kenneth D. Harris, Il Memming Park
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this work, we empirically investigate the advantages of an 1/n neural code, by enforcing the spectral properties of the biological visual system in artificial neural networks. To this end, we propose a spectral regularizer to enforce a 1/n eigenspectrum. Our results show that imposing the experimentally observed structure on artificial neural networks makes them more robust to adversarial attacks. 4 Experiments |
| Researcher Affiliation | Academia | Josue Nassar Department of Electrical and Computer Engineering Stony Brook University Piotr Aleksander Sokol Department of Neurobiology and Behavior Stony Brook University Sue Yeon Chung Center for Theoretical Neuroscience Columbia University Kenneth D. Harris UCL Institute of Neurology University College London Il Memming Park Department of Neurobiology and Behavior Stony Brook University |
| Pseudocode | No | The paper provides mathematical equations and descriptions of methods but does not contain a structured pseudocode or algorithm block. |
| Open Source Code | Yes | Code is available at https://github.com/josuenassar/power_law |
| Open Datasets | Yes | To empirically investigate the benefits of a power-law neural representation and of the proposed regularization scheme, we train a variety of models on MNIST [30]. [30] Y. Le Cun, L. Bottou, Y. Bengio, P. Haffner, et al. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278 2324, 1998. |
| Dataset Splits | No | The paper does not provide specific details on training, validation, and test dataset splits (exact percentages, sample counts, or citations to predefined splits) beyond implicitly using the MNIST dataset which has a standard split, and mentioning batch size. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory, cloud instances) used for running the experiments. |
| Software Dependencies | No | The paper mentions the use of Adam optimizer but does not specify software versions for any libraries, frameworks, or programming languages (e.g., PyTorch 1.x, Python 3.x). |
| Experiment Setup | Yes | For all experiments three different values of β were tested, β ∈ {1, 2, 5}, and the results of the best one are shown in the main text... The networks were optimized using Adam [29], where a learning rate of 10^-4 was chosen... For each network, the batch size is chosen to be 1.5 times larger than the widest layer in the network... where we set ε = 0.01 [for PGD]... βj = 0.01 [for Jacobian regularization]. |