Dynamics-inspired Neuromorphic Visual Representation Learning
Authors: Zhengqi Pei, Shuhui Wang
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimentally, we first validate Dy N on MNIST (Deng, 2012) to verify its capacity as a universal approximator. ... Then, we transform the weight-based layers of many pre-trained deep neural models, e.g., Dense Net (Huang et al., 2017) and Swin T (Liu et al., 2021), to the Dy N alternatives, followed by fine-tuning on Image Net (Deng et al., 2009). As a result, the parameters have been reduced by 5–10. Still, the transformed models outperform the original ones on both Image Net and Web Vision (Li et al., 2017). ... The main results on three datasets are presented in Table 2 and Table 3. |
| Researcher Affiliation | Academia | Zhengqi Pei 1 2 Shuhui Wang 1 3 1Institute of Computing Technology, Chinese Academy of Sciences 2School of Artificial Intelligence, University of Chinese Academy of Sciences 3Peng Cheng Laboratory. Zhengqi Pei <peizhengqi22@mails.ucas.cn>. Correspondence to: Shuhui Wang <wangshuhui@ict.ac.cn>. |
| Pseudocode | Yes | Algorithm 1 Dy N learning for general neural structure Input: Neuronal dynamics Q, Desired Output T repeat Dummy States A = Rel(Q, Q) via Eq. 5 Stress force F = Grad(A, T) via Eq. 7 Update Q = Reduce(F, Q) via Eq. 6 until Q reaches equilibrium |
| Open Source Code | Yes | Codes are available1. 1https://github.com/pzqpzq/flat-learning |
| Open Datasets | Yes | Experimentally, we first validate Dy N on MNIST (Deng, 2012)... followed by fine-tuning on Image Net (Deng et al., 2009). ... We evaluate Dy N on three visual classification datasets, MNIST (Deng, 2012), Image Net (Deng et al., 2009)) and Web Vision (Li et al., 2017). |
| Dataset Splits | No | The training/testing splits of all the datasets follow the official settings. While it mentions following official settings, it does not explicitly provide the specific percentages or sample counts for training, validation, and test splits within the paper itself. |
| Hardware Specification | Yes | We fine-tune the models with one NVIDIA RTX3090 24GB GPU on a cloud server. The inference stage is implemented on a laptop with 32GB memory. |
| Software Dependencies | No | The paper mentions using "torch.hub" to get pre-trained models but does not provide specific software dependencies with version numbers (e.g., Python, PyTorch, CUDA versions). |
| Experiment Setup | Yes | For a fair comparison, we set the model configuration of the original ANNs and their Dy N alternatives (e.g., the number of hidden units, validation criterion, SEED, etc.) to be the same. and The dimension d defaults to 9 unless otherwise noted. and This process continues until the stress force amongst sub-models is lower than a certain threshold, e.g., 10−3 of the normalized distances between sub-models. |