Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Improving Deep Learning Speed and Performance Through Synaptic Neural Balance
Authors: Antonios Alexos, Ian Domingo, Pierre Baldi
AAAI 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present theory of synaptic neural balance and we show experimentally that synaptic neural balance can improve deep learning speed, and accuracy, even in data-scarce environments. In our simulations, we systematically observe that: (1) Fully balancing before training results in better performance as compared to several other training approaches; (2) Interleaving partial (layer-wise) balancing and stochastic gradient descent steps during training results in faster learning convergence and better overall accuracy (with L1 balancing converging faster than L2 balancing); and (3) When given limited training data, neural balanced models outperform plain or regularized models; and this is observed in both feedforward and recurrent networks. The 'Experiments and Results' section further details empirical evaluation on various datasets and network architectures. |
| Researcher Affiliation | Academia | University of California, Irvine EMAIL, EMAIL, EMAIL |
| Pseudocode | No | The paper describes theoretical concepts and proofs (e.g., 'The complete theory is described in the Appendix with the detailed proofs of all the theorems.') and outlines methods in prose, but it does not contain any clearly labeled pseudocode or algorithm blocks in the main text. |
| Open Source Code | Yes | Our code is publicly available on github. Code https://github.com/antonyalexos/Neural-Balance |
| Open Datasets | Yes | We test all forms of neural balancing on the MNIST handwritten digit dataset exclusively through FCNs. Following the line of inquiry on the performance of neural balancing on FCNs trained on MNIST, we assess its performance on Fashion MNIST... We train a 3-layered RNN on the IMDB sentiment analysis dataset... |
| Dataset Splits | Yes | Specifically, the experiments are conducted using the MNIST dataset, wherein the training dataset is deliberately reduced to 600 samples, constituting only 1% of the original dataset. The test dataset remains unchanged... trained on the IMDB sentiment analysis dataset with 5% of the available training data. |
| Hardware Specification | No | The paper mentions 'computational intensity' in the context of accessing weights in a large network but does not provide any specific details about the hardware (e.g., GPU/CPU models, memory) used for conducting the experiments. |
| Software Dependencies | No | The paper does not explicitly state any software dependencies with specific version numbers (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | No | To ensure reproducibility and fairness, experiments comparing training methodologies use the same range of seeds, learning rates, and train/test splits. Every experiment was run with 8 different seeds and the result reported is the average of them. A more detailed description of our experimental setup can be found in the Appendix. While it mentions 'hyperparameters' and 'learning rates', it does not provide concrete values for these or other training parameters in the main text, instead deferring to an appendix. |