Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
BP-Modified Local Loss for Efficient Training of Deep Neural Networks
Authors: REN Lianhai, Qianxiao Li
ICLR 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Numerical experiments on full-tuning and Lo Kr tuning on the Res Net-50 model and Lo RA tuning on the Vi T-b16 model on CIFAR100 datasets show 20.5% test top-1 accuracy improvement for the Forward-Forward algorithm, 18.6% improvement for Lo Co algorithm and achieve only an average 7.7% of test accuracy loss compared to the BP algorithm, with up to 75% memory savings. |
| Researcher Affiliation | Academia | Lianhai Ren Department of Mathematics National University of Singapore 10 Lower Kent Ridge Road, Singapore 119076 EMAIL Qianxiao Li Department of Mathematics and Institute for Functional Intelligent Materials, National University of Singapore 10 Lower Kent Ridge Road, Singapore 119076 EMAIL |
| Pseudocode | Yes | Algorithm 1 BP-modified Local Loss Algorithm |
| Open Source Code | No | The paper does not contain any explicit statement about releasing their code or a link to a code repository for the methodology described. |
| Open Datasets | Yes | We conducted several experiments on CIFAR100 (Krizhevsky, 2009) and Image Net-Tiny (mnmoustafa & Ali, 2017) dataset |
| Dataset Splits | No | The paper mentions using CIFAR100 and Image Net-Tiny datasets but does not explicitly state the training, validation, or test splits used for the experiments. It implies standard usage but does not provide specific percentages or counts. |
| Hardware Specification | Yes | All the experiments were conducted using a single NVIDIA Ge Force RTX 3090 GPU using Py Torch (Paszke et al., 2019). |
| Software Dependencies | No | The paper mentions using PyTorch but does not specify a version number for PyTorch or any other software dependencies such as Python or CUDA. |
| Experiment Setup | Yes | All the experiments use batch size B = 64, learning rate 0.001 and epoch 30, and SGD optimizer with momentum 0.9. For the BP-modified local loss method, the batch size to compute the offset is B = 320 and split into mini-batches with size 8, and the period of the offset update is one epoch, i.e. K = N / B, where N is the size of the training dataset. The input image is resized to (224,224) and then normalized without any other enhancement before training. For the Lo RA and Lo Kr, the rank and alpha are set to be r = 1, α = 4. |