Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Progressive Data Dropout: An Embarrassingly Simple Approach to Train Faster
Authors: Shriram M S, Xinyue Hao, Shihao Hou, Yang Lu, Laura Sevilla-Lara, Anurag Arnab, Shreyank Gowda
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experimental results demonstrate that models trained with our approach achieve comparable or superior accuracy while requiring only 0.124 the number of effective epochs compared to standard training procedures. This efficiency gain is particularly notable for already-efficient architectures like Mobile Net and Efficient Former, where further reducing training costs can significantly impact deployment scenarios with constrained resources. We plot some of these results on CIFAR-100 Krizhevsky and Hinton [2009] in Figure 1. 5 Experimental Analysis |
| Researcher Affiliation | Academia | 1Department of Computer Science, University of Manchester 2School of Informatics, University of Edinburgh 3School of Informatics, Xiamen University 4School of Computer Science, University of Nottingham |
| Pseudocode | Yes | A full algorithm explaining the approach can be found in the supplementary material. Algorithm 1: Unified algorithm for all 3 variants, the common steps between DBPD and SMRD are in blue, specific steps for SRD is in violet and steps for SMRD is in red |
| Open Source Code | Yes | Code can be found here: https://github.com/bazyagami/ Learning With Revision. |
| Open Datasets | Yes | We conduct all classification experiments using three standard image classification benchmarks: CIFAR-10 Krizhevsky and Hinton [2009], CIFAR-100 Krizhevsky and Hinton [2009], and Image Net Deng et al. [2009]. |
| Dataset Splits | Yes | The CIFAR-10 and CIFAR-100 datasets each consist of 60,000 32 32 color images, split into 50,000 training and 10,000 test images. We also use the Image Net (ILSVRC2012) dataset, which contains over 1.28 million training images and 50,000 validation images across 1,000 object categories. |
| Hardware Specification | Yes | We use 8 A100 GPUs with 40GB memory. For Image Net fine-tuning we use 1 A100 GPU with 40GB memory. For CIFAR10 and CIFAR100, we use a 4060RTX GPU with 8GB memory. |
| Software Dependencies | No | We use an Adam W optimizer with a Step LR scheduler with a step size of 1. Our initial learning rate is 0.0003. All results have been conducted following official implementations on timm Wightman [2019]. |
| Experiment Setup | Yes | We follow the official implementation details for Image Net experiments, so for example Efficient Net is run for 350 epochs, while Res Net-50 is run for 100 epochs. However, for all CIFAR-100 experiments we run all models for 200 epochs and for all CIFAR-10 experiments we run them for 30 epochs. For our CIFAR experiments, we use an Adam W optimizer with a Step LR scheduler with a step size of 1. Our initial learning rate is 0.0003. For the self-supervised learning experiments with MAE He et al. [2022], we follow standard practice He et al. [2016] and run the pre-training phase for 800 epochs. We use a fixed batch size of 32 for all our experiments. |