Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Fast Training of Large Kernel Models with Delayed Projections
Authors: Amirhesam Abedsoltan, Siyuan Ma, Parthe Pandit, Misha Belkin
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this paper, we present a new methodology for building kernel machines that can scale efficiently with both data size and model size. Our algorithm introduces delayed projections to Preconditioned Stochastic Gradient Descent (PSGD) allowing the training of much larger models than was previously feasible. We validate our algorithm, Eigen Pro 4, across multiple datasets, demonstrating drastic training speedups without compromising the performance. Section 4 Numerical experiments |
| Researcher Affiliation | Collaboration | Amirhesam Abedsoltan UC San Diego EMAIL Siyuan Ma Google EMAIL Parthe Pandit IIT Bombay EMAIL Mikhail Belkin UC San Diego EMAIL |
| Pseudocode | Yes | Algorithm 1 Eigen Pro 4 Algorithm 2 Eigen Pro 4-Exact |
| Open Source Code | Yes | Our implementation is publicly available at: https://github.com/Eigen Pro/Eigen Pro. |
| Open Datasets | Yes | We evaluate several kernel methods on the following datasets: (1) CIFAR5M, (2) CIFAR5M2 [17], (3) Image Net1 [6], (4) Web Vision2 [12], and (5) Libri Speech [18]. Dataset details are provided in Appendix C. Appendix C.3 Datasets: CIFAR5M. Image Net. Webvision. Librispeech [18] is a large-scale (1000 hours in total) corpus of 16 k Hz English speech derived from audio books. |
| Dataset Splits | Yes | Librispeech [18] is a large-scale (1000 hours in total) corpus of 16 k Hz English speech derived from audio books. We choose the subset train-clean-100 and train-clean-300 (5M samples) as our training data, test-clean as our test set. |
| Hardware Specification | Yes | This work used the Extreme Science and Engineering Discovery Environment (XSEDE) [23]. We used machines with NVIDIA-V100, NVIDIA-A100 and NVIDIA-A40 GPUs, with a V-RAM up to 1.3 T, and 8x cores of Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz with a RAM of 100 GB. NOte that we had 1.3T of RAM for just one experiment CIFAR5M , for the rest of expermients we where constraint with 400G of RAM. |
| Software Dependencies | No | The paper mentions 'timm library [25]' and 'ESPnet toolkit [24]' but does not provide specific version numbers for these or other software used in their implementation. |
| Experiment Setup | Yes | Appendix C.4 Experiments details: Figure 1 This experiment used CIFAR5M data set... We set the bandwidth to 5.0 and use 1k Nystrom samples with preconditioning level of size 100. We used float16 for this experiment. Figure 5 This experiment has been run over Webvision data set... The bandwidth used is 5.0, 1k Nystrom samples with preconditioning level of size 100. We used float16 for this experiment. Figure 3 We follow the setting in [1]. The bandwith used here is 20 for Librispeach and Webvision and 16 for imagnet. Here again we used extracted feature of these datasets mentioned earlier. The precision used here is float32. with 10k Nystrom samples with preconditioning level of size 1000. Table 1 For all datasets here we used bandwidth of 5.0 with 1k Nystrom samples with preconditioning level of size 100. We used float16 for all dataset except for Librispeach where we used float32. |