Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Sinusoidal Initialization, Time for a New Start
Authors: Alberto Fernandez-Hernandez, Jose Mestre, Manuel F. Dolz, José Duato, Enrique Quintana-Orti
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through extensive empirical evaluations involving CNNs, including some efficient architectures, Vision Transformers, and language models, we rigorously demonstrate the empirical superiority of the Sinusoidal initialization over standard stochastic initializations, demonstrating faster convergence and higher final accuracy. On average, our experiments show an increase of 4.9% in final validation accuracy and 20.9% in convergence speed. |
| Researcher Affiliation | Collaboration | 1Universitat Politècnica de València 2Universitat Jaume I 3Openchip & Software Technologies S.L. |
| Pseudocode | No | The paper describes methods and theoretical foundations, but does not include any explicitly labeled pseudocode or algorithm blocks. It presents formulas and mathematical proofs. |
| Open Source Code | Yes | For full reproducibility, all code used to conduct the experiments is publicly available in the accompanying Git Hub repository: https://github.com/. |
| Open Datasets | Yes | Our experimental validation specifically addresses two main hypotheses: 1) Accuracy improvement: The Sinusoidal initialization achieves higher final validation accuracy compared to alternative initialization schemes in a significant proportion of cases. 2) Accelerated convergence: The Sinusoidal initialization converges faster than any other initialization considered, as measured by the Area Under the Curve (AUC) metric, which besides for classification evaluation, can be used to compare the training convergence speeds [Huang and Ling, 2005]. To evaluate these hypotheses rigorously, we conducted experiments involving five diverse configurations of DNNs and datasets: Res Net-50 [He et al., 2015a] trained on CIFAR-100 Krizhevsky et al. [2009], Mobile Net V3 [Howard et al., 2019] trained on Tiny-Image Net Le and Yang [2015], Efficient Net V2 [Tan and Le, 2021] trained on Tiny-Image Net, Vi T-B16 [Dosovitskiy et al., 2021] trained on Image Net-1K Russakovsky et al. [2015]), and BERT-mini [Devlin et al., 2019] trained on Wiki Text Merity et al. [2016]. |
| Dataset Splits | Yes | Note that all dataset splits used correspond to the default configurations |
| Hardware Specification | Yes | Table 5: Hardware and runtime per single training experiment. Model Dataset GPU Training time (hours) Res Net-50 CIFAR-100 NVIDIA A100-SXM4-80GB 2 Mobile Net V3 Tiny-Image Net NVIDIA A100-SXM4-80GB 4 Efficient Net V2 Tiny-Image Net NVIDIA A100-SXM4-80GB 4 Vi T-B16 Image Net-1k NVIDIA H100-PCIe-94GB 168 BERT-mini Wiki Text NVIDIA A100-SXM4-80GB 2 |
| Software Dependencies | No | The paper mentions Torchvision and Hugging Face for models and datasets, and refers to the authors original repository for LSUV (github.com/ducha-aiki/lsuv), but does not provide specific version numbers for these or other software components like Python, PyTorch, or CUDA. |
| Experiment Setup | Yes | The tables below detail the training hyperparameters (Table 4) and hardware (Table 5) used for each model-dataset pair. Table 4: Training hyperparameters for all evaluated models. Model Dataset Epochs Optimizers LR WD Batch size Res Net-50 CIFAR-100 100 SGD, Adam, Adam W 10 3 10 3 64 Mobile Net V3 Tiny-Image Net 200 SGD, Adam, Adam W 10 3 10 3 64 Efficient Net V2 Tiny-Image Net 200 SGD, Adam, Adam W 10 3 10 3 64 Vi T-B16 Image Net-1k 100 SGD 10 3 10 3 64 BERT-mini Wiki Text 200 Adam W 5 10 5 10 3 16 |