Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Revitalizing SVD for Global Covariance Pooling: Halley’s Method to Overcome Over-Flattening
Authors: Jiawei Gu, Ziyue Qiao, Xinming Li, Zechao Li
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on CNNs and transformer architectures show that Halley-SVD consistently and robustly outperforms i SQRT-COV at large model scales and batch sizes, achieving higher overall accuracy without mid-training switches or custom truncations. |
| Researcher Affiliation | Academia | 1School of Computer Science and Engineering, Nanjing University of Science and Technology, 2School of Computing and Information Technology, Great Bay University, 3Beijing University of Civil Engineering and Architecture EMAIL,EMAIL EMAIL, EMAIL |
| Pseudocode | No | The paper describes iterative methods using mathematical equations (e.g., Equation 7 for Halley's iteration) and textual descriptions of steps, but it does not include a dedicated 'Pseudocode' or 'Algorithm' block or figure. |
| Open Source Code | Yes | The code implementation of our Halley-SVD method is provided in the supplementary materials, including the core algorithm, training scripts, and evaluation protocols. This enables full reproducibility of our experimental results. |
| Open Datasets | Yes | All experiments use publicly available datasets (Image Net, CUB-200, Stanford Dogs, Stanford Cars). ... For large-scale classification, we utilize the Image Net-1k dataset [6]... Fine-grained visual classification (FGVC) experiments are performed on Caltech-UCSD Birds 200 (Birds) [45], Stanford Dogs (Dogs) [5], and Stanford Cars (Cars) [22]. |
| Dataset Splits | Yes | For large-scale classification, we utilize the Image Net-1k dataset [6], employing the standard training/validation split. |
| Hardware Specification | Yes | All experiments were conducted using Py Torch [32] (version 1.12 or later) and executed primarily on NVIDIA A100 GPUs. |
| Software Dependencies | Yes | All experiments were conducted using Py Torch [32] (version 1.12 or later) |
| Experiment Setup | Yes | For our proposed Halley-SVD, we employ K = 8 iterations for the matrix square root approximation, as determined by ablation studies (see Appendix G.7). The iteration is initialized with X0 = 10-3I. ... For Image Net-1k training: CNNs (Res Net-50/101): We train for 100 epochs using SGD with momentum 0.9 and weight decay 1e-4. We employ a Cosine Annealing learning rate schedule with an initial learning rate of 0.1 for a standard batch size (BS) of 256, linearly scaled (LR = 0.1 * BS/256) for larger batch sizes (e.g., 0.8 for BS=2048). A 5-epoch linear warmup is used. Transformers (Vi T-B/16, Swin-T): We train for 300 epochs using the Adam W optimizer [30] with weight decay 0.05. A Cosine Annealing schedule is used with an initial learning rate of 1e-3 for a standard batch size of 1024, linearly scaled for larger batches (e.g., 4e-3 for BS=4096). A 20-epoch linear warmup is applied. |