Imbalance Trouble: Revisiting Neural-Collapse Geometry
Authors: Christos Thrampoulidis, Ganesh Ramachandra Kini, Vala Vakilian, Tina Behnia
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We then present extensive experiments on synthetic and real datasets that confirm convergence to the SELI geometry. However, we caution that convergence worsens with increasing imbalances. We numerically test convergence to the SELI geometry in both synthetic and real classimbalanced datasets. For different imbalance levels, the learned geometries approach the SELI geometry significantly faster compared to the ETF geometry (Fig. 2). UFM: SGD solutions. Fig. 4 investigates whether the solutions found by SGD are consistent with the prediction of Theorem 1 about global minimizers of the UF-SVM. Deep-learning experiments. We investigate convergence to SELI in deep-net training of (R,ρ = 1/2)-STEP imbalanced MNIST, Fashion-MNIST and CIFAR10 datasets with Res Net-18 [10] and VGG-13 [29]; see Sec. G.1.1 for implementation details. |
| Researcher Affiliation | Academia | Christos Thrampoulidis , Ganesh R. Kini , Vala Vakilian , Tina Behnia University of British Columbia, University of California, Santa Barbara. |
| Pseudocode | No | The paper does not contain any sections or figures explicitly labeled as 'Pseudocode' or 'Algorithm'. |
| Open Source Code | Yes | [Yes] Our code scripts and instructions are included |
| Open Datasets | Yes | Deep-learning experiments. We investigate convergence to SELI in deep-net training of (R,ρ = 1/2)-STEP imbalanced MNIST, Fashion-MNIST and CIFAR10 datasets |
| Dataset Splits | No | The paper mentions 'zero training-error epoch' and 'zero training error' but does not explicitly specify validation dataset splits, percentages, or methodology for a validation set in the main text. |
| Hardware Specification | No | The paper states 'Supported by an NSERC Undergraduate Student Research Grant, an NSERC Discovery Grant, NSF CCF-2009030, a CRG8-KAUST award, and by UBC Advanced Research Computing services.' and in the ethics section confirms reporting of compute and resources, but the main text (Section 5) does not provide specific hardware details such as GPU/CPU models or memory amounts used for the experiments. |
| Software Dependencies | No | The paper mentions the use of 'CVX [6]' in Section 5, but it does not provide specific version numbers for this or any other software components used in the experiments. |
| Experiment Setup | Yes | UFM: SGD solutions. ... We fix k = 4 and, for each R, we select nmin so that n = ((R + 1)/2)knmin 400. The weights of the UFM are optimized using SGD with constant learning rate 0.4, batch size 4 and no weight decay. We train for 105 epochs, much beyond zero training error |