Precise asymptotic generalization for multiclass classification with overparameterized linear models
Authors: David Wu, Anant Sahai
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We study the asymptotic generalization of an overparameterized linear model for multiclass classification under the Gaussian covariates bi-level model introduced in Subramanian et al. (2022), where the number of data points, features, and classes all grow together. We fully resolve the conjecture posed in Subramanian et al. (2022), matching the predicted regimes for generalization. Furthermore, our new lower bounds are akin to an information-theoretic strong converse: they establish that the misclassification rate goes to 0 or 1 asymptotically. The key to our tight analysis is a new variant of the Hanson-Wright inequality which is broadly useful for multiclass problems with sparse labels. |
| Researcher Affiliation | Academia | David X. Wu Department of EECS UC Berkeley Berkeley, CA 94720 david_wu@berkeley.edu Anant Sahai Department of EECS UC Berkeley Berkeley, CA 94720 sahai@eecs.berkeley.edu |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any explicit statement or link for open-source code for the described methodology. |
| Open Datasets | No | The paper analyzes a theoretical 'Gaussian covariates bi-level model' and does not use or provide access to a public dataset for training. |
| Dataset Splits | No | The paper does not describe experiments with dataset splits for training, validation, or testing. |
| Hardware Specification | No | The paper is theoretical and does not report on experimental hardware specifications. |
| Software Dependencies | No | The paper is theoretical and does not list specific software dependencies with version numbers. |
| Experiment Setup | No | The paper is theoretical and does not include details about an experimental setup, such as hyperparameters or training settings. |