Hessian Eigenspectra of More Realistic Nonlinear Models
Authors: Zhenyu Liao, Michael W. Mahoney
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We also demonstrate our results empirically by showing that: 4. for a given response model (1), the Hessian eigenvalue distribution depends on the choice of loss function and the data/feature statistics in an intrinsic manner, e.g., bounded versus unbounded support and singleversus multi-bulk in Fig 2; and 5. there may exist two qualitatively different spikes one due to data signal µ and the other due to w or w and thus the underlying model which may appear on different sides of the main bulk, and their associated phase transition behaviors are characterized (Fig 4 versus 5). |
| Researcher Affiliation | Academia | Zhenyu Liao School of Electronic Information and Communications Huazhong University of Science and Technology, China zhenyu_liao@hust.edu.cn Michael W. Mahoney ICSI and Department of Statistics University of California, Berkeley, USA mmahoney@stat.berkeley.edu |
| Pseudocode | No | The paper does not contain pseudocode or a clearly labeled algorithm block. |
| Open Source Code | No | The paper does not provide an unambiguous statement or link for the open-source code of the methodology described. |
| Open Datasets | No | The paper uses synthetic data generated with parameters such as 'xi i.i.d. N(µ, C)' and specified dimensions (e.g., 'p = 800, n = 6 000'), but does not mention or provide access information for a publicly available or open dataset in the traditional sense. |
| Dataset Splits | No | The paper does not explicitly provide training/test/validation dataset splits, as its empirical components are demonstrations based on theoretical models and generated data rather than typical machine learning experiments with distinct data splits. |
| Hardware Specification | No | The paper does not explicitly describe the hardware used to run its experiments. |
| Software Dependencies | No | The paper does not provide a reproducible description of ancillary software with specific version numbers. |
| Experiment Setup | Yes | Fig 1: 'eigenspectral properties of the Hessian of G-GLMs with p = 800, n = 6 000 and C = Ip.' Fig 2: 'p = 800, n = 6 000, logistic model in (2) with µ = 0, C = Ip, w = 0 and w = [ 1p/2, 1p/2]/ p.' Fig 4: 'p = 512 and n = 2 048.' Fig 5: 'p = 800 and n = 8 000.' |