Bayesian Extensive-Rank Matrix Factorization with Rotational Invariant Priors
Authors: Farzad Pourkamali, Nicolas Macris
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We provide numerical checks which confirm the optimality conjecture when confronted to Oracle Estimators which are optimal by definition, but involve the ground-truth. Our derivation relies on a combination of tools, namely random matrix theory transforms, spherical integral formulas, and the replica method from statistical mechanics. This is corroborated by numerical calculations comparing our explicit RIEs with Oracle Estimators which are optimal by definition and involve the ground-truth matrices. We look at the asymptotic regime where all matrix dimensions and ranks tend to infinity at the same speed. In figure 1, the MSEs of these algorithmic estimators are compared with the one of Oracle estimator (3). |
| Researcher Affiliation | Academia | Farzad Pourkamali & Nicolas Macris School of Computer and Communication Sciences, Ecole Polytechnique Fédérale de Lausanne {farzad.pourkamali,nicolas.macris}@epfl.ch |
| Pseudocode | Yes | Therefore, given an observation matrix S, the spectral algorithm proceeds as follows: 1. Compute the SVD of S. 2. Approximate G µS(z) from the singular values of S. 3. Construct the RIEs for X, Y as proposed in paragraphs 3.2, 3.3. |
| Open Source Code | No | The paper does not provide an explicit statement about the release of source code for the described methodology, nor does it include a link to a code repository. |
| Open Datasets | No | The paper describes generating synthetic data using statistical distributions (e.g., 'i.i.d. Gaussian entries', 'Wishart matrix', 'shifted Wigner matrix', 'uniform distribution on [1, 3]'), but it does not mention or provide access to any pre-existing publicly available datasets, benchmarks, or corpora. |
| Dataset Splits | No | The numerical experiments describe averaging results over multiple runs with specified matrix dimensions (e.g., 'N = 2000, M = 4000') but do not specify training, validation, or test dataset splits in terms of percentages, sample counts, or predefined partition citations. |
| Hardware Specification | No | The paper does not provide any specific hardware details such as GPU models, CPU types, or memory specifications used for running the experiments. |
| Software Dependencies | No | The paper does not specify any software dependencies or their version numbers, such as programming languages, libraries, or frameworks used for implementation. |
| Experiment Setup | Yes | We consider the case where Y , W both have i.i.d. Gaussian entries of variance 1/N, and X is a Wishart matrix, X = HH with H RN 4N having i.i.d. Gaussian entries of variance 1/N. For various SNRs, we examine the performance of two proposed estimators, the RIE (7), and the square-root of the estimator (10) (since X is PSD), which is sub-optimal. In figure 1, the MSEs of these algorithmic estimators are compared with the one of Oracle estimator (3). We see that the average performance of the algorithmic RIE d Ξ X(S) is very close to the (optimal) Oracle estimator Ξ X(S) (relative errors are small and provided in the appendices) and we believe that the slight mismatch is due to the numerical approximations and finite-size effects. RIE is applied to N = 2000, M = 4000, and the results are averaged over 10 runs (error bars are invisible). |