CoLA: Exploiting Compositional Structure for Automatic and Efficient Numerical Linear Algebra
Authors: Andres Potapczynski, Marc Finzi, Geoff Pleiss, Andrew G. Wilson
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We showcase its efficacy across a broad range of applications, including partial differential equations, Gaussian processes, equivariant model construction, and unsupervised learning. We showcase the extraordinary versatility of Co LA with a broad range of applications in Section 3.2 and Section 4, including: PCA, spectral clustering, multi-task Gaussian processes, equivariant models, neural PDEs, random Fourier features, and PDEs like minimal surface or the Schrödinger equation. Not only does Co LA provide competitive performance to specialized packages but it provides significant speedups especially in applications with compositional structure (Kronecker, block diagonal, product, etc). |
| Researcher Affiliation | Academia | Andres Potapczynski 1 Marc Finzi 2 Geoff Pleiss3,4 Andrew Gordon Wilson1 1New York University, 2Carnegie Mellon University, 3University of British Columbia, 4Vector Institute |
| Pseudocode | Yes | Algorithm 1 Arnoldi iteration (Appendix C.3, page 10). Algorithm 2 Householder Arnoldi iteration (Appendix C.3, page 10). |
| Open Source Code | Yes | Our package is available at https://github.com/wilson-labs/cola. |
| Open Datasets | Yes | Most of the datasets are sourced from the University of California at Irvine s (UCI) Machine Learning Respository that can be found here: https://archive.ics.uci.edu/ml/datasets.php. Also, a community repo hosting these UCI benchmarks can be found here: https://github.com/treforevans/uci_datasets (we have no affiliation). |
| Dataset Splits | No | The paper describes the total size of datasets used (e.g., 'N = 1K' for synthetic Gaussian, 'N = 430K' for Buzz, 'N = 12.5K' for Elevators, 'm = 50K' for Neural-IVP) but does not provide explicit training, validation, or test splits (e.g., percentages, sample counts, or specific cross-validation schemes). |
| Hardware Specification | Yes | In terms of hardware, the CPU experiments were run on an Intel(R) Core(TM) i5-9600K CPU @ 3.70GHz and the GPU experiments were run on a NVIDIA Ge Force RTX 2080 Ti. |
| Software Dependencies | No | The paper mentions several software frameworks and libraries such as 'Py Torch', 'JAX', 'Sci Py', 'GPy Torch', and 'sklearn' but does not provide specific version numbers for any of them. |
| Experiment Setup | Yes | We used conjugate gradients (CG) as the iterative method, where we set the hyperparameters to a tolerance of 10-6 and to a maximum number of iterations to 1K. We used the exact same hyperparameters for Co LA. For the iterative method we used power iteration with a maximum number of iterations of 300 and a stop tolerance of 10-7. Co LA used SVRG also with the same stop tolerance and maximum number of iterations. Additionally, we set SVRG s batch size to 10K and the learning rate to 0.0008. |