Ridgeless Regression with Random Features
Authors: Jian Li, Yong Liu, Yingying Zhang
IJCAI 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 6 Experiments We use random Fourier feature defined in (8) to approximate the Gaussian kernel K(x, x ) = exp( σ2 x x 2/2). Note that, the corresponding random Fourier features (8) is with the frequency matrix Ω N(0, σ2). We implement all code on Pytorch 2 and tune hyperparameters over σ2 {0.01, , 1000}, λ = {0.1, , 10 5} by grid search. Table 1: Classification accuracy (%) for classification datasets. |
| Researcher Affiliation | Academia | Jian Li1 , Yong Liu2 , Yingying Zhang3 1Institute of Information Engineering, Chinese Academy of Sciences 2Gaoling School of Artificial Intelligence, Renmin University of China 3School of Mathematics and Information Sciences, Yantai University |
| Pseudocode | Yes | Algorithm 1 Ridgeless RF with Tunable Kernels (RFTK) Input: Training data (X, y) and feature mapping φ : Rd RD. Hyparameters σ2, β, T, b, γ, η and s. Output: The ridgeless RF model w T and the learned Ω. |
| Open Source Code | Yes | 2https://github.com/superlj666/Ridgeless-Regression-with-Random-Features |
| Open Datasets | Yes | We conduct experiments on the entire MNIST datasets. Table 1: Classification accuracy (%) for classification datasets. We bold the results with the best method and underline the ones that are not significantly worse than the best one. dna, letter, pendigits, segment, satimage, usps, svmguide2, vehicle, wine, shuttle, Sensorless. |
| Dataset Splits | Yes | To obtain stable results, we run methods on each dataset 10 times with randomly partition such that 80% data for training and 20% data for testing. |
| Hardware Specification | No | The paper states "We implement all code on Pytorch" but does not specify any hardware components such as GPU or CPU models, or memory specifications. |
| Software Dependencies | No | The paper mentions "Pytorch" as the implementation framework but does not provide a specific version number or other software dependencies with versioning (e.g., Python version, specific libraries). |
| Experiment Setup | Yes | We implement all code on Pytorch and tune hyperparameters over σ2 {0.01, , 1000}, λ = {0.1, , 10 5} by grid search. We set b = 32 and 100 epochs for the training, and thus the stop iteration is T = 100n/32. Before the training, we tune the hyperparameters σ2, λ, γ, β via grid search for algorithms on each dataset. |