Data-driven Random Fourier Features using Stein Effect
Authors: Wei-Cheng Chang, Chun-Liang Li, Yiming Yang, Barnabás Póczos
IJCAI 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our empirical results on six benchmark data sets demonstrate the advantageous performance of this approach over representative baselines in both kernel approximation and supervised learning tasks. |
| Researcher Affiliation | Academia | Wei-Cheng Chang LTI, CMU wchang2@cs.cmu.edu Chun-Liang Li MLD, CMU chunlial@cs.cmu.edu Yiming Yang LTI, CMU yiming@cs.cmu.edu Barnab as P oczos MLD, CMU bapoczos@cs.cmu.edu |
| Pseudocode | Yes | Algorithm 1 Data-driven random feature (SES) 1: procedure SES 2: Compute the Fourier transform p of the kernel 3: Generate a sequence w1, . . . , w M by QMC or MC 4: Compute un-weighted random feature z(x) by (4) 5: Calculate shrinkage weight by (8) 6: Compute data-driven random features by z (x) = diag( p |
| Open Source Code | No | The paper provides links to the implementations of baseline methods ('https://github.com/chocjy/QMC-features' for QMC and 'http://www.cs.toronto.edu/ duvenaud/' for BQ), but it does not state that the code for its own proposed method (SES) is open source or publicly available. |
| Open Datasets | Yes | We empirically demonstrate the benefits of our proposed method on six data sets listed in Table 1. The features in all the six data sets are scaled to zero mean and unit variance in a preprocessing step. |
| Dataset Splits | Yes | For regression task, we solve ridge regression problems where the regularization coefficient λ is tuned by five folds cross-validation. For classification task, we solve L2-loss SVM where the regularization coefficient C is tuned by five folds cross-validation. |
| Hardware Specification | Yes | All experiments code are written in MATLAB and run on a Intel(R) Xeon(R) CPU 2.40GHz Linux server with 32 cores and 190GB memory. |
| Software Dependencies | No | The paper states 'All experiments code are written in MATLAB' but does not specify a version number for MATLAB or any other software dependencies. |
| Experiment Setup | Yes | Gaussian RBF kernel is used through all the experiments where the kernel bandwidth σ is tuned on the set {2 10, 2 8, . . . , 28, 210} that is in favor of Monte Carlo methods. ... In BQ framework, we consider the GP prior with the covariance matrix k GP (w, w ) = exp( w w 2 2 2σ2 GP ) where σGP is tuned on the set {2 8, 2 6, . . . , 26, 28} over a subset of sampled pair data (x, x ). Likewise, regularization coefficient λβ of in the proposed objective function (6) is tuned on the set {2 8, 2 6, . . . , 26, 28} over the same subset of sampled pair data. For regression task, we solve ridge regression problems where the regularization coefficient λ is tuned by five folds cross-validation. For classification task, we solve L2-loss SVM where the regularization coefficient C is tuned by five folds cross-validation. |