Improved Kernel Alignment Regret Bound for Online Kernel Learning
Authors: Junfan Li, Shizhong Liao
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments In this section, we verify the following two goals. We adopt the Gaussian kernel κ(x, v) = exp( x v 2 2σ2 ). We choose three classification datasets (w8a:49,749, magic04:19,020, mushrooms:8,124) from UCI machine learning repository 4. More experimental results are shown in the supplementary materials. We do not compare with OGD, since it suffers O(dt) per-round time complexity which is prohibitive. We compare with some variants of OGD, including FOGD, NOGD (Lu et al. 2016) and Ske GD (Zhang and Liao 2019). We also compare with B(AO)2KS (Liao and Li 2021) which is an approximation of OMD. We exclude BOGD (Zhao et al. 2012), since its regret bound is same with FOGD and its performance is worse than FOGD. For all baseline algorithms, we tune the stepsize of gradient descent from 10[ 3:1:3]/ T. For the other parameters, we follow the original papers. For POMDR, we set B0 = 15 ln T , B = 400, M = 15, U = 25 and multiply by a constant c {0.05, 0.1} on the learning rate λt (see (10)). Such values of parameters do not change our regret bound. For the ALD condition (see (5)), we set the threshold 10T ζ, ζ {0.5, 2/3}. For all algorithms, we tune the kernel parameter σ 2[ 2:1:6]. We randomly permutate the examples in the datasets 10 times and report the average results. All algorithms are implemented with R on a Windows machine with 2.8 GHz Core(TM) i7-1165G7 CPU 5. The codes: https://github.com/Junf Li-TJU/OKL-Hinge |
| Researcher Affiliation | Academia | College of Intelligence and Computing, Tianjin University, Tianjin 300350, China |
| Pseudocode | Yes | Algorithm 1: POMDR |
| Open Source Code | Yes | The codes: https://github.com/Junf Li-TJU/OKL-Hinge |
| Open Datasets | Yes | We choose three classification datasets (w8a:49,749, magic04:19,020, mushrooms:8,124) from UCI machine learning repository 4. |
| Dataset Splits | No | The paper mentions randomly permutating examples in the datasets 10 times but does not specify explicit training, validation, or test dataset splits, nor does it explicitly mention a validation set. |
| Hardware Specification | Yes | All algorithms are implemented with R on a Windows machine with 2.8 GHz Core(TM) i7-1165G7 CPU 5. |
| Software Dependencies | No | The paper mentions implementation with R but does not provide specific version numbers for R or any other key software libraries or dependencies. |
| Experiment Setup | Yes | For POMDR, we set B0 = 15 ln T , B = 400, M = 15, U = 25 and multiply by a constant c {0.05, 0.1} on the learning rate λt (see (10)). Such values of parameters do not change our regret bound. For the ALD condition (see (5)), we set the threshold 10T ζ, ζ {0.5, 2/3}. For all algorithms, we tune the kernel parameter σ 2[ 2:1:6]. |