Fast-Rate PAC-Bayesian Generalization Bounds for Meta-Learning
Authors: Jiechao Guan, Zhiwu Lu
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments empirically show that our proposed meta-learning algorithms achieve competitive results with respect to latest works. For classification problems, we directly set our kl-bound (Theorem 3) and Catoni-bound (Theorem 4) as minimization objectives, with the bounded cross-entropy loss (Perez Ortiz et al., 2021) for model optimization. For regression problems, we develop a Gibbs optimal hyper-posterior algorithm with Bayesian neural networks (GOHP-NN). Finally, we conduct experiments on several benchmarks. The empirical results show that our proposed meta-learning algorithms achieve competitive performance with respect to latest works. |
| Researcher Affiliation | Academia | 1School of Information, Renmin University of China, Beijing, China 2Beijing Key Laboratory of Big Data Management and Analysis Methods, Beijing, China 3Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China. Correspondence to: Zhiwu Lu <luzhiwu@ruc.edu.cn>, Jiechao Guan <2014200990@ruc.edu.cn>. |
| Pseudocode | Yes | The detailed pseudo-code of our proposed meta-learning algorithms for classification and regression problems can be found in the Appendices E-F. Algorithm 1 Catoni-bound-minimizing meta-learning algorithm (meta-training phase). Algorithm 2 GOHP with SVGD approximation of Q (meta-training phase). Algorithm 3 GOHP with SVGD on the novel tasks (meta-test phase). |
| Open Source Code | No | The paper provides pseudocode in the appendices but does not explicitly state that source code is open or available, nor does it provide a link to a code repository. |
| Open Datasets | Yes | We conduct classification experiments in three different task environments, based on the augmentations of the MNIST dataset (Yann, 1998). For the second environment, we employ datasets corresponding to different calibration sessions of Swiss Free Electron Laser(Swiss FEL) (Milne et al., 2017). The other two environments are constructed by using the datasets from Physio Net 2012 challenge (Silva et al., 2012), which contains the time series of electronic health measurements from patients, in terms of the Glasgow Coma Scale (GCS) and the hematocrit value (HCT). Finally, we create the Berkeley-Sensor environment where the tasks need to make prediction of temperature measurements corresponding to the sensors installed in different places of one building (Madden, 2004). |
| Dataset Splits | No | The paper describes training and testing phases ('meta-training phase' and 'meta-test phase') but does not explicitly mention the use of a separate validation set for model tuning or evaluation. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used to run the experiments (e.g., GPU models, CPU types, or memory). |
| Software Dependencies | No | The paper mentions implementing algorithms for 'deep neural networks' and lists an optimizer 'ADAM', but it does not specify version numbers for any software, libraries, or frameworks used (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | All experiment details are set the same as that in (Amit & Meir, 2018). We set the same experiment setup as that in (Rothfuss et al., 2021). In practice, we set the parameters κP = 2000 and κQ = 0.001 respectively, and the confidence parameter δ = 0.1. ADAM is chose as the optimizer with learning rate of 10^-3 for all experiments. |