Loss Function Learning for Domain Generalization by Implicit Gradient
Authors: Boyan Gao, Henry Gouk, Yongxin Yang, Timothy Hospedales
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate the learned loss function on the four common DG benchmarks: VLCS (Fang et al., 2013), PACS (Li et al., 2018a), Office Home (Venkateswara et al., 2017), and Terra Incognita (Beery et al., 2018). |
| Researcher Affiliation | Collaboration | 1School of Informatics, University of Edinburgh 2Samsung AI Center, Cambridge. |
| Pseudocode | Yes | Further details can be found in (Lorraine et al., 2020), but we provide pseudo-code for computing the implicit gradient in Algorithm 2. |
| Open Source Code | No | The paper does not contain an explicit statement about releasing source code or a link to a code repository. |
| Open Datasets | Yes | We choose Rotated MNIST (Ghifary et al., 2015) as a small dataset suitable for loss learning. We evaluate the learned loss function on the four common DG benchmarks: VLCS (Fang et al., 2013), PACS (Li et al., 2018a), Office Home (Venkateswara et al., 2017), and Terra Incognita (Beery et al., 2018). |
| Dataset Splits | Yes | We train Res Net-18 with ITL on the training split and perform model selection using the validation set. |
| Hardware Specification | Yes | training our ITL required using Py Torch (Paszke et al., 2017) only required 8 hours on a single V100 GPU to complete 200 gradient descent steps on ω. |
| Software Dependencies | No | The paper mentions using PyTorch and cites its creators, but does not provide a specific version number for PyTorch or any other software dependency. |
| Experiment Setup | Yes | The learning rates in the inner loop and outer loop are both 0.01, a batch size of 32 is used for the inner loop, and the Neumann series used for approximating the inverse Hessian is truncated at 15 iterations. |