Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Domain Generalisation via Imprecise Learning
Authors: Anurag Singh, Siu Lun Chau, Shahine Bouabid, Krikamol Muandet
ICML 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Supported by both theoretical and empirical evidence, our work showcases the benefits of integrating imprecision into domain generalisation. |
| Researcher Affiliation | Collaboration | Anurag Singh is part of the Graduate School of Computer Science at Saarland University, Saarbr ucken, Germany. 1CISPA Helmholtz Center for Information Security, Saarbr ucken, Germany 2Department of Statistics, University of Oxford, UK. |
| Pseudocode | Yes | Algorithm 1 summarises the proposed algorithm. ... Algorithm 3 Sampling from a Beta Distribution using ICDF with Gradient Computation |
| Open Source Code | Yes | All proofs are in the appendix and we open-source our code at https://github.com/muandet-lab/dgil. |
| Open Datasets | Yes | We also experiment on the CMNIST dataset (Arjovski, 2021)... we use the UCI Bike Sharing dataset (Fanaee-T and Gama, 2014) |
| Dataset Splits | Yes | We consider 250 train and 250 test domains with 100 samples from each domain. ... We sample 10 training environments from a Beta(0.9, 1) distribution... we evaluate all the environments {0.0, 0.1, . . . , 0.9, 1.0}. ... The data is partitioned by season (1-4) and year (1-2) to create 8 different domains. Domains from the first year are used for training and the subsequent year as test domains. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used, such as GPU or CPU models, or memory specifications. |
| Software Dependencies | No | The paper mentions concepts like 'hypernetworks' and 'FILM layers' but does not specify software components with version numbers (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | We consider a linear model for each domain d: Yd = θd X + ϵ with X N(1, 0.5) and ϵ N(0, 0.1). ... We sample 10 training environments from a long-tailed Beta(0.9, 1) distribution... To implement the augmented hypothesis, we append FILM layers (Perez et al., 2018) to MLP architecture used in Eastwood et al. (2022a). ... For the initial 400 steps out of a 600-step training. ... To operationalise the imprecise risk optimization, we need to minimise (9) with respect to the family of probability distributions (Λ). Since for our case Λ = [0, 1], we parameterise the family of distributions with Beta(α, β). |