PGrad: Learning Principal Gradients For Domain Generalization
Authors: Zhe Wang, Jake Grigsby, Yanjun Qi
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on Domain Bed and WILDS benchmarks demonstrate that our approach effectively enables robust DG optimization and leads to smoothly decreased loss curves. Empirically, PGrad achieves competitive results across seven datasets, demonstrating its efficacy across both synthetic and real-world distributional shifts. We conduct empirical experiments to answer the following questions: Q1. Does PGrad successfully handle both synthetic and real-life distributional shifts? Q2. Can PGrad handle various architectures (Res Net and Dense Net), data types (scene and satellite images), and tasks (classification and regression)? Q3. Compared to existing baselines, does PGrad enable smooth decreasing loss curves and generate smoother parameter trajectories? Q4. Can PGrad act as a practical complementary approach to combine with other DG strategies? Q5. How do bottom eigenvectors in the roll-out trajectories affect the model s training dynamic and generalization ability? |
| Researcher Affiliation | Academia | Zhe Wang, Jake Grigsby, Yanjun Qi Department of Computer Science University of Virginia Charlottesville, VA 22903, USA {zw6sg, jcg6dn, yq2h}@virginia.edu |
| Pseudocode | No | The paper describes its method in detailed text and mathematical equations but does not include a structured pseudocode or algorithm block labeled 'Algorithm' or 'Pseudocode'. |
| Open Source Code | No | The paper does not provide any statement or link indicating that the source code for the methodology described is publicly available. |
| Open Datasets | Yes | The Domain Bed benchmark (Gulrajani & Lopez-Paz, 2021) is a popular suite designed for rigorous comparisons of domain generalization methods. WILDS (Koh et al., 2021) is a curated benchmark of 10 datasets covering real-life distribution shifts in the wild such as poverty mapping and land use classification. |
| Dataset Splits | Yes | We group 20% data from each training domain to construct validation set for model selection. In the main, we follow the popular setup where we sample 20% of the data from each training domain and group them as a validation set. The validation accuracy will be used as an indicator of the optimal model. |
| Hardware Specification | No | The paper mentions 'memory constraints on our hardware' and 'Our machine has', but does not provide specific hardware details such as GPU or CPU models, or memory specifications used for running experiments. |
| Software Dependencies | No | The paper mentions using 'Adam (Kingma & Ba, 2017) as the optimizer' but does not provide specific version numbers for Adam or any other software dependencies, libraries, or frameworks used. |
| Experiment Setup | Yes | For our method variation PGrad-B , we set B = 3 for all datasets except using B = 2 for Domain Net. We default to Adam (Kingma & Ba, 2017) as the optimizer to roll-out a trajectory. All experiments use the Domain Bed default architecture, where we finetune a pretrained Res Net50 (He et al., 2016). We list our hyperparameter search space in Table 6. |