Understanding the Generalization Benefit of Model Invariance from a Data Perspective
Authors: Sicheng Zhu, Bang An, Furong Huang
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In experiments on multiple datasets, we evaluate sample covering numbers for some commonly used transformations and show that the smaller sample covering number for a set of transformations (e.g., the 3D-view transformation) indicates a smaller gap between the test and training error for invariant models, which verifies our propositions. |
| Researcher Affiliation | Academia | Department of Computer Science University of Maryland, College Park {sczhu, bangan, furongh}@umd.edu |
| Pseudocode | Yes | In experiments, we use modified k-medoids [35] clustering method to find the approximation of N(ϵ, S, ρG) (see Algorithm 1). ... Algorithm 1 Sample Covering Number Estimation |
| Open Source Code | Yes | Code is available at https://github.com/bangann/understanding-invariance. |
| Open Datasets | Yes | We report experimental results on CIFAR-10 [29] and Shape Net [10] in this section |
| Dataset Splits | No | The paper describes sampling training images and reports results for different sample sizes per class, but it does not specify a distinct validation set or the exact percentages/counts for train/validation/test splits. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU models, CPU types, memory, or cloud instances) used to conduct the experiments. |
| Software Dependencies | No | The paper mentions using ResNet18 and methods like data augmentation and KL divergence regularization but does not list specific software libraries (e.g., PyTorch, TensorFlow) with their version numbers. |
| Experiment Setup | Yes | We use Res Net18 [25] on both datasets... A simple method to learn invariant models is to do data augmentation. The augmented loss function is Laug(x) = L(f(g(x)))... We use this method on CIFAR-10 and Shape Net... We further enforce the invariance using the invariance regularization loss similar to [48, 51]: L = Lcls(f(x)) + λKL(f(x), f(g(x))). ... Table 3 shows specific λ values. |