The Lie Derivative for Measuring Learned Equivariance
Authors: Nate Gruver, Marc Anton Finzi, Micah Goldblum, Andrew Gordon Wilson
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Using the Lie derivative, we study the equivariance properties of hundreds of pretrained models, spanning CNNs, transformers, and Mixer architectures. The scale of our analysis allows us to separate the impact of architecture from other factors like model size or training method. |
| Researcher Affiliation | Academia | Nate Gruver , Marc Finzi , Micah Goldblum, Andrew Gordon Wilson New York University |
| Pseudocode | Yes | Figure 3: Lie derivatives can be computed using automatic differentiation. We show how a Lie derivative for continuous rotations can be implemented in Py Torch (Paszke et al., 2019). The implementation in our experiments differs slightly, for computational efficiency and to pass secondorder gradients through grid_sample. |
| Open Source Code | Yes | We make our code publicly available at https://github.com/ngruver/lie-deriv. |
| Open Datasets | Yes | We evaluate the equivariance of pretrained models on 100 images from the Image Net (Deng et al., 2009) test set. |
| Dataset Splits | Yes | We evaluate the equivariance of pretrained models on 100 images from the Image Net (Deng et al., 2009) test set. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types with speeds, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper mentions 'Py Torch (Paszke et al., 2019)' in Figure 3, which indicates a software library, but does not specify a version number (e.g., PyTorch 1.x) for reproducibility. No other software components are mentioned with version numbers. |
| Experiment Setup | Yes | We define continuous transformations on images using bilinear interpolation with reflection padding. In total, we evaluate 410 classification models... We fine-tune a state-of-the-art vision transformer model pretrained with masked autoencoding (He et al., 2021) for 100 epochs on rotated MNIST. |