Improving Equivariant Model Training via Constraint Relaxation
Authors: Stefanos Pertigkiozoglou, Evangelos Chatzipantazis, Shubhendu Trivedi, Kostas Daniilidis
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We provide experimental results on different state-of-the-art network architectures, demonstrating how this training framework can result in equivariant models with improved generalization performance. Our code is available at https://github.com/StefanosPert/Equivariant_Optimization_CR |
| Researcher Affiliation | Academia | Stefanos Pertigkiozoglou University of Pennsylvania pstefano@seas.upenn.edu Evangelos Chatzipantazis University of Pennsylvania vaghat@seas.upenn.edu Shubhendu Trivedi Independent shubhendu@csail.mit.edu Kostas Daniilidis University of Pennsylvania Archimedes, Athena RC kostas@cis.upenn.edu |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available at https://github.com/StefanosPert/Equivariant_Optimization_CR |
| Open Datasets | Yes | We train on the Model Net40 dataset (Chang et al., 2015), which contains 12311 point clouds from 40 different classes. [...] We use the Steerable E(3) GNN (SEGNN) (Brandstetter et al., 2021) and we train it on the task of Nbody particle simulation (Kipf et al., 2018). [...] we train Equiformer (Liao & Smidt, 2023) on the task of molecular dynamics simulations for a set of molecules provided as part of the MD17 dataset (Chmiela et al., 2017). [...] We evaluate our method on the task of 2D smoke flow prediction described in Wang et al. (2022). |
| Dataset Splits | Yes | For the choice of λreg we perform hyperparameter grid search using cross-validation with an 80%-20% split of the original training set of Model Net40 into training and validation. [...] The other 20% of the data were held out as the validation set used to evaluate the model. |
| Hardware Specification | Yes | We run all the experiments on NVIDIA A40 GPUs. |
| Software Dependencies | No | The paper mentions optimizers like 'Adam optimizer (Kingma & Ba, 2015)' and 'Adam W optimizer (Loshchilov & Hutter, 2019)' and a framework 'Phi Flow (Holl et al., 2020)', but it does not specify version numbers for general software dependencies (e.g., Python, PyTorch, CUDA, specific library versions). |
| Experiment Setup | Yes | We fix the weight of the regularization term to be λreg = 0.01 for all the experiments. [...] For the relaxed version of VNPoit Net we train for 250 epochs using the Adam optimizer (Kingma & Ba, 2015), with an initial learning rate of 10-3, that we decrease every 20 epochs by a factor of 0.7, and weight decay equal to 10-4. For the relaxed version of VN-DGCNN we train for 250 epochs using stochastic gradient descent, with an initial learning rate of 10-3, that we decrease using cosine annealing, and weight decay equal to 10-4. The batch size used was 32. |