LieTransformer: Equivariant Self-Attention for Lie Groups
Authors: Michael J Hutchinson, Charline Le Lan, Sheheryar Zaidi, Emilien Dupont, Yee Whye Teh, Hyunjik Kim
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the generality of our approach by showing experimental results that are competitive to baseline methods on a wide range of tasks: shape counting on point clouds, molecular property regression and modelling particle trajectories under Hamiltonian dynamics. |
| Researcher Affiliation | Collaboration | 1Department of Statistics, University of Oxford, UK 2Deep Mind, UK. |
| Pseudocode | Yes | Algorithm 1 Lie Self Attention |
| Open Source Code | Yes | The code for our experiments is available at: https:// github.com/oxcsml/lie-transformer |
| Open Datasets | Yes | We apply the Lie Transformer to the QM9 molecule property prediction task (Ruddigkeit et al., 2012; Ramakrishnan et al., 2014). We test our model on the spring dynamics task proposed in Sanchez-Gonzalez et al. (2019). |
| Dataset Splits | No | For the shape counting task: "We first create a fixed training set Dtrain and test set Dtest of size 10,000 and 1,000 respectively." No explicit validation set is mentioned here. For QM9, it mentions hyperparameter search and differing splits in the original paper vs. their runs, but doesn't state their specific train/validation/test split percentages or counts. For Hamiltonian dynamics, it describes training data generation but not explicit splits. |
| Hardware Specification | No | The paper does not provide specific hardware details such as exact GPU/CPU models, processor types, or memory amounts used for running its experiments. |
| Software Dependencies | No | The paper mentions software packages like Pytorch, NumPy, SciPy, and Matplotlib, but does not provide specific version numbers for these dependencies. |
| Experiment Setup | No | The main text refers to Appendices (e.g., Appendix I.1, I.2, I.3) for experimental setup details, which are not provided. The main body does not explicitly list specific hyperparameter values, batch sizes, learning rates, or other detailed training configurations. |