Geometric Transformer with Interatomic Positional Encoding
Authors: Yusong Wang, Shaoning Li, Tong Wang, Bin Shao, Nanning Zheng, Tie-Yan Liu
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate Geoformer on several benchmarks, including the QM9 dataset and the recently proposed Molecule3D dataset. Compared with both Transformers and equivariant GNN models, Geoformer outperforms the state-of-the-art (So TA) algorithms on QM9, and achieves the best performance on Molecule3D for both random and scaffold splits. |
| Researcher Affiliation | Collaboration | 1 National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi an Jiaotong University 2 Microsoft Research AI4Science 3 Mila Québec AI Institute 4 Université de Montréal wangyusong2000@stu.xjtu.edu.cn, nnzheng@mail.xjtu.edu.cn shaoning.li@umontreal.ca {watong, binshao, tyliu}@microsoft.com |
| Pseudocode | No | The paper does not contain an explicitly labeled pseudocode or algorithm block. |
| Open Source Code | Yes | Codes are available at https://github.com/microsoft/AI2BMD/tree/Geoformer. |
| Open Datasets | Yes | Geoformer is evaluated on both QM9 dataset [35] that consists of 12 molecular properties and a large-scale Molecule3D dataset [51] derived from Pub Chem QC [32] with ground-state structures and the corresponding properties calculated at DFT level. |
| Dataset Splits | Yes | QM9 dataset consists of 130,831 small organic molecules with up to 9 heavy atoms. Each molecule is associated with 12 targets covering its energetic, electronic, and thermodynamic properties. We randomly split them in to 110,000 samples for training, 10,000 samples for validation and the remains for testing following the prior work [43]. The Molecule3D dataset consists of 3,899,647 molecules... The dataset is split into train, validation, and test set with the ratio of 6:2:2. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used, such as GPU or CPU models. |
| Software Dependencies | No | The paper does not provide specific version numbers for software dependencies (e.g., Python, PyTorch, CUDA versions). |
| Experiment Setup | Yes | All models are trained using the Adam W optimizer, and we use the learning rate decay if the validation loss stops decreasing. We also adopt the early stopping strategy to prevent over-fitting. The optimal hyperparameters such as learning rate and batch size are selected on validation sets. More detailed hyperparameters setting for Geoformer are provided in Appendix Table 4. |