Crystalformer: Infinitely Connected Attention for Periodic Structure Encoding
Authors: Tatsunori Taniai, Ryo Igarashi, Yuta Suzuki, Naoya Chiba, Kotaro Saito, Yoshitaka Ushiku, Kanta Ono
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Quantitative comparisons using Materials Project and JARVIS-DFT datasets show that the proposed method outperforms several neural-network-based state-of-the-art methods (Xie & Grossman, 2018; Schütt et al., 2018; Chen et al., 2019; Louis et al., 2020; Chen & Ong, 2022; Choudhary & De Cost, 2021; Yan et al., 2022) for various crystal property prediction tasks. |
| Researcher Affiliation | Collaboration | Tatsunori Taniai1, Ryo Igarashi1, Yuta Suzuki2, Naoya Chiba3, Kotaro Saito4,5 Yoshitaka Ushiku1, Kanta Ono5 1OMRON SINIC X Corp., 2Toyota Motor Corp., 3Tohoku University, 4Randeft Inc., 5Osaka University |
| Pseudocode | No | The paper describes methods and architectures but does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | We release our code online. |
| Open Datasets | Yes | Materials Project (MEGNet) is a collection of 69,239 materials from the Materials Project database retrieved by Chen et al. (2019)." and "JARVIS-DFT (3D 2021) is a collection of 55,723 materials by Choudhary et al. (2020). |
| Dataset Splits | Yes | Thanks to the great effort by Choudhary & De Cost (2021) and Yan et al. (2022), many relevant methods are evaluated on these datasets with consistent and reproducible train/validation/test splits." and "The three numbers below each property name indicate the sizes of training, validation, and testing subsets. |
| Hardware Specification | Yes | We evaluated the running times of all the methods by using a single NVIDIA RTX A6000 and a single CPU core on the same machine. |
| Software Dependencies | No | The paper mentions optimizers and activation functions but does not specify any software library names with version numbers required for reproduction. |
| Experiment Setup | Yes | For each regression task in the Materials Project dataset, we train our model by optimizing the mean absolute error loss function via stochastic gradient descent (SGD) with a batch size of 128 materials for 500 epochs. We initialize the attention layers by following Huang et al. (2020). We use the Adam optimizer (Kingma & Ba, 2015) with weight decay (Loshchilov & Hutter, 2019) of 10-5 and clip the gradient norm at 1. For the hyperparameters of Adam we follow Huang et al. (2020); we use the initial learning rate α of 5 × 10-4 and decay it according to α √4000/(4000 + t) by the number of total training steps t, and use (β1, β2) = (0.9, 0.98). |