Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
From Sequence to Structure: Uncovering Substructure Reasoning in Transformers
Authors: Xinnan Dai, Kai Yang, Jay Revolinsky, Kai Guo, Aoran Wang, Bohang Zhang, Jiliang Tang
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To this end, we vary the training set size from 100K to 400K and the number of Transformer layers from 2 to 5. For each substructure, we evaluate the extraction accuracy on 30K test graphs, averaging results over three runs. Table 1 shows the results for various substructure extraction tasks. |
| Researcher Affiliation | Academia | Xinnan Dai1 , Kai Yang2 , Jay Revolinsky1, Kai Guo1 , Aoran Wang3, Bohang Zhang2, Jiliang Tang1 1Michigan State University, 2Peking University, 3University of Luxembourg |
| Pseudocode | No | The paper includes theoretical definitions, lemmas, and theorems, but no explicitly labeled pseudocode or algorithm blocks are present. The methodology is described in prose and mathematical formulations. |
| Open Source Code | Yes | corresponding: EMAIL, equal contribution, code is available at https://github.com/DDigimon/From_Sequence_to_Structure |
| Open Datasets | Yes | We conduct experiments on QM9 [16] and PCBA [28]. |
| Dataset Splits | Yes | For each substructure, we evaluate the extraction accuracy on 30K test graphs, averaging results over three runs. ... For the hydroxyl group identification task, ... using 30,000 molecules for the training set and 3,000 for the test set. ... We use 10,000 molecules for training and 3,000 for testing in the carboxyl group recognition task. For the benzene ring recognition task, we construct a dataset with 30,000 molecules for training and 3,000 for testing. |
| Hardware Specification | Yes | All experiments are conducted on a machine equipped with 8 NVIDIA A6000 GPUs.e. |
| Software Dependencies | Yes | We use a lightweight version of the GPT-2 model, which is an implementation version of nano-GPT, with hyperparameters listed in Table 6. ... we visualize the fine-tuned LLa MA 3.1-8B-Instruct model on a triangle detection task (details in Appendix E.4.2)... |
| Experiment Setup | Yes | We use a lightweight version of the GPT-2 model, which is an implementation version of nano-GPT, with hyperparameters listed in Table 6. Table 6: Hyperparameter details heads embedding drop out rate batch size learning rate max epoch 12 384 0.2 2048 0.001 40000 |