Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Text-to-Code Generation for Modular Building Layouts in Building Information Modeling
Authors: YINYI WEI, Xiao LI
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To train and evaluate the framework, we curated a dataset of paired descriptions and ground truth layouts drawn from real-world modular housing projects. Performance was assessed using metrics for executable validity, semantic fidelity, and geometric consistency. |
| Researcher Affiliation | Academia | Yinyi Wei The University of Hong Kong EMAIL Xiao Li The University of Hong Kong EMAIL |
| Pseudocode | No | The paper provides examples of C# code snippets (e.g., in Table 1) as part of the code implementation and action sequences, but it does not contain a clearly labeled 'Pseudocode' or 'Algorithm' block with structured, abstract steps for a method or procedure. |
| Open Source Code | Yes | Our implementation is available at https://github.com/CI3LAB/Text2MBL. |
| Open Datasets | Yes | To support training and evaluation, we curated data pairing textual descriptions with action-based instructions in code format from real-world housing projects. |
| Dataset Splits | Yes | For the 198 MBL designs (396 pairs of descriptions and code), we partitioned them into training, development, and test sets using a 7:1:2 split, resulting in 138, 20, and 40 designs (276, 40, and 80 pairs). |
| Hardware Specification | Yes | All experiments were performed on a server equipped with four NVIDIA GeForce RTX 4090 GPUs and a 64-core Intel Xeon Platinum 8370C CPU operating at 2.80 GHz. |
| Software Dependencies | Yes | a proof-of-concept system was implemented in Autodesk Revit [3], a representative BIM platform, where custom classes and functions were developed using the Revit API [4] in C#. [3] Autodesk Inc. Autodesk revit. https://www.autodesk.com/, 2022. Version 2022. [4] Autodesk Inc. Revit api 2022. https://www.revitapidocs.com/2022/, 2022. Software Development Kit (SDK) for Autodesk Revit 2022. |
| Experiment Setup | Yes | Fine-tuning was conducted using a learning rate of 3e-4 and a batch size of 2; to effectively simulate a batch size of 4, gradient accumulation over 2 steps was applied. Training proceeded for 5 epochs, and model selection was based on the highest evaluation scores observed on the development set, where argument F1 was used for code-driven models and Io U was used for coordinate-driven models. Lo RA-specific hyperparameters were set with a rank of 64, a scaling factor (alpha) of 64, and an adapter dropout rate of 0.1. |