ExPT: Synthetic Pretraining for Few-Shot Experimental Design
Authors: Tung Nguyen, Sudhanshu Agrawal, Aditya Grover
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate Ex PT on few-shot experimental design in challenging domains and demonstrate its superior generality and performance compared to existing methods. |
| Researcher Affiliation | Academia | University of California, Los Angeles {tungnd,adityag}@cs.ucla.edu, sudhanshuagr27@g.ucla.edu |
| Pseudocode | No | The paper includes architectural diagrams (Figure 2) but no structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The source code is available at https://github.com/tung-nd/Ex PT.git. |
| Open Datasets | Yes | We consider 4 tasks from Design-Bench1 [58]. For each task, Design-Bench provides a public dataset, a larger hidden dataset which is used to normalize the scores, and an oracle. |
| Dataset Splits | No | The paper describes few-shot settings and data usage for pretraining and adaptation, but does not explicitly define or specify a separate validation dataset split for hyperparameter tuning or model selection in a reproducible manner. |
| Hardware Specification | No | No specific hardware details (e.g., GPU/CPU models, memory specifications) are provided for the experimental setup. |
| Software Dependencies | No | No specific software dependencies with version numbers (e.g., PyTorch 1.9, Python 3.8) are provided. |
| Experiment Setup | Yes | For each domain, we pretrain Ex PT for 10,000 iterations with 128 synthetic functions in each iteration...To increase the diversity of synthetic data, we randomize the two hyperparameters, length scale U[5.0, 10.0] and function scale σ U[1.0, 1.0], when generating each function. Additionally, we add Gaussian noises N(0, 0.1) to each input x sampled from Dunlabeled...For each generated function, we use 100 points as context points and the remaining 128 as target points, and train the model to optimize (2). |