Multi-Fidelity Residual Neural Processes for Scalable Surrogate Modeling
Authors: Ruijia Niu, Dongxia Wu, Kai Kim, Yian Ma, Duncan Watson-Parris, Rose Yu
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show that MFRNP significantly outperforms state-of-the-art in learning partial differential equations and a real-world climate modeling task. Our code is published at: github.com/Rose-STL-Lab/MFRNP. Section 5. Experiments, Section 5.3. Results, Section 5.4. Ablation Study. |
| Researcher Affiliation | Academia | 1Department of Computer Science and Engineering, University of California San Diego, La Jolla, California, USA 2Halıcıo glu Data Science Institute, University of California San Diego, La Jolla, California, USA 3Scripps Institution of Oceanography, University of California San Diego, La Jolla, California, USA. Correspondence to: Rose Yu <roseyu@ucsd.edu>. |
| Pseudocode | Yes | Algorithm 1 MFRNP Training Process. Algorithm 2 MFRNP Inference Process. |
| Open Source Code | Yes | Our code is published at: github.com/Rose-STL-Lab/MFRNP. |
| Open Datasets | Yes | We include Heat and Poisson s equations (Olsen-Kettle, 2011) from computational physics. We use numerical solvers to generate the ground-truth data with dense and coarse meshes for different fidelity levels. Climate Modeling: Earth Surface Temperature. ... together with 13 low-resolution computational climate model predictions from the Scenario MIP project(O Neill et al., 2016) at global scale. We provide details about the computational models used in Appendix A.1. ... and the up-to-date observation-calibrated ERA5-reanalysis dataset (Hersbach et al., 2020). |
| Dataset Splits | Yes | We use 10% of training data set as validation set. For context-target split, we randomly select 20% 25% of training data as our context set, the rest as our target set for each fidelity. |
| Hardware Specification | Yes | All models are trained on NVIDIA A100 GPU with 80GB memory. |
| Software Dependencies | No | We implement MFRNP with Py Torch (Paszke et al., 2019) and compare the average n RMSE in three random runs with following baselines. We use Adam optimizer (Kingma & Ba, 2014). The paper mentions PyTorch and Adam optimizer with citations but does not specify exact version numbers for PyTorch or any other software dependencies needed for reproducibility. |
| Experiment Setup | Yes | For training, we use Adam optimizer (Kingma & Ba, 2014) with base learning rate of 1e 3. We use 10% of training data set as validation set. For Heat and Poisson s equation, we run our model with latent dimension and encoder/decoder dimension of 32, and run our model with maximum epoch of 50000 and patience 10000. We use learning rate decay of 0.85 and stepsize 10000. We set the highest fidelity weight to 2 and lower fidelities to 1 in loss calculation to focus more on optimizing toward the highest fidelity. For Poisson5, using lower fidelity weight of 0.25 further improves performance. For context-target split, we randomly select 20% 25% of training data as our context set, the rest as our target set for each fidelity. For fluid simulation, we follow the same setup but use latent dimension and encoder/decoder dimension of 128. We set patience to 5000 with learning rate decay of 0.01. We normalize the data before training and measure n RMSE on the de-normalized space. For the baseline models, we follow the same setup as above. |