SineNet: Learning Temporal Dynamics in Time-Dependent Partial Differential Equations
Authors: Xuan Zhang, Jacob Helwig, Yuchao Lin, Yaochen Xie, Cong Fu, Stephan Wojtowytsch, Shuiwang Ji
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our method is rigorously tested on multiple PDE datasets, including the Navier-Stokes equations and shallow water equations, showcasing the advantages of our proposed approach over conventional U-Nets with a comparable parameter budget. We further demonstrate that increasing the number of waves in Sine Net while maintaining the same number of parameters leads to a monotonically improved performance. The results highlight the effectiveness of Sine Net and the potential of our approach in advancing the state-of-the-art in neural PDE solver design. Our code is available as part of AIRS (https://github.com/divelab/AIRS). |
| Researcher Affiliation | Academia | 1Department of Computer Science & Engineering, Texas A&M University 2Department of Mathematics, University of Pittsburgh |
| Pseudocode | No | The paper does not include a dedicated pseudocode section, algorithm block, or structured steps formatted like pseudocode. |
| Open Source Code | Yes | Our code is available as part of AIRS (https://github.com/divelab/AIRS). |
| Open Datasets | Yes | We use the dataset from Gupta & Brandstetter (2023), simulated with a numerical solver from the ΦFlow package (Holl et al., 2020a). ... We generate our CNS dataset using the numerical solver from Takamoto et al. (2022). ... We use the dataset from Gupta & Brandstetter (2023) for modeling the velocity and pressure fields for global atmospheric winds with a periodic boundary condition. |
| Dataset Splits | Yes | The data follow a train/valid/test split of 5,200/1,300/1,300, and the length of the time history h is 4 steps such that trajectories are unrolled for 10 steps during evaluation. ... The dataset is split as 5,400/1,300/1,300 and, following Takamoto et al. (2022), models use time history h = 10 such that trajectories are unrolled for 11 steps during evaluation. ... The data are split as 5,600/1,400/1,400, and h = 2 historical time steps are used such that trajectories are unrolled for 9 steps during evaluation. |
| Hardware Specification | Yes | Our code is implemented in Py Torch (Paszke et al., 2019). Models are trained and evaluated on 2 NVIDIA A100 80GB GPUs. All models are optimized for 50 epochs with batch size 32 and the model with the best validation rollout results is used for testing. |
| Software Dependencies | No | Our code is implemented in Py Torch (Paszke et al., 2019). While PyTorch is mentioned, a specific version number for PyTorch or other critical libraries/dependencies is not provided. The reference to Paszke et al. (2019) is a citation for PyTorch, not a version number. |
| Experiment Setup | Yes | All models are optimized for 50 epochs with batch size 32 and the model with the best validation rollout results is used for testing. ... All models are optimized with the Adam W optimizer (Kingma & Ba, 2015; Loshchilov & Hutter, 2019), using an initial learning rate of ηinit = 2 × 10−4, except for the F-FNO on SWE and CNS, where we found a larger learning rate to improve performance (see Table 6). The learning rate was warmed up linearly for 5 epochs from ηmin = 1 × 10−7 to ηinit before being decayed for the remaining 45 epochs using a cosine scheduler. |