Neural Operators with Localized Integral and Differential Kernels
Authors: Miguel Liu-Schiaffini, Julius Berner, Boris Bonev, Thorsten Kurth, Kamyar Azizzadenesheli, Anima Anandkumar
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Adding our layers to FNOs significantly improves their performance, reducing the relative L2-error by 34-72% in our experiments, which include a turbulent 2D Navier-Stokes and the spherical shallow water equations. |
| Researcher Affiliation | Collaboration | 1Department of Computing and Mathematical Sciences, California Institute of Technology, Pasadena CA 91125 2NVIDIA, Santa Clara, CA 95051. Correspondence to: Miguel Liu-Schiaffini <mliuschi@caltech.edu>, Julius Berner <jberner@caltech.edu>, Boris Bonev <bbonev@nvidia.com>. |
| Pseudocode | No | The paper describes mathematical formulations and architectures but does not include explicit pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code is available in the neuraloperator library at github.com/neuraloperator/neuraloperator and the torch-harmonics library at github.com/NVIDIA/torchharmonics. |
| Open Datasets | No | For the Darcy flow, the paper states 'we generate our data as described in Section 4.1' without mentioning public availability. For other datasets like Navier-Stokes, Diffusion-Reaction, Shallow Water, and Flow past a cylinder, the paper refers to 'the same experimental setup and dataset as Li et al. (2022a)', 'Takamoto et al. (2022)', 'Bonev et al. (2023)', and 'Rahman et al. (2024)' respectively, but does not explicitly state that these datasets are publicly available or provide direct access links/DOIs to the datasets themselves. |
| Dataset Splits | Yes | We use 10000 training samples and 2000 testing samples. (...) The dataset consists of 900 training samples and 100 validation samples discretized on a 128 128 regular grid with 100 equidistant time-steps in the interval [0, 5] and Gaussian initial conditions. |
| Hardware Specification | Yes | Training is conducted by minimizing the squared L2-loss for 70 epochs on a single NVIDIA P100 GPU (...) Training was performed for 136 epochs on a single NVIDIA RTX 4090 GPU (...) We train on a single NVIDIA RTX 4090 GPU for 500 epochs (...) Training was performed for 100 epochs on a single NVIDIA RTX A6000 GPU |
| Software Dependencies | No | The paper mentions using 'neuraloperator library' and 'torch-harmonics library' and adapting 'PDE Arena benchmark' but does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | We use a step learning rate decay scheduler, starting at 10 3 and halving every 10 epochs. (...) Training was performed for 136 epochs on a single NVIDIA RTX 4090 GPU with an exponentially decaying learning rate, starting at 10 3 and halving every 33 epochs. (...) We train on a single NVIDIA RTX 4090 GPU for 500 epochs with early stopping (...) Moreover, we use an exponentially decaying learning rate, starting at 10 4 and halving every 100 epochs. (...) The architecture is trained on a dataset of 250 samples for 50 epochs using the Adam optimizer and a learning rate of 2 10 3. |