Guaranteed Approximation Bounds for Mixed-Precision Neural Operators
Authors: Renbo Tu, Colin White, Jean Kossaifi, Boris Bonev, Gennady Pekhimenko, Kamyar Azizzadenesheli, Anima Anandkumar
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through extensive experiments on different state-of-the-art neural operators, datasets, and GPUs, we demonstrate that our approach reduces GPU memory usage by up to 50% and improves throughput by 58% with little or no reduction in accuracy. |
| Researcher Affiliation | Collaboration | Renbo Tu 1, Colin White 2, Jean Kossaifi3, Boris Bonev3, Gennady Pekhimenko1, Kamyar Azizzadenesheli3, Anima Anandkumar2 1 University of Toronto, 2 Caltech, 3 NVIDIA |
| Pseudocode | No | The paper describes methods and processes in text and flowcharts (Figure 2) but does not include formal pseudocode or algorithm blocks. |
| Open Source Code | Yes | We release our codebase and all materials needed to reproduce our results at https://github.com/neuraloperator/neuraloperator. |
| Open Datasets | Yes | We consider the Navier-Stokes equations... We use the same dataset as Kossaifi et al. (2023)... We use the same dataset as Li et al. (2021a)... We use the dataset from Bonev et al. (2023)... Our final two datasets are 3D real-world car dataset generated by prior work (Umetani & Bickel, 2018; Li et al., 2023)... Shape-Net (Chang et al., 2015)... |
| Dataset Splits | Yes | Navier-Stokes... 10 000 training samples and 2000 test samples... Darcy Flow... 5000 training samples and 1000 test samples... Spherical SWE... 120 training samples and 20 validation samples are generated on the fly... Shape-Net Car... 500 samples for training and the rest for the test set. For Ahmed-body, we have 500 for training and 51 for test. |
| Hardware Specification | Yes | All data are measured on the same hardware (RTX 3090 Ti) and the same virtual environment... On three different Nvidia GPUs, RTX 3090 Ti, V100, and RTX A6000, we demonstrate a consistent improvement in training throughput... |
| Software Dependencies | No | The paper mentions software like 'PyTorch', 'torch-harmonics package', 'opt-einsum', and 'Open-FOAM solver' but does not specify their version numbers for reproducibility. |
| Experiment Setup | Yes | We use the official implementation and default hyperparameters for all models... Batch sizes are selected to fully utilize each GPU... We train each model for 500 epochs... We run frequency modes {16, 32, 64, 128} on the Darcy flow dataset... |