Harmonic Neural Networks
Authors: Atiyo Ghosh, Antonio Andrea Gentile, Mario Dagrada, Chul Lee, Seong-Hyok Sean Kim, Hyukgeun Cha, Yunjun Choi, Dongho Kim, Jeong-Il Kye, Vincent Emanuel Elfving
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We benchmark our approaches against (quantum) physics-informed neural networks, where we show favourable performance. We exemplify our approaches on sample applications geared towards electrostatics (Sect.3.3), heat distribution (Sects.3.4&3.5), and robot navigation (Sect.3.6) and a further 3D fluid flow example in Sect.3.7. In order to provide robust benchmarks, we benchmark several forward solutions against finite element methods (FEM). We report in Table 1 the results as (i) root mean squared errors (RMSE) against FEM solutions, i.e. P (ϕ(xi) ϕFEM(xi))2/| Ω|, where Ω {xi} a set of collocation points uniformly sampled in the interior domain Ω, along with (ii) the expected absolute Laplacian over Ω. |
| Researcher Affiliation | Industry | 1PASQAL SAS, 2 av. Augustin Fresnel, Palaiseau, 91220, France 2LG Electronics, AI Lab, CTO Div, 19, Yangjae-daero 11-gil, Seocho-gu, Seoul, 06772, Republic of Korea 3POSCO Holdings, AI R&D Laboratories, 440, Tehera-ro, Gangam-gu, Seoul, 06194, Republic of Korea 4Pinostory Inc., 1-905 IT Castle, Seoul, 08506, Republic of Korea. |
| Pseudocode | No | The paper describes the proposed architectures and methods in prose and mathematical equations, but it does not include a structured pseudocode block or an explicitly labeled 'Algorithm' section. |
| Open Source Code | Yes | Supplementary code. doi: 10.6084/m9.figshare.23260154.v2. URL https://figshare.com/articles/software/Code_accompanying_the_paper_Harmonic_Neural_Networks_ICML_2023_/23260154. We provide implementations of our classical architectures in the supplementary material (SI) to facilitate the adoption of these inductive biases. |
| Open Datasets | No | The paper states 'For each application, we construct boundary conditions comprising of lines where we sample 100 uniformly-spaced points. To minimise physics-informed losses, we sampled 1024 collocation points randomly (uniformly) on the interior of each domain. These points are sampled once and then kept constant for each experiment to allow each run the same amount of information.' This describes data generation/sampling rather than the use of a pre-existing, publicly available dataset with concrete access information. |
| Dataset Splits | No | The paper describes sampling 'collocation points' and 'boundary conditions' for training and evaluating models. However, it does not explicitly provide percentages or counts for distinct training, validation, and test dataset splits, nor does it reference predefined splits from established benchmarks. |
| Hardware Specification | Yes | The classical neural networks we use are comparatively lightweight, and training for all conventional NNs was done on a single Apple M1 chip running Python 3.9.12 on mac OS 12.3.1. Equivalently, experiments involving QNNs were run on an AMD Ryzen 7 3700x processor, in a Python 3.9.5 Conda environment on Ubuntu 18.04. |
| Software Dependencies | Yes | We conduct all (both quantum and conventional) experiments in Python3 and make use of NumPy (Harris et al., 2020), PyTorch (Paszke et al., 2019) and Matplotlib (Hunter, 2007) packages. FEM ground truths were constructed using FEniCS (Alnæs et al., 2015). All the QNNs used in this paper are implemented with proprietary code, leveraging upon the packages PyTorch and Yao.jl (Luo et al., 2020). training for all conventional NNs was done on a single Apple M1 chip running Python 3.9.12 on mac OS 12.3.1. Equivalently, experiments involving QNNs were run on an AMD Ryzen 7 3700x processor, in a Python 3.9.5 Conda environment on Ubuntu 18.04. |
| Experiment Setup | Yes | We consistently use multilayer perceptrons with 3 hidden layers, width 32, initialised with Kaiming Uniform initialisers (He et al., 2015), optimised for 16,000 epochs over full-batches with an Adam optimizer (Kingma & Ba, 2014) with a learning rate (LR) of 10^-3. We use tanh activations for real-valued NNs and sin activations for holomorphic NNs. The training was performed by a hybrid optimization scheme comprising 1200 epochs of Adam (Kingma & Ba, 2014), followed by 300 epochs of L-BFGS (Liu & Nocedal, 1989), both with a learning rate 0.05. |