ReLUs Are Sufficient for Learning Implicit Neural Representations
Authors: Joseph Shenouda, Yamin Zhou, Robert D Nowak
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, we demonstrate that, contrary to popular belief, one can learn state-of-the-art INRs based on a DNN composed of only Re LU neurons. We substantiate our claims through experiments in signal representation, super resolution, and computed tomography, demonstrating the versatility and effectiveness of our method. |
| Researcher Affiliation | Academia | 1Department of Electrical and Computer Engineering, University of Wisconsin-Madison 2Department of Computer Sciences, University of Wisconsin-Madison. Correspondence to: Joseph Shenouda <jshenouda@wisc.edu>. |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code for all experiments can be found at https://github. com/joeshenouda/relu-inrs. |
| Open Datasets | Yes | In this experiment we simulated CT reconstruction by taking 100 equally spaced CT measurements of a 326 × 435 chest X-ray image (Clark et al., 2013). In this experiment we fit the four INR architectures to the standard 256 × 256 cameraman image. We implemented 4× super resolution on a single image from the DIV2K image dataset (Agustsson & Timofte, 2017). |
| Dataset Splits | No | The paper mentions training details like epochs and learning rates but does not explicitly provide training, validation, or test dataset split percentages or counts. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., CPU, GPU models, or memory) used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers (e.g., Python, PyTorch, CUDA versions). |
| Experiment Setup | Yes | For all experiments we utilized a three hidden layer DNN. The full training details for all experiments can be found in Appendix D. For the BW-Re LU DNN we used an initial learning rate of η0 = 4e-3 a scaling parameter applied to each neuron of c = 9. All models were trained for 1000 epochs and the decay rate for the learning rate was r = 0.1. The architecture consisted of 3 hidden layers and 300 neurons per layer. |