Meta-Learning Sparse Implicit Neural Representations
Authors: Jaeho Lee, Jihoon Tack, Namhoon Lee, Jinwoo Shin
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically demonstrate that meta-learned sparse neural representations achieve a much smaller loss than dense meta-learned models with the same number of parameters, when trained to fit each signal using the same number of optimization steps. |
| Researcher Affiliation | Academia | A School of Electrical Engineering, KAIST B Kim Jaechul Graduate School of AI, KAIST C Graduate School of Artificial Intelligence, UNIST D Department of Computer Science and Engineering, UNIST |
| Pseudocode | Yes | Algorithm 1 Meta-Sparse INR: Meta-learning Sparse Implicit Neural Representations |
| Open Source Code | Yes | Code: https://github.com/jaeho-lee/Meta Sparse INR |
| Open Datasets | Yes | Datasets. We focus on 2D image regression tasks using three image datasets with distinct characteristics: Celeb A (face data [21]), Imagenette (natural image [14]), and 2D SDF (geometric pattern [40]). |
| Dataset Splits | Yes | We use the default training and validation splits. All images are resized and cropped to have the resolution of 178 178. ... During the training phase, we meta-learn the initial INR from the training split of signals. During the test phase, we randomly draw 100 signals from the validation split of signals, and train for 100 steps using the full-batch Adam to fit the INR on each signal. |
| Hardware Specification | No | The paper does not explicitly describe the specific hardware (e.g., CPU, GPU models, memory, or cloud instances) used for running its experiments. |
| Software Dependencies | No | The paper mentions 'PyTorch implementation' in the acknowledgments but does not provide specific version numbers for PyTorch or any other software dependencies crucial for replication. |
| Experiment Setup | Yes | For other experimental details, including the data pre-processing steps and hyperparameters for the meta-learning phases, see Appendix A. ... Appendix A.1 Training Hyperparameters: We use the Adam optimizer for both inner and outer loops, with initial learning rates of 10^-3 and 10^-4 for inner and outer loops, respectively. The inner loop gradient steps t is set to 1. The outer loop batch size (i.e., the number of tasks sampled at each outer loop iteration) is set to 4. We meta-train our models for 50k steps. When fine-tuning, we take 100 full-batch Adam steps using the learning rate of 10^-4. |