Nonparametric Teaching of Implicit Neural Representations
Authors: Chen Zhang, Steven Tin Sui Luo, Jason Chun Lok Li, Yik Chung Wu, Ngai Wong
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5. Experiments and Results We begin by using a synthetic signal to empirically show the evolution consistency between parameter-based gradient descent (PGD) and functional gradient descent (FGD). Next, we assess the behavior of INT on a toy image-fitting instance and explore diverse algorithms with different INT frequencies and ratios. Lastly, we validate the INT efficiency in multiple modalities such as audio (-31.63% training time), images (-38.88%), and 3D shapes (-35.54%), while upkeeping its reconstruction quality. Detailed settings are given in Appendices C. |
| Researcher Affiliation | Academia | 1Department of Electrical and Electronic Engineering, The University of Hong Kong, HKSAR, China 2Department of Computer Science, The University of Toronto, Ontario, Canada. |
| Pseudocode | Yes | Algorithm 1 Implicit Neural Teaching Input: Target signal f , initial MLP fθ0, the size of selected training size k N, small constant ϵ > 0 and maximal iteration number T. Set fθt fθ0, t = 0. while t T and [fθt(xi) f (xi)]N 2 ϵ do The teacher selects k teaching examples: /* Examples corresponding to the k largest |fθt(xi) f (xi)|. */ {xi}k = arg max {xi}k {xi}N [fθt(xi) f (xi)]k 2. Provide {xi}k to the MLP learner. The learner updates fθt based on received {xi}k : // Parameter-based gradient descent. θt θt η xi {xi}k θL(fθt(xi), f (xi)). Set t t + 1. end |
| Open Source Code | Yes | Our project page is available at https://chen2hang.github. io/_publications/nonparametric_teaching_of_implicit_ neural_representations/int.html. |
| Open Datasets | Yes | Cameraman image (Van der Walt et al., 2014); Kodak dataset (Eastman Kodak Company, 1999); Librispeech (Panayotov et al., 2015); Pluto (NASA, 2018); Stanford 3D Scanning Repository (Stanford Computer Graphics Laboratory, 2007). |
| Dataset Splits | No | The paper discusses training and test sets, but it does not explicitly provide details about validation dataset splits (e.g., percentages, counts, or a clear strategy for creating a validation set). |
| Hardware Specification | Yes | This is necessary due to VRAM constraints of consumer-grade GPUs such as NVIDIA RTX3090 (24GB). |
| Software Dependencies | No | The paper mentions software like 'scikit-image' and 'Adam optimizer' but does not specify their version numbers. |
| Experiment Setup | Yes | We train SIREN models with 6 layers, each with 256 hidden units, with default settings as mentioned in Sitzmann et al., 2020b... We train the models with vanilla gradient descent without momentum for 5000 iterations. All models are trained using a cosine annealing scheduler with a starting learning rate of 1e-4 and a minimum learning rate of 1e-6. |