Scaling Laws For Deep Learning Based Image Reconstruction
Authors: Tobit Klug, Reinhard Heckel
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this work, we study whether major performance gains are expected from scaling up the training set size. We consider image denoising, accelerated magnetic resonance imaging, and super-resolution and empirically determine the reconstruction quality as a function of training set size, while simultaneously scaling the network size. |
| Researcher Affiliation | Academia | Tobit Klug & Reinhard Heckel Dept. of Computer Engineering Technical University of Munich Munich, Bavaria, Germany {tobit.klug,reinhard.heckel}@tum.de |
| Pseudocode | No | The paper describes algorithms and methods in prose and with mathematical formulas but does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | The repository at https://github.com/MLI-lab/Scaling_Laws_For_Deep_ Learning_Based_Image_Reconstruction contains the code to reproduce all results in the main body of this paper. |
| Open Datasets | Yes | To enable studying learned denoisers over a wide range of training set sizes we work with the Image Net dataset (Russakovsky et al., 2015). To explore the performance of learning based MR reconstruction over a wide range of training set sizes, we use the fast MRI multi-coil brain dataset (Knoll et al., 2020b). |
| Dataset Splits | Yes | We reserve 20 random classes for validation and testing. We design 10 training set subsets SN of sizes N [100, 100000] (see Fig. 1(a)) and with Si Sj for i j. For validation and testing we use 80 and 300 images respectively taken from 20 random classes that are not used for training. |
| Hardware Specification | Yes | The experiments were conducted on four NVIDIA A40, four NVIDIA RTX A6000 and four NVIDIA Quadro RTX 6000 GPUs. |
| Software Dependencies | No | The paper mentions software components like Adam optimizer, RMSprop optimizer, and torch.distributed package. However, it does not specify exact version numbers for these software dependencies, which are required for full reproducibility. |
| Experiment Setup | Yes | We use mean-squared-error loss and Adam (Kingma & Ba, 2014) optimizer with β1 = 0.9, β2 = 0.999. For moderate training set sizes up to 3000 images we find that heuristically adjusting the initial learning rate with the help of an automated learning rate annealing performs well. To this end, we start with a learning rate of 10 4 and increase after every epoch by a factor of 2 until the validation loss does not improve for 3 consecutive epochs. |