Evaluating Unsupervised Denoising Requires Unsupervised Metrics
Authors: Adria Marcos Morales, Matan Leibovich, Sreyas Mohan, Joshua Lawrence Vincent, Piyush Haluai, Mai Tan, Peter Crozier, Carlos Fernandez-Granda
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Controlled numerical experiments with synthetic noise confirm that they provide accurate approximations in practice. We validate our approach on real-world data from two imaging modalities: videos in raw format and transmission electron microscopy. Our results demonstrate that the proposed metrics enable unsupervised evaluation of denoising methods based exclusively on noisy data. |
| Researcher Affiliation | Academia | 1Center for Data Science, New York University, New York, NY 2Centre de Formaci o Interdisciplin aria Superior, Universitat Polit ecnica de Catalunya, Barcelona, Spain 3Radiomics Group, Vall d Hebron Institute of Oncology, Vall d Hebron Barcelona Hospital Campus, Barcelona, Spain 4Courant Institute of Mathematical Sciences, New York University, New York, NY 5School for Engineering of Matter, Transport & Energy, Arizona State University, Tempe, AZ. |
| Pseudocode | Yes | Algorithm 1 Bootstrap confidence intervals. Algorithm 2 Decomposition via spatial subsampling. |
| Open Source Code | Yes | Code to reproduce all computational experiments is available at https://github.com/adriamm98/umse |
| Open Datasets | Yes | We use a dataset of natural images (Martin et al., 2001; Zhang et al., 2017b; Franzen, 1993) corrupted with additive Gaussian noise... and a dataset of simulated electron-microscopy images (Vincent et al., 2021) corrupted with Poisson noise... We evaluate our proposed metrics on a dataset of videos in raw format, consisting of direct readings from the sensor of a surveillance camera contaminated with real noise at five different ISO levels (Yue et al., 2020). |
| Dataset Splits | Yes | The remaining 559 images were evenly split into validation and test sets. The data were divided into three sets: training & validation set, consisting of 70% of the data, and two contiguous test sets with pixel-wise correlation: one containing 155 images with moderate signal-to-noise ratio (SNR), and one containing 383 images with low SNR. |
| Hardware Specification | No | The paper mentions that "ASU Research Computing and NYU HPC for providing high performance computing resources" were used, but it does not specify any particular GPU/CPU models, memory details, or specific cluster configurations (e.g., "NVIDIA A100", "Intel Xeon E5"). |
| Software Dependencies | No | The paper mentions various software components and methods used, such as "Dn CNN", "UNet", "Dense Net", "Bilateral filter", and "skimage.restoration", but it does not provide specific version numbers for any of these components. |
| Experiment Setup | Yes | We use the pre-trained weights released in (Zhang et al., 2017a) and (Mohan et al., 2020). Bilateral filter with a filter diameter of 15 pixels and σvalue = σspace = 1. We set sigma=0.01. Trained for 1000 epochs using an Adam optimizer with an initial learning rate of 0.001, and scheduled reduction of the learning rate every 100 epochs. |