Towards Understanding the Invertibility of Convolutional Neural Networks
Authors: Anna Gilbert, Yi Zhang, Kibok Lee, Yuting Zhang, Honglak Lee
IJCAI 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we provide experimental validation of our theoretical model and analysis. We first validate the practical relevance of our assumption by examining the effectiveness of random filter CNNs, and then provide results on more realistic scenarios. In particular, we study popular deep CNNs trained for image classification on ILSVRC 2012 dataset [Deng et al., 2009]. We calculate empirical model-RIP bounds for W T , showing that they are consistent with our theory. Our results are also consistent with a long line of research shows that it is reasonable to model real and natural images as sparse linear combinations overcomplete dictionaries [Boureau et al., 2008; Le et al., 2013; Lee et al., 2008; Olshausen and others, 1996; Ranzato et al., 2007; Yang et al., 2010]. In addition, we verify our theoretical bounds for the reconstruction error x W T ˆz 2/ x 2 on real images. We investigate both randomly sampled filters and empirically learned filters in these experiments. Our implementation is based on Caffe [Jia et al., 2014] and Mat Conv Net [Vedaldi and Lenc, 2015]. |
| Researcher Affiliation | Collaboration | Anna C. Gilbert1 Yi Zhang1 Kibok Lee1 Yuting Zhang1 Honglak Lee1,2 1University of Michigan, Ann Arbor, MI 48109 2Google Brain, Mountain View, CA 94043 |
| Pseudocode | Yes | Algorithm 1 Model-based IHT |
| Open Source Code | No | The paper does not provide an explicit statement or link for the open-sourcing of its own code. |
| Open Datasets | Yes | To show the practical relevance of our theoretical assumptions on using random filters for CNNs as stated in Section 2.1, we evaluate simple CNNs with Gaussian random filters with i.i.d. zero mean unit variance entries on the CIFAR-10 [Krizhevsky, 2009]. |
| Dataset Splits | Yes | We tested different filter sizes (3, 5, 7) and numbers of channels (64, 128, 256, 1024, 2048) and report the best classification accuracy by cross-validation in Table 1. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU/GPU models, memory) used for running its experiments. |
| Software Dependencies | No | Our implementation is based on Caffe [Jia et al., 2014] and Mat Conv Net [Vedaldi and Lenc, 2015]. (No version numbers provided). |
| Experiment Setup | Yes | Specifically, we test random CNNs with 1, 2, and 3 convolutional layers followed by Re LU activation and 2 2 max pooling layer. We tested different filter sizes (3, 5, 7) and numbers of channels (64, 128, 256, 1024, 2048) and report the best classification accuracy by cross-validation in Table 1. We also report the best performance using learnable filters for comparison. More details about the architectures can be found in Appendix C.1. |