Nonlocal Neural Networks, Nonlocal Diffusion and Nonlocal Modeling
Authors: Yunzhe Tao, Qi Sun, Qiang Du, Wei Liu
NeurIPS 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this work, we aim to study the nature of nonlocal networks, namely, what the nonlocal blocks have exactly learned through training on a real-world task. and we train the nonlocal networks for image classification on the CIFAR-10 dataset [14], which consists of 50k training images from 10 classes, and do the spectrum analysis on the weight matrices of nonlocal blocks after training. |
| Researcher Affiliation | Collaboration | Yunzhe Tao School of Engineering and Applied Science Columbia University, USA y.tao@columbia.edu Qi Sun BCSRC & USTC Beijing, China sunqi@csrc.ac.cn Qiang Du School of Engineering and Applied Science Columbia University, USA qd2125@columbia.edu Wei Liu Tencent AI Lab Shenzhen, China wl2223@columbia.edu |
| Pseudocode | No | The paper provides mathematical formulations but no pseudocode or algorithm blocks are explicitly labeled or presented. |
| Open Source Code | No | The paper does not provide any statement or link regarding the availability of open-source code for the described methodology. |
| Open Datasets | Yes | we train the nonlocal networks for image classification on the CIFAR-10 dataset [14], which consists of 50k training images from 10 classes, and do the spectrum analysis on the weight matrices of nonlocal blocks after training. and To demonstrate the difference between two nonlocal networks and the effectiveness of our proposed method, we present the empirical evaluation on CIFAR-10 and CIFAR-100. |
| Dataset Splits | Yes | Following the standard practice, we present experiments performed on the training set and evaluated on the test set as validation. We compare the empirical performance of Pre Res Nets incorporating into the original nonlocal blocks [25] or the proposed nonlocal blocks in Eq. (8). Table 1 presents the validation errors of different models based on Pre Res Net-20 over CIFAR-10. |
| Hardware Specification | Yes | The model is with data augmentation [17] and trained on a single NVIDIA Tesla GPU. |
| Software Dependencies | No | The paper describes training parameters and hardware used but does not provide specific software dependencies with version numbers. |
| Experiment Setup | Yes | the training starts with a learning rate of 0.1 that is subsequently divided by 10 at 81 and 122 epochs (around 32k and 48k iterations). A weight decay of 0.0001 and momentum of 0.9 are also used. We terminate the training at 164 epochs. |