ARMA Nets: Expanding Receptive Field for Dense Prediction
Authors: Jiahao Su, Shiqi Wang, Furong Huang
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show both theoretically and empirically that the effective receptive field of networks with ARMA layers (named ARMA networks) expands with larger autoregressive coefficients. We also provably solve the instability problem of learning and prediction in the ARMA layer through a re-parameterization mechanism. Additionally, we demonstrate that ARMA networks substantially improve their baselines on challenging dense prediction tasks, including video prediction and semantic segmentation. |
| Researcher Affiliation | Academia | 1University of Maryland, College Park, MD USA 2Nanjing University, Nanjing, China |
| Pseudocode | No | The paper describes methods using mathematical equations and descriptions but does not include any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available on https://github.com/umd-huang-lab/ARMA-Networks. |
| Open Datasets | Yes | We evaluate our ARMA U-Net on the lesion segmentation task in ISIC 2018 challenge [38], comparing against a baseline U-Net [30] and non-local U-Net [36] (U-Net augmented with non-local attention blocks). ... We evaluate our ARMA-LSTM network on the Moving-MNIST-2 dataset [12] with different moving velocities, comparing against the baseline Conv LSTM network [6, 39] and its augmentation using dilated convolutions and non-local attention blocks [34]. |
| Dataset Splits | Yes | We include the detailed setups (datasets, model architectures, training strategies, and evaluation metrics) and visualization in Appendix A for reproducibility purposes. (Appendix A.2: We randomly split the ISIC 2018 dataset into training (1811), validation (259), and test (520) images. Appendix A.3: We sample 10000 training videos, 2000 validation videos, and 2000 test videos.) |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory, or specific computing platforms) used for running experiments. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers (e.g., Python 3.x, PyTorch 1.x, CUDA 11.x) that would allow for reproducible software environment setup. |
| Experiment Setup | Yes | We include the detailed setups (datasets, model architectures, training strategies, and evaluation metrics) and visualization in Appendix A for reproducibility purposes. (Appendix A.2: We optimize our models using Adam optimizer [18] with a learning rate of 10−4. The batch size is set to 8. We train models for 100 epochs. Appendix A.3: We optimize models using Adam [18] with a learning rate of 10−4, and the batch size is set to 8. We train 100 epochs for the original speed, 200 epochs for 2X speed, and 300 epochs for 3X speed.) |