Towards a Deep Network Architecture for Structured Smoothness
Authors: Haroun Habeeb, Oluwasanmi Koyejo
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The proposed model architecture outperforms conventional neural network architectures across a variety of simulated and real datasets with structured smoothness.We demonstrate the benefit of using FGL on simulated experiments and real neuroimaging data. We compare FGL against fully connected networks, convolutional neural networks, Coord Conv (Liu et al., 2018), and a closely related method proposed by Aydore et al. (2018). We extensively evaluate the performance of FGL on simulated and real brain imaging data showing improved performance. |
| Researcher Affiliation | Academia | Haroun Habeeb Department of Computer Science University of Illinois at Urbana Champaign Champaign, IL 61820 haroun7@gmail.com Sanmi Koyejo Department of Computer Science University of Illinois at Urbana Champaign Champaign, IL 61820 sanmi@illinois.edu |
| Pseudocode | No | No explicit pseudocode or algorithm block is provided in the paper. |
| Open Source Code | Yes | Code is available at https://www.github.com/anon/repo and a minimal version is provided in the supplementary.Our implementation of FGL using Py Torch is available at https://github.com/ anonymous/link. |
| Open Datasets | Yes | We evaluate our models on 5 datasets which were used by Mensch et al. (2017): Archi (Pinel et al., 2007), Brainomics (Orfanos et al., 2017), Cam-CAN (Shafto et al., 2014), LA5c (Poldrack et al., 2016) and HCP (Van Essen et al., 2012).Each dataset is publicly available on NeuroVault1 (Gorgolewski et al., 2015). |
| Dataset Splits | Yes | We create a test set using 20% of the simulated dataset. The remaining points are used for training.The split is done such that no subject in the test set appears in the training set. In each split, 30% of subjects are used for testing, and all or a part of the remaining subjects are used for training.These hyperparameters were selected by monitoring for overfitting on the training set (using a further validation split). |
| Hardware Specification | Yes | Training was done using Adam (Kingma and Ba, 2014) on 4 K80 GPUs. |
| Software Dependencies | No | We use Pytorch (Paszke et al., 2017) to implement models, and nilearn (Abraham et al., 2014) to preprocess and visualize f MRI images. (No version numbers are provided for Pytorch or nilearn). |
| Experiment Setup | Yes | Convolution based models were trained for 50 epochs, feedforward neural networks for 30 and FGL for 20.We use weight normalization (Salimans and Kingma, 2016)Training was done using Adam (Kingma and Ba, 2014). For FNNs, we tried 2 and 3 layer versions with intermediate sizes chosen from 64, 128, 256 and 512. The model with intermediate layer sizes of 512 and 128 worked best. The architecture is 5 3-D convolution layers of stride 2 and kernel size 4. The input images have 212455 voxels after masking...These voxels are then reduced to 1024 groups with feature vectors of length 8 each. Next, these groups are reduced to 256 variables with 64 features and finally to 32 variables with 128 features. |