Building Sparse Deep Feedforward Networks using Tree Receptive Fields
Authors: Xiaopeng Li, Zhourong Chen, Nevin L. Zhang
IJCAI 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We have carried out experiments to determine whether our method can produce high quality model structures. The TRFnets produced by our method are compared with FNNs obtained by manual grid search, and sparse FNNs obtained by weight pruning and weight regularization [Han et al., 2015; Collins and Kohli, 2014]. |
| Researcher Affiliation | Academia | Xiaopeng Li , Zhourong Chen , Nevin L. Zhang Department of Computer Science and Engineering The Hong Kong University of Science and Technology Clear Water Bay, Hong Kong {xlibo, zchenbb, lzhang}@cse.ust.hk |
| Pseudocode | Yes | Algorithm 1: TRF-NET(D, r, s, d) |
| Open Source Code | No | No explicit statement providing access to the source code for the described methodology. |
| Open Datasets | Yes | Six publically available datasets were use in the experiments. Tox21 challenge dataset.1 ... Text classification datasets.2 The five text datasets from [Zhang et al., 2015]: AG s News, DBPedia, Yelp Review, Yahoo!Answer and Sogou News. ... 1https://github.com/bioinf-jku/SNNs 2https://github.com/zhangxiangxiao/Crepe |
| Dataset Splits | Yes | The structures of FNNs are chosen by using validation data. |
| Hardware Specification | Yes | All neural networks are trained on a Tesla K20 GPU, while the structure learning is run on a CPU. |
| Software Dependencies | No | The paper mentions software components like "Re LUs", "Adam", and "dropout", and "word2vec model" but does not provide specific version numbers for any of them or for broader software dependencies like Python or deep learning frameworks. |
| Experiment Setup | Yes | When running the TRF-net algorithm on the Tox21 dataset, we used receptive fields with radius r = 5. Two different values, 1 and 2, were used for the stride s. ... The number of global neurons introduced at each layer is 10% of the number of TRF neurons on the same layer. ... we used Re LUs [Nair and Hinton, 2010] as the non-linear activation functions, and Adam [Kingma and Ba, 2015] as the optimizer, and we applied dropout [Srivastava et al., 2014] with rate 0.5. |