Lossless Compression of Structured Convolutional Models via Lifting
Authors: Gustav Sourek, Filip Zelezny, Ondrej Kuzelka
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To test the proposed compression in practice, we selected some common structured convolutional models, and evaluated them on a number of real datasets from the domains of (i) molecule classification and (ii) knowledge-base completion. The questions to be answered by the experiments are: 1. How numerically efficient is the non-exact algorithm in achieving lossless compression? 2. What improvements does the compression provide in terms of graph size and speedup? 3. Is learning accuracy truly unaffected by the, presumably lossless, compression in practice? |
| Researcher Affiliation | Academia | Gustav Sourek, Filip Zelezny, Ondrej Kuzelka Department of Computer Science Czech Technical University in Prague {souregus,zelezny,kuzelon2}@fel.cvut.cz |
| Pseudocode | No | The paper describes the two algorithms in narrative text but does not include any explicit 'pseudocode' or 'algorithm' blocks with structured formatting. |
| Open Source Code | No | The paper does not include an unambiguous statement or a link to open-source code for the methodology described. |
| Open Datasets | Yes | For structure property prediction, we used 78 organic molecule classification datasets reported in previous works (Ralaivola et al., 2005; Helma et al., 2001; Lodhi & Muggleton, 2005). For knowledge base completion (KBC), we selected commonly known datasets of Kinships, Nations, and UMLS (Kok & Domingos, 2007). |
| Dataset Splits | Yes | We then trained against MSE using 1000 steps of ADAM, and evaluated with a 5-fold crossvalidation. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running its experiments. |
| Software Dependencies | No | The paper mentions software frameworks like 'Py Torch Geometric (Py G)' and 'Deep Graph Library (DGL)' and 'ADAM' but does not specify their version numbers. |
| Experiment Setup | Yes | We approached all the learning scenarios under simple unified setting with standard hyperparameters, none of which was set to help the compression (sometimes on the contrary). We used the (re-implemented) LRNN framework to encode all the models, and also compared with popular GNN frameworks of Py Torch Geometric (Py G) (Fey & Lenssen, 2019) and Deep Graph Library (DGL) (Wang et al., 2019). If not dictated by the particular model, we set the activation functions simply as CW = 1 / (1+e^(W x)) and A = avg. We then trained against MSE using 1000 steps of ADAM, and evaluated with a 5-fold crossvalidation. For a more fair comparison, we further increased all (tensor) dimensions to a more common dim=10. |