Learning K-way D-dimensional Discrete Codes for Compact Embedding Representations
Authors: Ting Chen, Martin Renqiang Min, Yizhou Sun
ICML 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In our experiments with various applications from natural language processing to graph convolutional networks, the total size of the embedding layer can be reduced up to 98% while achieving similar or better performance. |
| Researcher Affiliation | Collaboration | 1Department of Computer Science, University of California, Los Angeles 2NEC Laboratories America. |
| Pseudocode | Yes | Algorithm 1 An epoch of code learning via Straight-through Estimator with Tempering Softmax. |
| Open Source Code | No | The paper does not include an explicit statement about releasing source code for the methodology described, nor does it provide a link to a code repository. |
| Open Datasets | Yes | For the language modeling task, we test on the widely used English Penn Treebank (Marcus et al., 1993) dataset, which contains 1M words with vocabulary size of 10K. |
| Dataset Splits | Yes | The training/validation/test split is provided by convention according to (Mikolov et al., 2010). |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper mentions general software components and frameworks like LSTM and Fast Text but does not provide specific version numbers for software dependencies. |
| Experiment Setup | Yes | By default, we use K = 32, D = 32 and pre-trained distillation guidance for the proposed method, and linear embedding transformation function with 1 hidden layer of 300 hidden units. |