Unsupervised Representation Learning via Neural Activation Coding
Authors: Yookoon Park, Sangho Lee, Gunhee Kim, David Blei
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical results show that NAC attains better or comparable performance on both tasks over recent baselines including Sim CLR and Distill Hash. In addition, NAC pretraining provides significant benefits to the training of deep generative models. |
| Researcher Affiliation | Academia | 1Computer Science Department, Columbia University, New York, USA 2Department of Computer Science and Engineering, Seoul National University, Seoul, South Korea. |
| Pseudocode | No | No pseudocode or algorithm blocks were found in the paper. |
| Open Source Code | Yes | Our code is available at https://github.com/yookoon/nac. |
| Open Datasets | Yes | linear classification on CIFAR-10 and Image Net-1K and (ii) nearest neighbor retrieval on CIFAR-10 and FLICKR-25K. |
| Dataset Splits | No | No explicit details on validation dataset splits (e.g., percentages, sample counts for a dedicated validation set, or a clear cross-validation setup) were found. |
| Hardware Specification | No | No specific hardware details (e.g., exact GPU/CPU models, memory amounts, or detailed computer specifications) were mentioned for running experiments, only 'multi-GPU training'. |
| Software Dependencies | No | The paper mentions using 'Res Net architecture' and 'LARS optimizer' but does not provide specific software dependencies with version numbers (e.g., Python 3.8, PyTorch 1.9, CUDA 11.1). |
| Experiment Setup | Yes | For optimization, we use LARS optimizer (You et al., 2017) with linear warmup for the first 10 epochs followed by cosine learning rate decay. We set weight decay to 10^-6. For CIFAR-10, we use a batch size of 1000 and train the encoder for 1000 epochs. The learning rate is set to 3.0 with momentum 0.9. For Image Net, we use a batch size of 512 and train the encoder for 200 epochs. The learning rate is set to 1.7 with momentum 0.9. |