Beyond Categorical Label Representations for Image Classification
Authors: Boyuan Chen, Yu Li, Sunand Raghupathi, Hod Lipson
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments reveal that high dimensional, high entropy labels achieve comparable accuracy to text (categorical) labels on the standard image classification task, but features learned through our label representations exhibit more robustness under various adversarial attacks and better effectiveness with a limited amount of training data. 4 EXPERIMENTAL SETUP |
| Researcher Affiliation | Academia | Boyuan Chen, Yu Li, Sunand Raghupathi, Hod Lipson Columbia University |
| Pseudocode | No | The paper provides the architecture of the high-dimensional label decoder in Table 5, but it does not contain any pseudocode or algorithm blocks. |
| Open Source Code | Yes | https://www.creativemachineslab.com/label-representation.html |
| Open Datasets | Yes | We evaluate our models on the CIFAR-10 and CIFAR-100 datasets (Krizhevsky, 2009). CIFAR-10 consists of 60, 000 images of size 32 32 uniformly distributed across 10 classes. The dataset comes with 50, 000 training images and 10, 000 test images. We use a 45, 000/5, 000 training/validation split. |
| Dataset Splits | Yes | We use the same training, validation and testing data split (45, 000/5, 000/10, 000) for all of our experiments. CIFAR-10 consists of 60, 000 images... The dataset comes with 50, 000 training images and 10, 000 test images. We use a 45, 000/5, 000 training/validation split. CIFAR-100 also comprises 60, 000 images... The dataset is split into 50, 000 training images and 10, 000 test images. We randomly select 5, 000 images from the training images to form the validation set. |
| Hardware Specification | Yes | All experiments are performed on a single Ge Force RTX 2080 Ti GPU. |
| Software Dependencies | No | We implement our models using Py Torch Paszke et al. (2019). While PyTorch is mentioned with a citation, a specific version number (e.g., PyTorch 1.x) is not provided, nor are other software dependencies with their versions. |
| Experiment Setup | Yes | We train the categorical models for 200 epochs with a starting learning rate of 0.1, and decay the learning rate by 0.1 at epoch 100 and 150. The high-dimensional models are trained for 600 epochs with the same initial learning rate, and we drop the learning rate by 0.1 at epoch 300 and 450. All models are trained with a batch size of 128 using the SGD optimizer with 0.9 momentum and 0.0001 weight decay. One exception is when train categorical models with the VGG19 image encoder, we use a larger weight decay, 0.0005. |