Efficiency for Free: Ideal Data Are Transportable Representations
Authors: PENG SUN, Yi Jiang, Tao Lin
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments across four widely-used datasets, seven neural network architectures, eight self-supervised learning algorithms demonstrate the effectiveness and efficiency of RELA. |
| Researcher Affiliation | Academia | Peng Sun1,2 Yi Jiang1 Tao Lin2, 1Zhejiang University 2Westlake University sunpeng@westlake.edu.cn, yi_jiang@zju.edu.cn, lintao@westlake.edu.cn |
| Pseudocode | Yes | Algorithm 1 Adaptive Loss Weighting Algorithm for RELA and Self-Supervised Learning |
| Open Source Code | Yes | Our code is available at: https://github.com/LINs-lab/Re LA. |
| Open Datasets | Yes | Datasets: For low-resolution data (32x32), we evaluate our method on two datasets, i.e., CIFAR-10 [35] and CIFAR-100 [34]. For high-resolution data, we conduct experiments on two large-scale datasets including Tiny-Image Net (64x64) [36] and full Image Net-1K (224x224) [20] |
| Dataset Splits | No | The paper refers to 'Validation Loss' in figures, implying a validation set is used, but does not explicitly provide specific dataset split information (exact percentages, sample counts, or detailed splitting methodology) for training, validation, or testing. |
| Hardware Specification | Yes | We implement our method through Py Torch [48], and all experiments are conducted on NVIDIA RTX 4090 GPUs. |
| Software Dependencies | No | The paper mentions 'PyTorch [48]' but does not provide a specific version number for this or any other software library. |
| Experiment Setup | Yes | This includes using Adam W as the optimizer, with a mini-batch size of 128 (except for Image Net-1K, where we use a mini-batch size of 512). ... Optimizer Adam W, Learning Rate 0.001, Weight Decay 0.01. ... Optimizer Adam (learning rate: 3e-4) |