Task-Induced Representation Learning
Authors: Jun Yamada, Karl Pertsch, Anisha Gunjal, Joseph J Lim
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To evaluate the effectiveness of different representation learning objectives, we compare common unsupervised approaches based on reconstruction, prediction and contrastive learning to four taskinduced representation learning methods. We perform experiments in four visually complex environments, ranging from scenes with distracting background videos in the Distracting DMControl bench (Stone et al., 2021) to realistic distractors such as clouds, weather and urban scenery in the CARLA driving simulator (Dosovitskiy et al., 2017). Across RL and imitation learning experiments we find that pre-trained representations accelerate downstream task learning, even in visually complex environments. Additionally, in our comparisons task-induced representation learning approaches achieve up to double the learning efficiency of unsupervised approaches by learning to ignore distractors and task-irrelevant visual details. |
| Researcher Affiliation | Collaboration | 1 University of Oxford, 2 University of Southern California, 3 Korea Advanced Institute of Science and Technology, 4 Naver AI Lab |
| Pseudocode | No | The paper describes algorithms and formulations mathematically but does not include explicit pseudocode blocks or figures labeled 'Algorithm' or 'Pseudocode'. |
| Open Source Code | Yes | We open-source our codebase with example commands to reproduce our results on the project website: clvrai.com/tarp. |
| Open Datasets | Yes | Distracting DMControl. We use the environment of Stone et al. (2021), in which the visual complexity of the standard DMControl Walker task (Tassa et al., 2018) is increased by overlaying randomly sampled natural videos from the DAVIS 2017 dataset (Pont-Tuset et al., 2017). |
| Dataset Splits | No | The paper describes the collection of training data and testing on downstream tasks but does not specify explicit train/validation/test splits with percentages or counts for the collected datasets within each task. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU models, CPU models, memory) used to run the experiments. It only lists hyperparameters for the software. |
| Software Dependencies | No | The paper mentions various software components and frameworks like 'SAC', 'PPO', 'beta-VAE', 'CQL', 'Adam', 'Re LU', 'LSTM' and 'PyTorch' (implied by ICLR 2022 context for deep learning papers) but does not provide specific version numbers for these dependencies. |
| Experiment Setup | Yes | We provide a detailed description of all used environments, the procedures for offline dataset collection, and descriptions of the downstream evaluation tasks in appendix, Section A. Furthermore, in appendix, Section D we list all hyperparameters used for the pre-training phase (i.e. task-induced and unsupervised representation learning) as well as for training of the RL policies. |