Lossless Adaptation of Pretrained Vision Models For Robotic Manipulation
Authors: Mohit Sharma, Claudio Fantacci, Yuxiang Zhou, Skanda Koppula, Nicolas Heess, Jon Scholz, Yusuf Aytar
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We perform a comprehensive investigation across three major model architectures (Vi Ts, NFNets, and Res Nets), supervised (Image Net-1K classification) and self-supervised pretrained weights (CLIP, BYOL, Visual MAE) in 3 task domains and 35 individual tasks, and demonstrate that our claims are strongly validated in various settings. |
| Researcher Affiliation | Collaboration | Mohit Sharma 12, Claudio Fantacci2, Yuxiang Zhou2, Skanda Koppula2, Nicolas Heess2, Jon Scholz2, Yusuf Aytar2 Carnegie Mellon University1, Deep Mind2 |
| Pseudocode | No | The paper does not contain any pseudocode or algorithm blocks. |
| Open Source Code | No | The paper states: 'Please see real world videos at https://sites.google.com/view/robo-adapters.' This link is for videos, not source code. |
| Open Datasets | Yes | In this work, we consider Metaworld (Yu et al., 2020), Franka-Kitchen (Gupta et al., 2019), and RGB-Stacking task suites (Lee et al., 2021). |
| Dataset Splits | No | The paper mentions using demonstrations for training and evaluating with rollouts, but does not specify explicit training, validation, or test dataset splits (e.g., percentages or counts for each split) for its own collected demonstration data. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used to run its experiments, such as GPU models, CPU types, or memory specifications. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software components or libraries used in the experiments. |
| Experiment Setup | Yes | Table 5: Training Details for each of the three different task suites used in our work. For each task within the task suite we use the same set of hyperparameters. Loss MSE MSE MSE Optimizer Adam Adam Adam Learning Rate 1e-4 1e-3 1e-4 Weight Decay 1e-6 1e-6 1e-6 Gradient Norm Clip 1.0 1.0 1.0 Training Steps 40K 40K 200K Learning Rate Schedule cosine cosine cosine Learning Rate Schedule Warmup Steps 5K 5K 10K Adapter Features Size 32 32 32 |