MEMO: Test Time Robustness via Adaptation and Augmentation
Authors: Marvin Zhang, Sergey Levine, Chelsea Finn
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In our experiments, we evaluate two baseline Res Net models, two robust Res Net-50 models, and a robust vision transformer model, and we demonstrate that this approach achieves accuracy gains of 1-8% over standard model evaluation and also generally outperforms prior augmentation and adaptation strategies. |
| Researcher Affiliation | Academia | Marvin Zhang1, Sergey Levine1, Chelsea Finn2 1UC Berkeley 2Stanford University |
| Pseudocode | Yes | Algorithm 1 Test time robustness via MEMO |
| Open Source Code | Yes | Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Yes] See supplementary material. |
| Open Datasets | Yes | We conduct CIFAR-10 [22] experiments on the CIFAR-10-C [12] and CIFAR-10.1 [37] test sets, and we conduct Image Net [38] experiments on the Image Net-C [12], Image Net-R [14], and Image Net-A [15] test sets. |
| Dataset Splits | Yes | Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Appendix A. |
| Hardware Specification | Yes | These experiments used four Intel Xeon Skylake 6130 CPUs and one NVIDIA TITAN RTX GPU. |
| Software Dependencies | No | The paper does not explicitly list specific software dependencies with version numbers (e.g., Python 3.x, PyTorch 1.x, CUDA 11.x). The checklist also states 'No' for 'total amount of compute and the type of resources used', which typically includes software details. |
| Experiment Setup | Yes | Full details on our experimental protocol are provided in Appendix A." and "Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Appendix A. |