How Mask Matters: Towards Theoretical Understandings of Masked Autoencoders
Authors: Qi Zhang, Yifei Wang, Yisen Wang
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we first present the main empirical results of our proposed U-MAE loss on different real-world datasets with different backbones. Then we conduct a series of experiments to understand how well the U-MAE loss works. |
| Researcher Affiliation | Academia | 1 Key Lab. of Machine Perception (Mo E), School of Intelligence Science and Technology, Peking University 2 School of Mathematical Sciences, Peking University 3 Institute for Artificial Intelligence, Peking University |
| Pseudocode | No | No structured pseudocode or algorithm blocks were found in the paper. |
| Open Source Code | Yes | Code is available at https://github.com/zhangq327/U-MAE. |
| Open Datasets | Yes | extensive experiments are conducted on CIFAR-10 [21], Image Net-100 [8], and Image Net-1K [8]. |
| Dataset Splits | No | No explicit details on train/validation/test splits (e.g., percentages, sample counts, or citations to predefined splits) were found in the main text. |
| Hardware Specification | No | No specific hardware details (e.g., exact GPU/CPU models, memory amounts, or detailed computer specifications) were found. |
| Software Dependencies | No | No specific ancillary software details, such as library or solver names with version numbers, were found in the paper. |
| Experiment Setup | Yes | The mask ratio is set to 0.75. For U-MAE, the coefficient of the uniformity term is set to 0.01. On CIFAR-10, we pretrain the model for 2000 epochs with batch size 4096 and weight decay 0.05. On Image Net-100 and Image Net-1K, we pretrain the model for 200 epochs with batch size 1024 and weight decay 0.05. |