Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Self-Supervised Learning via Maximum Entropy Coding
Authors: Xin Liu, Zhongdao Wang, Ya-Li Li, Shengjin Wang
NeurIPS 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments demonstrate that MEC learns a more generalizable representation than previous methods based on specific pretext tasks. It achieves state-of-the-art performance consistently on various downstream tasks, including not only Image Net linear probe, but also semi-supervised classification, object detection, instance segmentation, and object tracking. |
| Researcher Affiliation | Academia | Xin Liu Zhongdao Wang Yali Li Shengjin Wang Beijing National Research Center for Information Science and Technology (BNRist) Department of Electronic Engineering, Tsinghua University EMAIL EMAIL |
| Pseudocode | Yes | An overview of MEC is illustrated in Figure 2 and a Py Torch-like pseudocode is provided in Appendix A. |
| Open Source Code | Yes | Code and pre-trained models are available at https://github.com/xinliu20/MEC. |
| Open Datasets | Yes | We perform self-supervised pre-training using the proposed MEC on the training set of the Image Net ILSVRC-2012 dataset [17]. |
| Dataset Splits | Yes | We train a linear classifier on top of frozen representations of the pre-trained model on the Image Net training set, and report the top-1 accuracy on the Image Net validation set, which is a standard and important protocol in SSL [88, 12, 31, 14, 86]. ... We fine-tune the pre-trained model on a small subset of Image Net for classification task. Specifically, we adopt the same fixed splits of 1% and 10% of Image Net training set as in [12] and report both top-1 and top-5 accuracies in Table 2. |
| Hardware Specification | Yes | The total amount of compute used for this research is approximately 1,600 GPU hours per experiment on 8 NVIDIA V100 GPUs. |
| Software Dependencies | No | Our implementation is based on PyTorch [50] and uses Detectron2 [79] for object detection and instance segmentation experiments. We use a single ResNet-50 [33] backbone for all Image Net linear probing and semi-supervised classification experiments. No specific version numbers for PyTorch or Detectron2 are provided. |
| Experiment Setup | Yes | We implement our method based on Py Torch and train ResNet-50 models using SGD with momentum. We set the base learning rate to 0.05, batch size to 256, and weight decay to 1e-4 for 100 epochs unless specified otherwise. |