SimPer: Simple Self-Supervised Learning of Periodic Targets
Authors: Yuzhe Yang, Xin Liu, Jiang Wu, Silviu Borac, Dina Katabi, Ming-Zher Poh, Daniel McDuff
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on common real-world tasks in human behavior analysis, environmental sensing, and healthcare domains verify the superior performance of Sim Per compared to state-of-the-art SSL methods, highlighting its intriguing properties including better data efficiency, robustness to spurious correlations, and generalization to distribution shifts. |
| Researcher Affiliation | Collaboration | Yuzhe Yang1, Xin Liu2, Jiang Wu3, Silviu Borac3, Dina Katabi1, Ming-Zher Poh3, Daniel Mc Duff2,3 1MIT CSAIL 2University of Washington 3Google |
| Pseudocode | Yes | A PSEUDO CODE FOR SIMPER We provide the pseudo code of Sim Per in Algorithm 1. Algorithm 1 Simple Self-Supervised Learning of Periodic Targets (Sim Per) |
| Open Source Code | Yes | Code and data are available at: https://github.com/Yyz Harry/Sim Per. |
| Open Datasets | Yes | Rotating Digits (Synthetic Dataset) is a toy periodic learning dataset consists of rotating MNIST digits (Deng, 2012)., SCAMPS (Human Physiology) (Mc Duff et al., 2022), UBFC (Human Physiology) (Bobbia et al., 2019), PURE (Human Physiology) (Stricker et al., 2014), Countix (Action Counting). The Countix dataset (Dwibedi et al., 2020) is a subset of the Kinetics (Kay et al., 2017) dataset annotated with segments of repeated actions and corresponding counts., Land Surface Temperature (LST) (Satellite Sensing). LST contains hourly land surface temperature maps over the continental United States for 100 days (April 7th to July 16th, 2022). ... We created a snapshot of data from the NOAA GOES-16 Level 2 LST product... |
| Dataset Splits | Yes | SCAMPS: We randomly divide the whole dataset into training (2, 000 samples), validation (400 samples), and test (400 samples) set. ... Countix: The resulting dataset has 1, 712 training samples, 457 validation samples, and 963 test samples... |
| Hardware Specification | No | No specific hardware details (e.g., GPU/CPU models, memory) used for running the experiments are mentioned in the paper. |
| Software Dependencies | No | No specific software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow, CUDA versions) are mentioned. |
| Experiment Setup | Yes | Rotating Digits: In the supervised setting, we train all models for 20 epochs using the Adam optimizer (Kingma & Ba, 2014), with an initial learning rate of 10 3 and then decayed by 0.1 at the 12-th and 16-th epoch, respectively. We fix the batch size as 64 and use the checkpoint at the last epoch as the final model for evaluation. In the self-supervised setting, we train all models for 60 epochs, which ensures convergence for all tested algorithms. We again employ the Adam optimizer and decay the learning rating at the 40-th and 50-th epoch, respectively. Other training hyper-parameters remain unchanged. |