Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistency
Authors: Xiang Zhang, Ziyuan Zhao, Theodoros Tsiligkaridis, Marinka Zitnik
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate the new method on eight datasets, including electrodiagnostic testing, human activity recognition, mechanical fault detection, and physical status monitoring. Experiments against eight state-of-the-art methods show that TF-C outperforms baselines by 15.4% (F1 score) on average in one-toone settings (e.g., fine-tuning an EEG-pretrained model on EMG data) and by 8.4% (precision) in challenging one-to-many settings (e.g., fine-tuning an EEG-pretrained model for either hand-gesture recognition or mechanical fault prediction), reflecting the breadth of scenarios that arise in real-world applications. |
| Researcher Affiliation | Collaboration | Xiang Zhang Harvard University xiang_zhang@hms.harvard.edu Ziyuan Zhao Harvard University ziyuanzhao@college.harvard.edu Theodoros Tsiligkaridis MIT Lincoln Laboratory ttsili@ll.mit.edu Marinka Zitnik Harvard University marinka@hms.harvard.edu |
| Pseudocode | No | The paper does not contain any explicitly labeled pseudocode or algorithm blocks. It describes the methods and loss functions in prose and mathematical notation. |
| Open Source Code | Yes | The source code and datasets are available at https://github.com/mims-harvard/TFC-pretraining. |
| Open Datasets | Yes | The source code and datasets are available at https://github.com/mims-harvard/TFC-pretraining. ... (1) SLEEPEEG [61] has 371,055 univariate brainwaves (EEG; 100 Hz) collected from 197 individuals. ... (8) EMG [66] consists of 163 EMG samples with 3-class labels implying muscular diseases. Dataset labels are not used in pre-training. Further dataset statistics are in Appendix D and Table 3. |
| Dataset Splits | Yes | We randomly split the dataset into 80% for training, 10% for validation, and 10% for testing. |
| Hardware Specification | Yes | Computational resources: All experiments were conducted on NVIDIA A100 GPUs. |
| Software Dependencies | Yes | We use PyTorch [73] with Python 3.8. |
| Experiment Setup | Yes | All models are trained for 100 epochs with Adam optimizer (β1=0.9, β2=0.999), a weight decay of 1e−4, and a batch size of 128. For fine-tuning, we use a learning rate of 1e−3 and train for 20 epochs. The learning rate for pre-training is 1e−4. We randomly split the dataset into 80% for training, 10% for validation, and 10% for testing. |