REBAR: Retrieval-Based Reconstruction for Time-series Contrastive Learning
Authors: Maxwell Xu, Alexander Moreno, Hui Wei, Benjamin Marlin, James Matthew Rehg
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through validation experiments, we show that the REBAR error is a predictor of mutual class membership. Once integrated into a contrastive learning framework, our REBAR method learns an embedding that achieves state-of-the-art performance on downstream tasks across various modalities. |
| Researcher Affiliation | Academia | Maxwell A. Xu1, Alexander Moreno1, Hui Wei3, Benjamin M. Marlin3, James M. Rehg2 1 Georgia Tech, 2 UIUC, 3 UMass Amherst |
| Pseudocode | Yes | Algorithm 1 is the procedure for generating the confusion matrix from the REBAR Nearest Neighbor Validation Experiment detailed in Sec. 4.3 with Eq. 9. |
| Open Source Code | Yes | The full REBAR approach can be seen in Fig. 2, and our public code repository can be found here: https://github.com/maxxu05/rebar. Upon acceptance, we will release our Git Hub code publicly, which will have the set seeds and exact code we used to run our experiments. |
| Open Datasets | Yes | Data: We utilize 3 datasets from 3 different sensor domains with time-series that have their classification labels change over time: Human Activity Recognition (HAR) with accelerometer and gyroscopic sensors to measure activity (Reyes-Ortiz et al., 2015), PPG to measure stress (Schmidt et al., 2018), and ECG to measure heart condition (Moody, 1983). The datasets used are publicly available, and we describe how we curate each of them for our task in Appendix A.2. |
| Dataset Splits | Yes | The time-series are split into a 70/15/15 train/val/test split. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running experiments, such as particular GPU models, CPU specifications, or memory configurations. |
| Software Dependencies | No | The paper mentions the use of 'PyTorch' in Appendix A.3 but does not specify version numbers for PyTorch or any other software dependencies, which is necessary for reproducible environment setup. |
| Experiment Setup | Yes | For the PPG and HAR domains were trained with a learning rate of .00001 until convergence, and the ECG domain was trained with a learning rate of .000001 until convergence. TS2Vec... learning rate of .0001 and batch size of 16, in ECG, we have a learning rate of 0.00001 and batch size of 64, and in HAR, we have a learning rate of 0.00001 and batch size of 64. Our cross-attention model was trained to convergence and the dilated convolution block has an embedding size of 256 channels, initial kernel size of 15 and dilation of 1. The dilation gets doubled for each following layer. The input channel size of the first convolution is equal to the number of channels present in the data, and the bottleneck layer has size 32. For the PPG and ECG datasets, we have 6 dilated convolution layers to capture a larger receptive field of 883 and for the HAR dataset, we have 2 layers to capture a receptive field of 43. During training, our extended mask sizes for PPG, ECG, and HAR are 300, 300, and 15 respectively. During evaluation, the transient mask masked out 50% of the time points of the signal. During contrastive learning, for PPG data, we have a learning rate of .0001, 20 sampled candidates, τ is 10, α is 0.5, and batch size of 16, for ECG, we have a learning rate of .001, 20 sampled candidates, τ is .01, α is 1, and a batch size of 16, and for HAR, we have a learning rate of .001, 20 sampled candidates, τ is .1, α is 0, and a batch size of 64. |