Attend and Diagnose: Clinical Time Series Analysis Using Attention Models
Authors: Huan Song, Deepta Rajan, Jayaraman Thiagarajan, Andreas Spanias
AAAI 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Using the recent MIMIC-III benchmark datasets, we demonstrate that the proposed approach achieves state-of-the-art performance in all tasks, outperforming LSTM models and classical baselines with handengineered features. |
| Researcher Affiliation | Collaboration | Sen SIP Center, School of ECEE, Arizona State University, Tempe, AZ IBM Almaden Research Center, 650 Harry Road, San Jose, CA Lawrence Livermore National Labs, 7000 East Avenue, Livermore, CA |
| Pseudocode | Yes | The pseudocode to perform dense interpolation for a given sequence is shown in Algorithm 1. Denoting the hidden representation at time t, from the attention model, as st Rd, the interpolated embedding vector will have dimension d M, where M is the dense interpolation factor. Note that when M = T, it reduces to the concatenation case. The main idea of this scheme is to determine weights w, denoting the contribution of st to the position m of the final vector representation u. As we iterate through the timesteps of a sequence, we obtain s, the relative position of time step t in the final representation u and w is computed as w = (1 |s m| M )2. We visualize the dense interpolation process in Figure 2 for the toy case of T = 5, M = 3. The larger weights in w are indicated by darker edges while the lighter edges indicates lesser influence. In practice, dense interpolation is implemented efficiently by caching w s into a |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described. |
| Open Datasets | Yes | Using the recent MIMIC-III benchmark datasets, we demonstrate that the proposed approach achieves state-of-the-art performance in all tasks, outperforming LSTM models and classical baselines with handengineered features. Interestingly, these benchmarks are supported by the Medical Information Mart for Intensive Care (MIMIC-III) database (Johnson et al. 2016), the largest publicly available repository of rich clinical data currently available. Following (Harutyunyan et al. 2017), we used the cohort of 33, 798 unique patients with a total of 42, 276 hospital admissions and ICU stays. |
| Dataset Splits | Yes | The size of the benchmark dataset for each task is highlighted in Table 1. Table 1: Task-specific sample sizes of MIMIC-III dataset. Benchmark Train Validation Test Mortality 14,659 3,244 3,236 Decompensation 2,396,001 512,413 523,208 Length of Stay 2,392,950 532,484 525,912 Phenotyping 29,152 6,469 6,281 |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory amounts) used for running its experiments. |
| Software Dependencies | No | Note, in all cases, we used the Adam optimizer (Kingma and Ba 2014) with parameters β1 = 0.9, β2 = 0.98 and ϵ = 10 8. |
| Experiment Setup | Yes | The learning rate was set to 0.0005, batch size was fixed at 128 and a residue dropout probability of 0.4 was used. In this case, we set the batch size to 256, residue dropout to 0.3 and the learning rate at 0.0005. Our best results were obtained from training merely on about 25 chunks (batch size = 128, learning rate = 0.001) , when N = 1 and M = 10 (see Figure 3(e)), indicating that increasing the capacity of the model easily leads to overfitting. |