Reinforcement Learning based Disease Progression Model for Alzheimer’s Disease
Authors: Krishnakant Saboo, Anirudh Choudhary, Yurui Cao, Gregory Worrell, David Jones, Ravishankar Iyer
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We model Alzheimer s disease (AD) progression by combining differential equations (DEs) and reinforcement learning (RL) with domain knowledge. DEs provide relationships between some, but not all, factors relevant to AD. We assume that the missing relationships must satisfy general criteria about the working of the brain, for e.g., maximizing cognition while minimizing the cost of supporting cognition. This allows us to extract the missing relationships by using RL to optimize an objective (reward) function that captures the above criteria. We use our model consisting of DEs (as a simulator) and the trained RL agent to predict individualized 10-year AD progression using baseline (year 0) features on synthetic and real data. The model was comparable or better at predicting 10-year cognition trajectories than state-of-the-art learning-based models. |
| Researcher Affiliation | Collaboration | Krishnakant V. Saboo UIUC ksaboo2@illinois.edu Anirudh Choudhary UIUC ac67@illinois.edu Yurui Cao UIUC yuruic2@illinois.edu Gregory A. Worrell Mayo Clinic Worrell.Gregory@mayo.edu David T. Jones Mayo Clinic Jones.David@mayo.edu Ravishankar K. Iyer UIUC rkiyer@illinois.edu |
| Pseudocode | No | The paper describes the methodology conceptually, including equations and a workflow diagram (Fig. 3), but it does not provide any formal pseudocode or algorithm blocks. |
| Open Source Code | No | The paper mentions 'The garage contributors. Garage: A toolkit for reproducible reinforcement learning research. https://github.com/rlworkgroup/garage, 2019.' and 'We implement the simulator using Open AI s Gym framework [24].' These refer to external toolkits used, not the specific source code implemented by the authors for this paper. |
| Open Datasets | Yes | We validated the model on synthetic data and real-world data derived from the Alzheimer s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu) [25]. |
| Dataset Splits | Yes | We used 5-fold cross-validation for evaluation with a 64:16:20 split into training, validation, and testing sets. In each fold, each individual s data was only part of one of the three sets. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running its experiments. It mentions using Open AI's Gym framework, but no hardware specifications. |
| Software Dependencies | No | The paper mentions using 'Open AI s Gym framework [24]' and 'TRPO [23]' for the RL agent, but it does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | All the agent models are trained for 1 million episodes with a batch size of 1000. We clip the reward within a range of [-2000, 2000] with a KL-divergence of 0.01 and generalized advantage estimate discount factor of 0.97. We do not tweak any other hyperparameters of TRPO. The policy network is parameterized by a two-layer feedforward neural network with 32 hidden units each. We performed a grid-search for λ and I(0) and chose the value that minimized the validation set error. We set cognitive demand Ctask = 10 for all the experiments. |