Inverse Online Learning: Understanding Non-Stationary and Reactionary Policies
Authors: Alex Chan, Alicia Curth, Mihaela van der Schaar
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through application to the analysis of UNOS organ donation acceptance decisions, we demonstrate that our approach can bring valuable insights into the factors that govern decision processes and how they change over time. ... In this section we explore the explainability benefits of our method for evaluating real decision making. Given space constraints, we relegate validation of the method on synthetic examples to the appendix, and focus here on a real medical example of accepting liver donation offers for transplantation. ... In Table 2 we compare the predictive power of our method and all benchmarks on the task of imitation learning i.e. matching the actions of the real demonstrator in some held-out test set. |
| Researcher Affiliation | Academia | Alex J. Chan, Alicia Curth, and Mihaela van der Schaar University of Cambridge Department of Applied Mathematics and Theoretical Physics Cambridge, UK {ajc340, amc253, mv472}@cam.ac.uk |
| Pseudocode | Yes | Algorithm 1: Inverse Online Learning (IOL) Result: Parameters φ of variational distribution and θ of generative model Input: D, X, A, Y, learning rate η; Initialise φ, θ; while not converged do Evaluate pθ(mt|mt 1) ; Calculate forward model next step priors Evaluate qφ(mt|mt 1, h) ; Inference network to get posterior Sample ˆ m qφ( m| h) ; Straight-through MC sample NLLloss = Eqφ( m| h) pθ( a| m) ; KLloss = DKL qφ(m1| x)||pθ(m1) + PT t=2 E DKL qφ(mt|mt 1, h)||pθ(mt|mt 1) ; F(φ, θ) = NLLloss + KLloss; (φ , θ ) (φ, θ) η φ,θF(φ, θ) ; Gradient step for φ, θ φ, θ φ , θ end Return: φ, θ |
| Open Source Code | Yes | Code is made available at https://github.com/Xander JC/inverse-online, along with the group codebase at https://github.com/vanderschaarlab/mlforhealthlabpub. |
| Open Datasets | Yes | Our data focuses on data from the United Network for Organ Sharing (UNOS), the US non-profit that manages their national waiting list and maintains a database and record of transplantations that occur. ... All medical data used has undergone a de-identification process and consent was obtained by the relevant curators for their data to be publicly released. ... The ICU data covers the treatment of 23,106 of patients in the intensive care unit from Amsterdam UMC Elbers (2019). ... The CF data considers patients enrolled in the UK Cystic Fibrosis Registry Taylor Robinson et al. (2018), coviering around 5,800 patients over the course of multiple years. |
| Dataset Splits | Yes | In our action matching experiments we split the data into Train/Validation/Test folds by separating the data by centre. In particular we use centre CTR23901 with 114,314 patients for training; CTR124 with 21,067 patients for validation; and finally CTR279 with 14,295 patients for testing. |
| Hardware Specification | Yes | All experiments were performed on a 2016 Mac Book Pro, using a 2.9 GHz Dual-Core Intel Core i5 with 8GB of LPDDR3 RAM and no GPU acceleration. |
| Software Dependencies | No | The paper states "Code was written in Py Torch Paszke et al. (2019)". While it names PyTorch and cites its paper, it does not specify a version number for PyTorch (e.g., "PyTorch 1.9"), nor does it list any other software dependencies with their specific version numbers. |
| Experiment Setup | No | The paper mentions that "Hyperparameters were selected through grid search over a validation fold of the training data." However, it does not provide the specific values for these hyperparameters (e.g., learning rate, batch size, number of epochs) or other detailed training configurations (e.g., optimizer settings, network architecture specifics like number of layers or units). |