Zero-shot causal learning
Authors: Hamed Nilforoshan, Michael Moor, Yusuf Roohani, Yining Chen, Anja Šurina, Michihiro Yasunaga, Sara Oblak, Jure Leskovec
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results on real world datasets in large-scale medical claims and cell-line perturbations demonstrate the effectiveness of our approach. |
| Researcher Affiliation | Academia | Hamed Nilforoshan 1 Michael Moor 1 Yusuf Roohani2 Yining Chen1 Anja Šurina3 Michihiro Yasunaga1 Sara Oblak4 Jure Leskovec1 1Department of Computer Science, Stanford University 2Department of Biomedical Data Science, Stanford University 3School of Computer and Communication Sciences, EPFL 4Department of Computer Science, University of Ljubljana |
| Pseudocode | Yes | Algorithm 1 The Ca ML algorithm |
| Open Source Code | Yes | Code is available at: https://github.com/snap-stanford/caml/ |
| Open Datasets | Yes | We use data for 10,325 different perturbagens from the LINCS Program [74]. [74] Aravind Subramanian, Rajiv Narayan, Steven M Corsello, David D Peck, Ted E Natoli, Xiaodong Lu, Joshua Gould, John F Davis, Andrew A Tubelli, Jacob K Asiedu, David L Lahr, Jodi E Hirschman, Zihan Liu, Melanie Donahue, Bina Julian, Mariya Khan, David Wadden, Ian C Smith, Daniel Lam, Arthur Liberzon, Courtney Toder, Mukta Bagul, Marek Orzechowski, Oana M Enache, Federica Piccioni, Sarah A Johnson, Nicholas J Lyons, Alice H Berger, Alykhan F Shamji, Angela N Brooks, Anita Vrcic, Corey Flynn, Jacqueline Rosains, David Y Takeda, Roger Hu, Desiree Davison, Justin Lamb, Kristin Ardlie, Larson Hogstrom, Peyton Greenside, Nathanael S Gray, Paul A Clemons, Serena Silver, Xiaoyun Wu, Wen-Ning Zhao, Xiaohua Wu, Stephen J Haggarty, Lucienne V Ronco, Jesse S Boehm, Stuart L Schreiber, John G Doench, Joshua A Bittker, David E Root, Bang Wong, and Todd R Golub. A next generation connectivity map: L1000 platform and the first 1,000,000 profiles. Cell, 171(6):1437 1452.e17, November 2017. |
| Dataset Splits | Yes | We split all tasks into meta-training/meta-validation, and a hold-out meta-testing set for evaluating zero-shot predictions and We randomly sample 5% (1.52M patients) to use as controls, with a 40/20/40 split betweem metatrain/meta-val/meta-test. |
| Hardware Specification | Yes | Deep learning-based methods (i.e., Ca ML and its ablations, S-learner w/ meta-learning, T-learner w/ meta-learning, SIN, Graph ITE, Flex TENET, TARNet, and Dragon Net) were run on n1-highmem-64 machines with 4x NVIDIA T4 GPU devices. The remaining baselines (RA-learner, R-learner, X-learner, and T-learner) were run on n1-highmem-64 machines featuring 64 CPUs. |
| Software Dependencies | No | The paper mentions software like RDKit but does not provide specific version numbers for the software dependencies used in the experiments. |
| Experiment Setup | Yes | We perform hyper-parameter optimization with random search for all models, with the meta-testing dataset predetermined and held out. To avoid hyperparameter hacking , hyperparameters ranges are consistent between methods wherever possible, and were chosen using defaults similar to prior work [35, 25]. Choice of final model hyper-parameters was determined using performance metrics (specific to each dataset) computed on the meta-validation dataset, using the best hyper-parameters over 48 runs (6 servers x 4 NVIDIA T4 GPUs per server x 2 runs per GPU ) (Appendix C.4). and For all deep learning-based methods, we employed a batch size of 8,192, except for Graph ITE, where we were restricted to using a batch size of 512 due to larger memory requirements. |