Modern Hopfield Networks and Attention for Immune Repertoire Classification
Authors: Michael Widrich, Bernhard Schäfl, Milena Pavlović, Hubert Ramsauer, Lukas Gruber, Markus Holzleitner, Johannes Brandstetter, Geir Kjetil Sandve, Victor Greiff, Sepp Hochreiter, Günter Klambauer
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate that Deep RC outperforms all other methods with respect to predictive performance on large-scale experiments including simulated and real-world virus infection data and enables the extraction of sequence motifs that are connected to a given disease class. In this section, we report and analyze the predictive power of Deep RC and the compared methods on several immunosequencing datasets. The ROC-AUC is used as the main metric for the predictive power. |
| Researcher Affiliation | Collaboration | ELLIS Unit Linz and LIT AI Lab, Institute for Machine Learning, Johannes Kepler University Linz, Austria Department of Immunology, University of Oslo, Norway Department of Informatics, University of Oslo, Norway Institute of Advanced Research in Artificial Intelligence (IARAI) ... IARAI is supported by Here Technologies. |
| Pseudocode | No | The paper provides mathematical equations and an architecture diagram but does not include explicit pseudocode or algorithm blocks. |
| Open Source Code | Yes | Source code and datasets: https://github.com/ml-jku/Deep RC ... All datasets and code will be fully released at https://github.com/ml-jku/ Deep RC. |
| Open Datasets | Yes | Source code and datasets: https://github.com/ml-jku/Deep RC ... The CMV dataset is publicly available at https://clients.adaptivebiotech.com/ pub/Emerson-2017-Nat Gen. ... All datasets and code will be fully released at https://github.com/ml-jku/ Deep RC. |
| Dataset Splits | Yes | We used a nested 5-fold cross validation (CV) procedure to estimate the performance of each of the methods. All methods could adjust their most important hyperparameters on a validation set in the inner loop of the procedure. |
| Hardware Specification | No | The paper mentions 'increased GPU memory and increased computing power' in a general discussion about future impact, but does not specify any particular hardware (e.g., GPU/CPU models, memory) used for its experiments. |
| Software Dependencies | No | The paper mentions software components like 'Adam' optimizer and types of neural networks (1D convolutional network, self-normalizing layers) but does not provide specific version numbers for any software or libraries. |
| Experiment Setup | Yes | Deep RC is trained using standard gradient descent methods to minimize a cross-entropy loss... we train Deep RC using Adam (Kingma & Ba, 2014) with a batch size of 4 and dropout of input sequences (see Suppl. Sect. A3). ...computation of zi is performed in 16-bit and other computations in 32-bit precision... We used a nested 5-fold cross validation (CV) procedure... All methods could adjust their most important hyperparameters on a validation set in the inner loop... |