Extracting Biomolecular Interactions Using Semantic Parsing of Biomedical Text
Authors: Sahil Garg, Aram Galstyan, Ulf Hermjakob, Daniel Marcu
AAAI 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments show that our approach yields interaction extraction systems that are more robust in environments where there is a significant mismatch between training and test conditions. We conduct an exhaustive empirical evaluation of the proposed extraction system on 45+ research articles on cancer (approximately 3k sentences), containing approximately 20,000 positive-negative labeled biomolecular interactions1. Our results indicate that the joint extraction method that leverages both AMRs and SDGs parses significantly improves the extraction accuracy, and is more robust to mismatch between training and test conditions. |
| Researcher Affiliation | Academia | Sahil Garg, Aram Galstyan, Ulf Hermjakob, and Daniel Marcu USC Information Sciences Institute Marina del Rey, CA 90292 {sahil, galstyan, ulf, marcu}@isi.edu |
| Pseudocode | Yes | Algorithm 1 (a) Initialize: A(0) i = (XT i Xi) 1XT i Y i. (b) Iteratively update A(t+1) i until convergence: A(t+1) i = B[XT i Y i + j N (i) Zij i T Zij j A(t) j ] B = [XT i Xi + j N (i) Zij i T Zij i ] 1 (c) Set the inverse edge label matrices: Aiinv = A 1 i . |
| Open Source Code | Yes | 1The code and the data are available at https://github.com/sgarg87/big_mech_isi_gg |
| Open Datasets | Yes | AIMed: This is a publicly available dataset7, which contains about 2000 sentences from 225 abstracts. In contrast to Pub Med45, this dataset is very limited as it describes only whether a given pair of proteins interact or not, without specifying the interaction type. Nevertheless, we find it useful to include this dataset in our evaluation since it enables us to compare our results with other reported methods. 7http://corpora.informatik.hu-berlin.de |
| Dataset Splits | Yes | In a typical evaluation scenario, validation is performed by random sub-sampling of labeled interactions (at sentence level) for a test subset, and using the rest as a training set. This sentence-level validation approach is not always appropriate for extracting protein interactions (Tikk et al. 2010), since interactions from a single/multiple sentences in a document can be correlated. Such correlations can lead to information leakage between training and test sets (artificial match, not encountered in real settings). Therefore, for a realistic evaluation, we divide data sets at documents level into approx. 10 subsets such that there is minimal match between a subset, chosen as test set, and the rest of sub sets used for training a kernel classifier. In the Pub Med45 data sets, the 45 articles are clustered into 11 subsets by clustering Pub Med-Ids (training data also includes gold annotations). In AIMed, abstracts are clustered into 10 subsets on abstract-ids. In each of 25 independent test runs (5 for AIMed data) on a single test subset, 80% interactions are randomly sub sampled from the test subset and same percent from the train data. |
| Hardware Specification | No | The paper does not specify any hardware details such as CPU, GPU models, or memory for running the experiments. It only states that word vectors were learned 'on over one million Pub Med articles'. |
| Software Dependencies | No | For the classification, we use the LIBSVM implementation of Kernel Support Vector Machines (Chang and Lin 2011) with the sklearn python wrapper 8. While LIBSVM and scikit-learn are mentioned, specific version numbers for these software dependencies are not provided, which would be necessary for full reproducibility. |
| Experiment Setup | Yes | In the experiments below we set λ = 0.99 and α = 0.4. Also, consistent with previous studies, we normalize the graph kernel (e.g., kernel similarity K(Gi, Gj) is divided by the normalization term K(Gi, Gi)K(Gj, Gj)) to increase accuracy. Specifically, we used settings { probability = True, C = 1, class weight = auto}. |