Learning to Speed Up Structured Output Prediction
Authors: Xingyuan Pan, Vivek Srikumar
ICML 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our method on the task of entity and relation extraction and show that the speedup classifier outperforms even greedy search in terms of speed without loss of accuracy. and 5. Experiments We empirically evaluate the speedup based inference scheme described in Section 4 on the problem of predicting entities and relations (i.e. our running example). |
| Researcher Affiliation | Academia | 1School of Computing, University of Utah, Salt Lake City, Utah, USA. Correspondence to: Xingyuan Pan <xpan@cs.utah.edu>, Vivek Srikumar <svivek@cs.utah.edu>. |
| Pseudocode | Yes | Algorithm 1 Learning a speedup classifier using examples {xi}N i=1, and a black-box Solver Solve. |
| Open Source Code | No | The paper does not provide any statement regarding the release of source code or a link to a code repository for the described methodology. |
| Open Datasets | Yes | The dataset we used is from the previous work by Roth & Yih (2004). |
| Dataset Splits | No | The paper states: "We used 70% of the labeled data to train an ILP-based inference scheme... The remaining 30% labeled data are held out for evaluations." This indicates a train/test split but does not specify a separate validation set. |
| Hardware Specification | Yes | All our experiments were conducted on a server with eight Intel i7 3.40 GHz cores and 16G memory. We disabled multithreaded execution in all cases for a fair comparison. |
| Software Dependencies | No | The paper mentions software like "Gurobi solver" and "Stanford Named Entity Recognizer" but does not provide specific version numbers for these or any other software dependencies. |
| Experiment Setup | Yes | We trained a speedup classifier with two different beam sizes. Even with beam width b = 1, we are able to obtain > 95% validity ratio... With beam width b = 2, we recover the ILP model accuracy... We use the evaluation function pθ(v) in Eq. (15) with different values of θ to rank the search nodes. The results are given in Table 2. and The features we use for the speedup classifiers are counts of the pairs of labels of the form (source label, relation label), (relation label, target label), and counts of the triples of labels of the form (source label, relation label, target label). |