A Credit Assignment Compiler for Joint Prediction
Authors: Kai-Wei Chang, He He, Stephane Ross, Hal Daume III, John Langford
NeurIPS 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the feasibility of our approach on multiple joint prediction tasks. In all cases, we obtain accuracies as high as alternative approaches, at drastically reduced execution and programming time. We implement the credit assignment compiler in Vowpal-Wabbit (http://hunch.net/~vw/), a fast online learning library, and show that the credit assignment compiler achieves outstanding empirical performance both in accuracy and in speed for several application tasks. Details experimental settings are in appendices. |
| Researcher Affiliation | Collaboration | Kai-Wei Chang University of Virginia kw@kwchang.net He He University of Maryland hhe@cs.umd.edu Hal Daumé III University of Maryland me@hal3.name John Langford Microsoft Research jcl@microsoft.com Stephane Ross Google stephaneross@google.com |
| Pseudocode | Yes | Algorithm 1 MYRUN(X) % for sequence tagging, X: input sequence, Y: output |
| Open Source Code | No | Our approach is implemented using the Vowpal Wabbit [29] toolkit on top of a cost-sensitive classifier [3] trained with online updates [15, 24, 42]. The paper links to Vowpal Wabbit (http://hunch.net/~vw/), which is a third-party library they used, but does not explicitly state that *their specific implementation* of the credit assignment compiler is open-source or provide a direct link to it. |
| Open Datasets | Yes | Part of Speech tagging (POS) on the Wall Street Journal portion of the Penn Treebank; and sequence chunking problem: named entity recognition (NER) based on standard Begin-In-Out encoding on the Co NLL 2003 dataset. The parser is evaluated on the standard WSJ (English, Stanford-style labels), CTB (Chinese) datasets and the Co NLL-X datasets for 10 other languages.11 Our approach is implemented using the Vowpal Wabbit [29] toolkit on top of a cost-sensitive classifier [3] trained with online updates [15, 24, 42]. Details of dataset statistics, experimental settings, additional results on other applications, and pseudocode are in the appendix. ... 11PTB and CTB are prepared by following [8], and Co NLL-X is from the Co NLL shared task 06. |
| Dataset Splits | No | The paper mentions using 'holdout data' for tuning hyperparameters, but does not provide specific details on the size or methodology of the validation split in the main text. 'Details of dataset statistics, experimental settings... are in the appendix' indicates the information might be elsewhere, but it's not explicitly in the provided text. |
| Hardware Specification | No | No specific hardware details (e.g., CPU, GPU models, or memory specifications) used for running experiments were mentioned in the paper. |
| Software Dependencies | No | The paper mentions software like Vowpal-Wabbit, CRF++, CRF SGD, Structured Perceptron, Structured SVM, and Illioins-SL, but does not provide specific version numbers for these dependencies. |
| Experiment Setup | No | The paper states, 'Details experimental settings are in appendices,' and mentions that 'default hyperparameters or the tuned hyperparameters' were used. However, it does not provide specific hyperparameter values or detailed training configurations in the main text. |