Evaluation of Semantic Dependency Labeling Across Domains
Authors: Svetlana Stoyanchev, Amanda Stent, Srinivas Bangalore
AAAI 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this paper, we present a systematic evaluation of approaches to domain adaptation of generic semantic resources for SLU. We define a semantic dependency labeling (SDL) task... We present a statistical SDL system, One Par... For this purpose we manually annotated the unique sentences in the Communicator 2000 corpus... Using this data, we compare the accuracy of: (a) a generic model for One Par trained on Frame Net data; (b) the generic model plus some handwritten domain-specific constraints... (c) a model trained on Frame Net data and a small amount of domain-specific training data... and (d) a model trained only on domain-specific training data. |
| Researcher Affiliation | Industry | Svetlana Stoyanchev Interactions Labs 25 Broadway, New York, NY sstoyanchev@interactions.com Amanda Stent Yahoo Labs 229 W. 43rd St. New York, NY stent@yahoo-inc.com Srinivas Bangalore Interactions Labs 41 Spring Street, Murray Hill, NJ 07974 sbangalore@interactions.com |
| Pseudocode | No | No pseudocode or algorithm blocks are present. |
| Open Source Code | No | The paper states: 'new annotations on the Communicator 2000 corpus, which we will release to the research community.' This refers to data, not the open source code for their methodology (One Par). |
| Open Datasets | Yes | The Frame Net dataset (Lowe, Baker, and Fillmore 1997) contains 150K sentences with selective annotations of lexical units (LU) and 4K sentences with full text annotations of all predicates in each sentence (FT). ... The Communicator 2000 corpus consists of 662 human-computer spoken (telephone) dialogs in a travel booking domain. |
| Dataset Splits | No | The paper mentions a 'single test split of 1.6K utterances' and a 'model trained on 10% of the Communicator data (300 sentences)' but does not provide a complete and specific train/validation/test split for all data used. |
| Hardware Specification | No | No specific hardware details (e.g., CPU, GPU models, memory, or cluster specifications) are mentioned for the experimental setup. |
| Software Dependencies | No | The paper mentions using 'our own tools for syntactic processing' and 'maximum entropy classifier' but does not specify any software names with version numbers for reproducibility. |
| Experiment Setup | Yes | We use C=2 and T=0.2, values optimized on Frame Net data. |