tailieunhanh - Báo cáo khoa học: "Computing Confidence Scores for All Sub Parse Trees"

Computing confidence scores for applications, such as dialogue system, information retrieving and extraction, is an active research area. However, its focus has been primarily on computing word-, concept-, or utterance-level confidences. Motivated by the need from sophisticated dialogue systems for more effective dialogs, we generalize the confidence annotation to all the subtrees, the first effort in this line of research. | Computing Confidence Scores for All Sub Parse Trees Feng Lin Department of Computer Science and Engineering Fudan University Shanghai 200433 . China fenglin@ Fuliang Weng Research and Technology Center Robert Bosch LLC Palo Alto CA 94303 USA Abstract Computing confidence scores for applications such as dialogue system information retrieving and extraction is an active research area. However its focus has been primarily on computing word- concept- or utterance-level confidences. Motivated by the need from sophisticated dialogue systems for more effective dialogs we generalize the confidence annotation to all the subtrees the first effort in this line of research. The other contribution of this work is that we incorporated novel long distance features to address challenges in computing multi-level confidence scores. Using Conditional Maximum Entropy CME classifier with all the selected features we reached an annotation error rate of in the SWBD corpus compared with a subtree error rate of a closely related benchmark with the Charniak parser from Kahn et al. 2005 . 1 Introduction There has been a good amount of interest in obtaining confidence scores for improving word or utterance accuracy dialogue systems information retrieving extraction and machine translation Zhang and Rudnicky 2001 Guillevic et al. 2002 Gabsdil et al. 2003 Ueffing et al. 2007 . However these confidence scores are limited to relatively simple systems such as command-n-control dialogue systems. For more sophisticated dialogue systems . Weng et al. 2007 identi- fication of reliable phrases must be performed at different granularity to ensure effective and friendly dialogues. For example in a request of MP3 music domain Play a rock song by Cher if we want to communicate to the user that the system is not confident of the phrase a rock song the confidence scores for each word the artist name Cher and the whole sentence would not be enough. For .