tailieunhanh - Báo cáo khoa học: "WORD-SENSE DISAMBIGUATION METHODS USING STATISTICAL"

We describe a statistical technique for assigning senses to words. An instance of a word is assigned a sense by asking a question about the context in which the word appears. The question is constructed to have high mutual information with the translation of that instance in another language. When we incorporated this method of assigning senses into our statistical machine translation system, the error rate of the system decreased by thirteen percent. language model does not realize that take my own decision is improbable because take and decision no longer fall within a single trigram. . | WORD-SENSE DISAMBIGUATION USING STATISTICAL METHODS Peter F. Brown Stephen A. Della Pietra Vincent J. Della Pietra and Robert L. Mercer IBM Thomas J. Watson Research Center . Box 704 Yorktown Heights NY 10598 ABSTRACT We describe a statistical technique for assigning senses to words. An instance of a word is assigned a sense by asking a question about the context in which the word appears. The question is constructed to have high mutual information with the translation of that instance in another language. When we incorporated this method of assigning senses into our statistical machine translation system the error rate of the system decreased by thirteen percent. INTRODUCTION An alluring aspect of the statistical approach to machine translation rejuvenated by Brown et al. Brown et al. 1988 Brown et al. 1990 is the systematic framework it provides for attacking the problem of lexical disambiguation. For example the system they describe translates the French sentence .Je vais prendre la decision as I will make the decision correctly interpreting prendre as make. The statistical translation model which supplies English translations of French words prefers the more common translation take but the trigram language model recognizes that the three-word sequence make the decisionis much more probable than take the decision. The system is not always so successful. It incorrectly renders Je vats prendre ma propre decision as I will take my own decision. The language model does not realize that take my own decision is improbable because take and decision no longer fall within a single trigram. Errors such as this are common because the statistical models only capture local phenomena if the context necessary to determine a translation falls outside the scope of the models the word is likely to be translated incorrectly. However if the relevant context is encoded locally the word should be translated correctly. We can achieve this within the traditional paradigm of analysis