tailieunhanh - Báo cáo khoa học: "Evaluation of Machine Translations by Reading Comprehension Tests and Subjective Judgments"

This paper discusses the results of an experiment designed to test the quality of translations, in which human subjects were presented with IBM-produced machine translations of several passages taken from the Russian electrical engineering journal Elektrosviaz, and with human translations of some other passages taken from Telecommunications, the English translation of Elektrosviaz. | Mechanical Translation vol. 8 No. 2 February 1965 Evaluation of Machine Translations by Reading Comprehension Tests and Subjective Judgments by Sheila M. Pfafflin Bell Telephone Laboratories Incorporated Murray Hill New Jersey This paper discusses the results of an experiment designed to test the quality of translations in which human subjects were presented with IBM-produced machine translations of several passages taken from the Russian electrical engineering journal Elektrosviaz and with human translations of some other passages taken from Telecommunications the English translation of Elektrosviaz. The subjects were tested for comprehension of the passages read and were also asked to judge the clarity of individual sentences. Although the human translations generally gave better results than the machine translations the differences were frequently not significant. Most subjects regarded the machine translations as comprehensible and clear enough to indicate whether a more polished human translation was desirable. The reading comprehension test and the judgment of clarity test were found to give more consistent results than an earlier procedure for evaluating translations since the questions asked in the current series of tests were more precise and limited in scope than those in the earlier scries. In view of the considerable effort currently going into mechanical translation it would be desirable to have some way of evaluating the results of various translation methods. An individual who wishes to form his own opinion of such translations can of course read a sample but this procedure is unsatisfactory for many purposes. To indicate only one difficulty individuals vary widely in their reactions to the same sample of translation. However a previous attempt by Miller and Beebecenter1 to develop a more satisfactory approach gave discouraging results. When ratings of the quality of passages were used it was found that subjects had considerable difficulty in .