Đang chuẩn bị liên kết để tải về tài liệu:
Báo cáo khoa học: "A Cognitive Cost Model of Annotations Based on Eye-Tracking Data"

We report on an experiment to track complex decision points in linguistic metadata annotation where the decision behavior of annotators is observed with an eyetracking device. As experimental conditions we investigate different forms of textual context and linguistic complexity classes relative to syntax and semantics. Our data renders evidence that annotation performance depends on the semantic and syntactic complexity of the decision points and, more interestingly, indicates that fullscale context is mostly negligible – with the exception of semantic high-complexity cases. . | A Cognitive Cost Model of Annotations Based on Eye-Tracking Data Katrin Tomanek Language Information Engineering JULIE Lab Universitat Jena Jena Germany Udo Hahn Language Information Engineering JULIE Lab Universitat Jena Jena Germany Steffen Lohmann Dept. of Computer Science Applied Cognitive Science Universitat Duisburg-Essen Duisburg Germany Jurgen Ziegler Dept. of Computer Science Applied Cognitive Science Universitat Duisburg-Essen Duisburg Germany Abstract We report on an experiment to track complex decision points in linguistic metadata annotation where the decision behavior of annotators is observed with an eyetracking device. As experimental conditions we investigate different forms of textual context and linguistic complexity classes relative to syntax and semantics. Our data renders evidence that annotation performance depends on the semantic and syntactic complexity of the decision points and more interestingly indicates that fullscale context is mostly negligible - with the exception of semantic high-complexity cases. We then induce from this observational data a cognitively grounded cost model of linguistic meta-data annotations and compare it with existing non-cognitive models. Our data reveals that the cognitively founded model explains annotation costs expressed in annotation time more adequately than non-cognitive ones. 1 Introduction Today s NLP systems in particular those relying on supervised ML approaches are meta-data greedy. Accordingly in the past years we have witnessed a massive quantitative growth of annotated corpora. They differ in terms of the natural languages and domains being covered the types of linguistic meta-data being solicited and the text genres being served. We have seen large-scale efforts in syntactic and semantic annotations in the past related to POS tagging and parsing on the one hand and named entities and relations propositions on the other hand. More recently we are dealing with even more challenging issues such as