tailieunhanh - Báo cáo khoa học: "Creating a Gold Standard for Sentence Clustering in Multi-Document Summarization"

Sentence Clustering is often used as a first step in Multi-Document Summarization (MDS) to find redundant information. All the same there is no gold standard available. This paper describes the creation of a gold standard for sentence clustering from DUC document sets. The procedure of building the gold standard and the guidelines which were given to six human judges are described. The most widely used and promising evaluation measures are presented and discussed. regenerated from all/some sentences in a cluster (Barzilay and McKeown, 2005). . | Creating a Gold Standard for Sentence Clustering in Multi-Document Summarization Johanna Geiss University of Cambridge Computer Laboratory 15 JJ Thomson Avenue Cambridge CB3 0FD UK Abstract Sentence Clustering is often used as a first step in Multi-Document Summarization MDS to find redundant information. All the same there is no gold standard available. This paper describes the creation of a gold standard for sentence clustering from DUC document sets. The procedure of building the gold standard and the guidelines which were given to six human judges are described. The most widely used and promising evaluation measures are presented and discussed. 1 Introduction The increasing amount of online information and the growing number of news websites lead to a debilitating amount of redundant information. Different newswires publish different reports about the same event resulting in information overlap. Multi-Document Summarization MDS can help to reduce the amount of documents a user has to read to keep informed. In contrast to single document summarization information overlap is one of the biggest challenges to MDS systems. While repeated information is a good evidence of importance this information should be included in a summary only once in order to avoid a repetitive summary. Sentence clustering has therefore often been used as an early step in MDS Hatzi-vassiloglou et al. 2001 Marcu and Gerber 2001 Radev et al. 2000 . In sentence clustering semantically similar sentences are grouped together. Sentences within a cluster overlap in information but they do not have to be identical in meaning. In contrast to paraphrases sentences in a cluster do not have to cover the same amount of information. One sentence represents one cluster in the summary. Either a sentences from the cluster is selected Aliguliyev 2006 or a new sentence is regenerated from all some sentences in a cluster Barzilay and McKeown 2005 . Usually the quality of the sentence