tailieunhanh - Báo cáo khoa học: "Text Summarization Evaluation"

The T I P S T E R Text Summarization Evaluation (SUMMAC) has established definitively that automatic text summarization is very effective in relevance assessment tasks. Summaries as short as 17% of full text length sped up decisionmaking by almost a factor of 2 with no statistically significant degradation in Fscore accuracy. SUMMAC has also introduced a new intrinsic method for automated evaluation of informative summaries. | Proceedings of EACL 99 The TIPSTER SƯMMAC Text Summarization Evaluation Inderjeet Mani David House Gary Klein Lynette Hirschman The MITRE Corporation 11493 Sunset Hills Rd. Reston VA 22090 USA Therese Firmin Department of Defense 9800 Savage Rd. Ft. Meade MD 20755 USA Beth Sundheim SPAWAR Systems Center Code D44208 53140 Gatchell Rd. San Diego CA 92152 USA Abstract The TIPSTER Text Summarization Evaluation SUMMAC has established definitively that automatic text summarization is very effective in relevance assessment tasks. Summaries as short as 17 of full text length sped up decisionmaking by almost a factor of 2 with no statistically significant degradation in F-score accuracy. SUMMAC has also introduced a new intrinsic method for automated evaluation of informative summaries. 1 Introduction In May 1998 the . government completed the TIPSTER Text Summarization Evaluation SUMMAC which was the first large-scale developer-independent evaluation of automatic text summarization systems. The goals of the SUMMAC evaluation were to judge individual summarization systems in terms of their usefulness in specific summarization tasks and to gain a better understanding of the issues involved in building and evaluating such systems. Text Summarization Text summarization is the process of distilling the most important information from a set of sources to produce an abridged version for particular users and tasks Maybury 1995 . Since abridgment is crucial an important parameter to summarization is the level of compression ratio of summary length to source length desired. Summaries can be used to indicate what topics are addressed in the source text and thus can be used to alert the user as to source content the indicative function . In addition summaries can also be used to stand in place of the source the informative function . 202 Burlington Rd. Bedford MA 01730 They can even offer a critique of the source the evaluative function Sparck-Jones 1998 . Often summaries are .