Title: Using Parallel corpora for Multilingual (Multi-Document) Summarisation Evaluation
Citation: Multilingual and Multimodal Information Access Evaluation - ISBN: 978-3-642-15997-8; ISSN: 0302-9743 vol. 6360/2010 p. 52-63
Publisher: Springer-Verlag
Publication Year: 2010
JRC N°: JRC59027
URI: http://www.springerlink.com/content/l145412357783t24/
DOI: 10.1007/978-3-642-15998-5_7
Type: Articles in periodicals and books
Abstract: We are presenting a method for the evaluation of multilingual multi-document summarisation that allows saving precious annotation time and that makes the evaluation results across languages directly comparable. The approach is based on the manual selection of the most important sentences in a cluster of documents from a sentence-aligned parallel corpus, and by projecting the sentence selection to various target languages. We also present two ways of exploiting inter-annotator agreement levels, apply them both to a baseline sentence extraction summariser in seven languages, and discuss the result differences between the two evaluation versions, as well as between languages. The same method can in principle be used to evaluate single-document summarisers or information extraction tools.
JRC Directorate:Space, Security and Migration

Files in This Item:
There are no files associated with this item.

Items in repository are protected by copyright, with all rights reserved, unless otherwise indicated.