Title: Using Parallel corpora for Multilingual (Multi-Document) Summarisation Evaluation
Authors: TURCHI MARCOSTEINBERGER JOSEFKABADJOV MIJAILSTEINBERGER Ralf
Citation: Multilingual and Multimodal Information Access Evaluation - ISBN: 978-3-642-15997-8; ISSN: 0302-9743 vol. 6360/2010 p. 52-63
Publisher: Springer-Verlag
Publication Year: 2010
JRC N°: JRC59027
URI: http://www.springerlink.com/content/l145412357783t24/
http://publications.jrc.ec.europa.eu/repository/handle/JRC59027
DOI: 10.1007/978-3-642-15998-5_7
Type: Contributions to Conferences
Abstract: We are presenting a method for the evaluation of multilingual multi-document summarisation that allows saving precious annotation time and that makes the evaluation results across languages directly comparable. The approach is based on the manual selection of the most important sentences in a cluster of documents from a sentence-aligned parallel corpus, and by projecting the sentence selection to various target languages. We also present two ways of exploiting inter-annotator agreement levels, apply them both to a baseline sentence extraction summariser in seven languages, and discuss the result differences between the two evaluation versions, as well as between languages. The same method can in principle be used to evaluate single-document summarisers or information extraction tools.
JRC Institute:Institute for the Protection and Security of the Citizen

Files in This Item:
There are no files associated with this item.


Items in repository are protected by copyright, with all rights reserved, unless otherwise indicated.