Title: Conflation of expert and crowd reference data to validate global binary thematic maps
Authors: WALDNER FRANCOISSCHUCKNECHT ANNELESIV MYROSLAVAGALLEGO PINILLA FRANCISCOSEE LINDAPEREZ HOYOS ANAD'ANDRIMONT RAPHAELDE MAET THOMASLASO BAYAS JUAN CARLOSFRITZ STEFFENLEO OLIVIERKERDILES HERVEDIEZ MONICAVAN TRICHT KRISTOFGILLIAMS SVENSHELESTOV ANDRIILAVRENIUK MYKOLASIMOES MARTHARETHFERRAZ RODRIGOBELLON BEATRIZBEGUE AGNESHAZEU GERARDSTONACEK VACLAVKOLOMAZNIK JANMISUREC JANVERON SANTIAGO RDE ABBELLEYRA DIEGOPLOTNIKOV DMITYRMINGYONG LISINGHA MRINALPATIL PRASHANTZHANG MIAODEFOURNEY PIERRE
Citation: REMOTE SENSING OF ENVIRONMENT vol. 221 p. 235-246
Publisher: ELSEVIER SCIENCE INC
Publication Year: 2019
JRC N°: JRC114258
ISSN: 0034-4257 (online)
URI: https://www.sciencedirect.com/science/article/pii/S0034425718305017?via%3Dihub#s0125
http://publications.jrc.ec.europa.eu/repository/handle/JRC114258
DOI: 10.1016/j.rse.2018.10.039
Type: Articles in periodicals and books
Abstract: With the unprecedented availability of satellite data and the rise of global binary maps, the collection of shared reference data sets should be fostered to allow systematic product benchmarking and validation. Authoritative global reference data are generally collected by experts with regional knowledge through photo-interpretation. During the last decade, crowdsourcing has emerged as an attractive alternative for rapid and relatively cheap data collection, beckoning the increasingly relevant question: can these two data sources be combined to validate thematic maps? In this article, we compared expert and crowd data and assessed their relative agreement for cropland identification, a land cover class often reported as difficult to map. Results indicate that observations from experts and volunteers could be partially conflated provided that several consistency checks are performed. We propose that conflation, i.e., replacement and augmentation of expert observations by crowdsourced observations, should be carried out both at the sampling and data analytics levels. The latter allows to evaluate the reliability of crowdsourced observations and to decide whether they should be conflated or discarded. We demonstrate that the standard deviation of crowdsourced contributions is a simple yet robust indicator of reliability which can effectively inform conflation. Following this criterion, we found that 70% of the expert observations could be crowdsourced with little to no effect on accuracy estimates, allowing a strategic reallocation of the spared expert effort to increase the reliability of the remaining 30% at no additional cost. Finally, we provide a collection of evidence-based recommendations for future hybrid reference data collection campaigns.
JRC Directorate:Sustainable Resources

Files in This Item:
There are no files associated with this item.


Items in repository are protected by copyright, with all rights reserved, unless otherwise indicated.