Title: Quality Assessment of Classification and Cluster Maps without Ground Truth Knolwedge
Authors: BARALDI ANDREABRUZZONE LorenzoBLONDA Palma
Citation: IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING vol. 43 no. 4 p. 857-873
Publisher: IEEE GEOSCIENCE AND REMOTE SENSING SOCIETY
Publication Year: 2005
JRC Publication N°: JRC32634
URI: http://publications.jrc.ec.europa.eu/repository/handle/JRC32634
Type: Articles in Journals
Abstract: This work focuses on two challenging types of problems related to quality assessment and comparison of thematic maps generated from remote sensing (RS) images when little or no ground truth knowledge is available. These problems occur when: 1) competing thematic maps, generated from the same input RS image, assumed to be available, must be compared, but no ground truth knowledge is found to assess the accuracy of the mapping problem at hand, and 2) the generalization capability of competing classifiers must be estimated and compared when the mall/unrepresentative ground truth problem affects the RS inductive learning application at hand. Specifically focused on badly posed image classification tasks, this paper presents an original data-driven (i.e., unsupervised) thematic map quality assessment (DAMA) strategy complementary (not alternative) in nature to traditional supervised map accuracy assessment techniques, driven by the expensive and error-prone digitization of ground truth knowledge. To compensate for the lack of supervised regions of interest, DAMA generates so-called multiple reference cluster maps from several blocks of the input RS image that are clustered separately. Due to the unsupervised (i.e., subjective) nature (ill-posedness) of data clustering, DAMA provides no (absolute) map accuracy measure. Rather, DAMA’s map quality indexes are to be considered unsupervised (i.e., subjective) relative estimates of labeling and segmentation consistency between every competing map at hand and the set of multiple reference cluster maps. In two badly posed RS image mapping experiments, DAMA’s map quality measures are proven to be: 1) useful in the relative comparison of competing mapping systems; 2) consistent with theoretical expectations; and 3) in line with mapping quality criteria adopted by expertphotointerpreters. Documented limitations of DAMA are that it is intrinsically heuristic due to the subjective nature of the clustering problem, and like any evaluation measure, it cannot be injective.
JRC Institute:Institute for the Protection and Security of the Citizen

Files in This Item:
There are no files associated with this item.


Items in repository are protected by copyright, with all rights reserved, unless otherwise indicated.