Geography
Title
Death to Kappa: Birth of quantity disagreement and allocation disagreement for accuracy assessment
Document Type
Article
Abstract
The family of Kappa indices of agreement claim to compare a map's observed classification accuracy relative to the expected accuracy of baseline maps that can have two types of randomness: (1) random distribution of the quantity of each category and (2) random spatial allocation of the categories. Use of the Kappa indices has become part of the culture in remote sensing and other fields. This article exam- ines five different Kappa indices, some of which were derived by the first author in 2000. We expose the indices' properties mathematically and illustrate their limitations graphically, with emphasis on Kappa's use of randomness as a baseline, and the often-ignored conversion from an observed sample matrix to the estimated population matrix. This article concludes that these Kappa indices are useless, mis- leading and/or flawed for the practical applications in remote sensing that we have seen. After more than a decade of working with these indices, we recommend that the profession abandon the use of Kappa indices for purposes of accuracy assessment and map comparison, and instead summarize the cross-tabulation matrix with two much simpler summary parameters: quantity disagreement and alloca- tion disagreement. This article shows how to compute these two parameters using examples taken from peer-reviewed literature. © 2011 Taylor & Francis.
The available download on this page is the author manuscript accepted for publication. This version has undergone full peer review but has not been through the copyediting, typesetting, pagination and proofreading process.
Publication Title
International Journal of Remote Sensing
Publication Date
1-1-2011
Volume
32
Issue
15
First Page
4407
Last Page
4429
ISSN
0143-1161
DOI
10.1080/01431161.2011.552923
Repository Citation
Pontius, Robert Gilmore and Millones, Marco, "Death to Kappa: Birth of quantity disagreement and allocation disagreement for accuracy assessment" (2011). Geography. 760.
https://commons.clarku.edu/faculty_geography/760
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Copyright
This is an Accepted Manuscript of an article published by Taylor & Francis in International Journal of Remote Sensing on 08/07/2011, available at: https://doi.org/10.1080/01431161.2011.552923