Geography
A call to interpret disagreement components during classification assessment
Document Type
Article
Abstract
This foresight manuscript proposes several ideas concerning how to conduct insightful classification assessment. Authors should report disagreement components that relate to the research question, without anointing results as acceptable or good. This manuscript reviews the citations of the 2011 manuscript entitled ‘Death to Kappa: Birth of Quantity Disagreement and Allocation Disagreement for Accuracy Assessment’, which gave two recommendations: (1) do not use Kappa and (2) use disagreement components. We analyzed 200 articles that cited the Death to Kappa manuscript. A quarter of the articles followed both recommendations, another quarter followed only the first recommendation, another quarter followed only the second recommendation, and the last quarter followed neither recommendation. The attempt to replace Kappa with disagreement components has been partially effective, while Kappa continues to haunt several professions. We discuss misguided uses of Percent Correct and Kappa in Remote Sensing and Land Change Modeling. The concepts are general and thus relate to additional fields. Authors frequently use arbitrary thresholds of metrics to claim that results are acceptable. However, the notion that results can be acceptable or not is inherently unscientific. Scientists must use a metric that addresses a clear research question in which the scientists have no vested interest in the results.
Publication Title
International Journal of Geographical Information Science
Publication Date
3-2025
Volume
14
Issue
4
ISSN
1365-8816
DOI
10.1080/13658816.2025.2469830
Keywords
accuracy, error, model
Repository Citation
Pontius, Robert Gilmore; Francis, Thomas; and Millones, Marco, "A call to interpret disagreement components during classification assessment" (2025). Geography. 1003.
https://commons.clarku.edu/faculty_geography/1003