Geography

Recommendations for using the relative operating characteristic (ROC)

Document Type

Article

Abstract

The relative operating characteristic (ROC) is a widely-used method to measure diagnostic signals including predictions of land changes, species distributions, and ecological niches. The ROC measures the degree to which presence for a Boolean variable is associated with high ranks of an index. The ROC curve plots the rate of true positives versus the rate of false positives obtained from the comparison between the Boolean variable and multiple diagnoses derived from thresholds applied to the index. The area under the ROC curve (AUC) is a summary metric, which is commonly reported and frequently criticized. Our manuscript recommends four improvements in the use and interpretation of the ROC curve and its AUC by: (1) highlighting important threshold points on the ROC curve, (2) interpreting the shape of the ROC curve, (3) defining lower and upper bounds for the AUC, and (4) mapping the density of the presence within each bin of the ROC curve. These recommendations encourage scientists to interpret the rich information that the ROC curve can reveal, in a manner that goes far beyond the potentially misleading AUC. We illustrate the benefit of our recommendations by assessing the prediction of land change in a suburban landscape. © 2014 Springer Science+Business Media Dordrecht.

Publication Title

Landscape Ecology

Publication Date

3-1-2014

Volume

29

Issue

3

First Page

367

Last Page

382

ISSN

0921-2973

DOI

10.1007/s10980-013-9984-8

Keywords

accuracy, AUC, index, land change, map, prediction, ROC, threshold, uncertainty

Link to Full Text

Share

COinS