Computer Science

Training ensembles using max-entropy error diversity

Document Type

Conference Paper

Abstract

Ensembles provide a powerful method for improving the performance of automated classifiers by constructing piecewise models that combine individual component classifier hypotheses. Together, the combined output of the component classifiers is more capable of fitting the type of complex decision boundaries in data sets where class boundaries overlap and class exemplars are disperse in feature space. A key ingredient to ensemble classifier induction is error diversity among component classifiers. Work in the ensemble literature suggests that ensemble construction should consider diversity even at some expense to individual classifier performance. To make such tradeoffs, a component classifier inducer requires knowledge of the choices made by its peers in the ensemble. In this work, we present a method called MaxEnt-DiSCO that trains component classifiers collectively using entropy as a measure of error diversity. Using the maximum entropy framework, we share information on instance selection among component classifiers collectively during training. This allows us to train component classifiers collectively so that their errors are maximally diverse. Experiments demonstrate the utility of our approach for data sets where the classes have a moderate degree of overlap. © 2009 American Institute of Physics.

Publication Title

AIP Conference Proceedings

Publication Date

2009

Volume

1193

First Page

202

Last Page

209

ISSN

0094-243X

ISBN

9780735407299

DOI

10.1063/1.3275615

Keywords

classification and classification systems, computer science and technology

APA Citation

Holness, G. F., & Utgoff, P. E. (2009, December). Training Ensembles using Max‐Entropy Error Diversity. In AIP Conference Proceedings (Vol. 1193, No. 1, pp. 202-209). American Institute of Physics.

Link to Full Text

Share

COinS