|
NeuroCOLT
Technical Report NC-TR-97-003
Constructing
Bayesian finite mixture models by the EM algorithm
Petri
Kontkanen, Petri Myllymäki, Henry Tirri
University of Helsinki
Finland
Abstract
In this paper we explore the use of finite mixture models for
builing decision support systems capable of sound probabilistic inference.
Finite mixture models have many appealing properties: they are computationally
efficient in the prediction (reasoning) phase, they are universal
in the sense that they can approximate any problem domain distribution,
and they can handle multimodality well. We present a formulation of
the model construction problem in the Bayesian framework for finite
mixture models, and describe how Bayesian inference is performed given
such a model. The model construction problem can be seen as missing
data estimation and we describe a realization of the Expectation-Maximization
(EM) algorithm for finding good models. To prove the feasibility of
our approach, we report crossvalidated empirical results on several
publicly available classification problem datasets, and compare our
results to corresponding results obtained by alternative techniques,
such as neural networks and decision trees. The comparision is based
on the best results reported in the literature on the datasets in
question. It appears that using the theoretically sound Bayesian framework
suggested here the other reported results can be outperformed with
a relatively small effort.
Download Compressed
Postscript
Title
Page
|