Iterative estimation of Robust Gaussian mixture models in heterogeneous data sets

Abstract

Density estimation is the process of estimating the parameters of a probability density function from data. The Gaussian mixture model (GMM) is one of the most preferred density families. We study the estimation of a Gaussian mixture from a heterogeneous data set that is de ned as the set of points that contains interesting points that are sampled from a mixture of Gaussians as well as non-Gaussian distributed uninteresting ones. The traditional GMM estimation techniques such as the Expectation-Maximization algorithm cannot e ectively model the interesting points in a heterogeneous data set due to their sensitivity to the uninteresting points as outliers. Another potential problem is that the true number of components should often be known a priori for a good estimation. We propose a GMM estimation algorithm that iteratively estimates the number of interesting points, the number of Gaussians in the mixture, and the actual mixture parameters while being robust to the presence of uninteresting points in heterogeneous data. The procedure is designed so that one Gaussian component is estimated using a robust formulation at each iteration. The number of interesting points that belong to this component is also estimated using a multi-resolution search procedure among a set of candidates. If a hypothesis on the Gaussianity of these points is accepted, the estimated Gaussian is kept as a component in the mixture, the associated points are removed from the data set, and the iterations continue with the remaining points. Otherwise, the estimation process is terminated and the remaining points are labeled as uninteresting. Thus, the stopping criterion helps to identify the true number of components without any additional information. Comparative experiments on synthetic and real-world data sets show that our algorithm can identify the true number of components and can produce a better density estimate in terms of log-likelihood compared to two other algorithms.