Robust fitting of regression mixtures to contaminated trade data with high density regions

Robust methods are needed to t regression lines when outliers are present. In a clustering framework, outliers can be extreme observations, high leverage points, but also data points which lie between two groups. Outliers are also of paramount importance in the analysis of international trade data, because they may provide information about anomalies like fraudulent transctions. In this work we show that robust techniques can fail when a large proportion of non-contaminated observations fall in a small region, which is a likely occurrence in many international trade data sets. In such instances, the eect of a high-density region is so strong that it can override the benets of trimming or other robust devices. We propose to solve the problem by sampling a much smaller subset of observations which preserves the cluster structure and retains the main outliers of the original data set. This goal is achieved by dening the retention probability of each point as an inverse function of the estimated density function for the whole data set. We motivate our proposal as a thinning operation on a point pattern generated by dierent components. We then apply robust tting methods to the thinned data set for the purposes of classication and outlier detection. We show the advantages of our method both in empirical applications to international trade examples and through an extensive simulation study.

PERROTTA Domenico;  CERIOLI Andrea; 

2014-08-12

SPRINGER HEIDELBERG

JRC77592

1862-5347,   

http://dx.doi.org/10.1007/s11634-013-0151-5,    https://publications.jrc.ec.europa.eu/repository/handle/JRC77592,   

10.1007/s11634-013-0151-5,