Tools for Monitoring Robust Regression in SAS IML Studio
S, MM, LTS, LMS and Especially the Forward Search
This report focuses on robust regression tools that are at the core of a JRC system for the routine generation and dissemination of EU import prices and the detection of patterns of anti-fraud relevance in large volumes of trade. These tools have been implemented in SAS in the context of a project supported by the Hercule III program of the European Commission. Although the development framework is very specific to anti-fraud, the applicability of the SAS package is much wider and the underlying models (that are by the academic co-authors of the report) are very general.
The forward search (FS) is a general method of robust data fitting that moves smoothly from very robust to maximum likelihood estimation. The regression procedures are already included in a MATLAB toolbox, FSDA, developed by the same authors of this report. The work on a SAS version of the FS originates from the need for the analysis of large data sets expressed by law enforcement services operating in the
European Union that can use our SAS software for detecting data anomalies that may point to fraudulent customs returns. The series of fits provided by the FS leads to the adaptive data-dependent choice of highly efficient robust estimates. It also allows monitoring of residuals and parameter estimates for fits of differing robustness. Our SAS package applies the idea of monitoring to several robust estimators for regression for a range of values of breakdown point or nominal efficiency, leading to adaptive values for these parameters.
Examples in the report are for S estimation and, not included in FSDA, for Least Median of Squares (LMS) and Least Trimmed Squares (LTS) regression.
Specific to our SAS implementation, we describe the approximations used to provide fast analyses of large datasets using a FS with batches. We also present examples of robust transformations of the response in regression. Further, our package provides the SAS community with methods of monitoring robust estimators for multivariate data, including multivariate data transformations.
TORTI Francesca;
PERROTTA Domenico;
ATKINSON Anthony C.;
CORBELLINI Aldo;
RIANI Marco;
2020-09-07
Publications Office of the European Union
JRC121650
978-92-76-21438-0 (online),
1831-9424 (online),
EUR 30341 EN,
OP KJ-NA-30341-EN-N (online),
https://publications.jrc.ec.europa.eu/repository/handle/JRC121650,
10.2760/35922 (online),
Additional supporting files
| File name | Description | File type | |
|---|
