Robust neural estimation and diagnostics

Bolland, Peter (1998) Robust neural estimation and diagnostics. Doctoral thesis, University of London: London Business School.

Abstract

Neural networks are powerful -non-parametric statistical estimators that have been successfully applied to many problem domains across a broad range of disciplines. In this-thesis we investigate a number of issues relating to their robustness to outliers, influential observations and leverages. It is shown that although neural networks have been heralded as being robust estimators there is a methodological gap in terms of a rigorous statistical procedure to justify these claims. Neural models are susceptible to outliers, influential observations and leverages in much the same way as conventional parametric models. Ill-conditioned data within the empirical distribution can have a catastrophic impact on the estimated models, significantly distorting -the estimation of parameters; model diagnostics (e. g. variable relevance); and performance metrics (e. g. model fit, generalisation measures). The problems associated with outliers are exacerbated for high-dimensional problems with unknown relationships, which is precisely the, problem domain of neural networks. At present, neural network practitioners have to blindly rely on untested modelling assumptions. Even in circumstances when discrepancies between assumptions and reality are known, the remedies are either not available or inappropriate for the special properties of neural networks whose flexibility introduces new methodological issues. The aim of this thesis is to place neural estimation within a framework of robust statistical inference and to develop the theoretical foundations for the identification and treatment of illconditioned data (outliers and leverage/influential observations). We derive influence functions, leverage metrics, and residual diagnostics for neural networks, which provide an array. of tools for identifying data points which deviate from the assumed distribution, or exert disproportionate influence on the estimated model. We show-how the properties of the estimator can be tailored to offer the desired robustness. characteristics. We design a set of M-estimators and GM-estimators in analogy to those described for linear regression. Whilst the M-estimators are robust to the corrupting influence outliers, GM-estimators also offer protection against leverages and influential observations. For time series applications of neural models we develop robust data filtering methods, based on a non-Gaussian extended Kalman filter. Within the state space framework multivariate non-linear time series are iteratively filtered/cleaned of ill-conditioned data. We analyse and evaluate diagnostic procedures and the robust estimators empirically both under controlled simulation with synthetic data and in the setting of real life problems drawn from ill-conditioned financial data series.