Higher-order robustness

(1)

Conference Presentation

Reference

Higher-order robustness

LA VECCHIA, Davide, RONCHETTI, Elvezio, TROJANI, Fabio

Abstract

The higher–order robustness for M–estimators is introduced and defined. The conditions needed to ensure higher stability of the asymptotic bias are provided by refining the Von Mises bias expansion. Admissible M–estimators featuring second–order robustness are thus introduced. Then, a saddle-point argument is applied in order to approximate the finite sample distribution of second–order robust M–estimators. The link between the stability of this approximation and the second–order robustness is explored. Monte Carlo simulation provides evidence that second–order robust M–estimators perform better than the MLE and Huber–type estimators, even in moderate to small sample sizes and/or for large amounts of contamination.

LA VECCHIA, Davide, RONCHETTI, Elvezio, TROJANI, Fabio. Higher-order robustness. In:

2nd International Workshop of the ERCIM Working Group on Computing & Statistics, Limassol (Cyprus), 29-31 October 2009, 2009, p. 31

Available at:

http://archive-ouverte.unige.ch/unige:75154

Disclaimer: layout of this document may differ from the published version.

(2)

Higher-Order Robustness

Davide La Vecchia,

^∗

Elvezio Ronchetti,

^†

and Fabio Trojani

^‡

Abstract

Using the von Mises expansion, we study the higher-order infinitesimal robustness of a general M-functional and characterize its second-order properties. We show that second-order robustnessisequivalenttotheboundednessofboththeestimator’sestimatingfunction andits derivative with respect to the parameter. It implies, at the same time, (i) variance-robustness and (ii) robustness of higher-order saddlepoint approximations to the estimator’s finite sample density. The proposed constructionof second-order robustM-estimators is fairly general and potentially useful in a variety of relevant settings. Besides the theoretical contributions, we discussthemain computationalissuesandprovide analgorithmfortheimplementationof second-order robust M-estimators. Finally, we illustrate our theory by Monte Carlo simulation and in a real-data estimation of the maximal losses of Nikkei 225 index returns. Our findings indicate that second-orderrobustestimatorscanimproveonother widely-applied robustesti- mators, in terms of efficiency and robustness, for moderate to small sample sizes and in the presenceofdeviationsfrom idealparametricmodels.

MSC: 30E15,41A60, 62F10, 62F35,62J05, 62J12

Keywords: Von Mises Expansion, M-estimator, Robustness, Saddlepoint, Generalized extreme valuedistribution.

∗Lecturer in the Dept. of Econometrics and Business Statistics, Monash University, Melbourne, Australia.

†Professor at Research Center for Statistics and Dept. of Economics, University of Geneva, Switzerland, and Faculty of Economics, Universit`a della Svizzera Italiana, Via G. Buffi 13, Lugano, Switzerland.

‡Professor at Universit`a della Svizzera Italiana, Via G. Buffi 13, Lugano, Switzerland and Research Fellow at the Swiss Finance Institute. The authors thank the Editor, the Associate Editor, an anonymous Referee, M. Hallin, F. Peracchi, J. Fan, J. Kolassa, seminar participants at Princeton University, Rutgers University, HEC Paris, Monash University, and at the Winter Meeting of the Econometric Society in Rome, for useful comments which improved an earlier version of the paper. Financial support by the Swiss National Science Foundation Pro*Doc Program 114533 and NCCR-FinRisk is gratefully acknowledged.

(3)

1 INTRODUCTION

Many authors have studied the robustness of estimators and test statistics in a variety of settings, under different forms of deviations from ideal models. Nowadays, the need for a robust statistical approach, one which limits the sensitivity to small and moderate deviations from the theoretical model, is well recognized. Robust statistics can then be viewed as an extension of parametric statistics, taking into account that parametric models are only approximations of reality. Corresponding theories for robust inference have been developed. Tukey (1960), Huber (1964) and Hampel (1968) laid the foundations of modern robust statistics. Book- length expositions are in Huber (1981) (2nd edition by Huber & Ronchetti (2009)), Hampel, Ronchetti, Russeeuw & Stahel (1986), and Maronna, Martin & Yohai (2006).

The first line of research formalizing the robustness problem was Huber’s minimax theory (Huber 1964). In this context, the statistical problem is a game between the Nature, which chooses a distribution in the neighborhood of the model, and the statistician, who chooses a statistical procedure in a given class. The statistician achieves robustness by a minimax procedure that minimizes a loss criterion (e.g., the asymptotic variance) at the least favorable distribution in the neighborhood. A second line of research opened by Hampel has considered local robustness, or infinitesimal robustness, i.e., the impact of moderate distributional deviations from ideal models on a statistical procedure. Here the quantities of interest (for instance the bias or the variance of an estimator) are functionals of the underlying distribution, and their linear approximation is used to study the behavior in a neighborhood of the ideal model. In this setting, a key tool is a functional derivative, the Influence Function (IF), which describes the local stability of the functional. The generality of this approach has enabled authors to develop robust methodologies applicable to a wide variety of models, both in iid (see, e.g., Hampel et al. (1986) for an overview), and time series (see, e.g., Martin & Yohai (1986), K¨unsch (1984), La Vecchia & Trojani (2010)). Using the concept of breakdown point (see Hampel (1968), Donoho & Huber (1983)), a third stream of literature has focused on global robustness features, i.e., the potential impact of large contaminations by outliers on a statistical procedure. In this literature, so-called high-breakdown estimators have been proposed, in order to cope with a large fraction of observations not consistent with the ideal model. An overview of this approach for the linear regression model can be found in Rousseeuw & Leroy (1987) and more extensively in Maronna et al. (2006, Chapter 4-5). Fi- nally, partially more recent developments following a different direction to robust estimation include approaches based on the max-bias curve in Huber (1981) and Martin, Yohai & Zamar

(4)

(1989) (see also section 2 below), the minimum distance approach initiated in Beran (1977), Donoho & Liu (1988) and Lindsay (1994), and the related weighted likelihood method in Markatou, Basu & Lindsay (1997) and Markatou, Basu & Lindsay (1998).

This paper studies the higher-order infinitesimal robustness properties of a statistical functional in neighborhoods of a given reference model P0, extending the approaches in the literature on first-order differentiable functionals. For instance, Hampel (1974) introduces bounded-influence estimators, with a bounded first-order von Mises (von Mises 1947) kernel, the estimator’s IF. In this approach, the first-order von Mises kernel is also the key tool to robustify estimators having an unbounded IF and optimal first-order robust estimators are constructed by down-weighting observations that are too influential with respect to the first-order kernel.

An important motivation of our higher-order robust approach is the fact that the charac- terization of the functional’s behavior by means of the first-order von Mises (von Mises 1947) kernel can rapidly become inaccurate, even for small deviations from the ideal model. A simple example within a linear regression model,y=α+βx+u illustrates this point. In this context, we can consider a bivariate outlier in (x, y) coordinates, for an increasing probability of contamination ε. In FigureA, the continuous line plots the “asymptotic bias” (henceforth called the bias, for the sake of brevity) of the Least Squares estimator, as ε increases. The dashed-dotted curve plots the bias approximation provided by the first-order von Mises kernel, while the dashed line plots the bias according to a second-order approximation. Even in this simple setting, the first-order von Mises kernel produces large approximation errors.

Following the motivation in the above example, we study the higher-order robustness of a general M-functional and we rely on higher-order von Mises (von Mises 1947) expansions to characterize in detail the second-order robustness ofM-functionals. We provide a number of novel findings. First, we show that second-order robustness is equivalent to the boundedness of both the estimating function and its derivative with respect to the parameter. Second, we prove that second-order robustness implies both the robustness of the second-order bias of an estimator (second-order B-robustness) and the robustness of its asymptotic variance functional (V-robustness). This last property is not shared in general by first-order robust functionals; see Hampel et al. (1986). Third, second-order robustness allows one to go beyond bias and variance characterizations, in order to study also procedures for refining finite sample approximations to the density of an estimator (see, e.g., Daniels (1954) and Field & Ronchetti (1990)). We show that second-order robustness implies the robustness of the corresponding saddlepoint density approximation and its relative errors. Fourth, we introduce a new class of

(5)

second-order robust M-estimators, show that their estimating function can be redescending, and provide an algorithm for their implementation. This opens up a wide field of possible applications. Finally, we study the properties of second-order robustM-estimators by Monte Carlo simulation and in a real-data application to the estimation of the tail of maximal losses of Nikkei 225 index returns. Monte Carlo simulations, in the linear regression and the Generalized Extreme Value (GEV) setting, show that our second-order robust M-estimator can improve on optimal first-order robust estimators (i) by allowing for a better control of the damaging effects of outliers and (ii) by producing a lower Mean Squared Error in presence of even moderate contamination. The real-data application shows that second-order robust M-estimators can better control the influence of tail observations that artificially inflate the estimated tail index. This allows for a more robust quantification of tail risk.

The rest of the paper is organized as follows. Section 2 introduces the higher-order von Mises approximation of a statistical functional. Section 3and 4provide the main theoretical results on second-order robust M-functionals. Section 5 provides an algorithm for their implementation and tackles the main computational aspects. Section 6 and 7 apply our second-order robust methodology in Monte Carlo simulations and to real-data. Assumptions and proofs are available online. Figures and Tables are collected in Appendix A.

2 HIGHER-ORDER EXPANSION FOR STATISTICAL FUNCTIONALS

Let M be the family of all probability measures on Z ⊂ R^m and let T : dom(T) → R^p be a statistical functional, defined on dom(T) ⊂ M. For P ∈ M, T(P) can represent any characteristic ofP, e.g., the location, a tail area or a more general quantity of interest. Robust statistics is concerned with the smoothness of T(·) in neighborhoods of a fixed ideal model P0 ∈ M. Intuitively, small deviations from P0 should imply small functional variations. We consider the contamination neighborhood:

U^η(P0) :={Pε,G := (1−ε)P0+εG : 0≤ε < η, η≪1 and G∈ M}, (2.1) and study the behavior of T(P_ε,G) as a function of P_ε,G ∈ Uη(P₀). Ideally, this would require computing T(Pε,G) for each G∈ U^η(P0), a task that is unfeasible in many settings. In some models, the worst case bias over the neighborhood can be studied by means of the max-bias

(6)

curve (see Huber (1981) and Martin et al. (1989)):

maxbias(ε;T, P₀) = sup

G kT(P_ε,G)−T(P₀)k , (2.2) which is available in closed form for some specific models, such as linear regression. However, even (2.2) cannot be computed explicitly in most models. Therefore, one relies on asymptotic expansions in a neighborhood of the ideal modelP0. The first-order robust approach linearizes T(Pε,G) using a first-order von Mises expansion, in which the linear term is the first-order von Mises kernel:

T₁(P_ε,G−P₀) = Z

Z

ϕ₁(z₁;P₀)d(P_ε,G−P₀)(z₁);ϕ₁(z₁;P₀) = ∂

∂ε1

T(P₀+ε₁(δz1−P₀))|ε1=0, where δz₁ is a Dirac mass at z₁ ∈ Z. The first-order kernel ϕ₁(z1;P0) is the IF of T(·) at P0 (see Hampel (1974) and Hampel et al. (1986)) and its boundedness implies a first-order robust statistical functional. Higher-order terms are linked to the higher-order (Gˆateaux) derivatives of T(·); for a formal definition, see von Mises (1947) and Fernholz (2001). The von Mises expansion of T(·) at the k-th order reads:

T(Pε,G) =T(P0)+εT1(Pε,G−P0)+ε²T2(Pε,G−P0)+...+ε^kTk(Pε,G−P0)+O d(Pε,G, P0)^k+1 , (2.3) where T_k(·) is the k-th order derivative of T(·) and it depends on the higher-order kernel ϕ_k, and d(·,·) is a suitable metric on the family of probability distributions over the sample space Z, e.g., Kolmogorov or L´evy metric.

If ε is small, the first-order kernel might provide a good approximation. In general, higher-order kernels can be considered. This analysis can give additional insights about the local robustness properties of T(·). In the worst case scenario over (2.1), the expansion up to the second order provides a better approximation of (2.2) than the standard first-order approximation.

Example 1: Logistic regression. Consider for simplicity a logistic regression model with one covariate: Y|X = x ∼ Bin(1,Λ(xβ)), where Λ(xβ) = exp(xβ)/(1 + exp(xβ)), β ∈ R and x ∈ R. The Maximum Likelihood estimator of β₀ is the M-functional T(·) such that EP0[ψ(Y, X;β0)] = 0 iff T(P0) = β0, where ψ(y, x;β0) := (y − Λ(xβ0))x. It is well-known that the first-order kernel is ϕ1(y1, x1;P0) = M⁻¹(ψ;P0)ψ(y1, x1;β0), where M(ψ;P0) = EP0[Λ(Xβ0) (1−Λ(Xβ0))X²]. After some algebra, we obtain the second-order

(7)

von Mises kernel as:

ϕ2(y1, y2, x1, x2;P0) = ∂²

∂ε1∂ε2

T((1− X2

i=1

εi)P0+ X2

i=1

εiδzi)|^ε1=0,ε2=0

=M⁻¹(y1−Λ(x1β0))x1+M⁻¹(y2−Λ(x2β0))x2

+W M⁻²(y1−Λ(x1β0))x1x2(y2−Λ(x2β0))

−M⁻²Λ(x1β0) (1−Λ(x1β0))x²₁x2(y2−Λ(x2β0))

−M⁻²Λ(x2β0) (1−Λ(x2β0))x²₂x1(y1−Λ(x1β0)), (2.4) where M =M(ψ;P₀) and W =−E_P₀[Λ(Xβ₀) (1−Λ(Xβ₀)) (1−2Λ(Xβ₀))X³]. In contrast to ϕ₁(y₁, x₁;P₀), the dependence on y and x in equation (2.4) is quadratic and cubic. This feature can better capture a nonlinearity of T(Pε,G) in ǫ, which is directly related to the higher order moments of Y and X under the probabilityG.

RemarkA special case arises forPε,G ≡Pn, withPnthe empirical distribution ofz1, . . . ,zn. Then, equation (2.3) for k= 1 is the first-order approximation of T(Pn)−T(P0):

T(P_n)−T(P₀) = 1 n

Xn i=1

ϕ₁(z_i;P₀) +O_p(n⁻¹). (2.5) Condition kϕ₁(z;P0)k< b <∞, for allz ∈Z and some b >0, ensures a bounded sensitivity of the linearized empirical statistical functional to perturbations of z1, . . . ,zn, where k · k denotes the Euclidean norm for a vector and the Frobenius norm for a matrix. Using the theory of U-statistics (see Hoeffding (1948)), the second-order kernel sheds additional light on the behavior of T(Pn). Equation (2.3) for k= 2 gives:

T(Pn)−T(P0) = 1 n

Xn i=1

ϕ₁(zi;P0) + 1 2n²

Xn i=1

Xn l=1

ϕ₂(zi,zl;P0) +Op(n^−3/2). (2.6) Thus, the joint analysis of ϕ₁ and ϕ₂ can provide a more accurate description of the finite sample behavior of T(Pn); cf. also Mallows (1975).

Higher-order von Mises expansions like (2.3) can be applied to several statistical functionals. Among these, M-functionals play a crucial role in statistics and econometrics. Their behavior in terms of the first-order kernel approximation has been analyzed in great detail in the literature; see, among others, Hampel et al. (1986). However, the example in the Introduction and Example 1 suggest that additional information is gained by an analysis involving ϕ₂(z1,z2;P0).

(8)

3 SECOND-ORDER ROBUST M -FUNCTIONALS

3.1 Definition

Given an estimating function ψ : Z ×R^p → R^p, the M-functional T(·) is implicitly defined by the unique root of the moment conditions: EP[ψ(Z;T(P))] = 0. Given observations z1,z2, ...,zn, an M-estimator is implicit solution of the finite sample equations Pn

i=1ψ(zi;θˆn) = 0, where θˆn :=T(Pn). When the function ψ satisfies the assumptions B1- B4 and N1-N4 in Huber (1981), implying that ψ is continuous, θˆ_n is root-n consistent and asymptotically normal. Weaker conditions for asymptotic normality have been introduced by He & Shao (1996). The assumptions in Huber (1981) are reproduced for completeness in the online Appendix.

M-functionals provide a convenient setting for the computation of first- and second- order kernels. The first-order kernel is proportional to the estimating function: ϕ₁(z;θ0) = M⁻¹(ψ;θ₀)ψ(z;θ₀), where M(ψ;θ₀) := E_P₀[−∇_θ′ψ(Z;θ₀)] and θ₀ := T(P₀). Thus, B- robust M-functionals have a bounded estimating function. Using results in Appendix A of Gatto & Ronchetti (1996), the second-order kernel is:

ϕ₂(z1,z₂;θ₀) = ϕ₁(z1;θ₀) +ϕ₁(z2;θ₀) +M⁻¹(ψ;θ₀)γ(z1,z₂;θ₀)

+M⁻¹(ψ;θ₀) [∇_θ′ψ(z2;θ₀)ϕ₁(z1;θ₀) +∇_θ′ψ(z1;θ₀)ϕ₁(z2;θ₀)](3.1), where

γ(z1,z2;θ0)^′ =







ϕ^′₁(z2;θ0)EP0

h ∂²

∂θ∂θ^′ψ⁽¹⁾(Z;θ0)i

ϕ₁(z1;θ0) ...

ϕ^′₁(z2;θ₀)EP0

h ∂²

∂θ∂θ^′ψ^(p)(Z;θ₀)i

ϕ₁(z1;θ₀)





 , (3.2)

with ψ^(j) the j-th component of ψ.

Remark. If the estimating function ψ is continuous, ∇_θ′ψ and the derivatives in (3.2) are classical derivatives P0-a.s. Under the weaker assumption that ψ is only P0-a.s. continuous, then ∇_θ′ψ and ∂²ψ^(j)/∂θ∂θ^′, for j = 1, ..., p, must be interpreted as distributional derivatives; see Example 2.

Higher-order kernels can be computed with the recursive formula in Withers (1983). Note that ϕ₂(z1,z2;θ0) in equation (3.1) depends onψ(z;θ0) and its first derivative with respect toθ. More generally, thek−th order kernelϕ_k(z1, ...,zk;θ0) depends on the estimating function and its derivatives up to order k−1. Consequently, a bounded (continuous) estimating function ψ(z;θ₀) with bounded (continuous) derivatives up to order (k −1) ensures that

(9)

the k-th order von Mises approximation (2.3) of functional T(P_ǫ,G) is a bounded function of G ∈ M. We call these functionals k-th order B-robust M-functionals. For brevity, we also denote byBias⁽²⁾(ε;ψ, G;θ₀) the second-order von Mises approximation ofT(Pǫ,G)−T(P0), implied by von Mises expansion (2.3) fork = 2.

It follows that second-order B-robust M-functionals are characterized by a bounded second-order von Mises expansion (2.3), i.e., a bounded (continuous) estimating function with bounded derivative. We summarize this finding in the next proposition.

Proposition 1 Let T(·) be a Fisher consistent M-functional defined by a continuous esti- mating function ψ :Z×R^p →R^p satisfying the assumptions in the online Appendix and the two following conditions:

(i) sup

z∈Zkψ(z;θ₀)k<∞ ; (ii) sup

z∈Zk∇_θ′ψ(z;θ₀)k<∞. (3.3) Then, Bias⁽²⁾(ε;ψ,·,θ₀) is a bounded function of G∈ M.

Remark If ψ is P0-a.s. continuous, then some care is needed in Proposition 1. Let D(ψ) denote the set of discontinuity points ofψ. Then, ϕ₂(z¹, z²;θ0)has an irregular part consisting of delta functions, and Bias⁽²⁾(ε;ψ,·,θ0) is unbounded for contaminations with positive mass on D(ψ). In these cases, a smooth version of ψ can be defined to ensure the continuity. In Example 2 and 6, we introduce such regularizations for specific models.

The class of estimating functions in Proposition 1 defines the second-order robust M- estimators. By definition, second-order B-robustness implies first-order B-robustness. How- ever, since ϕ₂(z₁, z₂;θ⁰) depends on ϕ₁(z₁;θ⁰), it is also interesting to study when first- order robustness implies second-order robustness.

Example 2: Univariate location model Let Z ∼ φ(z−θ0), where φ is the standard normal density. Then, the log-likelihood score is s(z;θ0) =z−θ0 and the Maximum Likeli- hood M-functional (the arithmetic mean) has an unbounded first-order kernel. First-order B-robustness is ensured by a M-functional with a bounded estimating function satisfying the assumptions in the online Appendix, implying a bounded first-order von Mises kernel ϕ₁(z₁;θ₀). The properties of the second-order kernel depend on the continuity of ψ(z−θ₀).

Let D(ψ) be the finite set of the points in Z where the ψ is discontinuous. Then, the derivative ψ^′ :=∂θψ is a Schwartz distribution (see, e.g., Hampel et al. (1986, p. 127)):

ψ^′ =ψ^′I_Z\D(ψ)+ X

c∈D(ψ)

(ψ(c+)−ψ(c−))∆_c, (3.4)

(10)

which is the sum of a regular part and a linear combination of delta functions ∆_c, with

∆_c =∞ if z =c and zero otherwise. Then, formula (3.1) reads:

ϕ2(z1, z2;θ0) = 1−M⁻¹(ψb;θ0)ψ^′(z2−θ0)

ϕ1(z1;θ0)+ 1−M⁻¹(ψb;θ0)ψ^′(z1−θ0)

ϕ1(z2;θ0).

If ψ is a continuous bounded function (e.g., Tukey’s biweight, Andrew’s sine and Huber- estimating functions), kernel ϕ2 is regular and the first-order B-robust location estimator is also second-order B-robust. If ψ is only P0-a.s. continuous (e.g., in the case of the skipped mean), ϕ2 contains irregular parts and contaminations with non zero mass on D(ψ) imply an infinite Bias⁽²⁾(ε;ψ,·, θ0), even when ψ is bounded. This problem can be avoided using a regularized estimating function. Precisely, given an estimating function ψ discontinuous on a set of zero P0-measure, a regularized estimating function ˜ψ can be obtained, e.g., by the convolution with a symmetric kernel, as illustrated in Hampel, Henning & Ronchetti (2011):

ψ(x) =˜ R

Zψ(x+u)dQ(u), where Q(u) is N(0, V) and V is the smoothing parameter.

Beyond location functionals, there is a wide class of first-order robust M-estimators that are not second-order robust. To explore the link between first and second-order robustness more broadly, we start from the standard construction of optimal first-order robust M- estimators; see, e.g., Hampel et al. (1986). Given an unbounded estimating functionψ(z;θ₀) such that EP0[ψ(Z;θ₀)] = 0, an optimal first-order B-robust M-estimator is defined by the following estimating function:

ψ_b(z;θ₀):=A(θ₀)[ψ(z;θ₀)−τ(θ₀)] min(1;b/kA(θ₀)[ψ(z;θ₀)−τ(θ₀)]k) (3.5) where the matrix A(θ0) is such that either

EP0[−∇_θ_′ψ_b(Z;θ₀)] =I (unstandardizedcase) (3.6) or

EP0[ψ_b(Z;θ₀)ψ^′_b(Z;θ₀)] =I (self −standardizedcase) (3.7) and vectorτ(θ₀)satisfies: EP0[ψ_b(Z;θ₀)] = 0. In order to study the second-order robustness properties, we consider the derivative ∇_θ′ψ_b(z;θ₀), which is piecewise given by:

∇_θ_′ψ_b(z;θ0) =

( Dθ^′(A, v)+A[∇_θ′ψ−∇_θ′τ] for kvk ≤b bkvk⁻¹ I −vv^′kvk⁻²

(Dθ^′(A, v)+A[∇_θ_′ψ−∇_θ_′τ]) for kvk> b , (3.8) where v := A(θ₀)[ψ(z;θ₀)−τ(θ₀)] and D_θ^′(A, v) is a p×p matrix with (i, l)-th component given by Pp

j=1

∂Aij

∂θl (ψ^(j)−τ^(j)); see also Lˆo & Ronchetti (2012) for an application

(11)

in the context of empirical likelihood-type estimators. Therefore, ∇_θ_′ψ_b(z;θ₀) is bounded

provided that: (

k∇_θ_′ψk is bounded for kvk ≤b

k∇_θ_′ψk kvk⁻¹ is bounded for kvk> b . (3.9) The following examples illustrate condition (3.9) for some well-known M-functionals.

Example 3: Linear regression and GLMLety=x^′β+u, whereu∼N(0,1) andx∈R^p. Forz = (y,x) the maximum likelihood score iss(y,x;β) = (y−x^′β)x. The (optimal) first- order B-robust estimating function is (3.5) with ψ(z;θ):=s(y,x;β), where τ = 0 and A is constant. It is easy to see that condition (3.9) is violated, since ∇_β_′s(y,x;β) = −xx^′. Thus, this estimator is not second-order robust. For instance, for a sequence (y_n,x_n) such that kx_nk → ∞ and kAs(yn,x_n;β)k is bounded from below by b and from above by a second constant, we find thatk∇_β_′s(yn,xn;β)k/ks(yn,xn;β)k=O(kxnk²), which violates the second requirement in condition (3.9).

Example 4: Generalized Extreme Value (GEV) distribution The GEV family is parameterized by a parameter θ₀ = (ξ₀, µ₀, σ₀)^′ and densities of the form:

p0(z;ξ0, µ0, σ0) = 1 σ0

exp

"

−

1 +ξ0

z−µ0

σ0

−_ξ¹

0#

1 +ξ0

z−µ0

σ0

−_ξ¹

0−1

. (3.10) For ξ → 0, ξ > 0 and ξ < 0 equation (3.10) corresponds, respectively, to the Gumbel, Fr´echet and Weibull family. In this model, the log-likelihood score s(z;θ) is unbounded and the Maximum Likelihood estimator is not robust. An optimal first-order B-robust estimator of form (3.5) was proposed in Dupuis (1997) and Dupuis & Field (1998) forψ(z;θ) = s(z;θ).

Analytical calculations show that the second requirement in condition (3.9) is violated, so that this estimator is not second-order robust. This finding can be illustrated numerically for the parameter choice µ0 = 0, σ0 = 1 and ξ0 =−0.1. The function k∇_θ′sk is continuous over the support of GEV(ξ0, µ0, σ0). Thus, it has a finite maximum on the compact set

|z| ≤ 9 (namely when ksk ≤2500). Therefore, the first requirement of (3.9) is satisfied. In contrast, the function k∇_θ_′sk ksk⁻¹ is unbounded, when |z| > 9 (namely for ksk > 2500), which violates the second requirement of condition (3.9).

(12)

3.2 Main properties

We derive additional features of second-order robustM-estimators, which are directly related to their mean square error and finite sample robustness properties. The optimalM-estimators derived in Section 4 will have these robustness features. For brevity, we assume that the function ψ satisfies the assumptions stated in the online Appendix.

3.2.1 B-robustness and V-robustness

In contrast to first-order robust estimators, second-order robust estimators also imply the robustness of their asymptotic variance functional. Let this variance functional beV(ψ;Pε,G) = EPε,G[ϕ₁(Z;T(Pε,G))ϕ^′₁(Z;T(Pε,G))] and consider (i) the Change of Variance Function as given by CVF(z;ψ, P0) =∂V(ψ;Pε,δz)/∂ε|^ε=0 :=∂εV(ψ;Pε,δz),as well as (ii) the standardized Change of Variance Sensitivity:

κ^∗(ψ;P0) := sup

z∈Z

tr CVF(z;ψ, P0)/trV(ψ;P0). (3.11) where tr denotes the trace of a matrix. Expanding log trV(ψ;Pε,G) atε= 0 and taking sup_G over (2.1), we get

sup

G

trV(ψ;P_ε,G)∼tr[V(ψ;P₀)] exp[εκ^∗(ψ;P₀)], (3.12) An M-functional such that κ^∗(ψ;P0)<∞ is called V-robust, i.e. the (asymptotic) variance functional of the estimator is robust. V-robust M-functionals are first-order B-robust, but in general the converse implication does not hold; see again Hampel et al. (1986). Instead, as stated by the next proposition, second-order robustness is a sufficient condition for V- robustness. Therefore, first-order B-robust M-functionals are V-robust if an only if the derivative of their estimating function is bounded.

Proposition 2 A second-order B-robust M-functional is V-robust.

3.2.2 Robust saddlepoint density approximation

Bias and variance provide important information the the first-order asymptotic (normal) distribution ofM−estimators. Very accurate approximations of the finite sample distribution of an estimator can be obtained by means of saddlepoint-type expansions; see Field & Ronchetti (1990), among others. When the data are generated from a distribution Pε,G ∈ U^η(P0) satisfying the Assumptions in Field & Ronchetti (1990, page 62), the exact finite sample density

(13)

f(t;n, ε, G) of theM-functional can be approximated by a saddlepoint expansion as follows:

f(t;n, ε, G) =g(t;n, ε, G){1 +a₁(t;P_ε,G)n⁻¹+O(n⁻²)}, (3.13) where

g(t;n, ε, G) = (n/2π)^p/2c⁻ⁿ(α(t;Pε,G);Pε,G)

detM˜(t;Pε,G)

detQ(t;˜ Pε,G)

−1/2

, (3.14) M˜(t;P_ε,G) = E_H_ε,G;t[−∇_t′ψ(Z;t)] and Q(t;˜ P_ε,G) = E_H_ε,G;t[ψ(Z;t)ψ^′(Z;t)]. In this approximation, a1(t;Pε,G) depends, among other things, on the standardized third and fourth cumulants of ψunder Pε,G. The vector α(t;Pε,G) is the solution of the saddlepoint equation EHε,G;t[ψ(Z;t)] = 0, with Hε,G;t(·) the conjugate measure of Pε,G, defined by the density:

dHε,G;t/dPε,G(z) =c(α(t;Pε,G);Pε,G) exp{α^′(t;Pε,G)ψ(z;t)}, where c(α(t;Pε,G);Pε,G)⁻¹=EPε,G[exp{α^′(t;Pε,G)ψ(Z;t)}].

For a given distributionPε,G, the leading term g(t;n, ε, G) in equation (3.14) provides a relative approximation error of order O(n⁻¹). We are interested in the robustness properties of functionals g(t;n, ε, G) and a1(t;Pε,G), and in features of anM-estimator that guarantee robustness of these functionals.

Consider the following von Mises expansion of g(t;n, ε, G) (and similarly for a1(t;Pε,G)):

g(t;n, ε, G) =g(t;n,0, P0) +ε∂εg(t;n, ε, G) +O(ε²),where ∂εg(t;n, ε, G) is the derivative of g(t;n, ε, G) evaluated at ε = 0. ∂εg(t;n, ε, G)/g(t;n,0, P0) is a standardized sensitivity of the saddlepoint approximation g(t;n, ε, G) to ε contaminations of the reference modelP0 in direction G∈ M. These sensitivities are bounded for second-order robust M-estimators.

Proposition 3 Let ψ(z;θ₀) be an estimating function satisfying the conditions in (3.3), then (i) sup_G|∂_εg(t;n, ε, G)|<∞ and (ii) sup_G|∂_εa₁(t;P_ε,G)|<∞.

Proposition3directly implies that the saddlepoint approximation of second order robust M-estimators, as well as its relativeO(n⁻¹)-error, are bounded over the neighborhoodU^η(P0).

This property does not hold in general for first-order robust M-functionals.

Remark Ronchetti & Ventura (2001) analyze the effects of a model misspecification on the accuracy of saddlepoint density approximations for univariate estimators of location. They find that small deviations from the parametric model can easily wipe out the improvements in finite sample accuracy, obtained by the saddlepoint method, for classical estimators like

(14)

the arithmetic mean. In contrast, the saddlepoint approximations implied by Huber-type M-estimator of location provide a robust second-order accuracy. According to Example 2, this robust location estimator is second-order robust. Therefore, Proposition 3 provides a theoretical justification for these findings.

4 CONSTRUCTION OF SECOND-ORDER ROBUST M -FUNCTIONALS

4.1 Optimal M -functionals

Let Ψ be the class of functionsψ(z;θ) :Z×R^p →R^psatisfying the Assumptions in the online Appendix and such that EP0[ψ(z;θ0))] = 0. An optimal second-order robust M-estimator is an estimator with an estimating function ψ^∗ ∈ Ψ that minimizes trV(ψ;θ0), subject to a constraint on the estimating function and its derivative with respect to θ (see Proposition 1). Thus, an optimal second-order M-functional solves the minimization problem:

minψ∈ΨtrV(ψ;θ₀), s.t. sup

z∈Zkψ(z;θ₀)k ≤b and sup

z∈Zk∇_θ′ψ^(j)(z;θ₀)k ≤c^(j), (4.1) for given positive constant b and c⁽¹⁾, ..., c^(p), possibly dependent onθ.

Remark If c^(j) → ∞ for each j, problem (4.1) becomes the standard first-order robust optimization problem, with one robustness constraint on the Euclidean norm of the estimator IF. The solution to this problem is the well-known estimator implied by: ψ^∗ = A(s− τ) min(1;b/kA(s−τ)k), with Asuch that EP0[−∇_θ_′ψ^∗] =I. In order to derive an explicit optimal solution to (4.1) for a well-known benchmark, we study in more detail the one- dimensional location problem.

Proposition 4 Let Z ∼ φ(z −θ0), where φ is the standard normal density, Z ⊂ R and θ0 ∈Θ⊂R. The likelihood score function is s(z;θ0) :=∂θlogφ(z−θ0) = z−θ0. Forb∈R⁺, define

ψ˜b,c(z−θ0) =





c(z−θ0) min

1;_|Ac(z−θ^b ₀_)|

0< c <1 (z−θ0) min

1;_|A(z−θ^b

0)|

c≥1. (4.2)

The M-estimator implied by the estimating function ψb,c(z−θ0) =Aψ˜b,c(z−θ0), where A is such thatEP0[−∂θψb,c]=1, is second-order robust and minimizesV(ψ;θ0)among all functions ψ ∈Ψ, such that both |ψ(z;θ0)| ≤b and |∂θψ(z;θ0)| ≤Ac, for all z ∈ Z.

(15)

Since in the location problem the estimating function has the form ψ(z−θ₀), it follows that |∂_zψ(z −θ₀)| = |∂_θψ(z−θ₀)|. Eq. (4.2) points out that the constraint on the second- order kernel is binding for c∈(0,1). In this case, Eq. (4.2) defines a new estimator having a tighter bound on the derivative. This bound implies a stronger control for the impact that contaminations around the symmetry center θ0 can have on the second-order von Mises kernel. For c≥ 1, since sup_z|∂θψ˜b,c(z −θ0)| = 1, the solution to the minimization problem (4.1) is the (unstandardized) first-order robust M-functional of Huber-type (see Eq. (3.6)).

For general estimation problems, problem (4.1) is difficult to solve in closed form. There- fore, we propose a class of second-order robust estimators that are more broadly applicable.

4.2 M -functionals for general settings

Given an (unbounded) estimating function ψ defining a Fisher consistent M-functional, we assume the existence of two functions hi : Z → R⁺, i = 1,2, such that (i)k∇_θ_′ψkkψk⁻¹ = O(h₁) and (ii) k∇_θ′ψk = O(h₂) and denote by g := max(h₁, h₂) the maximum of these functions. An estimating function implying bounded first and second-order von Mises kernels can be then constructed according to the next proposition.

Proposition 5 Given positive constants b and c, define the bounded estimating function:

ψ_b,c :=A(ψ−τ) min

1; b

kA(ψ−τ)k

min

1;c g

, (4.3)

where the matrix A and the vector τ are solutions of the equations:

EP0

ψ_b,c

= 0 , (4.4)

EP0

−∇_θ_′ψ_b,c

= I. (4.5)

Then, the function ψ_b,c satisfies condition (3.3).

Remark In the location setting, the M-estimator defined by ψ_b,c coincides with the optimal estimator defined in Proposition 4, e.g., for: c < 1,g ≡1, and b/A in (4.3) equal to b/(Ac) in (4.2). See, also Example 6 below for a graphical illustration.

By construction, as b, c → ∞ the estimating function ψ_b,c in Eq. (4.3) converges to the estimating function ψ. For fixed b and as c → ∞ the bound on the second-order kernel is relaxed and the function ψ_b,c converges to ψ_b, i.e., the estimating function of the standard first-order B-robust M-estimator. The matrix A depends on the norm used to measure the bound on the first order kernel ϕ₁(z1;P0). While equation (4.5) implies a bound defined by

(16)

the Euclidean norm, other standardizations are possible. For instance, a bound defined by a self-standardized metric is obtained by a condition of the form (see also equation (3.7) and Section 4.3 in Hampel et al. (1986)):

EP0

ψ_b,cψ^′_b,c

=I . (4.6)

Overall, Proposition 1 and 5 give rise to a new class of second-order robust M-estimators.

Corollary 6 The M-estimator θˆn defined by the solution to Pn

i=1ψ_b,c(zi;θˆn) = 0, where ψ_b,c is defined by equation (4.3), is second-order robust.

Example 6: Second-order robustness in location models In the setting of Exam- ple 2, the second-order robust estimator of Proposition 5 can impose a tighter bound on the second-order von Mises kernel, in dependence of function g and tuning constant c. A regularized Huber estimator can be also implemented, to eliminate possible points of non- differentiability, while controlling the bound on the first derivative. For illustration, Figure 2 compares the estimating function of the following M-estimators: the standard first-order robust M-estimator (dashed line) and two second-order robust M-estimators, obtained for g(z) = z², b = 2.5, c= 6.5 (continuous line) and for g(z) = 1, b = 4.545 and c= 0.55 (dot- dashed line). We also plot the case of a regularized Huber M-estimator ˜ψ, with V = 2.046, as proposed in Hampel et al. (2011) (dotted line). For g(z) ≡ c, our second-order robust M-estimator coincides with the standard first-order robust M-estimator. For c = 0.85, we obtain an estimating function that behaves like the Huber estimating function in the extreme regions of the state space, but with a smaller first derivative over the compact set [−b, b].

Finally, a polynomial function g(z) yields a redescending estimating function.

Example 7: Second-order robustness in linear regression models Consider again Example 3 with ψ(z;θ) = s(y,x;β) = (y−x^′β)x and τ = 0. Condition (3.9) is violated for kvk > b when kxk → ∞ and the residual (y −x^′β) is bounded. The largest speed at which the ratio k∇_β′s(y,x;β)kkvk⁻¹ can diverge as kxk → ∞ is kxk². This suggests that we can set h₁(x) = kxk². Similarly, condition (3.9) can be violated for kvk ≤ b when kxk → ∞ and the likelihood score (y−x^′β)x is bounded. In this case, the speed of divergence of k∇_β′s(y,x;β)k is kxk². This suggests that we can set h2(x) = kxk² as well, i.e., g(z) = kxk². The resulting second-order robust M-estimator (4.3) has an estimating function given by:

ψ_b,c(y,x;β) =Axumin

1; b

|u|kAxk

min

1; c kxk²

, (4.7)

(17)

where u = y−x^′β and matrix A is determined either by equation (3.6) or equation (3.7).

Under the assumption of a symmetric distribution of the error u at the ideal model, one can set τ ≡ 0 in equation (4.3). This feature simplifies the implementation of the estimator. Estimating function (4.7) corresponds to the estimating function of an optimal B-robust Hampel-Krasker (unstandardized) or Krasker-Welsch (self-standardized) estimator, with ad- ditionalMallows-type weights on thexvariables. Therefore, largexobservations are typically down-weighted more than in the optimal first-order B-robust estimator.

5 IMPLEMENTATION

5.1 Algorithm

Robust M-estimator (4.3) is computed with an iterative algorithm. We present the self- standardized version implied by equation (3.7). Given constant b ≥ √p (see Hampel et al.

(1986), p. 228), set ωci:= min (1;c/gi), gi =g(zi) and perform the following steps:

1. Set initial values θ⁽⁰⁾ and τ⁽⁰⁾ =0; Given (unbounded) estimating functionψ, set:

A⁽⁰⁾A⁽⁰⁾^′−1

= 1 n

Xn i=1

ψ⁽⁰⁾_i ψ^(0)′_i

where, for brevity, ψ⁽⁰⁾_i :=ψ

z_i;θ⁽⁰⁾

. Moreover, let:

ω_bi⁽⁰⁾ := min



1; b

A⁽⁰⁾

ψ⁽⁰⁾_i −τ⁽⁰⁾



.

2. Computeτ⁽¹⁾ as:

τ⁽¹⁾ = E_θ(0)

ψ⁽⁰⁾_i ω_bi⁽⁰⁾ω_ci E_θ⁽⁰⁾

ω_bi⁽⁰⁾ωci

(5.1)

and A⁽¹⁾ as

A⁽¹⁾A⁽¹⁾^′−1

= 1 n

Xn i=1

(ω_bi⁽⁰⁾ωci)²

ψ⁽⁰⁾_i −τ⁽⁰⁾ ψ⁽⁰⁾_i −τ⁽⁰⁾^′ .

3. Givenτ⁽¹⁾andA⁽¹⁾, compute the parameterθ⁽¹⁾as the solution of the implicit equation:

0= Xn

i=1

A⁽¹⁾ ψ

z_i;θ⁽¹⁾

−τ⁽¹⁾ min



1; b

A⁽¹⁾

ψ⁽⁰⁾_i −τ⁽¹⁾



wci (5.2)

(18)

Given θ⁽¹⁾, set ψ⁽¹⁾_i :=ψ(zi;θ⁽¹⁾) and:

ω⁽¹⁾_bi := min



1; b

A⁽¹⁾

ψ⁽¹⁾_i −τ⁽¹⁾



 .

4. Go back to Step 2 and replace ω⁽⁰⁾_bi by ω_bi⁽¹⁾,ψ⁽⁰⁾_i by ψ⁽¹⁾_i and τ⁽⁰⁾ by τ⁽¹⁾. Then iterate Steps 2. and 3. until convergence of the sequences {θ^(j)},{A^(j)}and {τ^(j)}.

Remark For complex models, the algorithm can be demanding, because the weights in (4.3), the matrixA(·) and the vectorτ(·) depend nonlinearly on the estimator itself. In Step 1., selecting the initial value θ⁽⁰⁾ with a high breakdown-point estimator or another robust M-estimator is a wise choice to improve the convergence. Alternatively, the re-sampling techniques in Markatou et al. (1998, section 5.1) can be applied. In Step 2., τ^(j) is equation (5.1) is the ratio of two expectations. Often, these integrals are not available in closed-form and Monte Carlo methods or Gaussian quadrature techniques can be applied. In symmetric models (e.g., location or linear regression settings),τ^(j) is zero for everyj and the implementation is simpler. When in Step 3. equation (5.2) has multiple solutions, a selection criterion is needed. Our advice is to take the solution closest to the initial estimate in Step 1.

5.2 Detailed computational aspects

Specification of function g in (4.3): In many applications, the likelihood score (s) is available in closed-form and some guidelines for selecting g are easily settled. (i) specify ψ = s in (4.3); (ii) compute functions h1 and h2 by means of k∇_θ′skksk⁻¹ = O(h1) and k∇_θ′sk=O(h2); (iii) setg := max(h1, h2). If the likelihood score is not analytically available, other characteristics (e.g., the central moments) can be exploited to define an (unbounded) estimating function ψ in (i). Our advice is to choose ψ as the estimating function implied by a Pseudo Maximum Likelihood method or the Method of Moments. Specifying h1 and h2

as polynomials of sufficient degree (or as exponentials) is often a wise choice yielding second- order weights (min(1;c/g)) that strongly control the impact of influential observations. Then, the impact of g on ψ_b,c can be tuned by c; see, e.g., Examples 6 and 7 above.

Selection of the tuning constants: A first selection criterion can be based on the ratio of the MSE of the most efficient benchmark (i.e., the MLE) and the MSE of the second- order B-robust M-functional, both evaluated under P0. This ratio gives information about the cost (in terms of efficiency loss) that is paid for robustness. Defining rMSE(ψ_b,c;θ₀) =

(19)

MSE(s;θ₀)/MSE(ψ_b,c;θ₀), on can select (b, c) to achieve a pre-specified level of efficiency loss. This rMSE criterion is easy to implement and relies only on computations at the reference model P0. Alternatively, one can study the MSE over contamination neighborhoods.

Higher-order expansions for the MSE of location M-estimators on shrinking neighborhoods are provided by Ruckdeschel (2011). Ferrari & La Vecchia (2012) provide approximations for the maximal MSE (MMSE) under contamination, based on maximal bias and maximal variance approximations over contamination neighborhoods. Minimizing the MMSE then automatically takes into account the interplay of bias and variance. In practice, the criterion depends on the information available. If no knowledge about the size of the contamination neighborhood is available, then rMSE is the natural candidate. Otherwise, the MMSE criterion can be a viable alternative.

Multiple minima: In settings where ψ_b,c is redescending, multiple solutions to the estimating equations can arise. This issue can be tackled, e.g, by a standard one-step approach. Under mild regularity conditions, a one-step approximation is asymptotically equivalent to iterating the algorithm in Section 5.1 until convergence; see, e.g., van der Vaart Van der Vaart (1998, p. 73). Thus, the one-step method can be applied in cases where the iteration of the algorithm in Section 5 is problematic. The implementation of this method builds on a preliminary root-n consistent M-estimator, say eθn. Then, θˆn = θen− Pn

i=1∇_θ′ψ_b,c(z_i,eθ_n)−1Pn

i=1ψ_b,c(z_i,θe_n). θe_n can be a robust M-estimator available in common statistical packages, such as, e.g., the MM-estimator in Salibian-Barrera & Yohai (2006) for the linear regression setting (4.7). One-step methods have been successfully exploited in linear models by Stefanski, Carroll & Ruppert (1986) and Agostinelli & Markatou (1998), among others.

6 MONTE CARLO EVIDENCE

We study by Monte Carlo simulation the finite sample behavior of our second-order robust M-estimator in two model settings. For both examples, we have written a code in MATLAB^TM and run it on a Windows machine with CPU Intel Xeon X7460 @ 2.66GHz.

6.1 Linear regression model

Consider the linear regression model y = α +x^′β +u, where u ∼ N(0,1), α = 0.5 and β = (3,1)^′. We simulate both clean and contaminated samples. Clean samples are generated

(20)

according to the regression model, with x ∼ N(0,I) where I is the 2×2 identity matrix.

Contaminated samples feature contaminations in Y- and X-space. Y-contaminations are generated by a contaminated error in the response variable ˜u = Hyu+ (1 −Hy)ζu, where Hy is a Bernoulli random variable such that P(Hy = 0) = εy, ζu ∼ δu, and δu is a Dirac mass at −2.5. For each j-th component x^(j) of vector x, X-contaminations are simulated as x˜^(j) = Hxx^(j) + (1−Hx)ζx, where ζx ∼ δq, with δq a Dirac mass at 2.5, and Hx is a Bernoulli random variable such that P(H_x = 0) = ε_x,y. To each contaminated regressor realization, we associate a response variable with value equal to −2.5. In this way, we simulate with probability εx,y contaminations in (X, Y)-space modeling bad leverage points.

In summary, our simulation covers different potential data structures: (i) bad leverage points with large regression residuals (points with εx,y-contamination), (ii) contaminated response variables and good regressors (points with ǫy-contamination), (iii) good response variables and good regressors (points with no contamination). We simulate under increasing amounts of contamination: (a) ε_y =ε_x,y = 0%, (b) ε_y =ε_x,y = 5%, (c) ε_y =ε_x,y = 7.5%. We consider Maximum Likelihood, Mallows-type, first-order and second-order robust M-estimators, as well as the highly-efficient/high breakdown-point MM-estimator in Salibian-Barrera & Yohai (2006). The first-order robust M-estimator is the Hampel-Krasker estimator with b = 4, while the second-order robust M-estimator is the estimator (4.7), with A(θ) as in Eq. (3.6) and tuning constants b = 4 and c = 2.95. The Mallows-type estimator is defined in Eq.

(6.3.14)-(6.3.16) of Hampel et al. (1986), withb= 4 andc= 2.95 and the matrixB estimated by the MCD estimator of scatter, while the MM-estimator used in our simulations implies a breakdown point of 30%. For all infinitesimally robust estimators, we determine tuning constants that ensure a good performance in the presence of contamination, while preserving a reasonable Asymptotic Relative Efficiency (ARE) at the ideal model. The Monte Carlo size is 1000 and we consider sample sizes n = 90,250,550. Table 1 reports the MSE of the different estimators for clean and contaminated samples. Results for n= 30 are similar and are not shown.

The MLE has typically very high MSE’s except under ideal conditions. The second-order robust M-estimator systematically improves over the first-order robust estimator, in some cases by a large extent, in presence of a contamination. In Table 2, an analysis of the robust weights of first- and second-order robust M-estimators clarifies their different MSE’s properties. Overall, all observations downweighted by the first-order robust M-estimator are also downweighted by the second-order robustM-estimator, but with a smaller weight. Given the downweighting ofX-observations, the Mallows-typeM-estimator has a similar MSE behavior

(21)

as the second order robust estimator, but with larger MSE’s, especially for contaminations of the Y variable alone and for sample sizes n = 90,250. Finally, the MM-estimator is slightly more efficient than the second-order robust estimator at the ideal model and in presence of (X, Y)-contaminations, but it implies a larger MSE for simple Y-contaminations.

6.2 Generalized extreme value distribution

Setting and estimation results. We estimate the parameters of a GEV distribution, where clean samples are generated according toz ∼GEV(ξ0, µ0, σ0), withξ0 =−0.1, µ0 = 0, σ0 = 1, as in Example 4. Contaminated samples are generated by the replacement model ˜z =zHz+ (1−Hz)ζq, where Hz is a Bernoulli random variable such that ǫz = P(Hz = 0) = 0.09 and ζ_q ∼ δ_q, with δ_q the Dirac mass at q = 15. The sample sizes are n = 27,30,45,54 and the Monte Carlo size is 2500. The focus is on small samples, since in typical applications the parameters of the GEV distribution are necessarily estimated based on a few extreme values (see, e.g., Dupuis & Field (1998) for a discussion). Dupuis & Field (1998) show that the self-standardized first-order robust M-estimator has good reliability. We go one step further and implement the self-standardized second-order robust M-estimator in equation (4.3). For comparison, we also implement the Probability Weighted Moments (PWM) estimator in Hosking, Wallis & Wood (1985). Table 3 presents the results. We find that all estimators have a similar performance when εz = 0. In presence of a contamination, different MSE patterns arise: Outliers badly affect the MSE of the MLE, while all robust estimators show a more resistant MSE profile, with the second-order robust M-estimator having the lowest MSE under contamination.

Specific implementation aspects. The tuning constants b = 2.2 and c = 15 have been selected with the rMSE criterion to ensure an ARE of about 90%, for both the first- and the second-order robust M-estimators. Given the analytical likelihood score in this setting, the second-order robust M-estimator is based onψ(z;θ) =s(z;θ), with matrix A(θ) given in equation (3.7) and function h1 =h2 = exp(|z|). Vector τ in (5.1) is computed by Monte Carlo simulation. We also examined in more detail the performance of the algorithm of Sec- tion 5.1 in our Monte Carlo study. The algorithm performance depends on both sample size and the degree of contamination ε. Large contaminations tend to slow-down the algorithm.

For instance, while forn = 54 the average computation time is 3.96 seconds (with a standard deviation of 33.94) when ε = 10% the average computation time increases to 5.03 seconds (with an increased standard deviation of 40.13). On the other hand, given a fixed contami-