A CLASS OF THE BEST LINEAR UNBIASED ESTIMATORS IN THE GENERAL LINEAR MODEL

A CLASS OF THE BEST LINEAR UNBIASED ESTIMATORS IN THE GENERAL LINEAR MODEL

GABRIELA BEGANU

The existence of the best linear unbiased estimators (BLUE) for parametric es- timable functions in linear models is treated in a coordinate-free approach, and some relations between the general linear model and certain particular models are derived. A class of the BLUE is constructed using the maximal covariance operator. It is shown that a BLUE exists for each parametric estimable function and is unique in this class.

AMS 2000 Subject Classification: 62H12, 47A05.

Key words: best linear unbiased estimator, maximal covariance operator, finite- dimensional inner product space, orthogonal projection.

1. INTRODUCTION

The coordinate-free approach is used in this paper to treat the existence of the best linear unbiased estimators of parametric estimable functions in the general linear model, thus extending and unifying some results previously obtained [7], [8], [10], [16], [17]. This approach, which may also be called the geometric approach, for linear regression models has added the understanding to the problems of estimation and hypothesis testing and it was used at least since linear algebra and the theory of Hilbert spaces have been developed in operators form rather than in matrix form. One of the initial authors who treated in a coordinate-free approach the problem of the existence of BLUE for the expected mean in the general linear model is Kruskal [12]. He gave a necessary and sufficient condition for BLUE to be equal to the ordinary least squares estimators. This condition became the foundation for the alternative forms, as well as their extensions, proved in [1], [2], [3], [4], [5], [6], [9], [11], [15].

The purpose of the paper is to construct a limited class of the best linear unbiased estimators for each linear parametric function using the coordinate- free approach which allows an atractive and simple computational form.

The paper is structurated as follows. The general linear model describ- ing the functional relationship disturbed stochastically is presented from the coordinate-free point of view in Section 2. Some relations between the general

MATH. REPORTS9(59),4 (2007), 327–333

linear model and particular linear models regarding the existence of BLUE are also deduced.

In Section 3, a class of the best linear unbiased estimators of parametric estimable functions is constructed. It is shown that a BLUE in this class exists for each parametric function in the linear model and this BLUE is unique in this class.

2. RELATIONS BETWEEN LINEAR MODELS REGARDING THE EXISTENCE OF BLUE

Let (E, S) be a measurable space and P = {P_θ | θ ∈ Θ} a family of associated probability measures. The complete structure ofPis not necessarily known, but Θ is assumed to be a given parameter space.

A random real-valued element y is a transformation y : E → K, where (K,(·,·)) is a finite-dimensional inner product space. The expectation ofy is the unique elementµ_θ ∈ Ksuch that

(1) E_θ(a, y) = (a, µ_θ)

while the covariance operatorV ofyis the unique symmetric and non-negative mapping fromK to K such that

(2) cov_θ((a, y),(b, y)) = (a, V b)

for all a, b ∈ V. The expectation (1) and the covariance (2) of y exist with respect to allP_θ∈ P, i.e., for all θ∈Θ.

Let us introduce some notation: R(A) is the image of a linear mapping A:L → K, or the column space of the matrixA; spanAis the linear manifold spanned by an arbitrary setA; A^⊥ is the orthogonal complement of a linear manifold A in a finite-dimensional inner product space. Let (L,[·,·]) and (K,(·,·)) be two finite-dimensional inner product spaces. Then A : K → L is the adjoint of the linear operator A, i.e., (Al, k) = [l, Ak] for all l ∈ L, k ∈ K; N(A) is the null space of A. If A is the matrix corresponding to a linear operator A, then A is its transpose, A⁻ is a generalized inverse of A(AA⁻A = A) and A⁺ is the Moore-Penrose inverse uniquely determined byA.

Let Ω ⊆ K. The statistical model M(Ω, V) is the set of all random elements y withE_θy =µ_θ ∈Ω and cov_θy =V for allθ ∈Θ. If Ω is a linear subspace ofK, thenM(Ω, V) is called a linear model.

When Ω = span{E_θy=µ_θ |θ∈Θ} and V = span{cov_θy=V |θ∈Θ}, the general linear model is denoted M(Ω,V).

In the sequel, the general linear regression model will be considered.

Then Ω =R(X), the range of a linear mapping X:L → K, where (L,[·,·]) is a finite-dimensional Euclidean vector space.

A parametric function g(θ) is said to be estimable if and only if there exists an elementa∈ K such that

(3) E_θ(a, y) = (a, µ_θ) =g(θ) for allθ∈Θ.

Let us assume that Ω ⊆ R(V) and let [λ, β] be a linear parametric estimable function. It is well known that a BLUE of [λ, β] is (a, y) iffXa=λ and cov_θ(a, y) = (a, V a) has to be minimized in the class of all linear unbiased estimators, i.e., a∈M ={d∈ K |Xd=λ}. Since M−M =X⁻¹(0), (a, y) is a linear unbiased estimator of minimum covariance iff (V a, d) = 0 for all d∈X⁻¹(0), or, equivalently, iffV a=Xbfor someb∈ L.

The main question in the theory of linear models is the existence of minimum covariance linear unbiased estimators for parametric functions. One of the classical results, which will be used, is a theorem of Lehmann and Scheff´e [14]: for the linear model M(Ω,V), (a, y) is a BLUE ofE_θ(a, y) for all θ∈Θ if and only ifV a∈Θ for all V ∈ V.

Let V₀ be a maximal element of V, i.e., R(V) ⊆ R(V₀) for all V ∈ V. Such an element always exists ([13]).

Using the Lehmann-Scheff´e theorem, another proof will be given for the result below obtained in [8].

Proposition 1. Let V₀ be the maximal element of V. A BLUE of a parametric estimable function exists in the modelM(Ω,V) if and only if

(4) V₀⁻¹(Ω)⊆V⁻¹(Ω)

for allV ∈ V.

Proof. V₀ being the maximal element ofV, we deduce that ifb ∈R(V) for allV ∈ V then b∈R(V₀).

Assume now that (a, y) is a BLUE in the model M(Θ,V). By the Lehmann-Scheff´e theorem this condition is equivalent to b = V a ∈ Ω for all V ∈ V. Thus, R(V₀) ⊆ Ω. Since Ω ⊆ R(V₀) for all V ∈ V, we deduce relation (4).

Conversely, if (4) holds, then V₀a ∈ Ω implies V a ∈ Ω for all V ∈ V, which is the condition for (a, y) to be a BLUE in the modelM(Ω,V).

Proposition2. If (a, y) is a BLUE of a parametric estimable function in the modelM(Ω, V₀), then (a, y) is a BLUE in the model M(Ω,V) provided that aBLUE in this model exists.

Proof. By the Lehmann-Scheff´e theorem in the modelM(Ω, V₀) we have V₀a ∈ Ω. Since a BLUE exists in the model M(Ω,V), this and relation (4) yield a ∈ V⁻¹(Ω) for all V ∈ V, which means that (a, y) is a BLUE in the modelM(Ω,V).

Proposition3. (a, y) is aBLUE of a parametric estimable function in the model M(Ω, V₀) if and only if (a, y) is a BLUE in the model M(Ω, W), where

(5) W =V₀+P

is a symmetric non-negative definite operator and P is the orthogonal projec- tion onΩ.

Proof. (a, y) is a BLUE of a linear parametric function (3) in the model M(Ω, V₀)if and only if V₀a∈ Ω. Since P is the orthogonal projection on Ω, we haveP a∈Ω. These conditions are equivalent toW a∈Ω.

Remark. The orthogonal projection P = XX⁺ on Ω in (5) can be re- placed by every symmetric and non-negative definite operator T such that R(T)⊆Ω. A choice of T can be XX.

Proposition 4. A BLUE of a parametric estimable function (3) exists in the model M(Ω,V) if and only if

(6) V W⁻(Ω)⊆Ω

for allV ∈ V, where W⁻ is a generalized symmetric inverse ofW.

Proof. From Propositions 1, 2 and 3 we have that the condition (a, y) is a BLUE of a parametric function g(θ) for all θ∈Θ in the model M(Ω,V), is equivalent to the condition that (a, y) is a BLUE of g(θ) in the model M(Ω, W). This last statement is equivalent to W a=b∈Ω.

It follows from (5) that

(7) R(V)⊆R(V₀)⊆R(W)

for allV ∈ V. Hence V =W W⁻V or, by transposing, V =V W⁻W and we can write

V a=V W⁻W a=V W⁻b∈V W(Ω)

for allV ∈ V. SinceV a∈Ω for all V ∈ V, relation (6) follows.

3. A CLASS OF BLUE

In the sequel, the question of the existence of the best linear unbiased estimators forg(θ),θ∈Θ, will be treated in the general linear model imposing a restrictive condition.

Theorem1. If(a, y)is aBLUEofE_θ(a, y)in the linear modelM(Ω,V), then there exists a₁ ∈ R(W) such that (a₁, y) is a BLUE of E_θ(a, y) for all θ∈Θ.

Proof. Let (a, y) be a BLUE of E_θ(a, y), θ ∈ Θ, such that a /∈ R(W).

Then a∈ K can be uniquely written as a = a₁+a₂, where a₁ ∈ R(W) and a₂ ∈R(W)^⊥.

Since Ω⊆R(W) (by (7)), we haveR(W)^⊥⊆Ω^⊥, which meansa₂ ∈Ω^⊥. Hence

(8) E_θ(a, y) = (a, Xβ) = (a₁, Xβ) =E_θ(a₁, y) for allθ∈Θ.

Moreover, by the Farkas-Minkowski theorem and the symmetry of the operatorsV and W, relation (7) implies R(W)^⊥=N(W) ⊆R(V)^⊥ =N(V) for allV ∈ V. Therefore, a₂ ∈N(V) or, equivalently, V a₂ = 0 for all V ∈ V. This result allows one to write

(9) cov_θ(a, y) = (a₁+a₂, V(a₁+a₂)) = (a₁, V a₁) = cov_θ(a₁, y) for allV ∈ V and θ∈Θ.

Relations (8) and (9) show that (a₁, y) with a₁ ∈ R(W) is a BLUE of E_θ(a, y), which means that a BLUE of g(θ) with a₁ ∈ R(W) exists when a BLUE ofg(θ) witha∈ K exists in the modelM(Ω,V) for all θ∈Θ.

If there exista, b∈ Ksuch thatXa=Xb=λ, then (a, y) and (b, y) are linear unbiased estimators for [λ, β]. Although both are unbiased estimators for the same parametric function, it is not necessarily true that (a, y) = (b, y), hence the linear unbiased estimator is not in general unique. However, if N(X) ={0} then Xa=Xbimplies that a−b∈N(X), so that a=b and the linear unbiased estimator of [λ, β] is unique. Note that this uniqueness is dependent upon the conditionR(X) = Ω =K.

The problem of the uniqueness of the BLUE (a, y) with a ∈ R(W) is solved by the result below

Theorem2. If(a, y)and(b, y)witha, b∈R(W)are best linear unbiased estimators of a parametric function g(θ) in the linear model M(Ω,V) for all θ∈Θ, then a=b.

Proof. (a, y) and (b, y) being unbiased estimators of the same parame- tric function, we have E_θ(a, y) = E_θ(b, y) for all θ ∈ Θ, which means that (a−b, Xβ) = 0, and this relation is equivalent toa−b∈Ω^⊥.

Since (a, y) and (b, y) are minimum variance estimators in the model M(Ω,V), we have

cov_θ(a, y) = (a, V a) = (a, V b) = cov_θ(b, y)

for allV ∈θ and θ∈Θ. The same equations are verified by V₀, the maximal element ofV and, by Proposition 3, by W. Hence

(10) (a−b, W(a−b)) = 0,

which yieldsW(a−b) = 0 becauseW is a non-negative definite operator. Now, by the Farkas-Minkowski theorem, the relationsW(a−b) = 0 andW(a−b)∈Ω imply

a−b∈W⁻¹(0)∩W⁻¹(Ω) =R(W)^⊥∩[W(Ω^⊥)]^⊥=

= [R(W)∪W(Ω^⊥)]^⊥=R(W)^⊥.

Buta, b∈R(W), hencea−b∈R(W). It follows thata−b= 0.

REFERENCES

[1] S.F. Arnold,A coordinate-free approach to finding optimal procedures for repeated mea- sures designs. Ann. Statist.7(1979), 812–822.

[2] J.K. Baksalary and A.C. van Eijnsbergen, A comparison of two criteria for ordinary least squares estimators to be best linear unbiased estimators. Amer. Statist.42(1988), 205–208.

[3] G. Beganu, Estimation of regression parameters in a covariance linear model. Stud.

Cerc. Mat.39(1987), 3–10.

[4] G. Beganu,The existence conditions of the best linear unbiased estimators of the fixed effects. Econom. Comput. Econom. Cybernet. Stud. Res.36(2003), 95–102.

[5] G. Beganu,On Gram-Schmidt orthogonalizing process of design matrices in linear mod- els as estimating procedure of covariance components. Rev. Real Acad. Cienc. Ser. A Mat.99(2005), 187–194.

[6] G. Beganu, A two-stage estimator of individual regression coefficients in multivariate linear growth curve models. Rev. Acad. Colombiana Cienc.30(2006), 1–6.

[7] H. Drygas,Estimation and prediction for linear models in general spaces. Math. Opera- tionsforsch. Statist. Ser. Statist.8(1975), 301–324.

[8] H. Drygas,Multivariate linear models with missing observations. Ann. Statist.4(1976), 770–787.

[9] M.L. Eaton,Gauss-Markov estimation for multivariate linear models: A coordinate-free approach. Ann. Statist.2(1974), 528–538.

[10] S. Gnot, W. Klonecki and R. Zmyslony,Uniformly minimum variance unbiased estima- tion in Euclidean vector spaces. Math. Operationsforsch. Statist. Ser. Statist.8(1975), 199–210.

[11] S.I. Haberman, How much do Gauss-Markov and least squares estimates differ? A coordinate-free approach. Ann. Statist.3(1975), 982–990.

[12] W. Kruskal, When are Gauss-Markov and least squares estimators identical? A coordinate-free approach. Ann. Math. Statist.39(1968), 70–75.

[13] L.R. Lamotte,Admissibility in linear estimation. Ann. Statist.10(1982), 245–255.

[14] E. Lehmann and H. Scheff´e, Completeness, similar regions and unbiased estimation.

Sankhya10(1950), 305–340.

[15] S. Puntanen, G.P.H. Styan and Y. Tian, Three rank formulas associated with the co- variance matrices of the BLUE and the OLSE in the general linear model. Econometric Theory21(2005), 659–664.

[16] J. Seely,Linear spaces and unbiased estimation. Ann. Math. Statist.41(1970), 1725–

1734.

[17] J. Seely,Linear spaces and unbiased estimation. Application to the mixed linear models.

Ann. Math. Statist.41(1970), 1735–1748.

[18] G. Zyskind,On canonical forms, non-negative covariance matrices and best and simple least squares linear estimators in linear models. Ann. Math. Statist.38(1967), 1092–

1109.

Received 8 February 2007 Academy of Economic Studies

Department of Mathematics Piat¸a Romanˇa nr. 6 010374 Bucharest, Romania

gabriela beganu@yahoo.com