Heterogeneous variances in Gaussian linear mixed models

HAL Id: hal-00894091

https://hal.archives-ouvertes.fr/hal-00894091

Submitted on 1 Jan 1995

HAL is a multi-disciplinary open access

archive for the deposit and dissemination of

sci-entific research documents, whether they are

pub-lished or not. The documents may come from

teaching and research institutions in France or

abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est

destinée au dépôt et à la diffusion de documents

scientifiques de niveau recherche, publiés ou non,

émanant des établissements d’enseignement et de

recherche français ou étrangers, des laboratoires

publics ou privés.

Heterogeneous variances in Gaussian linear mixed

models

Jl Foulley, Rl Quaas

To cite this version:

Original

article

Heterogeneous

variances

in

Gaussian

linear mixed models

JL

_Foulley

RL

_Quaas

1

Institut

national de la recherche

_agronomique,

station de

_génétique

quantitative

et

_appliquée,

78352

_{Jouy-en-Josas,}

_Fra!ece;

2

Department

of

Animal

_Science,

Cornell

_{University, Ithaca,}

NY

_14853,

USA

(Received

28

_{February 1994; accepted}

29 November

₁₉₉₄₎

Summary - This paper reviews some

problems

encountered in

estimating heterogeneous

variances in Gaussian linear mixed models. The one-way and

multiple

classification

cases are considered. EM-REML

_algorithms

and

Bayesian procedures

are derived. A

structural mixed linear model on

_log-variance

_components is also

_presented,

which allows identification of

_meaningful

sources of variation of

heterogeneous

residual and

_genetic

components of variance and assessment of their

_magnitude

and mode of action.

heteroskedasticity

/

mixed linear

_{model / restricted}

maximum likelihood

_/

Bayesian statistics

Résumé - Variances _{hétérogènes} en modèle linéaire mixte _gaussien. Cet article

fait

le point sur un certain nombre de

_problèmes

qui surviennent lors de l’estimation de variances

_{hétérogènes}

dans des modèles linéaires mixtes

_gaussiens.

On considère le cas

d’un ou

_{plusieurs facteurs}

d’hétéroscédasticité. On

_développe

des

_algorithmes

EM-REML

et

_bayésiens.

On propose

également

un modèle linéaire mixte structurel des

_logarithmes

des variances qui permet de mettre en évidence des sources

_{significatives}

de variation des variances résiduelles et

_génétiques

et

_{d’appréhender}

leur importance et leur mode d’action.

hétéroscédasticité

_/

modèle linéaire mixte

_/

maximum de vraisemblance résiduelle

_/

statistique bayésienne

INTRODUCTION

Genetic evaluation

_procedures

in animal

_{breeding rely mainly}

on best linear

unbi-ased

_prediction

_(BLUP)

and restricted maximum likelihood

_(REML)

estimation of

parameters

of Gaussian linear mixed models

_(Henderson,

_1984).

_Although

BLUP

mixed-model

_{methodology postulate}

_homogeneity

of variance

components

across

subclasses involved in the stratification of data.

_However,

there is now a

_great

deal

of

_experimental

evidence of

_{heterogeneity}

of variances for

_{important production}

traits of livestock

_(eg,

milk

_yield

and

_growth

in

_cattle)

both at the

_genetic

and envi-ronmental levels

_(see,

for

_example,

the reviews of Garrick et

_{al, 1989,}

and Visscher et

_al,

_1991).

As shown

by

Hill

_(1984),

_{ignoring heterogeneity}

of variance decreases the

ef-ficiency

of

_genetic

evaluation

_procedures

and

_consequently

response to

_selection,

the

_importance

of this

_phenomenon

_depending

on

_assumptions

made about sources

and

_magnitude

of

_{heteroskedasticity}

_(Garrick

and Van

Vleck, 1987;

Visscher and

Hill,

1992).

Thus,

making

correct inferences about heteroskedastic variances is crit-ical. To that

_{end, appropriate}

estimation and

testing

procedures

for

_{heterogeneous}

variances are needed. The _purpose of this _paper is an

_attempt

to describe such pro-cedures and their

principles.

For

_pedagogical

reasons, the

presentation

is divided

into 2

_parts

_according

to whether

_{heteroskedasticity}

is related to a

single

or to a

multiple

classification of factors.

THE ONE-WAY CLASSIFICATION

Statistical model

The

_population

is assumed to be stratified into several

_{subpopulations}

_(eg,

_herds,

regions,

etc)

indexed

_by

i =

_{1, 2, ... , I, representing}

a

_potential

source of

hetero-geneity

of variances. For the sake of

simplicity,

we first consider a _{one-way random}

model for variances such as

where _yi is the

_(n

₂

_{x 1)}

data vector for

_{subpopulation}

_i,

₁₃

is the

_{(p x 1)}

vector of

fixed effects with incidence matrix

_X

_i

_,

is *u a

(q

x

₁₎

vector of standardized random effects with incidence matrix

_Z

_i

and ei is the

(n

i

x 1)

vector of residuals.

The usual

_assumptions

of

_normality

and

_independence

are made for the

distri-butions of the random variances u* and _ei, ie u* !

_{N(0, A) (A}

_positive

definite

matrix of coefficients of

_{relationship)}

and eiN

NID(O,

er! 1!;)

and

Cov(e

i

,

u*!)

= 0 so that _y2N

_N(X

_il

_3,

_{a u 2i Z’AZI i +0,2 ei 1,}

where

_or

₂

_ei

_and

_{o, ui 2}

are the residual and

u-components

of variance

_pertaining

to

_{subpopulation}

i. A

_{simple example}

for

_[1]

is a

_2-way

additive mixel model _Yij =

p,+hi+as!8!

+ e

zjk

with fixed herd

_(hi)

and random sire

_(<7..;,!)

effects. Notice that model

_[1]

includes the case of fixed effects

nested within

subpopulations

as observed in many

applications.

EM REML estimation of

heterogeneous

variance

_components

To be consistent with common

_practice

for estimation of variance

_components,

we chose REML

(Patterson

and

_Thompson,

_1971;

_Harville,

1977)

as the basic

estimation

procedure

for

_{heterogeneous}

variance

_components

_(Foulley

et

_al,

_1990).

A convenient

_algorithm

to

_compute

such REML estimates is the

be based on the

_general

definition of EM

_(see

_{pages 5} and 6 and formula 2.17 in

Dempster

et

_al,

₁₉₇₇₎

which can be

explained

as follows. L tt’

₍

1 1 1 1

)1

2

{ 2}

l 2

{ 2}

d 2

_{( 2’}

Lettln

g

y = y 1 , Y 2, ... , Y i, ... , Y I ,

(y

e

2

=

1

0

,

2

ei 1, U2

,

Ui

2

=

o

f

u

and

U2

=

(

g2

&dquo; e

9

u 2’),

the derivation of the EM

_algorithm

for REML stems from a

_complete

data set defined

_by

the vector x

= (y’,

*/

I

)

,

U

1

13

and the

corresponding

likelihood function

L(

6

2

;x)

=

Inp(xlc¡

2

).

In this

presentation,

the vector

(3

is treated in a

_Bayesian

manner as a vector of random effects with variance fixed at

_infinity

_(Dempster

et

_al,

1977; Foulley,

1993).

A

_frequentist

_{interpretation}

of this

_algorithm

based on error

contrasts can be found in De Stefano

(1994).

A similar derivation was

_given

for the

homoskedastic case

_by

Cantet

(1990).

As

_usual,

the EM

algorithm

is an iterative one

_consisting

of an

_{’expectation’}

_(E)

and of a ’maximization’

(M)

_step.

Given the

current estimate

c¡

2

=

_c¡

_2[t]

at iteration

_[t],

the E

_step

consists of

computing

the conditional

_expectation

of

_{L( c¡}

₂

_{; x),}

ie

given

the data vector y and

()&dquo;2

= ()&dquo;2[t].

The M

_step

consists of

_choosing

the next value

()&dquo;2[

t+l]

of

U

2

_by

maximizing

Q()&dquo;

2

1()&dquo;

2[t]

)

with

respect

to

U

2

Since

_{In p(xl(T2)}

_{= ln p (y ! (3, u* ,}

_{(T2)+ln p(l3, u* 1(T2) with In p(l3, u}

_*

_{I (}

_T

₂

₎

_providing

no information about

o-

2

,

Q( (

T

2

1

(T2[

t]

)

can be

replaced by

Under model

_!1!,

the

_expression

for

_Q

_*

_(c

_r

_21U

₂

_[l]

₎

reduces to

where

E!t!(.)

indicates a conditional

_expectation

taken with

_respect

to the

distribu-tion of

_[3,

_U

_*

_{I y,}

6

2

=

(J’

2[t]

.

This

posterior

distribution is multivariate normal with

mean

E(l3ly,

6

2

)

= BLUE

(best

linear unbiased

estimate)

of

j3,

E(u!y,

(

7

’)

= BLUP

of u, and

Var(l3, uly,

(J’2)

= inverse of the mixed-model coefficient matrix.

The

_system

of

equations

åQ

*

( (J’21

o’!)/9o’!

= 0 _can be written _as follows: With

respect

to the

_u-component,

we have

For the residual

_component,

Since

E!t] (e!ei)

is a function of the unknown Qui

_only,

_equation

[5]

_depends

_only

on that unknown whereas

_equation

[6]

_depends

of both variance

_components.

We

then solve

_[5]

first with

_respect

to _{Ju, , and then solve}

_[6]

second

_substituting

the solution

a!t+1!

to _{o,,,, back} into

E!t](e!ei)

of

_(6!,

ie with

Hence

It is worth

_noticing

that formula

_[7]

_gives

the

_expression

of the standard deviation of the

_u-component,

and has the form of a

_regression

coefficient estimator.

Actually

Ju

, is the coefficient of

regression

of any element of yi on the

corresponding

element

of Zju

*

.

Let the

_system

of mixed-model

_equations

be written as

and

C

= [

C

,

3,3

C

,

3

.

J =

g inverse of the coefficient matrix.

L

C.,3

CUU

I

=

-The elements of

_[7]

and

_[8]

can be

_expressed

as functions of _y,

_(3,

u and blocks of

C as follows

For readers interested in

_applying

the above

formulae,

a small

_example

is the

is worth

_noticing

that formulae

_[7]

and

_[8]

can also be

_applied

to the homoskedastic

case

_{by considering}

that there is

_just

one

_{subpopulation}

_{(I = 1).}

The

_resulting

algorithm

looks like a

_regression

in contrast to the conventional EM whose formula

(a![t+1]

=

E

l

t]

(u’A-

1

u)/q)

where u is not standardized

_(u

=

cr!u*)

is in _terms of a variance. Not

_only

do the formulae look

_{quite different,}

but

_they

also

perform

quite

differently

in terms of rounds to _convergence. The conventional EM tends to

do

_quite

_poorly

if

_{or »}

_{o, and}

_(or)

with little

information,

whereas the scaled EM is at its best in these situations. This can be demonstrated

_{by examining}

a

balanced

_paternal

half-sib

_design

_(q

_{families with progeny group size n}

_each).

This is convenient because in this case the EM

_algorithms

can be written in terms of

the between- and within-sire sums of _{squares and convergence}

_performance

checked

for a

_variety

of situations without

_simulating

individual records. For this

_simple

situation

_performance

was

_fairly

well

predicted

by

the criterion

R

2

=

_n/(n

₊

_a),

where a =

a2/0,2 .

_Figure

1 is _a

plot

of rounds _to

convergence for the scaled and usual EM

_algorithms

for an

_arbitrary

set of values of n and a. As noted

_by

_Thompson

and

_Meyer

_(1986),

the usual EM

_performs

very

poorly

at low

_R

₂

_,

eg, n = 5 and

h

2

=

_4/(a

₊

₁₎

= 0.25 _{or n} = 33 and

h

2

=

_0.04,

_ie

R

2

=

_0.25,

but

very well at the other end of the

_{spectrum: n}

= 285 and

h

2

= 0.25 or n = 1881 and

h

2

=

_0.04,

ie

R

2

= 0.95. The

_performance

of the scaled version is the exact

_opposite.

Interestingly,

both EM

_algorithms

_perform

_similarly

for

R

2

values

typical

of _many animal

_breeding

data sets

_(n

= 30

and h

2

=

0.25,

ie

R

2

=

2/3).

Moreover,

solutions

_given

_by

the EM

_algorithm

in

_[7]

and

_[8]

turn out to be within the

_parameter

space in the homoskedastic case

(see

_proof

in the

Appendix)

Bayesian approach

When there is little information per

subpopulation

(eg,

herd or herdx

_management

unit),

REML estimation of

_Q

_e

_i

and

_Q

_u

_z

can be unreliable. This led Hill

₍₁₉₈₄₎

and Gianola

₍₁₉₈₆₎

to

suggest

estimates

shrunken towards some common mean

natural extension of the EM-REML

technique

described

_previously.

The

parameters

ol2 ei

and

_o,

_U

_,

₂

are assumed to be

_{independently}

and

_identically

distributed random variables with scaled inverted

chi-square density functions,

the

_parameters

of which

are

s2,,

q

,

e

and

su,

_r!!

respectively.

The

_parameters

se and

s! are

location

parameters

of the

_prior

distributions of variance

components,

and TIe

and 7

7

,,

(degrees

of

belief)

are

quantities

related to the

squared

coefficient of variation

(cv)

of true variances

by

qe =

(2/cve )

₊ 4

and

q

u

=

(2/cufl)

₊ 4

respectively:

Moreover,

let us assume as in Searle et al

_(1992,

page

99),

that the

priors

for residual and

_u-components

are assumed

independent

so that

_{p (,72i, U2i) = p(,71i)p(0,2i).}

The

₂

_Q

_t

_[

_])

_{1 O’}

₂

_{( 0’}

_@

function to maximize in order to

_produce

the

posterior

mode

of

o-

2

is now

(Dempster

et

_al,

1977,

page

6):

with

Equations

based on first derivatives set to zero are:

Using

!l2ab!,

one can use the

_following

iterative

algorithm

Comparing

[13b]

and

_[14]

with the EM-REML formulae

_[7]

and

_[8]

shows how

prior

information modifies data information

_(see

also tables I and

_II).

In

_particular

when

_TJe

_(TJ

_u

₎

= 0

(absence

of

knowledge

on

_prior

_variances),

formulae

_[13b]

and

_[14]

are _very similar to the EM-REML formulae.

_They

would have been

_exactly

the

same if we had considered the

_posterior

mode of

log-variances

instead of

_variances,

!7e and !7.!

replacing

17, + 2 and _{!7! +} 2

_respectively

in

_!11!,

_and,

_consequently

also in the denominator of

_[13b]

and

_!14!.

In

_contrast,

if

_!7e(!/u)

---> 00

_(no

_{variation among}

variances),

estimates tend to the location

parameters

s!(s!).

Extension to several

_u-components

The EM-REML

_equations

can

_easily

be extended to the case of a linear mixed

model

_including

several

independent

u-components

_{(uj; j}

=

1, 2, ... ,

J),

ie

In that _case, it can be shown that formula

[7]

is

_{replaced by}

the linear

_system

The formula in

_[8]

for the residual

_components

of variance remains the same.

This

_algorithm

can be extended to

_{non-independent}

u-factors. As in a

_sire,

maternal

_grand

sire

_model,

it is assumed that correlated

factors j and

k are such

that

_Var(u*)

=

Var(u*)

=

A,

and

Cov(uj, u!/)

=

p

jk

A

with

_dim(u!)

= _m for all

j.

Let

a

2

=

(or2&dquo; or2&dquo;

p’)

with p =

vech(S2),

S2

being

_a

(m

x

_m)

correlation matrix with _pjk as element

_jk.

The

Q#(êT2IêT

2[t]

)

function to maximize can be written here as

where

The first term

_{Q7 (u! ] 8&dquo;!°! )}

=

ErJ[lnp(yll3,u*,(J’!)]

has the _same form _as with

the case of

_independence

_except

that the

_expectation

must taken with

_respect

to the distribution of

_(3,

_u

_*

_Iy,

Õ

’

2

=

Õ’

2[t]

.

The second term

Q!(plÕ’2[t])

=

E

l

c

’

]

) [In p(u

*

1

&2

)]

can be

expressed

as

where D =

{uj’ A

-l

uk}

is a

(J

x

_J)

_symmetric

matrix.

The maximization of

Q#(¡

2

1(¡

2[t]

)

with

_respect

to

6

2

can be carried out in 2

_stages:

_i)

maximization of

Qr(¡

2

1(¡2

[t]

)

with

respect

to the vector

!2 of

variance

components

which can be solved as

_above;

and

_ii)

maximization of

Q#(p 1&211,)

with

_respect

to the vector of correlation coefficients p which can be

_performed

via

THE MULTIPLE-WAY CLASSIFICATION

The structural model on

_{log-variances}

Let us assume as above that the

a2s

(u

and e

types)

are a

_priori

_{independently}

distributed as inverted

chi-square

random variables with

_parameters

5!

(location)

and _riz

_(degrees

of

_belief),

such that the

_density

function can be written as:

where

_r(

_x

₎

is the _gamma function.

From

_!19),

one can

_{alternatively}

consider the

_density

of the

_log-variance

1n

Q2

,

I or more

_{interestingly}

that

of v

z

=

ln(a2/s2).

In

addition,

it can be assumed that 7

1i = ! for all

i,

and that

lns2

can be

_decomposed

as a linear combination

p’S

of

some vector 5 or

_explanatory

variables

_(p’

_being

a row vector of

_incidence),

such that

with

For v

i

-->

_0,

the kernel of the distribution _in

[21]

_{tends towards}

exp( -r¡v’f /4),

_thus

leading

to the

following

normal

_{approximation}

where the variance a

_priori

_(!)

of true variances is

_{inversely proportional}

_to

_q

(!

= 2/?!), !

also

_being

_{interpretable}

as the _square coefficient of variation of

log-variances.

This

_{approximation}

turns out to be excellent for most situations encountered in

_practice

_{(cv ! 0.50).}

Formulae

_[20]

and

_[21]

can be

_naturally

extended to several

_independent

classi-fications in v =

(v!, v2, ... ,

vj, ... ,

v!)’

such that

with

where

_K

_j

=

dim(v

j

)

and J1 =

(t,’, v’)’

is the _vector of

dispersion

_parameters

and

C’

=

(p!, q

’)

is the

_{corresponding}

_{row vector} of incidence.

This

presentation

allows us to mimick a mixed linear model structure with fixed 5 and random v effects on

_{log-variances,}

similar to what is done on cell means

(vt

i

=

x!13

₊

z’u

=

t!0),

and thus

justifies

the choice of the

log

_as the link function

(Leonard,

1975; Denis,

1983;

Aitkin, 1987;

Nair and

_Pregibon,

₁₉₈₈₎

to use for this

generalized

linear mixed-model

_approach.

where y! =

_{in 0,2i 1,}

ye =

in or’ 1;

P

u

,

P

e

are incidence matrices

_pertaining

to fixed effects

₅

_u

_,

_be

_{respectively; Q}

_u

_,

_Qg

are incidence matrices

_pertaining

to random effects vu =

(V!&dquo;V!2&dquo;&dquo;,V!j&dquo;&dquo;)’

and Ve =

(V!&dquo;V!2&dquo;&dquo;,V!jl)’

with

v! -Nid(0,!I!.)

J J _UJ

and V

el

J

_{-NID(0,!,I! ,)}

J _ej’

respectively.

Estimation

Let A =

_{(À!, A}

_’)’

_and

_(ç!,

_{g[I’ where}

_g

_u

₌

_{ç

_uJ

_and

_Ç

_e

=

!ei, 1.

Inference

u e

e

>

about 71 is of an

_{empirical Bayes type}

and is based on the mode a of the

_posterior

density

p(Àly, E,

=

i;) given I = I

its

marginal

maximum likelihood

estimator,

ie

Maximization in

_[26ab]

can be carried out

_according

to the

_procedure

described

by

Foulley

et al

₍₁₉₉₂₎

and San Cristobal et al

_(1993).

The

_algorithm

for

_computing

X

can be written as

(from

iteration t to t +

₁₎

where

z =

(z’

u

,

z!)

are

_working

variables

_updated

at each iteration and such that

w =

W

-- Wue J is

a (2I, 21)

matrix of

_weights

described in

_Foulley

et al

W

eu

Wee

(1990, 1992)

for the environmental variance

part,

and in San Cristobal et al

₍₁₉₉₃₎

for the

_general

case.

where

9tl ,

9t/

are

solutions of

_[26]

for ! =

![nl

and

H[n]

H!n!e!&dquo;

are blocks of

were _{uj ’}

_ej’

are so utlonS 0 lor L, =

L, , an VjVj’ ej,ej&dquo; are oc SO

the inverse of the coefficient matrix of

_[27]

_pertaining

to

_v

_U

_j’

_Vej’

_{respectively.}

Testing procedure

The

_procedure

described

_previously

reduces to REML estimation of J1 when flat

priors

are assumed for _v! and _v,, or

equivalently

when the structural model for

log-variances

is a

purely

fixed model. This

_property

allows derivation of

testing

procedures

to

_{identify significant}

sources of

heteroskedasticity.

This can be achieved

via the likelihood ratio test

_(LRT)

as

_proposed

_by

_Foulley

et al

_{(1990, 1992)}

and San

Cristobal et al

₍₁₉₉₃₎

but in

_using

the residual

_(marginal)

_{log-likelihood}

function

L(7!;

y)

instead of the full

_{log-likelihood}

_L(71,

[3;

y).

Let

_o

_:À

_H

c

!lo

be the null

_hypothesis

and

_:À

_H

₁

E Il -

_A

_o

its

_alternative,

where

A refers to the

_complete

_parameter

space and

!lo,

a subset of

_{it, pertains}

to

_H

_o

_.

Under

_H

_o

_,

the statistic

has an

_{asymptotic chi-square}

distribution with r

degrees

of

freedom;

r is the

difference in the number of estimable

parameters

involved in A and

Il

o

,

eg,

r =

rank(K)

when

H

o

is defined

by

K’71 =

_0,

K’

being

a full row rank matrix.

Fortunately,

for a Gaussian linear

model,

such as the one considered here

y 2

N

N(Xil3, a!iZ!AZi

+

_U2i

_j&dquo;

_i

_),

_{L(À; y),}

can be

_{easily expressed}

as

1

0

0

where N =

! ni,

p = rank(X),

Ti

=

(

x

i,or.

i

zi),

E- -

[0 0]

and where N

= L

ni,

P =

rank(X),

Ti

=

(Xi,

aUiZi),

!- =

10

A

_J

and

<=1

! !,

e =

(13, û

*

’)

is _a solution of the

_system

of mixed-model

equations

in

[9],

ie

I

1 I

[t a;:2T!Ti + !-] ê = t a;:2T!Yi’

L

i=1

&dquo; i

_

i=l

&dquo; iyi

CONCLUSION

This _{paper is} an

_attempt

to

_synthesize

the current state of research in the field of statistical

_analysis

of

_{heterogeneous}

variances

_arising

in mixed-model

_methodology

and in its

_application

to animal

_breeding.

For

_pedagogical

_reasons, the _paper

successively

addressed the cases of _one-way and

_multiple-way

classification. In

any case, the estimation

procedures

chosen were REML and its natural

Bayesian

extension,

the

posterior

mode

_using

_{inverted gamma}

prior

distributions. For a

single classification, simple

formulae for

computing

REML and

_Bayes

estimations

For

_{multiple classifications,}

the

_key

idea

_underlying

the structural

_approach

is to render models as

_parsimonious

as

possible,

which is

_especially

critical with the

large

data sets used in

_genetic

evaluation

_having

a

_large

number of subclasses.

Consequently,

it was shown how

_{heterogeneity}

of

_log-residual

and

_u-components

of variance can be described with a mixed linear model structure. This makes the

corresponding

estimation

_procedures

a natural extension of what has been done for

decades

_by

BLUP

_techniques

on subclass means.

An

_important

feature of this

_procedure

is its

_ability

to assess via likelihood ratio

tests the effects of

_potential

sources of

_{heterogeneity}

considered either

_marginally

or

_{jointly. Although computationally demanding,}

these

_procedures

have

_begun

to be

_applied.

San Cristobal et al

₍₁₉₉₃₎

_applied

these

_procedures

to the

_analysis

of muscle

_development

scores at

_weaning

of 8 575 progeny of 142 sires in the Maine

Anjou

cattle breed where

_{heroskedasticity}

was found both for the sire and residual

components.

Weigel

et al

₍₁₉₉₃₎

and DeStefano

₍₁₉₉₄₎

also used the structural

approach

for sire and residual variances to assess sources of

_{heterogeneity}

in within-herd variances of milk and fat records in Holstein. Herd size and within-within-herd means were associated with

_significant

increases in residual variances as well as various

management

factors

_(eg,

_milking

_system).

_{Approximations}

for the to estimation of within

_{region-herd-year-parity}

_phenotypic

variances were also

_proposed

for

_dairy

cattle evaluation

_{by Wiggans}

and Van Raden

₍₁₉₉₁₎

and

_Weigel

and Gianola

(1993).

These

_techniques

also _open new

_prospects

of research in different fields of

quantitative genetics,

eg,

analyses

of

genotype

x environment interactions

_(Foulley

et

_al,

_1994;

Robert et

_al,

_1994),

_{testing procedures}

for

_genetic

_parameters

_(Robert

et

al,

1995ab),

crossbreeding

experiments

and

_QTL

detection.

However,

research is

still needed to

_improve

the

_methodology

and

_efficiency

of

_algorithms.

ACKNOWLEDGMENTS

This paper draws from material

presented

at the 5th World

_Congress

of Genetics

_Applied

to Livestock

_{Production, Guelph, August 7-12,}

1994.

_Special

thanks are

_expressed

to AL

_DeStefano,

V

Ducrocq,

RL

_Fernando,

D

_Gianola,

D

_H6bert,

CR

_Henderson,

S

_Im,

C

Robert,

M

_Gaudy-San

Cristobal and K

Weigel

for their valuable contribution to the discussion and to the

publication

of papers on this

_subject.

The assistance of C Robert in

the

_computations

of the numerical

_example

is also

_{greatly acknowledged.}

REFERENCES

Aitkin M

₍₁₉₈₇₎

_Modelling

variance

_{heterogeneity}

in normal

_{regression using}

GLIM.

_Appl

Stat

36,

332-339

Cantet RJC

₍₁₉₉₀₎

Estimation and

prediction problems

in mixed linear models for maternal

_genetic

effects. PhD

_{thesis, University}

of

Illinois, Urbana, IL,

USA

Dempster AP,

Laird

_NM,

Rubin DB

₍₁₉₇₇₎

Maximum likelihood from

_incomplete

data

via the EM

_algorithm.

J R Stat Soc B _39, 1-38

Denis JB

₍₁₉₈₃₎

Extension du mod6le additif

_d’analyse

de variance par mod6lisation

multiplicative

des variances. Biometrics

_39,

849-856

DeStefano AL

₍₁₉₉₄₎

_Identifying

and

quantifying

sources of

_{heterogeneous}

residual and

Foulley

JL

₍₁₉₉₃₎

A

_simple

_argument

_showing

how to derive restricted maximum likeli-hood. J

_Dairy

Sci

_76,

2320-2324

Foulley JL,

Gianola

_D,

San Cristobal

_M,

Im S

₍₁₉₉₀₎

A method for

_assessing

extent and

sources of

_{heterogeneity}

of residual variances in mixed linear models. J

_Dairy

Sci

_73,

1612-1624

Foulley JL,

San Cristobal

_M,

Gianola

_D,

Im S

₍₁₉₉₂₎

_Marginal

likelihood and

_Bayesian

approaches

to the

_analysis

of

_{heterogeneous}

residual variances in mixed linear Gaussian models.

Comput

Stat Data Anal 13, 291-305

Foulley JL,

H6bert

_{D, Quaas}

RL

₍₁₉₉₄₎

Inferences on

_homogeneity

of

_{between-family}

components of variance and covariance among environments in balanced cross-classified

designs.

Genet Sel

Evol 26,

117-136

Garrick

_DJ,

Van Vleck LD

₍₁₉₈₇₎

_Aspects

of selection for

_performance

in several

environ-ments with

_{heterogeneous}

variances. J Anim Sci

_65,

409-421

Garrick

_DJ,

Pollak

_{EJ, Quaas RL,}

Van Vleck LD

₍₁₉₈₉₎

Variance

_{heterogeneity}

in direct and maternal

_weight

traits

_by

sex and _percent

_purebred

for Simmental sired calves.

J Anim Sci

_67,

2515-2528

Gianola D

₍₁₉₈₆₎

On selection criteria and estimation of _parameters when the variance is

heterogeneous.

Theor

_Appl

Genet

_72,

671-677

Gianola

_{D, Foulley JL,}

Fernando

_RL,

Henderson

_{CR, Weigel}

KA

₍₁₉₉₂₎

Estimation of

heterogeneous

variances

_{using empirical}

_Bayes

methods: theoretical considerations. J

_Dairy

Sci 75, 2805-2823

Graybill

FA

₍₁₉₆₉₎

Introduction to Matrices with

_Applications

to Statistics. Wadsworth

Publishing Co, Bemont, CA,

USA

Harville DA

₍₁₉₇₇₎

Maximum likelihood

_approaches

to variance component estimation

and related

_problems.

J Am Stat Assoc 72, 320-340

Henderson CR

₍₁₉₈₄₎

_{Applications of}

Linear Models in Animal

_{Breeding. University}

of

Guelph, Guelph, Ontario,

Canada.

Hill WG

₍₁₉₈₄₎

On selection among groups with

heterogeneous

variance. Anim Prod 39, 473-477

Johnson

_{DL, Thompson}

R

₍₁₉₉₄₎

Restricted maximum likelihood estimation of variance

component for univariate animal models

_using

_{sparse matrix}

_techniques

and a

quasi-Newton

_procedure.

In: 5th World

_Congress

on Genetics

_Applied

to Livestock Production

(C

Smith et

_al,

_eds),

_University

of

_{Guelph, Guelph, ON, Canada,}

vol

_18,

410-413 Leonard T

₍₁₉₇₅₎

A

_{Bayesian approach}

to the linear model with

_unequal

variances.

Technometrics

_17,

95-102

Liu

_C,

Rubin DB

₍₁₉₉₄₎

_Application

of the ECME

_algorithm

and Gibbs

sampler

to

_general

linear mixed model. In: Proc XVllth International Biometric

Conference,

McMaster

University, Hamilton, ON, Canada,

vol 1, 97-107

Meyer

K

₍₁₉₈₉₎

Restricted maximum likelihood to estimate variance _components for animal models with several random effects

using

a derivative-free

algorithm.

Genet

Sel

Evol 21,

317-340

Nair VN,

Pregibon

D

₍₁₉₈₈₎

_{Analysing dispersion}

effects from

_replicated

factorial

exper-iments. Technometrics

30,

247-257

Patterson

HD, Thompson

R

₍₁₉₇₁₎

_Recovery

of interblock information when block sizes

are

_unequal.

BioTn,etrika 58, 545-554

Robert

_{C, Foulley JL, Ducrocq}

V

₍₁₉₉₄₎

Estimation and

_testing

of a constant

_genetic

correlation among environments. In: 5th World

Congress

on Genetics

Applied

to

Livestock Production

_(C

Smith et

_al,

_eds),

_University

of

Guelph, Guelph, ON, Canada,

vol

18,

402-405

Robert

_{C, Foulley JL, Ducrocq}

V

_(1995a)

Genetic variation of traits measured in

Robert

_{C, Foulley}

JL,

Ducrocq

V

_(1995b)

Genetic variation of traits measured in

several environments. II. Inference on between-environments

_homogeneity

of intra-class correlations. Genet Sel

_{Evol 27,}

125-134

B

San Cristobal M,

Foulley JL,

Manfredi E

₍₁₉₉₃₎

Inference about

_{multiplicative}

het-eroskedastic _components of variance in a mixed linear Gaussian model with an

ap-plication

to beef cattle

_breeding.

Genet Sel Evol

_25,

3-30

Searle

_SR,

Casella

G,

McCulloch CE

₍₁₉₉₂₎

Variance

_Components.

J

_Wiley

and

Sons,

New

_York,

USA

Thompson R, Meyer

K

₍₁₉₈₆₎

Estimation of _{variance components:} what is

_missing

in the

EM

_algorithm?

J Stat

Comput

Simul

_24,

215-230

Visscher PM, Hill WG

₍₁₉₉₂₎

_{Heterogeneity}

of variance and

_dairy

cattle

_breeding.

Anim Prod 55, 321-329

Visscher

PM, Thompson R,

Hill WG

₍₁₉₉₁₎

Estimation of

genetic

and environmental variances for fat

_yield

in individual herds and an

_{investigation}

into

_{heterogeneity}

of

variance between herds. Livest Prod Sci 28, 273-290

Weigel KA,

Gianola D

₍₁₉₉₃₎

A

_{computationally simple Bayesian}

method for estimation

of

_{heterogeneous}

within-herd

_phenotypic

variances. J

_Dairy

Sci

_76,

1455-1465

Weigel KA,

Gianola

_D,

Yandel

_BS,

Keown JF

₍₁₉₉₃₎

Identification of factors

_causing

heterogeneous

within-herd variance _components

_using

a structural model for variances.

J

_Dairy

Sci

_76,

1466-1478

Wiggans GR,

VanRaden PM

₍₁₉₉₁₎

Method and effect of

_adjustment

for

_{heterogeneous}

variance. J

Dairy

Sci

74,

4350-4357

APPENDIX

A

proof

that

_equation

_[7]

is

_non-negative

in the homoskedastic case is as follows.

The numerator of

_[7]

can be written as:

where e =

y -

XI3,

Ê= y -

X(3

=

_VPy,

and u =

-

(y -

1

a

y

Z’

u

X!)

=

u

,,ZPy

with V =

Var(y),

P =

y-1- Y-1X(X’y-1X)-X’Y-1,

and

K’y,

_an

N-rank(X)

vector of linear contrasts. Now

so that

Moreover

V-’ -

P =

V-’X(X’V-’X)-X’V-

1

_is

then

nnd

matrices,

then

_{tr(AB) >}

0

_{(Graybill, 1969).}

COMMENT

Robin

_Thompson

Roslin

_{Institute, Roslin,}

Midlothian EH25

_9PS,

UK

This _paper is a

synthesis

of recent work of the authors and their co-workers in the area of

_{heterogeneous}

variances. I think it is a valuable review

_giving

a

logical

presentation

showing

how

_{heterogeneous}

variance

_modelling

can be carried out.

The

parallels

between estimation of linear

parameters

and variance

_parameters

is

highlighted. My

comments relate to

_{transformation,}

_convergence,

_simplicity

and

utility.

Transformation

Emphasis

is on the

_genetic

standard deviation as a

_parameter

rather than the

genetic

variance. Such a

_{parameterization}

has been used to allow estimation of

binary

variance

_components

_(Anderson

and

_Aitkin,

_1985).

_Using

scaled variables

can also allow reduction of the dimension of search in derivative-free methods of multivariate estimation

_(Thompson

et

_al,

_1994).

It is also a

_special

case of the Choleski

_{transformation,}

which has been

_suggested

to

_speed

_up estimation and

keep

parameters

within bounds for

_repeated

measures data

_(Lindstrom

and

_Bates,

1988)

and multivariate

_genetic

data

_{(Groeneveld, 1994).}

Reverter et al

₍₁₉₉₄₎

have

_recently

_suggested

_regression

_type

methods for estimation of variance

_parameters.

As the authors

_point

_out,

there is a natural

regression interpretation

to the similar

_equations

_[7]

and

_[8].

However

_[7]

and

[8]

include trace terms that

_essentially

correct for attenuation or

_uncertainty

in

knowing

the fixed or random effects.

Convergence

In the discussion of

Dempster

et al

₍₁₉₇₇₎

I

_pointed

out that the rate of convergence for a balanced one-way

_analysis

is

(in

the authors’

_notation)

_{approximately}

1-

_R

₂

_,

which,

I

_{think, explains}

one of the

graphs

in

figure

1. In the time

_available,

I have

not been able to derive the rate of _convergence for the scheme based on

[7]

and

[8],

but if

_Q

_e

is known and

_[7]

is used to estimate

_(T2i,

which should be a

_good

approximation,

then the rate of _convergence is 1 -

2R

2

(1 -

_R

_Z

_).

This is in

_good

agreement

with

_figure

_1,

_suggesting

_symmetry

about

R

2

= _{0.5 and}

_equality

_{of the}

speed

of convergence

when R

2

=

2/3.

As someone who has never understood the EM

_algorithm

or its

_popularity,

I would have

_thought

schemes based on some form of second differentials would be

more

useful,

especially

as some of the authors’ schemes allow

_negative

standard

deviations.

Simplicity

highlight

the need for a

_{heterogeneous analysis,}

both for

fitting

the data and

measuring

the

possible

loss of response.

Utility

I can see the use of these methods at a

phenotypic level,

but I am less clear if it is

realistic to

attempt

to detect differences at a

_genetic

level

(Visscher, 1992).

At the

simplest

level if the residuals are

heterogeneous

can one

_{realistically}

discriminate

between models with

_homogeneous

genetic

variances or

_homogeneous

heritabilities ?

Additional references

Anderson

DA,

Aitkin MA

₍₁₉₈₅₎

Variance _component models with

_binary

response:

interviewer

variability.

J R Stat Soc B

_47,

203-210

Groeneveld E

₍₁₉₉₄₎

A

_{reparameterization}

to

_improve

numerical

optimization

in

multi-variate REML

_(co)variance

component estimation. Genet Sel

Evol 26,

537-545

Lindstrom

MJ,

Bates DM

₍₁₉₈₈₎

Newton-Raphson

and EM

_algorithms

for linear mixed-effects models for

repeated

measures data. J Am Stat Assoc _83, 1014-1022

Thompson R, Crump RE, Juga J,

Visscher PM

₍₁₉₉₅₎

_Estimating

variances and

covari-ances for bivariate animal models

using

scaling

and transformation. Genet Sel

Evol 27,

33-42

Reverter

_A,

Golden

BL,

Bourdon

_RM,

Brinks JS

₍₁₉₉₄₎

Method R variance components

procedure: application

on the

_{simple breeding}

value model. J Anim Sci

72,

2247-2253

Visscher PM

₍₁₉₉₂₎

Power of likelihood ratio tests for

_{heterogeneity}

of intraclass

correla-tion and variance in balanced half-sib

_designs.

J

_Dairy

Sci

_75,

1320-1330

COMMENT

Daniel Gianola

Department

of

Meat and Animal

_{Science, University}

_of

Wisconsin-Madison,

USA

This _paper reviews and extends

_previous

work done

_by

the senior author and collaborators in the area of inferences about

heterogeneous

variance

_components

in animal

_breeding.

The authors

_impose

a model on the variance

_structure,

and

suggest

estimation and

_{testing procedures,}

so as to arrive to

_explanatory

constructs that have as few

_parameters

as

_possible.

Their

approach

is

_systematic,

in contrast to

other

_suggestions

available in the literature

_(some

of which are referenced

_{by Foulley}

and

_G!uaas),

where a model on the variances is

_adopted

implicity

without reference to the amount of

support

that is

_{provided by}

the data at

_hand,

and often on the

basis of mechanistic or ad hoc considerations. In this

front,

their

developments

are

welcome.

They

adopt

either a likelihood or a

_{Bayesian viewpoint,}

and restrict themselves

to

_conducting

a search of the

_appropriate

maximizers

(REML

or

_{posterior modes,}

respectively)

via the EM

_algorithm,

which is

_implemented

in a

clear, straightforward

way. In so

_doing,

_they

arrive at a formulation

(’scaled’

EM)

which exhibits a

different numerical behavior from that of the ’standard’ EM in a

simple

model.

at

_{estimating equations}

that are similar to those of Gianola et al

_(1992),

cited

_by

the authors. It is now well known that

_setting

all

degree

of belief

_parameters

to

zero leads to an

_{improper posterior distribution;}

_{surprisingly,}

the authors

(as

well as Gianola et

al,

1992)

do not alert the readers about this

pitfall.

The authors do not

_provide

measures of the curvature of the likelihood or of the

pertinent posterior

distribution in the

_neighborhood

of the maximum. It is useful

to recover second derivatives either to

_implement

_fully

the maximum likelihood

analysis,

or to

_approximate

the

_joint

_{(or marginal)}

_posterior

with a Gaussian distribution. Gianola et al

₍₁₉₉₂₎

gave

expressions

for second differentials for some

simple

heterosckedastic

_models,

and an extension of these would have been an

interesting

contribution.

Their

_{testing procedure}

relies on

_{asymptotic properties}

of the

_(marginal)

like-lihood ratio test. I wonder how accurate this test is in situations where a

het-eroskedastic model of

_high

_{dimensionality}

may be warranted. In this

situation,

the

asymptotic

distribution of the test criterion _may differ

_drastically

from the exact

sampling

distribution. It is somewhat

_surprising

that the authors do not discuss

Bayesian

test and associated methods for

_{assessing uncertainty;}

some readers _may

develop

the false

_impression

that there is a theoretical vacuum in this domain

_(see,

for

example, Robert,

1992).

There is a lot more information in a

_posterior

(or

normalized

likelihood)

than

that contained in first and second differentials. In this

_respect,

an

_{implementation}

based on Monte-Carlo Markov chain

_(MCMC)

methods such as the Gibbs

_sampler

(eg,

Tanner,

1993)

can be used to estimate the whole set of

_posterior

distributions in the _presence of

_{heterogeneous}

variances. In some

_simple

heteroskedastic linear

models it can be shown that the random walk involves

_simple

chains of normal

and inverted

_chi-square

distributions.

_Further,

it is

_possible

to arrive at the exact

(within

the limits of the Monte-Carlo

_error)

_posterior

distributions of linear and nonlinear functions of fixed and random effects. In the

sampling

theory

framework

one encounters

_immediately

a Behrens-Fisher

_problem,

even in a

_simple

contrast between ’treatments’. The

Bayesian

approach

via MCMC would allow an exact

analysis of,

for

_example,

breed

_comparison

_experiments,

when the sources of

germplasm

involved have

_{heterogeneous}

and unknown

_dispersion.

I have been

_intrigued

for some time about the

_possible

consequences of hetero-geneous covariance matrices in animal evaluation in a

multiple-trait

analysis.

If

there is

_{heterogeneous}

variance there must be

_{heterogeneity}

in covariances as well !

Perhaps

the consequences on animal

ranking

are even more subtle than in the

uni-variate case. It is not obvious how the structural model

_approach

can be

_generalized

here,

and this is a

_challenge

for future research.

Additional References