On a m a x i m i z i n g p r o b l e m with several applications in statistical theory

A R K I V F O R M A T E M A T I K B a n d 4 n r 2 9

C o m m u n i c a t e d 22 F e b r u a r y 1961 b y H . CRAM]tR a n d O. FROST~A~r

On a m a x i m i z i n g p r o b l e m with several applications in statistical theory

By

S U M M A R Y

For a certain distribution, the three problems of determining

(i) optimum spacings of observations in linear estimation of location and scale p~rameters (ii) optimum strata limits in proportionate sampling

(iii) optimum interval boundaries in grouping a sample are closely related to each other.

In section two of this paper, two basic lemmas are stated. Applied to problem (i) above, these lemmas enable us to make conclusions about the rate at which the efficiency of linear estimates increases with the number of observations used to form the estimate. Similarly, in problem (ii), statements can be made concerning the gain in accuracy due to increasing the number of strata.

Furthermore, the laborious procedure of calculating the optimum solutions of the above mentioned problems is replaced by a simplified approximation technique which will cause negligible loss of accuracy.

2. T w o l e m m a s

L e t H r (2), (r = 1, 2), be r e a l - v a l u e d f u n c t i o n s , d e f i n e d f o r 0 ~ 2 ~ 1, a n d f o r m t h e s u m s

k+l [ H , (2,) - H , (2,_1)] [ H , (2,) - H , (25-1)]

K, (21, ..., 2~)-- Y~

T w o g r o u p s of c o n d i t i o n s a r e i m p o s e d u p o n H r (2), (r = 1, 2), a n d its deri- v a t i v e s .

A1. T h e f u n c t i o n s H r (2), ( r = 1, 2), a r e c o n t i n u o u s f o r 0 ~<2-~< 1 a n d t e n d t o z e r o as 2 - + 0 a n d 2 - > 1 .

A2. T h e d e r i v a t i v e s H v (2), ( r = 1, 2), of o r d e r 1 . . . 5 a r e c o n t i n u o u s a n d l i m i t e d f o r 0 < 2 < 1.

A3. T h e i n t e g r a l s

f

e~ = H ; ( 0 s ( 0 d t , (r, s ; 1 , 2), (2)

o e x i s t a n d e n e29. - e~2 > 0.

C.-E. S.~RNDAL, On a maximizing problem with several applications A4. e~x [H~' (2)] 2 - 2 el~ ' HI" (2) H~' (2) + e~ [H~" (2)3 2 =P 0 for all 2; 0 < 2 < 1 .

A5. The integral

c = f ~ {exl [H~' (t)] * - 2 el, H i ' (t) H~' (t) + e~ [ H i ' (t)]2} } d t (3) exists.

I n t h e second group of conditions, the subscript r can t a k e on either of the values 1 and 2.

B1. The function H , (2)is continuous for 0~<2~< 1 a n d tends to zero as 2-->0 and 2--> 1.

B2. The derivatives H~ (2) or order 1 . . . . , 5 are continuous a n d limited for 0 < 2 < 1 .

]33. The integral

e . = jo" [H~ (0] 2 d t exists and err > O.

B4. H ~ ' ( 2 ) # 0 for all 2; 0 < 2 < 1 . ]35. The integral

c~ = I ~ [Hr' (t)]t dt (4)

J o exists.

Also, define

z r ~ - k + 1, i ( i = 0 , 1 . . . . k , k + l ) . (5) We can now state the two l e m m a s (although the proofs are o m i t t e d here).

b m m a 1. I/ conditions A 1 - A 5 are /ul/illed, then the determinant

= K l l K12 K (21 . . . 2k) /~21 K22 attains a maximum /or

2 , = Q ( e r , ) + O ( k + I ) 2, ( i = l , . . . , k ) , (6) where ~ is defined by (5) and

Q (y) = M -1 (y) is the inverse /unction o/

1 ( ~ [ 2 (t)]~-2el2H1 (t)H~

M (Y)=c,3o (ell H . . . . . . (t)-4-e2~[H'l" (t)]~}t dt, where c is given by (3).

3 8 6

ARKIV FOR MATEMATIK. Bd 4 nr 29 Furthermore, the m a x i m u m value o/ K is

Kraax ~ e l l e22 -- e122 -- c 3 12 (k + I) ~ The value taken by K /or

+ o (k + 1) -~. (7)

~, = Q ( ~ ) , (i = 1 . . . k), (8)

can also be written in the /orm (7).

L e m m a 2. I / conditions B 1 - B 5 with r = 1 or 2 are /ul/illed, then the s u m K , , ( r = 1 or 2), attains a m a x i m u m /or

2 , = Q , ( ~ ) + 0 ( k + l ) -a, ( i = 1 . . . k ; r = l or2), (9) where ze~ is defined by (5) and

is the inverse /unction o/

Qr (y) = M ; 1 (y)

M~(y) = 1 f~ [H'r' (t)]'

The m a x i m u m value o/ Krr is grr, max : err The value o/ Kr, /or

d

12 (k +

2~ = Qr (rot), (i = 1 . . . k; r -~ 1 or 2), can also be expressed in the form (10).

( l l )

3. O p t i m u m spacing in linear e s t i m a t i o n

A large sample of n observations from a population with a continuous c d f of t y p e F [ ( z - ~l)/:q] is available. The u n k n o w n parameters ~1 a n d ~ are to be estimated b y linear estimates using only k o u t of the n observation~ n a m e l y

z(~0 <z(~,) < ... <z(~). (12)

These are to be optimally selected. I f x = ( z - ~ l ) / a 2 , we have

E x(.,) = x~ + 0 (n- t),

where x~=G(2~), 2 ~ = n ~ / / ( n + l ) , a n d G ( x ) = F - l ( x ) is the inverse of F.

C.-E, S~-RNDA.L, On a maximizing problem with several applications Also define

HI (4) = / [G (~)],

H~ (4) = G (4) / [G (4)], ⁽¹³⁾

where / ( x ) = F ' (x) is the density function.

The general form of linear unbiased estimates al and a2, based on the k sample quantiles (12), was given b y 0 g a w a [1]. We are now going to investigate the asymptotic efficiencies of these estimates. Let Kr~ and er~, (r, s = 1, 2), be defined b y (1) and (2), respectively, with Hr (2), (r= 1, 2), given b y (13). Three cases are distinguished.

Case 1. B o t h 0~ 1 and c~ 2 unknown.

The joint asymptotic efficiency of a~ and ~ is

K11K2~. - K122 E12 e l l e22 - e122 9

The quantities ~. which maximize El2 will be called the joint o p t i m u m spacings for ~1 and ~2, and the corresponding estimates will be termed the joint optimum linear estimates (JOLE) of ~1 and ~2. B y application of lemma 1, we have

T h e o r e m 1. I] Hr (2), (r = 1, 2),/ul/ills conditions A1-A5, then the joint optimum spacings /or ~1 and o~ are given by (6), and the joint asymptotic e//iciency o/the JOLE o/ o~ 1 and o~ 2 is

1 c 3

EI~=I 1 2 ( k + 1)~ elt %2_ e~ + ~ ( k + 1) -~"

Cases 2 a and 2b. 0 n l y one of the two parameters at, ( r = 1 or 2), is un- known; the remaining one is known.

The asymptotic efficiency of a*, (r= 1 or 2), is then Er =Kr-~, (r= l or 2).

err

The quantities )~ which maximize Er will be called the optimum spacings for

~r, and the corresponding estimate will be called the optimum linear estimate (OLE) of ~T- From lemma 2 we conclude

Theorem 2. I / Hr (~t), (r = 1 or 2), /ullills conditions B1-B5, then the optimum spacings /or o~ r are given by (9) and the asymptotic e//iciency o/ the OLE o/ar is

E r = l 1 c~ ~_o(k+l)_2, ( r = l or 2).

12 (k + 1) 2 err

Under conditions, slightly more rigorous t h a n those needed in proving lemmas 1 and 2, one can also state

3 8 8

AI~KIV FOR MATEMATIK. Bd 4 nr 29 Theorem 3. I n order to obtain an asymptotically e/ficient estimate o/ ~1 (~2 known) or 0% (~I known), or joint asymptotically e/]icient estimates o] ~1 and ~ , it is su]/icient to utilize only k = k (n) out o/ the n observations available, where, as n--->oo, k ( n ) - > o o in the weakest possible way. (This means that, when n-->oo, the relative number o[ observations k (n)/n-->O.)

For small values of k, optimum spacings were laboriously calculated by Ogawa [2] and Sarhan, Greenberg and Ogawa [3] in the following estimation cases :

(a) normal distribution, a 1 unknown, ~2 known, (b) normal distribution, ~2 unknown, ~1 known, (c) exponential distribution, g2 unknown, a 1 known.

A simplified method will be described here. Comparison of formulas (9) and (11) suggests t h a t an estimate which m a y be termed the nearly optimum linear estimate (NOLE) of ar can be based on the spacings (11). These latter will be called the nearly optimum spacings for at. Likewise, the nearly joint o p t i m u m estimates (NJOLE) of a 1 and a 2 will be defined as the estimates which are based on the nearly joint optimum spacings (8). (Asymptotically, the efficiencies of the N J O L E and the N O L E behave according to the formulas of Theorems 1 and 2, respectively.)

To illustrate the advantage of the NOLE, we choose the estimation cases (a) and (c) mentioned above. I n fact, the nearly optimum spacings are:

case (a):

; t , = r 1 6 2 ( i = 1 . . . . , k), where

and r its inverse;

(x) = ( : ~ e-t'/2dt J _ ~ V2 :~

case (c): A~ = 1 - (1 - n ~ ) a, (i = 1, . . . , k), where g~ is defined b y (5).

The easily calculated nearly o p t i m u m spacings yield estimates slightly less efficient t h a n the OLE's, a fact which is illustrated by Table 1. (The effi- eiencies of the OLE are taken from [2] and [3].)

Table 1. Asymptotic efficiency of the OLE as compared to the NOLE.

Estimation case

Asymptotic efficiency

k = 5 k = 1 0

OLE NOLE OLE NOLE

Normal, ~i (~2 known) ] 0.9420 0.9387 0.9808 0.9800

t

Exponential, a~ (al known) [ 0.9476 0.9448 0.9832 0.9826

C.-E. SA-RNDAL, On a maximizing problem with several applications

Still better results can be obtained, if we change the definition of the quantities

~ into

i - a

~ = k - a - b + 1' (14)

where the constants a and b can be properly chosen in each estimation case.

4. Optimum stratification in proportionate sampling

Consider a population with the frequency function /(x), defined over the interval A < x ~< B. The mean of x,

m = f ~ t / ( t ) d t , is assumed to exist.

means of the points

The population is to be stratified into ( k + l ) strata b y A =xo < xl < ^{. . .} <Xk<Xk+I=B.

I f proportionate allocation is used, and we omit the finite population cor- rection factor, then the variance of the usual estimate

can be written

k + l

~ = ~ P~ ~

~=1

v (~) = ~ . . . .

where n is the sample size and K l i is given b y (1) with H I (2) = t / (t) d t - m 2.

Compared to the spacing problem dealt with in the preceding section, the problem of this section is much simpler to handle. To meet conditions B1, B3 and B4, we need only presuppose t h a t /(x) is continuous and limited for A ~< x ~< B, and t h a t the variance of x exists. Under these assumptions we have

Theorem 4.

o~ 9 is

I/ conditions B2 and B5 are /ul/illed, then the minimum variance

V ( ~ ) ~ = 1 [12 ( - - ~ 1)~ + o (k + 1)-2], the optimum strati/ication points being

390

ARKIV FOR MATEMATIK. B d 4 n r 2 9

where 2~ is given by (9).

x, = a (~t~), (15)

F o r c e r t a i n d i s t r i b u t i o n s a n d for small v a l u e s of k, D a l e n i u s [4] a n d Z i n d l e r [5]

h a v e c a l c u l a t e d o p t i m u m s t r a t i f i c a t i o n points. One a t t e m p t t o get r e a s o n a b l e a p p r o x i m a t i o n s was b r i e f l y o u t l i n e d b y E k m a n [6]9

H e r e we define n e a r l y o p t i m u m s t r a t i f i c a t i o n p o i n t s a c c o r d i n g t o (15) w i t h hi given b y (11). F o r t h e p o p u l a t i o n / ( x ) = e -~, t a b l e 2 gives a c o m p a r i s o n b e t w e e n o p t i m u m a n d n e a r l y o p t i m u m s t r a t i f i c a t i o n points. These l a t t e r are g i v e n b y

x~ = 3 log 1 - ~ ' 1

a n d we h a v e chosen t o define g~ a c c o r d i n g t o (14) w i t h a=O, b = - 0 . 5 , cor- rections which are a p p r o p r i a t e in this case. (The resulting s t r a t i f i c a t i o n p o i n t s m a y s u i t a b l y be t e r m e d a, b-corrected n e a r l y o p t i m u m 9

Table 2. O p t i m u m a n d a, b-corrected n e a r l y o p t i m u m s t r a t i f i c a t i o n p o i n t s for /(x) =e -~ a n d t h e c o r r e s p o n d i n g variances.

Number of strata

O p t i m u m a , b - c o r r e c t e d 9 n e a r l y o p t i m u m

X 1

19 1.533

Optimum 1.018

a, b-corrected

nearly optimum 1.009

Optimum 09

a, b-corrected nearly optimum 0 9

Points of stratification

X2 ~3

2.611 2.542

1.772 3.365 1.763 3.296

n v (~) 09 09 09 0.1800 0.1090 0.1091

S o m e t i m e s t h e spacing p r o b l e m of section 3 a n d t h e s t r a t i f i c a t i o n p r o b l e m of this section y i e l d t h e s a m e o p t i m u m solutions 2~ (and also t h e same n e a r l y o p t i m u m solutions) 9 These cases can be s u m m a r i z e d as follows:

Distribution

Normal: ~ 1 e -89 {(z-~,)/~,]l

~2 V2

Spacing problem which is equivalent to the stratification problem

Estimation of ~1; c% known

G a m m a : - -1

~ F ( p ) [(Z -- ~1)/0~2] p - 1 e -[< . . . . )]g,] Estimation of ~2; ~ known9

C . - E . SARNDAL, On a maximizing problem with several applications 5. Determination of optimum interval limits in grouping

Here the problem is to group a sample from a certain distribution b y means o f optimally determined group limits.

Suppose first t h a t the grouped sample is used for estimation purposes. Kull- dorff [7], [8], [9], investigated the m a x i m u m likelihood estimates of al (:r known) and a2 (al known) in the normal distribution and of as (~1 known) in the exponential distribution. I n these cases, the asymptotic efficiencies of the m a x i m u m likelihood estimates are identical with the quantities E~ for the corresponding spacing problem. Therefore, nearly optimum group limits can be derived by use of the technique described in section 3.

Cox [10], using a different approach, assumed t h a t the grouping is done merely for convenience of exposition. B y means of a quadratic loss function, the total loss of information due to grouping can be expressed. The problem of mini- mizing this loss can be directly transferred into the stratification problem of section 4, and the approximation technique used there can be applied for determining nearly optimum group limits.

6. Remarks

A detailed discussion of the results given in this paper will be presented sub- sequently. Special emphasis will be put on the spacing problem, and attention will also be paid to those frequently occurring estimation cases, where one or more of the conditions imposed in section 2 fail to be fulfilled.

This work was sponsored b y the U.S. A r m y Research Office.

R E F E R E N C E S

1. OGAWA, J . , C o n t r i b u t i o n to t h e t h e o r y of s y s t e m a t i c s t a t i s t i c s , I. O s a k a M a t h . J o u r n a l , Vol. 3 (1951).

2. - - , C o n t r i b u t i o n to t h e t h e o r y of s y s t e m a t i c s t a t i s t i c s , I I . O s a k a M a t h . J o u r n a l , Vol. 4 (1952).

3. SARHAN, A. E . , GR~.~.~BERG, B. G., a n d OaAWA, J . , Simplified e s t i m a t e s for t h e e x p o n e n t i a l d i s t r i b u t i o n . S u b m i t t e d to A n n . M a t h . Star.

4. D~u~ivIUS, T., S a m p l i n g in S w e d e n . S t o c k h o l m , 1957.

5. ZINDLE~, H . J . , ~ b e r F a u s t r e g e l n z u r o p t i m a l e n S c h i c h t u n g bei N o r m a l v e r t e i l u n g . Allgem.

Stat. A r c h l y , 40 (1956).

6. EKMAN, G., A p p r o x i m a t e e x p r e s s i o n s for t h e c o n d i t i o n a l m e a n a n d v a r i a n c e o v e r s m a l l i n t e r v a l s of a c o n t i n u o u s d i s t r i b u t i o n . A n n . . M a t h . Star. 30 (I959).

7. KULLDORFF, G., M a x i m u m likelihood e s t i m a t i o n of t h e m e a n of a n o r m a l r a n d o m v a - riable w h e n t h e s a m p l e is g r o u p e d . S k a n d . A k t u a r i e t i d s k r i f t 41 (1958).

8. - - , M a x i m u m likelihood e s t i m a t i o n of t h e s t a n d a r d d e v i a t i o n of a n o r m a l r a n d o m v a r i - able w h e n t h e s a m p l e is g r o u p e d . S k a n d . A k t u a r i e t i d s k r i f t 41 (1958).

9. - - , M a x i m u m likelihood e s t i m a t i o n for t h e e x p o n e n t i a l d i s t r i b u t i o n w h e n t h e s a m p l e is g r o u p e d . M i m e o g r a p h e d , L u n d , 1958.

10. C o x , D. R . , N o t e o n g r o u p i n g . J . A m . Star. Assoc. 52 (1957).

392

Tryekt den 17 november 1961 Uppsala 1961. A]mqvist & Wiksells Boktryckeri AB