A refined saddle point approximation

A refined saddle point approximation

T H O M A S ~-~ O G L U N D

W e consider high c o n v o l u t i o n powers

f f * ( x ) = ~ f ( x l ) . . . f ( x , , )

Xl+ . . . +xn=x

o f a n a r b i t r a r y f u n c t i o n f: Z - + [0, ~ ) w i t h finite s u p p o r t S = {x E Z I f ( x ) > 0}, a n d our aim is to o b t a i n a n a p p r o x i m a t i o n of fn*(x) which is s h a r p for all x C Z.

I t will be easy t o see t h a t o u r a p p r o x i m a t i o n below (properly i n t e r p r e t e d ) holds w h e n S is a one p o i n t set. To a v o i d trivialities we e x c l u d e this ease f r o m t h e re- m a i n d e r o f t h e t e x t . L e t x a n d 2 s t a n d for t h e smallest a n d largest p o i n t o f S, respectively, a n d let I d e n o t e t h e c o n v e x hull of S considered as a subset o f R, . _ x = m i n S , 2 = m a x S , I ~ l-X, 2].

I n o r d e r t o be able to give t h e a n n o u n c e d a p p r o x i m a t i o n we h a v e t o require that the support of f n , is a convex subset of Z for all n sufficiently large. This is the case i f and only i f both -X + 1 and 2 - - 1 belong to S. F o r if 2 - - 1, say, does n o t belong to S t h e n n2 - - 1 $ supp i f * for all n, a n d henc~ t h e s u p p o r t o f f n . n e v e r becomes a c o n v e x subset of Z. (Note t h a t t h e integer i n t e r v a l [-xn, 2hi is t h e c o n v e x hull o f s u p p f " * considered as a subset of Z.) Conversely, if x + 1 a n d 2 - 1 belong to S t h e n supp f " * contains all points of f o r m

]ClX @ ]c2(_x @ 1) @ ]ca(2 - - l) @ k42 = n x + (k a @ k4)(2 -- x -- l) + k 2 + k4, w i t h k 1 .. . . , k~ n o n - n e g a t i v e integers satisfying /c 1 + . . . + k 4 = n. T h a t is, s u p p f ~ contains all points of f o r m n _ x + h l ( 2 - - x - - 1) + h 2 , w i t h 0 < h l < n , 0 < h g . < n , a n d h e n c e s u p p f ~ = [nx, n2] for all n > m a x ( 1 , 2 - - x - - 2 ) .

L e t /, s t a n d for t h e m e a s u r e on Z which assigns t h e weight f ( x ) to t h e p o i n t x,

#(E) = ~ f(x).

x c E

T h e a p p r o x i m a t i o n will be i b r m u l a t e d in t e r m s of q u a n t i t i e s which are n a t u r a l l y t i e d to t h e f a m i l y of p r o b a b i l i t y distributions whose densities r e l a t i v e to t h e m e a s u r e

# consist of t h e closure o f t h e e x p o n e n t i a l f a m i l y p,(x) = e"~/cf(a), a C R. H e r e

174 T H O M A S /-IO G L U N D

~(a) = ~ e"~f(x) is t h e L a p l a c e t r a n s f o r m of f. T o c o m p l e t e t h e f a m i l y we t h u s h a v e to a d d t h e t w o p r o b a b i l i t y d i s t r i b u t i o n s given b y t h e densities

p_+(x) = lim pa(X) = ( ~ x x / f ( x ) a n d p+~(x) = lim p~(x) ~-- (~;,/f(2)

(the K r o n e c k e r delta).

T h e a b o v e - m e n t i o n e d q u a n t i t i e s are a m o n g t h e following ones. T h e meanvalue

t h e variance

f d

m a : xpa(x)d#(x) = da log qJ(a),

f

^d2

V a - - - - (X -- m.)2pa(x)d/u(x) ~- ~a2 log q~(a),

a n d t h e entropy

= - - f p a ( X )

H= log pa(x)d#(x)

J = log ~(a) - - am~.

I f we n o t e t h a t m + : _ x , m + ~ - - - - 2 a n d d m a / d a = v ~ > O for a C R , we see t h a t t h e m e a n v a l u e m a p s 1~ o n t o I in a one to one m a n n e r . T h e f o u r t h q u a n t i t y we will n e e d is t h e maximum likelihood estimator, d = m -1, which is a one to one m a p p i n g o f I o n t o R. I t s n a m e comes f r o m t h e i d e n t i t y

p3(x)(x) = m a x pa(x).

aeR L e t us f i n a l l y i n t r o d u c e t h e a n a l y t i c f u n c t i o n

i f

~(~) - - 2 ~ e x p [ Z ( e ~ - - 1 - - i ~ ) ] d ~ :

The function ~(2), ~ ~ 0, is strictly positive and satisfies

e(0) = ~ _> e(~) = ( 2 z ~ ) - ' / 2 ( ~ § O ( U ~ ) ) , as ~ - ~ ^{~ .} This will be p r o v e d a t t h e e n d o f t h e paper.

I t is clear t h a t f f * ( x ) = 0 w h e n x / n ~ I , a n d t h a t f " * ( x ) > 0 w h e n x / n C I , p r o v i d e d n _> m a x (1, 2 - - x - - 2). T h e local central limit t h e o r e m says

f~*(x) = cH~189 [-- 89 -- mo)2/Vo] ~- O(n-1))

u n i f o r m l y in x E Z, a n d h e n c e it tells us n o t h i n g m o r e t h a n f~*(x) = O(e"H~

e x c e p t w h e n x/n belongs t o a s u b i n t e r v a l of I o f l e n g t h O(~loog n/n) (centred a r o u n d m0). R i c h t e r [3] gave a n a p p r o x i m a t i o n w h i c h holds w h e n x/n belongs to a s u b i n t e r v a l of l e n g t h o(1). B u t t h e r e are still b e t t e r results. So-called saddle

A ~ E F I ~ E 9 S A m ) ~ P o I ~ T APP~OXIMATIO~ ¹⁷⁵ p o i n t a p p r o x i m a t i o n s h a v e long since b e e n u s e d in s t a t i s t i c a l m e c h a n i c s . A rigorous r e s u l t of t h a t k i n d w a s g i v e n b y M a r t i n - L S f [2]:

f"*(x)

= e x p

[nH~(x/n)](2~nVa(x/,))-}(1 q- O(1/n)),

u n i f o r m l y in x 6 Z as

x/n

is w i t h i n b u t s t a y s a w a y f r o m t h e b o u n d a r y o f I . I t is also clear t h a t

f"*(x) = f(x/n F

= e x p

[nH~(,/,)]

w h e n

x/n

b e l o n g s t o t h e b o u n d a r y of I .

T h e r e m a i n i n g g a p is filled in b y

the refined saddle point approximation.

THEOREM. A s n

---> oo,

ff*(x)

= e x p

[nH~(~l,)]~(nv~(~/~))(1 ~- O(lln)) uniformly for x/n 6 I , and fn*(x) = 0 when x/n ~ I.

I t will follow f r o m t h e details b e l o w t h a t t h e s t a t e m e n t a b o v e is still t r u e if we r e p l a c e t h e e r r o r

O(1/n)

b y O(min

(nv~(~/,O,

2 I/n)).

T h e p r o o f s t a r t s w i t h t h e i d e n t i t y

1

j e-(a+ia)xq~( a -~ i~)'~d~.

f f * ( x ) -

^2~

W e p u t a =

d(x/n)

a n d c o n c l u d e

1 ^{f D}

J r~c~/,)(~Fd~,

ff*(x) =

e x p

[nH,~(,I,) ] G

w h e r e

W e h a v e to s h o w t h a t

= f ).

Z

u n i f o r m l y in a f o r - - oo < a _~ oo a n d a of t h e f o r m a =

d(x/n).

T h e p r o o f will be s e p a r a t e d i n t o t w o eases: 0 < a < oo a n d - - oo < a < 0 , a n d t h e second case will be o m i t t e d since it is quite a n a l o g o u s to t h e f i r s t one. T h e w o r d c o n s t a n t (in f o r m u l a s w r i t t e n Const.) will be u s e d for n u m b e r s w h i c h do n o t d e p e n d on a, ~, or n.

176

Le~

Then c 2 ( a ) ~ - v a and

T E O ~ A S ~ O G L U N D

c~(a) = f (x - ~ o ) % ( x ) @ ( x ) . Z

~o(~) = 1 + ~ - - ~ ok(a).

L E P T A 1. Ick(a) - - ( - - 1)ke-af@ -- 1)/f@)l ~ Cke -2a, ]c - ~ 2, 3 . . . . f o r s o m e c o n s t a n t C > 1 a n d a l l a > 0 .

This and the lemma below will be proved at the end of the paper. P u t

(i~)k

ra(~) ~- ~ ~-. (ok(a) - - ( - - ])kVa), k ~ 3

I t follows from the lemma t h a t

]Ck(a ) - - ( - - ])kval ~ 2Cke -2a ~ Const. Ckv2,,

and hence also that f%(~)l--~Const, v2,1~l 3, for l~l < ~ . This motivates the decomposition

~a(~) = ~ § v~ - ~ - 1 § i~,) § ra(O,).

Note t h a t r~(~) is negligible compared to the other terms not only when v~ is small b u t also otherwise provided lal is small. Small ~ give the main contribution to

7r n

f _ = y,(~) d~ in the latter ease.

Introduce the abbreviations

~(i~) ~

= ~ ( ~ ) , q = Va(e - ~ - - 1 + i~,), r = to(c,), s = ~ (c~(a) - - ( - - U~vo), k ~ 4

and M ~ max (]Yl, le~-ll, leU+l~ll, e-"a~/2" ~ ).

Then 7 ~ l § 2 4 7 r and

y~ __ e ~ = (yn _ en(v-1)) § (en(v-1) __ e.q( 1 § nr))

(This complicated decomposition is unnecessary if we are content with the error O ( n - 8 9 instead of O(]/n).) The inequalities

ix ~ - - y~] < n i x - - Yl (max (/xl, l y l ) ) n - 1 and le ~ - - 1 - - x l ~ Ixl:elXl/2 together with the familiar estimate l Y - l l ~ v,~2/2 yields

A R E F I N E D S A D D L E P O I N T A P P R O X I I ~ f A T I O N 177

< n M , - 1 ] y _ d - i ] < n M ~ 1]y ~ 112d-1/2 < Const. M " ( n v , s 2 ) 2 / n f o r Is[ ~< :~.

(Note t h a t supava < co, a n d hence also 1 / M <_ expv~s2/2 <_ Const.) I n similar w a y we o b t a i n

]e "(r-l) - - e"q(] + nr)[ < Const M~(nv,s2)3/n a n d

[nr(e ~q - - exp (-- nvaa2/2))] <_ Const. M~(nv~a2)a/n f o r I ~[ < z . (Here we used t h e i n e q u a l i t y ]e i~ - - 1 - - i s - - (is)2/2] _< ls[z/6.) F i n a l l y

Ins exp (-- nv~s2/2)] <_ Const. M~(nVaS2):/n.

(The e s t i m a t e Is[ < Const. v:a[s] 4 is a consequence of l e m m a 1.)

a

LEMMA 2. T h e r e are p o s i t i v e constants e a n d 6 such that M < exp (-- e:xZv.) f o r all - - ~ ~ a < ~ a n d - - :r < s < :~ s a t i s f y i n g v~[s I < d.

W e use t h e fact f ~ = (r -- s) exp (-- nVaS2/2)da = O, m a k e t h e s u b s t i t u t i o n s --> -- s in the integral representation of ~(2), a n d split the domains of i n t e g r a t i o n in t h e expression D into two parts: Is] _< rain (~, ~/Va) a n d 5/Va < IS I __< :r. The result is

D _< Const. g(nv~)/n + f (lye(s)[" + [exp (e i~ -- 1 is) p~v~)ds,

~/v~ <_ 1~[ _<

where

/ ,

g(a) = / e - ~ ' [ ( a ~ y + (a~Y]&.

I t is clear t h a t g(2) < Const. (22 -[- 2a), 2 ~ 0, a n d t h e s u b s t i t u t i o n s - + ~ 2 89 shows t h a t g(2) < Const. 2-89 Hence g(2) < Const. rain (1, 22)Q(~) for all 2 > 0.

The second t e r m in t h e s u m d o m i n a t i n g D is non-zero o n l y if the d o m a i n of i n t e g r a t i o n is n o n - e m p t y , i.e. o n l y w h e n v~ > 6/x. According to l e m m a 1 this implies t h a t a belongs to a compact subset, K , of R. W e also k n o w t h a t r / = i n f ~ 5 / v , > O, a n d it is wellknown t h a t [ya(S)] < 1 for all a C R a n d o < I~l < ~.

(a,

~) - +

r-(~)

(See l e m m a 3, p. 475 of Feller [1].) I t is easy to see t h a t t h e function is continuous,

([>( ~ ) - r # ) [ _< f

^Ip~ - pb(x)Ir

+ b, - ;~ I),

a n d hence also

sup

a. C K

]7~(~)[ < 1.

1 7 8 THOMAS H 6 G L U ~ D

T h e e s t i m a t e

(valid w h e n d/v~ < Is l < ~r) f i n a l l y shows t h a t t h e r e is a integral in q u e s t i o n is d o m i n a t e d b y

Const. e - ~ < Const. e-~'~? < Const. rain (1, (nv~)~)~(nv,)/n, ( R e m e m b e r t h a t Va ~ 5/~7"~.)

Proof of lemma 1. W e p o i n t o u t t h e fact t h a t

[e~l = e ~ ~ = e x p ( - v~(1 - - c o s s ) ) _< e - ~ ( : . . . . ~)/~

$ > 0 such t h a t t h e

a S

where ~" < ~/max~ v,.

> po(x) = ~-~ + o(e-~

a - ~ ~ . I t is clear t h a t m a = 2 - ~ O ( e - ~ ) . Also

]ck(a ) - - ( - - 1 ) U e - a f ( 2 - 1)If(2)[ <_ ] ( 2 - ma)~po(2)[ +

x - 2

+ 1(2 - : - m o ) k p o ( 2 - - ~) - ( - 1 ) % - ~ f ( 2 - 1 ) / f ( 2 ) 1 + I ~ (x - m . ) k p a ( x ) ] .

T h e f i r s t e x p r e s s i o n t o t h e r i g h t is d o m i n a t e d b y (Const. e-a) k, t h e t h i r d b y (Const.)ke -2a, a n d t h e second b y

](2 - - 1 - - ma) ~ - - ( - - 1)klpa(2 - - 1) -~ Const. e -2a

< (Const.) k ]2 -- m~Ie -a -~ Const. e -2a < (Const.) ~ e -2a.

T h e l e m m a follows.

Proof of lemma 2. W e h a v e

R e (y~(s) - - 1) = R e V a ( e - i ~ - - 1 ~ is) + R e r,(s) < -- Va(1 - - cos s) -~-

]ra(S)]

< - - v,s22/• 2 + Const. v:~lsl a < - - v,s'~/10,

p r o v i d e d [sl ~< x a n d v, la I is s u f f i c i e n t l y small. H e r e we h a v e used t h e i n e q u a l i t y 1 - t o s s _>~e2/~2, v a l i d for [~[ _<~. These estimates t a k e care of ]e~-:[ a n d [e~+lrt].

I n o r d e r to e s t i m a t e [y~(s)] we n o t e t h a t Jlog (1 ~- z) - - z] < ]z[ 2 for lzJ < 1 B u t lya(~) - - iI ~< v~x2/2 <_ 1 for I~l < z a n d Val~l ~< l/Jr. H e n c e

]ya(S)[ = le~~ _< [exp (y - - i ~- V2a~4/4)[ <

e x p ( - - G~2/lO -~ v2,~4/4) < e x p ( - -

Va~2/20),

p r o v i d e d Is[ ~ z a n d v,]~l is s u f f i c i e n t l y small.

The function o(~). I t is clear t h a t le(2)l _< Q(0) = 1 for ~ >_ 0. I n o r d e r to show t h a t ~ ( ~ ) > 0 for 2 > 0, we n o t e t h a t

~(~) = e-~ e~(k-~)d~.

A R E F I N E D SADDLE POINT APPROXIlVfAT~ON 179 H e n c e

for 4 = 0, 1, 2 , . . . , and

otherwise.

4 C R - - Z .

o(2)=e - ~ . > o

4 k sin zt(2 -- k) e-~o~ k! ~(2 -- k)

B u t sin ~(2 -- k) = (-- 1) k sin ~2, a n d sgn (sin ~4) = (-- 1) [~d for I t therefore follows from l e m m a 3 t h a t ~(4) > 0 also for 0 < 4 $ Z.

L]~MMA 3. P u t

S(4) k=o )-[' k! 2 - - k ' 4 C R - - Z .

T h e n S(2) > 0 i f [4] is even a n d non-negative, a n d S(2) < 0 otherwise.

Proof. Observe t h a t

~ { ( - - 2) 2k 1 (-- 2) 2k+1 1

S ( 2 ) = ~ 0 ~ (2k~. 2 - - 2k @ (2k @ 1)! 2 - - 2 k - - 1) --

22k(1 1 )

= - k = o ~ ~ 2 k + 1 + (2 - 2k)(2 - 2k - 1) '

a n d t h a t ( 2 - - 2 k ) ( 2 - - 2 k - - 1 ) > 0 if a n d only if 4 - - 2 k < 0 or 2 - - 2 k > 1.

H e n c e S(2) < 0 if ( 2 - - 2 k ) ( 2 - - 2 k - - 1) > 0 2 < 0 or [2] is odd.

Similarly

1 ~ (((~k 2)2k+1 1 ( - - 2 ) 2k+2 1 )

S ( 2 ) = ~ +~o~ + 1)! 4 - - 2 k - - 1 + ( 2 k + 2)! 2 - - 2 k - - 2 =

1 ~ 42k+1 ( 1 1 i)

- 4 +~0= ( 2 k + 1 ) ! 2 k + 2 + ( 4 - - 2 k - - 1 ) ( 4 - - 2 k - - 2 ' a n d hence S(2) > 0 if [2] is even a n d non-negative. The l e m m a follows.

Since

(2~4)-89 = - - e- 89 + 2(i~)a/6)d~

2~ R

it remains to show t h a t the difference b e t w e e n this integral a n d t h e integral defining

~o(2) is 0(2 -3/2) as 2---~ oo. I t should be clear t h a t t h e contribution to these integrals from values of ~ satisfying [~ I > 1 is less t h a n 0(2-a/2). P u t sk(~) = e '~ - - ~ 1 o (i~)J/j!. T h e n Isk(~)l < I~lk/k!, a n d

for all k = 0 , 1 , 2 , . . . , i.e. if

180 TI{O~AS I~6GLUND

i f i

[~().) - - ( 2 z A ) - k l ~-- ~ e - l - ~ ( e ~'~"(~)- 1 - -

A(i~)3/6)da +

0 ( 2 - 3 / 2 ) . E~[<a

T h e i n e q u a l i t y [e ~ - 1 - - yI ~ ] e~ - l - - x ] -~- I x - - y ] ~ [x]%l~1/2 ~- ]x - - yr, ap- plied to x--~ 2s3(~ ) and y ~-- A(i~)3/6, shows t h a t t h e i n t e g r a n d to t h e r i g h t is d o m i n a t e d b y

[ ( ~ ) ~ e -:'~',:~ +

(~o~)~e-~:~]/,~

p r o v i d e d [~] ~ 1. H e n c e @(A) = (2n~)- 89 Jr 0(1/~)).

R e f e r e n c e s

1. FELLER, W., A n introduction to probability theory and its applications, Vol. 2. Wiley, New York, 1966.

2. MARTI~-LSF, P., Statistiska modeller, anteclcningar frdn seminarier 14s~ret 1969--70, ut- arbetade av 1~. Sundberg (in Swedish). University of Stockholm, Department of M a t h e m a t i c a l statistics.

3. RICHTER, W . , Local limit theorems for large deviations. Theor. Probability Appl., 2 (1957), 206--220.

Received September 3, 1973 T h o m a s H S g l u n d

Institut for M a t e m a t i s k StatiStik H . C. Orstsd Institut

Universitetsparken 5 D K - 2 1 0 0 K o b e n h a v n D a n m a r k