EX S=~(al, a2, a3 .... ), EX=lim ~ ~ X (ak) EX Case Institute of Technology, Cleveland, Ohio, U.S.A.Q) COMPUTABLE PROBABILITY SPACES

COMPUTABLE PROBABILITY SPACES

BY

B A Y A R D R A N K I N

Case Institute of Technology, Cleveland, Ohio, U.S.A.Q)

1. Introduction

One use of t h e a x i o m s a n d d e f i n i t i o n s of t h i s p a p e r is f o u n d in t h e s t u d y of c e r t a i n subclasses of r a n d o m v a r i a b l e s , for e x a m p l e t h o s e d e f i n e d on a p r o b a b i l i t y space (R, B, P ) , R real, t h a t a r e f i n i t e a n d c o n t i n u o u s a l m o s t e v e r y w h e r e as c o n t r a s t e d w i t h f i n i t e a n d a l m o s t c o n t i n u o u s (i.e. m e a s u r a b l e ) . W i t h t h i s a v e n u e o p e n all e x p e c t a t i o n s E X of s o m e i m p o r t a n t subclasses of r a n d o m v a r i a b l e s can be e v a l u a t e d as a s y m p t o t i c a v e r a g e s :

I N

E X = l i m ~ ~ X (ak) (1)

on a single f i x e d sequence S=~(al, a2, a3 . . . . ), o r o b t a i n e d as s i m p l e e x t e n s i o n s of such e v a l u a t i o n s . I n o r d e r t h a t t h e s e subclasses can b e s t u d i e d w i t h o u t considering t h e i r b e h a v i o r off t h e f i x e d sequence S w h i c h b e h a v i o r seems i n c o n s e q u e n t i a l from t h e p r o b a b i l i s t i e p o i n t of v i e w - - i t is n e c e s s a r y t o i n c l u d e t h e m in a s e t t i n g m o r e g e n e r a l t h a n m e a s u r e - t h e o r e t i c p r o b a b i l i t y t h e o r y . This is t h e ease b e c a u s e t h e class of f u n c t i o n s for which (1) converges on a sequence S m a y be m o r e g e n e r a l t h a n a n y class of r a n d o m v a r i a b l e s on a p r o b a b i l i t y space of t h e f o r m (S, B, P ) for w h i c h (1) is t h e i r m a t h e m a t i c a l e x p e c t a t i o n . T h e m o t i v a t i o n s for t h i s a p p r o a c h are as follows.

H i s t o r i c a l l y a n d i n t u i t i v e l y t h e e x p e c t a t i o n E X of a r a n d o m v a r i a b l e X is e q u a t e d w i t h a n a s y m p t o t i c a v e r a g e of t h e f o r m (1), t h e v a r i a b l e X b e i n g c o n s i d e r e d as a f u n c t i o n on a sequence S ~ (al, a2, a3 . . . . ) of e l e m e n t a r y e v e n t s or s a m p l e p o i n t s . This e q u a l i t y is r e c o v e r e d in t h e s t r o n g l a w of large n u m b e r s a n d t h e ergodie t h e o r e m of m o d e r n m e a s u r e - t h e o r e t i c p r o b a b i l i t y . I n t h e m o d e r n version, of course, t h e p o i n t s % are o b t a i n e d as i m a g e s (1) Acknowledgement: This paper has been made possible by a grant from the Case Research Fund and by the added support of the Case Computing Cen~er.

9{) BAYARD RANKIN

of a point a I under a group of transformations on a probability space. The group must have special properties to insure t h a t the equality holds and the initial point a 1 must be chosen outside a measurable set M of probability measure zero, where M in general depends

o n X .

The whole modern reformulation of the equality (1) is a natural one once a random variable is defined as a finite measurable function on a probability space. There are limita- tions, however9 Primarily, it is intuitively unsatisfactory that the sequence S for which (1) holds should be different for different random variables (a consequence of the fact t h a t the exceptional set M depends on X). One quite naturally asks, " I s there a probability space (S, B, P) in which S ~ (a l, a s, a s . . . . ) is a single abstract sequence and on which (1) holds for all random variables X ? " The answer to the question is in the affirmative 9 At the same time a n y random variable on such a probability space must have at most a countable number of values (because S is countable) and thus a distribution function which is at most a step function 9

Another limitation is the absence of any computable structure in the space S of elementary events, leaving only the expectation attainable b y computable methods but not the sample values of the random variables9 Again, one quite naturally asks, " I s there a probability space (S, B, P) in which S is composed of a subset of the real numbers com- putable in the Turing sense?" The answer is once again in the affirmative. At the same time, because there are at most countably m a n y computable numbers, a n y random variable on such a probability space must again be elementary 9

Thus, while the answers are not trivial--and, in fact, will require a detailed explanation - - w e are twice led to answers of limited generality after posing questions of basic interest.

I t is our present purpose to find answers of greater generality b y asking the same two questions with more flexible phrasing. Inevitably this means formulating and accepting a flexible probability theory t h a t augments the measure-theoretic approach.

2. The elementary case

Let (S, B, P) be a probability space and let IA be an indicator function of the set A E B (i.e., a function defined on S t h a t takes the value 1 on A and 0 elsewhere)9

D E f i n I T I O N 1. (S, B, P ) is called a sequence probability space i/S==-(al, as, a s . . . . ) is an abstract sequence and

9 ] 3,

P A = l l m ~ IA (ak) (2)

/or all A E B.

C O M P U T A B L E P R O B A B I L I T Y S P A C E S 91 T~]~ OREM 1. There exists a sequence probability space ( S, B, P) in which B is uncountably in/inite.

The proof of this theorem is accomplished b y construction and is inspired, as is much of this paper, b y H. Weyl's work on uniformly dense sequences [9]. A sequence S - - (a 1, a~, aa . . . . ) is uniformly dense in the closed unit interval [0, 1] if for all intervals i contained in [0, 1]

# ( i ) = l i m N ~ I~(ak) (3)

# being the Lebesque measure. Weyl has proved, for example, t h a t the sequence S ~ (al, a~, a 3 .. . . ) defined b y ak = ~k (rood 1), ~ irrational, has the property of being uniformly dense in [0, 1].

To construct a sequence probability space, let S be a uniformly dense sequence in [0, 1]. L e t H be a countable partition of [0, 1] composed of non-trivial intervals and let B be constructed b y intersecting with S the smallest a-algebra, A, of sets in [0, 1] containing II: B = ~4 N S. E v e r y m e m b e r A ' of A is at most a countable union of non-trivial, disjoint intervals and A is in one-to-one correspondence with B. Because A ' is a countable union of intervals there are for E > 0 two members .4', _A' of A, each composed of finite disjoint unions of intervals such t h a t

i) A ' ~ A' c ~ i ' ii) # A ' - / ~ A ' < e.

Thus,

l i m N 1~ IA, (ak) < l i m ~ ~ I_A, (ak) + e =/~ A' + e 1 N

1 N ] N _

lira ~ 1~

IA,

(ak) > lim ~r ~ I~" (ak) -- e =/~ A ' - e and it follows t h a t

9 1 N

# A ' = h m ~ ~ IA, (ak).

Because of the one-to-one correspondence between A and B we can define P A = # A' for A = A' N S with the consequence t h a t for all A E B

I N P A = lira ~-, ~ I a (a~).

2 u

]8 is a a-algebra of sets in S and P is a measure 9 (S, B, P) is then a sequence probability space. Q.E.D.

7 - - 6 0 3 8 0 7 Acta mathematica. I 0 3 . T m p r i m ~ le 19 m a r s 1960

92 B A Y A R D R A N K I N

T h e p r o p e r t y of o n e - t o - o n e - n e s s b e t w e e n A a n d B p l a y e d a n e s s e n t i a l role in t h e a b o v e p r o o f a n d p e r m i t t e d t h e t r a n s f e r of a m e a s u r e # on sets in [0, 1], t o a m e a s u r e P on sets in S. I f t h e a - a l g e b r a A h a d b e e n g e n e r a t e d n o t f r o m YI b u t f r o m t h e class of all i n t e r v a l s in [0, 1] t h e a r g u m e n t w o u l d h a v e b r o k e n down. T h i s is a p o i n t w h i c h will h a v e b e a r i n g on t h e w a y s in which one can a n d c a n n o t g e n e r a l i z e t h e i d e a of a sequence p r o - b a b i l i t y space.

T h e t h e o r e m j u s t p r o v e d t e l l s in w h a t sense a sequence p r o b a b i l i t y s p a c e is non- trivial. T h e n e x t t h e o r e m t e l l s in w h a t sense t h e c o n c e p t is l i m i t e d .

T H E O R E M 2. Let X be a random variable de/ined on the sequence probability space (S, 73, P ) and let F x ( x ) be its distribution/unction.

F x (x) is a step/unction.

This t h e o r e m , w h i c h m a y be considered obvious, requires a l i t t l e discussion. I t is r e c a l l e d t h a t in t h e p r o o f of t h e l a s t t h e o r e m t h e a-field, B, in a sequence p r o b a b i l i t y space c o u l d n o t b e t o o rich, t h a t is, c o u l d n o t i n c l u d e t o o m a n y s u b s e t s of S. I n p a r t i c u l a r , B c o u l d n o t i n c l u d e all p o i n t s ak of a n i n f i n i t e sequence S of d i s t i n c t p o i n t s , for in s u c h a case P a k = 0 for e a c h k, b e c a u s e of (2), a n d P S = ~ P a k = 0--- a c o n t r a d i c t i o n .

T h i s p r o v e s t h a t t h e r e is no r a n d o m v a r i a b l e on a sequence p r o b a b i l i t y s p a c e w h i c h h a s a d i f f e r e n t v a l u e a t e a c h p o i n t of S, unless S h a s f i n i t e l y m a n y p o i n t s .

T h e p r e s e n t t h e o r e m s a y s more, n a m e l y , t h a t e a c h r a n d o m v a r i a b l e on a sequence p r o b a b i l i t y s p a c e is e q u i v a l e n t t o one t h a t i n d u c e s a p a r t i t i o n of S i n t o sets of p o s i t i v e m e a s u r e , t h e e l e m e n t s of t h e p a r t i t i o n b e i n g of t h e f o r m A = {a k : X = const.}. T h e proof, on t h e o t h e r h a n d , does n o t r e q u i r e t h e special f o r m of m e a s u r e g i v e n b y (2), b u t is b a s e d s i m p l y on t h e f a c t t h a t t h e r a n g e of a r a n d o m v a r i a b l e on a sequence p r o b a b i l i t y s p a c e is a c o u n t a b l e set a n d n o n o n - t r i v i a l c o n t i n u o u s d i s t r i b u t i o n f u n c t i o n c a n h a v e p o i n t s of i n c r e a s e o n l y on such a set. T h e p r o o f follows.

Proo/. B e c a u s e S is c o u n t a b l e , X can h a v e a t m o s t a c o u n t a b l e n u m b e r of values. F r o m t h e d e c o m p o s i t i o n of d i s t r i b u t i o n f u n c t i o n s

F x (x) = F ~ (~) (x) + F x (s) (x)

w h e r e F x (~) (x) is a s t e p f u n c t i o n a n d F x (~) (x) is a s i n g u l a r f u n c t i o n which is c o n t i n u o u s w i t h p o i n t s of increase b e l o n g i n g t o a c o u n t a b l e set. T h e m e a s u r e #(~) on t h e r e a l line t h a t c o r r e s p o n d s t o F x (~ (x) assigns t h e m e a s u r e zero t o e v e r y c o u n t a b l e set a n d t h u s Fx(~)(x) -- O.

Q . E . D .

C O M P U T A B L E P R O B A ] 3 I L I T Y S P A C E S 93 The two theorems to follow display the special advantages of sequence probability spaces9 For one of these advantages having to do with computability, it is convenient to define some notions.

THEOREM 3.

I / X is a bounded random variable defined on the sequence probability space (S, B, P), then

E X = l i m l ~ x (ak).

Pro@

Let 1-Ix be the partition of S induced b y X. T h a t is A E H x if and only if for some real r,

A : X : r ) .

We m a y assume without loss of generality t h a t

P A

> 0 for all A.

There exists for e > 0 a set A~ composed of a finite union of elements of II x such t h a t

PA~

> 1 - e. This can be demonstrated b y letting Ak be the union of all A E I I x for which

2-(~-1) ~> p A > 2 ~.

For each k, A~ i s at most a finite union sets in l-Ix and O

Ak=--S.

Because P is competely

1 M

additive, for some M depending on

s, P [ U Az] > 1 - r

1

Now define for e > 0

X (ak) = {B X(ak) forf~ ak E A ~ a ~ r

{ X ; ~ ) f o r a k E A ~

X(ak) = for

akqA~

where B is a n y real n u m b e r such t h a t [ X ] < B < ~ . I t follows t h a t

ii)

E.X - E _X K 2 B e

and since X and _X have at most a finite n u m b e r of values

9 1 N - -

iii) E ) ~ = h m ~ . ~ X (%)

iv)

Therefore,

9 1 N

E X = hm ~ ~ _X (a~).

T

94 B A Y A R D R A N K I N l N

lim ~ X ( a k ) ~ E X < ~ E X _ + 2 B e < ~ E X + 2 B e

1 N m

lim ~ I X ( a k ) > ~ E X _ > ~ E X - 2 B e > ~ E X - 2 B e .

Q.E.D.

D E F I N I T I O N 2. A sequence S = (al, a s, a 3 .. . . ) is said to be computable, i/ it is real and there is a Turing machine(1) which,/or any pair o/positive integers M , •, will print out in order the first M digits o/all ak, k = 1, 2 . . . N in a ]inite number o/steps.

D E F I N I T I O N 3. A sequence probability space (S, B, P) is called a computable probability space, i / S is computable.

Because a n y T u r i n g machine can be simulated b y a m o d e r n all-purpose c o m p u t e r t h e n o t i o n of a c o m p u t a b l e p r o b a b i l i t y space makes possible t h e investigation of w h e t h e r t h e sample values of a sequence of statistically i n d e p e n d e n t r a n d o m variables can be c o m p u t e d . Such an investigation m i g h t yield results for actual statistical calculations, where at present r a n d o m n u m b e r s generated b y physical processes or p s e u d o - r a n d o m n u m b e r s generated b y computers m u s t be used. Such a n investigation m i g h t also yield results for t h e t y p e of processes studied in statistical mechanics. More will be said a b o u t these implications later.

D E F I N I T I O N 4. A random variable X de]ined on the computable probability space (S, B, P) is called a computable random variable, i/ the sequence o/ sample values X (al) , X (a2), X (a3) . . . . is computable.

D E F I N I T I O N 5. A sequence o/ random variables X 1 , X 2 , X 3 . . . . defined on the com- putable probability space (S, B, P) is called computable, i] there is a Turing machine which, ]or any triplet o/positive integers M, N, Q, will print out in some speci]ied order the ]irst M digits o/all X~ (as), ] = 1, 2, 3 . . . N, i = 1, 2, 3 . . . Q in a ]inite number o/steps.

THEOREM 4. There exists a computable probability space (S, B, P) in which B is un- countably infinite.

The proof is t h e same as for t h e existence of a sequence p r o b a b i l i t y space w i t h t h e a d d e d r e m a r k t h a t t h e sequence a~ = ~]c (mod 1) /c = l, 2, 3 . . . on which t h e space is

(1) For the definition of a Turing machine see [3, 8].

COMPUTABLE P R O B A B I L I T Y SPACES 95 built, is computable if the irrational ~ is computable. Such an ~ might be chosen to be

~, e ol- ~/2.

T H E 0 R E M 5. There exists/or each positive integer N a computable sequence o/statistically independent random variables X1, X 2 .. . . X N defined on a computable probability space (S, B, P).

For a proof, it is sufficient to let S ~ (a 1, a 2, aa . . . . ) be the uniformly dense sequence in the unit interval defined by ak = z k (mod 1). For given integer N, let B be the smallest a-algebra containing all sets Ai, N of the form

Aj ,N= { ak. ~ <~ a~ < J~--~l } , j = O,1,'" , 2N --1

and let P be defined b y (2).

The N identically distributed random variables 2i-I_i

X~ = ~ IA2j, ,, i = 1,2,..., N ]=o

are statistically independent and computable. The independence follows from direct computation and the computability follows from the fact t h a t for each k and each i a finite number of digits of ak determines whether X~(ak) has value 0 or 1. Q.E.D.

As will be seen later it is only for simplicity and not out of necessity that for the proof we have chosen to construct random variables which have only two values. I n fact, the proof of the following theorem will be obvious from later results.

T~EO~EM 6. For any distribution [unction Fx(t ) (o/a random variable) with at most a finite number o/points o/increase, and/or any positive integer N, there exist N identically distributed, statistically independent, computable random variables Xx, X2, ..., X N de/ined on a computable probability space (S, B, P) such that Fxk(t ) = Fx(t), k = O, 1 . . . N.

As a special sequence t h a t leads to simple computations the following sequence of rationals, S ~ ~ (al0 , a20 , aa o .. . . ) t h a t is uniformly dense in the unit interval will be found extremely useful. S ~ will also be found useful in the proofs of some later theorems.

a l ~ = 0

0 1

a ~ = ai + 2 N + l ,

i = 1,2,-..,2 N, N = 0 , 1,2,..-

9 6 B A Y A R D R A N K I N

The first few terms of S o are:

0, 76, Y6, i~, i~" 25

The proof t h a t S ~ is uniformly dense in the unit interval is accomplished by showing first, t h a t (3) holds for all diadic intervals, that is intervals whose end points are of the form

k/2 N,

where k and N are positive integers 0 ~< k ~< 2 N. The second part of the proof uses the fact t h a t all intervals in the unit interval can be approximated arbitrarily closely from above and below b y diadic ones.

If in the proof of Theorem 5 we take as the computable sequence S o instead of S ~ (z(mod 1), 2 z ( m o d 1) . . . . ), the sample values of the statistically independent random variables defined there can immediately be displayed for a n y N:

Table o/ ten sample values, X t (aOk), o] five statistically independent and computable indicator random variables

1 2

1 0

1 1

1 1

1 1

1 1

3 4 5 6 7 8 9 l 0

1 0 1 0 1 0 1 0

0 0 1 1 0 0 1 1

1 1 0 0 0 0 1 1

1 1 1 1 1 1 0 0

1 1 1 1 1 1 1 1

The use of such sample values in monte Carlo calculations will be discussed later.

3. Probability functional spaces

I t was chiefly to obtain theorems of the type 3 and 6 t h a t computable probability spaces were introduced. Yet the random variables appearing in these theorems must be elementary as we learned from Theorem 2. Now because of a desire to reformulate Theorems 3 and 6 in a more general context, we define what is meant b y "probability functional spaces". The specific use of probability functional spaces in extending the computability notions will be left for a later section.

Before closing this section we will show that the present ideas are completely consistent with measure-theoretic probability and, in fact, are more general b y lacking only one postulate. The missing postulate is the familiar complete additivity, or continuity, postulate for an additive set function. One might look upon the definitions to follow as a system for defining a class of random variables without calling upon the complete additivity postulate, t h a t is, without calling upon the whole mechanism of Lebesque theory. I t is

C O M P U T A : B L E P R O B A B I L I T Y S P A C E S 97 interesting to recall t h a t t h e c o n t i n u i t y a x i o m is t h e only a x i o m i n t r o d u c e d b y K o l m o g o r o v in his basic p a p e r of 1933 [4] of which he said, " I t is a l m o s t impossible to elucidate its empirical m e a n i n g " . I t is because of its e x t r e m e i m p o r t a n c e in facilitating t h e m a t h e m a t i c s t h a t t h e a x i o m is k e p t t h r o u g h o u t m o s t of p r o b a b i l i t y t h e o r y .

L e t S ~ (a} be a n a b s t r a c t collection of elements a, [0, 1] ~ the closed N - d i m e n s i o n a l u n i t cube, ~N the e x t e n d e d N - d i m e n s i o n a l real space a n d H N a subset of /~N. W e write C ( H ~) for t h e class of all functions defined on H ~ to [0, 1] a n d continuous on H ~. W e will also write R for/~1.

L e t /1,/~ . . . /~ be 57 functions, each defined on S t o ~). I f H N is t h e range of t h e v e c t o r function (/1,/2 . . . ]N), a finite composition c(/1,/3 . . . . ,/N) of t h e v e c t o r function (/1,/~ . . . IN) with t h e continuous function c is defined for each c E C (GN), H N c G N, b y c (/1,/3 . . . / ~ ) (a) = c (/2 (a),/3 (a) . . . / ~ (a)).

A class :~ of functions, each defined on S t o [0, 1] is said to be closed under all finite compositions with continuous /unctions, if c (/2,/3 . . . . , IN)E ~ for all finite collections/1,/3, . . . . /~ chosen f r o m :~ a n d for all c E C ([0, lJN).

L e t us denote b y " ~ (:~)" t h e smallest linear space containing a class :~, of functions defined on S to [0, 1], t h a t is, t h e space of c o m p l e x v a l u e d functions w i t h d o m a i n S such t h a t :

i) IEs if/E:~

ii) klg 1 + k3g3Es ) if gl, g3E~(:~) a n d kl, k 3 are finite complex n u m b e r s iii) s c s (y) if s (:~) is a n y other space satisfying i), ii).

D E F I N I T I O N 6. Let :~ be a class o/ /unctions de/ined on S to [0, 1] that contains Is.

A probability/unctional associated with the class ~ is a real-valued/unctional E de/ined on (:~) and 8atis]ying the/ollowing three properties/or all/1,/2 E ~ and all/inite complex kl, k~:

i)

0<E/1<1 ii)

E ! s = 1

iii) E(]Cl/: -b ]c2/2) : k l E / 1 § Ic3E/2.

T h e following p o s t u l a t e will be essential for defining a p r o b a b i l i t y functional space.

P O S T U L A T E 1. :~ is a non-empty class o/ /unctions de/ined on S to [0, 1] and closed under all finite compositions with continuous/unctions.

D E F I N I T I O N 7. A probability/unctional space is the triplet (S, ~, E) in which S is an abstract space, :~ is a class o//unctions de/ined on S to [0, 1] that satis/ies Postulate 1 and E is a probability/unctional associated with ~.

98 BAYJLV~D RANXrS

DE:FINITION

8. A random variable defined on the probability /unctional space

(S,

5, E) is a real, finite/unction X whose domain is S and/or which

:~x= {c(X): c~C(~)} c :~.

Henceforth, we will always use the symbol

":~x2x~

... x f f ' for the class of composi- tions

{c(X1,

X2 . . . XN) : cEC(RN)).

E X A M P L E 1. As an example of a probability functional space let S be

the unit interval [0, 1], :~ the class of continuous functions defined on and to [0, 1] and E / t h e R i e m a n n integral of /E ~:. I n this ease the class of r a n d o m variables X defined on (S, :~, E) coincides with the class of real continuous functions defined on [0, 1].

EXAMPLE 2. As another example let S be the unit interval [0, 1], ~ the class of R i e m a n n integrable functions defined on and to [0, 1] and

E l

t h e R i e m a n n integral of /E 5. I n this case the class of r a n d o m variables X defined on (S, :~, E) coincides with the class of real finite functions defined on S which are continuous almost everywhere.

We know from Lusin's Theorem t h a t the class of real finite functions defined on [0, 1] which are almost continuous coincide wiVh a class of r a n d o m variables (measurable functions) defined on a real probability space ([0, 1], B, P). F r o m this fact it is clear how t h e r a n d o m variables X of this example compare with r a n d o m variables defined on ([0, l/

B, P).

EXAMPLE 3. As another example let (S, B, P) be an abstract probability space, the class of measurable functions defined on S to [0, 1] and

E l

the integral or m a t h e m a t i c a l expectation of / on the a b s t r a c t space

(S, 73, P).

I n this case the class of r a n d o m variables defined on (S, 5, E) coincides with the class of random variables defined on (S, B, P) and if AE]~, then IAE:~ and

EIA = P A .

EXAMPLE 4. As in the proof of Theorem l, let

S = (a 1, a2,

a a . . . . ) be a uniformly dense sequence in the closed unit interval [0, 1]. L e t ~ be the class of R i e m a n n integrable functions defined on and to [0, 1] and restricted to S. F o r /E :~, let

E / = l i m l ~ / (ak).

A slight extension of the proof of Theorem 1 will show t h a t

El,

as defined here, equals the R i e m a n n integral of a n y R i e m a n n integrable function on [0, 1] whose restriction to S i s / . E can immediately be extended to E(:~) and (S, :~, E) is a probability functional space.

The class of r a n d o m variables defined on (S, 5, E) can be obtained b y restricting to S the class of real finite functions on [0, 1] which are continuous almost everywhere.

COMPUTABLE PROBABILITY SPACES 99 A n i m p o r t a n t thing displayed in t h e first t w o examples is t h a t t h e class of r a n d o m variables defined on a p r o b a b i l i t y functional space m a y coincide with a subset of t h e class of r a n d o m variables defined on a p r o b a b i l i t y space. T h e t h i r d example shows t h a t the concept of r a n d o m variables on a probability functional space, t h o u g h a generalization, is n o t inconsistent with the measure-theoretic concept. T h e f o u r t h example suggests in w h a t sense probability functional spaces p e r m i t a generalization of sequence a n d c o m p u t a b l e p r o b a b i l i t y spaces. More will be said a b o u t this later.

The following t h e o r e m shows in w h a t sense there is closure of t h e class of r a n d o m variables defined on a probability functional space.

THEOR~,M 7. Let X1, X 2 .. . . . X N be random variables defined on the probability/unc- tional space (S, :~, E). The composition

Y = c (X 1, X 2 .. . . . XN)

is a random variable i] c is a / u n c t i o n defined on ~ N to ~ 1 / o r which i) c is finite on the finite part o / ~ t ~

ii) lim c(~) = c(~0)/or ~2, ~ o E R N.

x-->x0

Proo/. Y can be written Y = 5(U1, U~ . . . UN) where Uk = %(Xk), ck is a continuous one-to-one function m a p p i n g R onto [0, 1] a n d 5 is the function defined on [0, 1] N to ~N t h a t is given b y

c ( U 1, U 2 , " ", UN) = c [ c ; 1 (U1) , c21 (U2), . . . , CN 1 (UN) ]

where %-1 is t h e inverse of %. T h u s c(Y), ~ E C ( R ) , is a finite composition of t h e v e c t o r function (U1, U 2 .. . . . UN) with t h e continuous f u n c t i o n ~ (5) a n d belongs to :~ because each Uk belongs to ~ a n d P o s t u l a t e 1 is satisfied. Q.E.D.

T h e o r e m 7 can be c o m p a r e d to t h e m o r e general s t a t e m e n t in measure-theoretic probability t h e o r y t h a t Baire functions of r a n d o m variables are r a n d o m variables. The fact t h a t limits of r a n d o m variables as defined here m a y n o t be r a n d o m variables p r e v e n t s t h e m o r e general statement. I n fact, in the setting of E x a m p l e s 1, 2 or 4 r a n d o m variables can be displayed such t h a t a composition with some function possessing a single disconti- n u i t y fails to p r o d u c e a r a n d o m variable.

TI4EOR~M 8. Let X~, X 2 .. . . . X N and Y = c(X1, X~ . . . XN) be as in Theorem 7 with c satis]ying conditions i) and ii).

i) ~x, c ~ x . x ... XN ii) J r c ~ x , . x , . . . X N.

I 0 0 B A Y A R D I ~ A N X I N

T h e T h e o r e m is an i m m e d i a t e consequence of t h e definitions. I t displays how t h e classes of functions ~ z b e h a v e relative to each other like t h e algebras of sets t h a t are generated b y r a n d o m variables defined on a p r o b a b i l i t y space.

W e n o w wish to e x t e n d t h e functional E to r a n d o m variables X defined on a p r o b a b i l i t y functional space (S, :~, E) so t h a t a m a t h e m a t i c a l e x p e c t a t i o n of r a n d o m variables will be defined. To do this we consider two cases.

CASE 1: X is non-negative. L e t X (~)= / X for X < ~ n [ n otherwise.

E a c h composition in the sequence X r X r . . . . belongs t o s and, thus, E X r exists a n d is finite for each integer n. E X (1), E X ~2~ . . . . is a non-decreasing sequence of finite real numbers. W e define t h e mathematical expectation of X b y

E X = l i m E X r

n

CASE 2: X is an a r b i t r a r y r a n d o m variable defined on (S, :~, E).

I n this case, X can be w r i t t e n X = X + - X - where

{ X for X>~O { 0 X for X < O

X + = , X - = 9

0 otherwise otherwise

B y T h e o r e m 7, X + a n d X - are r a n d o m variables, a n d t h e y are non-negative. T h e m a - t h e m a t i c a l e x p e c t a t i o n s E X + , E X - are defined b y Case 1. If at least one of t h e n u m b e r s E X +, E X - is finite, we define the mathematical expectation of X b y

E X = E X + - E X - .

If E X exists a n d is finite, X is said to be integrable. Because of T h e o r e m 7 a n d t h e a b o v e definitions all of t h e following expectations are defined in which X a n d Y are r a n d o m variables, defined a n d integrable on a p r o b a b i l i t y functional space:

E ( X + Y),

EIx I, lXl

^{E x} r > O, E ( X Y).

iOx-I X I

- o o < O < c o , belongs to C ( : ~ x ) f o r real r a n d e - I x [ r > 0 , is m o n o t o n i c

B e c a u s e ~ ~

in r, 4Px(0 ) = l i m E e , - c ~ < 0 < + c~, exists a n d is defined for all r a n d o m variables X defined on a p r o b a b i l i t y functional space. T h e formal similarity between (I)x(0) a n d charac- teristic functions of r a n d o m variables on a p r o b a b i l i t y space leads us t o call this the characteristic/unction of X. T h e question of whether this function has t h e analytic pro- perties of a characteristic function required in t h e measure theoretic case, t h a t is, c o n t i n u i t y a n d n o n - n e g a t i v e definiteness, remains to be answered b y proof, however. W e will answer this question later in special cases.

COMPUTABLE PROBABILITY" SPACES 101 T h e similarity between the w a y the a b o v e extensions were m a d e a n d the w a y t h e Lebesque integral or t h e Danicll integral is define~l is obvious. T h o u g h E X can be inter- preted as an integral over a measure space for t h e proper choice of (S, ~-, E ) , some choices of (S, :~, E) p r e v e n t this. Some exceptional cases in which E X is n o t an integral over a measure space a n d c a n n o t be e x t e n d e d to be an integral arise because no p o s t u l a t e cor- responding to complete a d d i t i v i t y of a set function (such as c o n t i n u i t y of t h e linear func- tional E) has been assumed. I t is precisely t h e exceptional case t h a t is of greatest interest to this paper. I t arises when, in generalizing c o m p u t a b l e p r o b a b i l i t y spaces, S is chosen to be a sequence.

E X A M P L E 1. (continuation) I n this case every r a n d o m variable, X , is b o u n d e d a n d E X is t h e R i e m a n n integral of X. (I)x is a characteristic function.

E X A M P L E 2. (continuation) As s t a t e d earlier t h e class of r a n d o m variables, X, in this example do n o t comprise t h e class of finite measurable functions on t h e real p r o b a b i l i t y space ([0, 1], B, P) b u t are a subclass of them. On t h e other hand, E e x t e n d e d t o t h e r a n d o m variables of this example is such t h a t E X is t h e Lebesque integral of X over ([0, 1], B, P) a n d is t h u s a m a t h e m a t i c a l e x p e c t a t i o n in t h e measure-theoretic sense. (])z is a characteristic function.

E X A M P L E 3. (continuation) I n this case, where the class of r a n d o m variables defined on (S, :~, E) coincides with the class of r a n d o m variables defined o n (S, B, P), E X coin- cides with the m a t h e m a t i c a l e x p e c t a t i o n or integral of X over (S, B, P), a n d (])z with the characteristic function.

E X A M P L E 4. (continuation) H e r e S is a sequence. Unlike E x a m p l e s 1 a n d 2 the p r o b a b i l i t y functional extended t o X , E X , is n o t a Lebesque integral of a measurable function on a p r o b a b i l i t y space. T a k e for example t h e sequence X1, X ~ . . . . where

1 k < n

Xn ( a k ) =

0 otherwise

E a c h Xn is a r a n d o m variable on (S, 5 , E ) since it coincides with t h e restriction t o S of a R i e m a n n integrable function whose d o m a i n is [0, 1]. E X n = 0 for all finite n. T h u s

lim E X,~ - O.

However, at the same time, t h e sequence is m o n o t o n e non-decreasing with lim X~ = I s . T h u s

E lim X n = 1.

Monotone convergence does n o t hold a n d E is n o t a Lebesque integral.

M a n y of the properties of e x p e c t a t i o n as defined in measure-theoretic probability

102 BAYARD RANKIN

t h e o r y are t r u e of e x p e c t a t a t i o n as defined in t h e p r e s e n t context. S o m e t y p i c a l properties are s t a t e d in t h e n e x t t h e o r e m .

T H E O R E M 9. Let X and Y be random variables de/ined on the probability/unctional space (S, :~, E) and let r, s be real/inite numbers.

i) X is integrable, i/ and only i/ I X I is integrable

ii) 1 / X and Y are inteffrable, r X + s Y is integrable and E ( r X § s Y) = r E X § s E Y iii) I / X <~ Y, then E X <~ E Y

iv) 1] I X I <~ y and Y is integrable, then X is integrable

v) IEXI<EIX I

vi) 1 / I X l " is integrable,

I x I s

i8 integrable /or 0 < s <~ r vii) (HSlder Inequality) 1 / 1 / r + 1 I s = 1, r > 1,

EIXYI E"r IXI'E 'Sl YI

vii) (Minkowski Inequality) 1 / r >~ l,

E~/rIX § YI r <~ E v r l X t r + E~'r[ y t ~-

T h e proofs are similar t o those of t h e corresponding t h e o r e m s f o u n d in Lo~ve's b o o k [5]

or K o l m o g o r o v ' s m o n o g r a p h [4].

Because in t h e p r e s e n t e o n t e x t only t h e compositions c (X) of r a n d o m variables defined in T h e o r e m 7 can be g u a r a n t e e d t o yield r a n d o m variables, o p e r a t i o n s like Eltx<t ] for real t are undefined unless, as in t h e discrete case, Itx<t ] h a p p e n s t o belong t o ft. I t is also t h e case t h a t t h e w e a k closure of t h e class of r a n d o m variables on a p r o b a b i l i t y func- tional space, as set f o r t h in T h e o r e m 7, restricts t h e t y p e of limits t h a t yield r a n d o m variables.

A few of t h e m o s t i m p o r t a n t ideas a n d t h e o r e m s for p r o b a b i l i t y functional spaces will n o w be developed.

D E F I N I T I O N 9. Let X , X 1 , X 2 . . . . be random variables de/ined on the probability /unctional space (S, :~, E). The sequence X1, X~ . . . . converges in probability to X and we write

P

Xn ~ X

i/ and only i / / o r arbitrary a > O, and/or any/unction h E C ( R) such that h (x) = 0 for I x [ < a lim E h ( X . - X ) = O.

n-~oo

C O M P U T A B L E P R O B A B I L I T Y S P A C E S 103 I t is e a s y t o show t h a t if " p r o b a b i l i t y s p a c e " is s u b s t i t u t e d for " p r o b a b i l i t y functional s p a c e " in this definition t h a t t h e criterion is e q u i v a l e n t to convergence in p r o b a b i l i t y of r a n d o m variables defined on a p r o b a b i l i t y space.

Uni/orm convergence a n d convergence in the r-th mean h a v e t h e c u s t o m a r y definitions a n d for these we write

u

X~ ~ X

T

X~ 84 ~ X ,

the l a t t e r being defined for r a n d o m variables with finite r t h absolute m o m e n t s .

F o r sequences which are m u t u a l l y c o n v e r g e n t in a n y of t h e a b o v e senses we write

P

X~ - X ~ ~ 0

u

X~ - X ~ ~ 0

r

X,~ - X., ~ 0

T h e basic inequalities of p r o b a b i l i t y t h e o r y can be r e s t a t e d in t h e following form.

(Compare K o l m o g o r o v [4] or Lo~ve [5].)

T H E O R E M 10. Let X be a random variable defined on the probability ]unctional space (S, ~, E) and let g be an even, non-decreasing, %on-negative ]unction satis/ying i) and ii) o]

Theorem 7, N = 1. Let h E C (R) be such that h (X) = 0 i] I X] < a, a > O.

i) E g (X! >~ E h (X)

g (a)

ii) I] g is bounded by K and 1 - g(a)/g (X) <~ h (X), then E h (X) >i E g ( X ) - g (a)

K

iii) I f ] X I is b o u n d e d b y L a n d 1 -: g (a)/g (X) < h (X), t h e n E h (X) >I E g (X) - g(a)

g (L)

Proo/. U n d e r the respective a s s u m p t i o n s of i), ii) a n d iii) of t h e Theorem:

i) E g (X) >~ E g (X) h (X) >1 g (a) E h (X).

ii) E g ( X ) = E g ( X ) h ( X ) + E g ( X ) [ 1 - h(X)] < K E h ( X ) + g(a).

iii) E g ( X ) = E g ( X ) h ( X ) + E g ( X ) [ 1 - h ( X ) ] ~ g ( L ) E h ( X ) § Q . E . D .

1 0 4 B A Y A R D R A N K I N

THEO~tEM 11. Let X , X1, X= . . . . be random variables delined on the probability lunc- tional space (S, :~, E).

r P

11 X , ~ X then X~ ~ X

i)

ii) iii)

satislying i), ii) o I Theorem 7, N = 1, such that g (0)= O.

P

X= ~ X i I and only i I E g ( X N - X)

P r

11 the Xn are unilormly bounded and i I X , ~ X then Xn ~ X

Let g be an even bounded, non-negative 1unction, monotonic increasing on [0, co ],

~ 0 . T h e proof follows directly f r o m t h e previous theorem.

COROLLARY. Let X , X1, X 2 .. . . be random variables on the probability/unctional space

(& :~, E).

P f x n - x l

i)

Xn

- - X i I

and

only i l E i + i X _ Xn I

ii) X , , - X ~ ~ O i l a n d o n l y i l E i + l X m _ X ~ l - - > O

1 P

11 X= ~ X or i I X1, X2,... are unilormly bounded and X~ ~ X , then E X~-+E X

I t is seen t h a t if a sequence of X~ converges in p r o b a b i l i t y or in t h e r t h m e a n to X

iii)

and

a n d also to Y t h e n

/ X - Y /

E I + I X - - ~ [ O.

Thus, as in measure-theoretic p r o b a b i l i t y t h e o r y (see Lo~ve [5]) where convergence is convergence of equivalence classes to equivalence classes of r a n d o m variables, we defined equivalent r a n d o m variables on a probability functional space in terms of t h e a b o v e metric.

D E F I N I T I O N 10. Let X and Y be random variables de/ined on the probability lunctional space (S, :~, E). X and Y are called equivalent (X = Y) i/:

E t x - r l - o

I + I X - Y ]

Of course, this criterion implies t h a t X equals Y almost everywhere if t h e y h a p p e n also to be r a n d o m variables defined on a p r o b a b i l i t y space as in E x a m p l e 3. T h e definition

C O M P U T A B L E 1 ) I ~ O B A B I L I T Y S P A C E S 105 permits the equivalence classes of r a n d o m variables on a p r o b a b i l i t y functional space t o be viewed as elements of a metric space in w h i c h distance is defined b y

I x - y l d ( X , Y ) = E l + l X _ y l "

The question of completeness of these metric spaces arises. T h o u g h in some i m p o r t a n t special cases the space is complete, one can discover counterexamples t o completeness in the general case. A n y sequence of r a n d o m variables in E x a m p l e 1 t h a t converges every- where to a discontinuous function converges m u t u a l l y in t h e sense of t h e a b o v e metric b u t does n o t converge to a r a n d o m variable defined on t h e space of E x a m p l e 1.

T ~ E O R E M 12.

Let X and Y be random variables de/ined on the probability/unctional space (S, 5, E).

i)

X ~ - Y, i / a n d only i / E I X - Y I = 0

ii)

X=-- Y implies E X = E Y and EIX ] = E I r l . Proo/of

(i). Using the n o t a t i o n following T h e o r e m 8,

EIX-Yl I X - Y I + < I x - Y l

- ~ < E 1 E 0

l + n + I X - Y [ (n) "~" I + I X - Y I

E I X - Yl

= l i m

E I X -

YI <", = 0

Proo[

of (ii).

I E I X I - EI Y[ I <~ EI [X[ - [ Y] ] ~< E ] X - Y I , I E X " E YI <~ E I X - YI- Q-E'D"

As will be seen in t h e following discussion, t h e concept of sets of probability measure zero, t h o u g h definable in probability functional spaces, does n o t in general lead t o a stronger form of convergence t h a n convergence in p r o b a b i l i t y or to a useful t y p e of equivalence of r a n d o m variables as happens in t h e special case of probability spaces.

D E F I N I T I O N 11.

Let (S, 5, E) be a probability Junctional space. A set A ~ S is said to have probability/unctional measure zero, i/

i) IAE:~

ii)

E I A = 0

Convergence almost surely

for a sequence of r a n d o m variables on a probability func- tional space c a n n o w be defined in the c u s t o m a r y way, t h e only difference being t h a t t h e exceptional set where the sequence m a y n o t converge m u s t h a v e t h e above definition. The n o t a t i o n for convergence to a r a n d o m variable a n d m u t u a l convergence in this sense is:

a , 8 .

X~ ~ X

a . 8 ,

Xn - Xm ~0.

106 BAYARD RANKIN

W e will also s a y t h a t a relation b e t w e e n r a n d o m v a r i a b l e s d e f i n e d on a p r o b a b i l i t y f u n c t i o n a l s p a c e holds almost surely (a. s.) if i t holds e x c e p t on a set of p r o b a b i l i t y f u n c t i o n a l m e a s u r e zero. These d e f i n i t i o n s coincide w i t h t h e m e a s u r e - t h e o r e t i c d e f i n i t i o n s in t h e special case d i s p l a y e d in E x a m p l e 3.

THa~OR~M 13. Let (S, ~, E) be a probability/unctional space and let A c S be a set o/

probability/unctional measure zero. For all random variables X , Y de/ined on (S, ~ , E):

i) ~ l x f i ~ = o

ii) I] X = Y a.s. then X=-- Y.

Proo] of (i). F o r a n a r b i t r a r y r a n d o m v a r i a b l e X on (S, :~, E), a n d n > 1, E [ ] X [ I a ] ( , ) = n E [ I X [ Ia](") n E [ I X ] ( n ) I a ! < ~ n E i A = 0

n n

Thus,

E I X ] IA = lim E [ I X I IA] (~) = 0.

P r o o / o f (ii). I f X = Y a . s . t h e n b y (i):

E I X - rI = E I X - rlI~=O

a n d b y T h e o r e m 12, X ~ Y. Q . E . D .

I t is seen h o w e q u i v a l e n t r a n d o m v a r i a b l e s a r e o b t a i n e d f r o m X b y m o d i f y i n g X on a set of m e a s u r e zero. I n m e a s u r e - t h e o r e t i c p r o b a b i l i t y t h i s m e t h o d y i e l d s t h e e n t i r e class of v a r i a b l e s e q u i v a l e n t t o X , I n t h e p r e s e n t c o n t e x t i t is e a s y t o show, however, t h a t t w o r a n d o m v a r i a b l e s X a n d Y d e f i n e d on a p r o b a b i l i t y f u n c t i o n a l space m a y differ e v e r y - w h e r e on S a n d still b e e q u i v a l e n t . L e t (S, :~, E) b e t h e space d e f i n e d in E x a m p l e 4, w h e r e for t h e s a k e of t h e p r e s e n t a r g u m e n t , S is t h e special sequence d e f i n e d i n S e c t i o n 2 a n d l a b e l e d " S o''. I t is i m p o r t a n t t h a t e v e r y p o i n t of S ~ is r a t i o n a l . N o w t a k e for X t h e f u n c t i o n i d e n t i c a l l y zero on So a n d for X ' t h e following:

X ' (0) = 1

X ' (a ~ = 1/n, a ~ ES ~ a ~ r O,

where a ~ = re~n, n > 0, a n d m a n d n are i n t e g e r s w i t h o u t a n y c o m m o n divisor. X ' is t h e r e s t r i c t i o n t o S o of a well k n o w n p o s i t i v e f u n c t i o n on [0, 1] w h i c h is c o n t i n u o u s on all i r r a t i o n a l s , d i s c o n t i n u o u s on all r a t i o n a l s , a n d whose R i e m a n n i n t e g r a l o v e r [0, 1] v a n i s h e s . I t is, thus, a r a n d o m v a r i a b l e w i t h E IX' ] = 0 a n d is t h e r e f o r e e q u i v a l e n t t o X .

I n a similar e x a m p l e t h a t will come l a t e r i t wiil b e seen t h a t a sequence X 1, X 2 .. . . of r a n d o m v a r i a b l e s c a n be d e f i n e d on a p r o b a b i l i t y f u n c t i o n a l space for w h i c h

p X n ~ a > 0 a n d

t ~ . 8 .

X , - - - - * O

C O M P U T A B L E P R O C A B I L I T Y S P A C E S 107 Therefore, c o n v e r g e n c e a l m o s t s u r e l y on a p r o b a b i l i t y f u n c t i o n a l space does n o t i m p l y c o n v e r g e n c e in p r o b a b i l i t y .

E X A M P L E 1. (continued) I n t h i s e x a m p l e t h e r e a r e no sets of p r o b a b i l i t y f u n c t i o n a l m e a s u r e zero.

E X A M P L E 2. (continued) H e r e a set A c S has p r o b a b i l i t y f u n c t i o n a l m e a s u r e zero if a n d o n l y if i t h a s R i e m a n n c o n t e n t zero.

E X A M P L E 3. (continued) H e r e t h e c o n c e p t of p r o b a b i l i t y f u n c t i o n a l m e a s u r e zero coincides w i t h t h a t of p r o b a b i l i t y m e a s u r e zero.

E X A M P L E 4. (continued) I n t h i s case a set A chosen f r o m t h e sequence S - - (a I a 2 .. . . ) has p r o b a b i l i t y f u n c t i o n a l m e a s u r e zero if a n d o n l y if i t h a s d e n s i t y zero.

T H E O R E M 14. ( D o m i n a t e d Convergence) ]/ Y, X , X1, X 2 .. . . are random variables de/ined on the probability/unctional space (S, 5, E) and i/

I x n l <<- r a s . , Iil< Y a s . , /or some r > 1,

P

then X n ~ X implies E X n ~ E X and ~E IXn t - - ~ E [ X ] .

1

Proo/. W e shall p r o v e X n ~ X a n d use t h e C o r o l l a r y to T h e o r e m 11. F o r a r b i t r a r y a > 0, l e t h E C (1~) b e such t h a t

i) h ( X ) = O for

IX[<a

if) 1 - a / [ X ] < h ( X ) . Then,

m I X n - X] = E [ X , -

X[h(X.

- x ) + E I X , - X [ [1 - h ( X n - X)]

<~ E~lr[Xn - X [ r" E1/Sh~(X, - X ) + a

where 1/r + 1Is = 1 a n d r > 1. W e h a v e u s e d t h e H61der i n e q u a l i t y of T h e o r e m 9. B e c a u s e [ X n _ / [ r ~ (2 y ) r a.s., E l ~ f I X n - X [ r is a finite n u m b e r , s a y K . C o n s e q u e n t l y ,

EIX. -X]

<~KE1/*he(X, - X ) + a .

P

N o w X , ~ X a n d h e E C (R) w i t h h e ( i ) = 0 for IX[ < a, a n d we h a v e E1/e h e ( X n - X)--> 0

a is a r b i t r a r y a n d t h e t h e o r e m is p r o v e d . Q . E . D .

D E F I N I T I O ~ 12. Let (S, 5, E) be a probability /unctional space and X1, X 2 .. . . a sequence o / r a n d o m variables de/ined on it.

i) X 1 and X 2 are called statistically independent i/ E / 1 / 2 = E / 1 E [ ~ /or all

/I~X~,

12 e.

~x,.

8 - - 603807 A c t a m a t h e m a t i c a . 103. I m p r i m 6 le 21 m a r s 1960

108 B A Y A R D R A N K I N

i i ) X 1 , X 2 . . . . are called statistically independent i / /or a n y /inite subcollection X I ' , X~', . . . . X N ' , E / 1 / 2 " ' " /N = E / 1 E / 2 "'" E / ~ /or a l l / k E ~ x ~ , k = 1, 2 . . . N .

L E M M A . Let X be a non-negative random variable defined on the p r o b a b i l i t y / u n c t i o n a l space (S, :~, E).

~ - - , o l i m E X ( ~ ) = E X w h e r e X ( ~ X otherwise.i/X>Je Proo/. X(~) = X - X (~) +

If X is integrable, 0 ~< E X ( o - E X = e - E X (~) <~ e.

If X is not integrable, the L e m m a is true by Theorem 9 (iii).

THEOREM 15. Let X and Y be statistically independent random variables de/ined on the probability/unctional space (S, :~, E),

i) Eg~g 2 = E g ~ E g 2 /or all g~CI2(Jx), g 2 E t : ( J r )

ii) cl ( X ) and c2( Y) are statistically i n d e p e n d e n t / o r a n y / u n c t i o n s c 1 and c 2 satis/ying i) and ii) o / T h e o r e m 7, w i t h N = 1

iii) (I)x+ r (0) = C x ( O ) C g r ( O ) , - c ~ < 0 < ~ , . i [ X Y >10 iv) I / X and Y are integrable, E X Y = E X E Y

v) I / X and Y are integrable, a~x+ y = a x 2 + a r 2 where a x e = E [ X - E X ] 2.

Proo] of (iv).

Case I . X >~ O, Y >~ O. First,

X(V~) y(v~) <~ [ X Y](~), n > 1.

Because X (Vn) and y(vn) belong t o / 2 (~x) and t: (:~r), respectively, and because of (i):

E XO"n) E Y(v~) ~ E [ X y](n) and

E X E Y < ~ E X Y . Next,

[ X Vl(n~) ~ y ( n ) v ( n ) )> O.

Again

X(~) E/2 (~x) and v(~) e E (:~r), (s) ~(e)

Therefore,

~ (e) ~(~)

and

E X Y ~ E X ( ~ ) E Y(~).

From the Lemma, one obtains after letting e approach zero, E X Y < ~ E X E Y

and with the above reverse inequality the theorem follows.

C O M P U T A B L E P R O B A B I L I T Y S P A C E S 109 Case I I . X a n d Y are integrable.

F r o m (ii) it is clear t h a t the pairs (X +, Y+), ( X - , Y-), (X +, Y-), (X-, Y + ) ( I X I , I Y I ) are statistically independent. I t follows t h a t X Y is integrable a n d we have:

E X Y = E [ X Y]+ -- E [ X Y ] -

= E [X+ y + + X - Y-] - E [ X - Y+ + X + Y-]

- ( E X + - E X - ) ( E Y + - E

Y-)

= E X E Y . Q.E.D.

I n this paper t h e special case i n t r o d u c e d in E x a m p l e 4 is of particular interest a n d will provide the f r a m e w o r k for t h e discussion remaining after t h e n e x t section. There are a large n u m b e r of definitions a n d theorems with which one could continue t h e discussion of probability functional spaces. As one m i g h t suspect, the m a j o r i t y of these are f o r m u l a t e d b y recasting the measure-theoretic ideas in terms of the above notions. Proofs m u s t be carried o u t in the more general d o m a i n b u t often are suggested b y t h e measure-theoretic ones.

4. A logical algebra of functions

I t is i m m e d i a t e l y noticed t h a t probability functional spaces, a n d r a n d o m variables on them, h a v e been i n t r o d u c e d w i t h o u t a n y reference t o logical operations on t h e class of functions f analogous to t h e logical operations of complementation, union a n d inter- section on a class of sets. This section introduces such operations.

NOTATZON. T h e operations /1 + / 2 , / 1 - - / 2 on real v a l u e d functions are the usual pointwise sum a n d difference. T h e r e l a t i o n s / 1 =/2, /1 ~</2 etc. between functions defined on S are u n d e r s t o o d to hold pointwise for all points in S. The functions m i n [/1,/2], m a x [/1,/2], where ]1, ]2 are functions defined to [0, 1] are u n d e r s t o o d to be t h e compositions cl [/1,/2], c2 [/1,/2], where Cl, c 2 E C ([0, 1] 3) a n d c l/x, y / = m i n [x, y], c 2 [x, y / = m a x [x, y], 0 < x, y ~< 1. " r is a s y m b o l for "if a n d only if". " ~ " is a s y m b o l for "implies".

DEFIS~ITION 13. A logical algebra of/unctions ( f , U, ~ is a non-empty collection f o/functions defined on an abstract space S to [t3, 1], a binary operation U de/ined for each pair of/unctions in f , and a unary operation c defined/or each element in f , /or which the following eight postulates are satis/ied for all /1,/e,/a Eft. I n the postulates the definitions

/1 n h = (1,~ u 1/)~

1r = I s ~ are used.

110 B A Y A R D R A N K I N

i) 1/~:~

ii) llUlse:~

iii) 11 U 11 c = I s

iv) 11 f'111 c = I~

v ) / 1 u ls = Is u 1~

vi) 11 u (Is u 1~) = (11 u 1,) u 1~

vii) /1 N Ir = I~

v i i i ) 1, u l~ = 1, + 1 ~ ~ 1, n 1~ = 1 ~ .

The operations U, N, c will be called union, intersection a n d complement, respectively.

I f the distributive postulates ix) 11 U (1~ n ]3) = (1, U 1~) n (h u 1~)

x) 1~ n (1~ u 1~) = (h n 1~) u (h n 1~)

were added, (:~, U, c) would be Boolean, with or without postulate (viii). E x a m p l e s of systems satisfying all postulates (i)-(x) can be displayed b u t do not include some i m p o r t a n t systems, as the following discussion will show. The formal similarity between postulate (viii) and the additivity p r o p e r t y of additive set functions is worth noticing and, in fact, the postulate is brought in to m a k e probability functionals behave formally on a logical algebra of functions like set functions on an algebra of sets.

A justification for the choice of the axioms will be seen in Theorem 19 and Theorem 21.

These axioms lead to m a n y of the properties of Boolean algebra, though the following theorems do not emphasis the Boolean characteristics.

D E F I N I T I O N 14. Let (:~, U, ~) be a logical algebra o~/unctions. For all/1,/~E:~

T~]~oR~.~ 16. Let { ~ , U, c) be a logical algebra o//unctions. $' or all 11, ls E ~

i) 11 c = I s - I1

ii) (11 + t~) ~ = 11 ~ + 1r - I~ if 11 + Is e 3 ~ iil) (11 - 1~)~ = 1 ( - Is ~ + z~ if 11 - I~ ~ :~

iv) (rain [ / . / ~ ] y = m a x / / 1 ~, ls c / i f min[l~, I s / E 3 ~

Theorem 16 permits us to state the following principle for a logical algebra of functions.

PRINCIPLV, OF DUALITY. Let (5, U, c) be a logical algebra o/ /unctions. A n y = , c or ~ relation that is uniwrsally true between elements o/ :~ and that is ]ormed with the use o / t h e symbols

COMPUTABLE PROBABILITY SPACES

U, N, ~, + , - , Is,

Ir = , ~ , ~ , min, max

l l l

becomes another true relation when these symbols are replaced, respectively, by el, U, c, - - I s + , + I s - , Ir Is,

= , c , ~ , max, min,

complement and equality remaining unchanged.

T~EOREM 17.

Let (:~, U, c) be a logical algebra o/ /unctions. For all/1,/2E~

i) /1 ~ /2 <::> /1 n / 2 c = /1 -- /2 ~r = /2 [j (/2 c N/1) ii) /1 ~ / 2 * / 1 >~/2

iii) 11 U/2 : / 1 -~ (/1 U/2) N/1 c

THEOREM 18.

Let (:~, U, ~) be a logical algebra o/ /unctions. For all/1,/2 E:~

i) /t U/2 ~< min [/t § I s / ii)(1) /1 U/2 = / : + / 2 - / 1 N/3

The proof uses the statements of Theorem 17.

I t should be remarked t h a t Theorem 18 does not require min [/1 +/2,

Is/

or/1 +/2 to belong to :~. Theorem 18, ii obviously implies

/= 89

for all /E :~ which is the replacement for idempotency in this algebra.

I t will be of interest for us to investigate in the next theorem conditions under which equality holds in Theorem 18, i. 5Tot only will such conditions provide us with an algorithm for c o m p u t i n g / t U/2 but they will also bring into the algebra other useful properties.

At this point it m a y be helpful t o warn the reader of certain desirable properties familiar from Boolean algebra which are

not

in general true in a logical algebra of functions.

These include among others the distribution, absorption and idempotency laws involving union and intersection.

TtrEOREM 19.

There is one and only one/unction h on

[0, 1] 2

to

[0, 1]

such that/or the binary operation /1 U/2 = h

(/1,/3)

and every class :~ satis/ying Postulate I (o/Section

2)

/or some S, (:~, U, c) is a logical algebra o//unctions.

The unique binary operation may be written

/1 U/3 = min [/1 § /2' Is]"

Pro@

For any class ~ satisfying Postulate 1, min [/1 +/2,

Is]

will belong to ~ when /i a n d / 2 belong, and it can easily be verified b y direct computation t h a t (~, 0 , ~ will be (1) This relation has appeared in a number of different studies generally concerned with multivMued logies and valuated algebras.

1 1 2 ' ~ A Y A R D R A N X : N

a logical a l g e b r a of functions. I t will suffice t o p r o v e u n i q u e n e s s for a n y p a r t i c u l a r choice of t h e class 9: s a t i s f y i n g P o s t u l a t e 1.

W e choose for t h e s a k e of t h e p r o o f t h e class 9: of all R i e m a n n i n t e g r a b l e f u n c t i o n s d e f i n e d on [0, 1] t o [0, 1]. (See E x a m p l e 2, Section 3.) F o r t h i s class, consider a logical a l g e b r a of f u n c t i o n s (9:, U, ~) t h a t is a r b i t r a r y e x c e p t for t h e r e s t r i c t i o n /1 U ]~ = h(/1,/2) for a l l / 1 , / 2 E 9: a n d some f u n c t i o n h on [0, 1] 3 t o [0, 1]. B e c a u s e of P o s t u l a t e (ii) of Defini- t i o n 13, h (x, y) m u s t be c o n t i n u o u s w h e n c o n s i d e r e d as a f u n c t i o n on [0, 1] 2. W e wish t o show t h a t h(/1,/2) h a s t h e u n i q u e f o r m m i n [/1 + / 2 , I s / for all /1,/2 E ~ . This will be p r o v e d if i t can b e shown t h a t h h a s t h e s t a t e d u n i q u e f o r m for all c o n s t a n t f u n c t i o n s in Y.

Choose n o w a n y t w o c o n s t a n t f u n c t i o n s x, y in 9: such t h a t x + y = I s . F o r t h o s e func- tions P o s t u l a t e (iii) i m p l i e s

x U y = x U x ~ = I s .

B e c a u s e y U I s = y a n d y U y~ = I s a n d b e c a u s e y U v, as a f u n c t i o n of c o n s t a n t a r g u m e n t s v, is continuous, t h e r e is for e a c h c o n s t a n t f u n c t i o n z ~> y, z E g:, some c o n s t a n t f u n c t i o n w E 9: such t h a t z -- y U w. Therefore,

x U z = xU ( y U w ) = ( x U y ) U w = Is.

W e conclude t h a t for t h e c o n s t a n t f u n c t i o n s x a n d z in 9:

x U z - I s p r o v i d e d x + z >~ Is.

T h e o r e m 16 a n d t h e d e f i n i t i o n of i n t e r s e c t i o n we h a v e for F r o m P o s t u l a t e (viii),

x, z E g :

A d d i n g t h e f a c t

xCUz c = x c ~-zC<=~xUz = I z.

x c + z c <~ I s ~ x + z >~ I s

we h a v e for c o n s t a n t s x', z ~ E 9:

x' U z' = x' § z' p r o v i d e d x' + z' <~ Is.

Thus, i f / 1 , / 2 a r e c o n s t a n t f u n c t i o n s in

/1 U/2 = m i n [11 + f2, Is]

a n d t h i s i m p l i e s t h e result. Q . E . D .

I t can be seen t h a t t h e i n c l u s i o n r e l a t i o n for a logical a l g e b r a of f u n c t i o n i m p l i e s t h e

>~ r e l a t i o n a n d p r o v i d e s a p a r t i a l o r d e r i n g of ~. I n t h e special case w h e r e t h e u n i o n o p e r a - t i o n is chosen t o b e 0 , t h e s t r o n g e r r e s u l t s of t h e n e x t t h e o r e m c a n be o b t a i n e d .

T~EOR]~M 20. F o r a logical algebra o/ / u n c t i o n s o/ the type ( ~ , O, c), i n which satis/ies Postulate 1, we h a v e / o r a l l / 1 , / 2 E :~