RANDOM VARIABLES AND EXPECTATION
Corollary 4.5.2 If a and b are constants, then
4.5.1 Expected Value of Sums of Random Variables
The two-dimensional version of Proposition 4.5.1 states that if X and Y are random variables andgis a function of two variables, then
E[g(X,Y)] =
A similar result can be shown in the discrete case and indeed, for any random variablesX andY,
E[X +Y] =E[X] +E[Y] (4.5.1)
By repeatedly applying Equation 4.5.1 we can show that the expected value of the sum of any number of random variables equals the sum of their individual expectations.
For instance,
E[X +Y +Z] =E[(X +Y)+Z]
=E[X +Y] +E[Z] by Equation 4.5.1
=E[X] +E[Y] +E[Z] again by Equation 4.5.1 And in general, for anyn,
E[X1+X2· · · +Xn] =E[X1] +E[X2] + · · · +E[Xn] (4.5.2) Equation 4.5.2 is an extremely useful formula whose utility will now be illustrated by a series of examples.
EXAMPLE 4.5e A construction firm has recently sent in bids for 3 jobs worth (in profits) 10, 20, and 40 (thousand) dollars. If its probabilities of winning the jobs are respectively .2, .8, and .3, what is the firm’s expected total profit?
SOLUTION LettingXi,i=1, 2, 3 denote the firm’s profit from jobi, then total profit=X1+X2+X3
and so
E[total profit] =E[X1] +E[X2] +E[X3] Now
E[X1] =10(.2)+0(.8)=2 E[X2] =20(.8)+0(.2)=16 E[X3] =40(.3)+0(.7)=12 and thus the firm’s expected total profit is 30 thousand dollars. ■
EXAMPLE 4.5f A secretary has typedN letters along with their respective envelopes. The envelopes get mixed up when they fall on the floor. If the letters are placed in the mixed-up envelopes in a completely random manner (that is, each letter is equally likely to end up in any of the envelopes), what is the expected number of letters that are placed in the correct envelopes?
SOLUTION LettingX denote the number of letters that are placed in the correct envelope, we can most easily computeE[X]by noting that
X =X1+X2+ · · · +XN
where
Xi =
1 if theith letter is placed in its proper envelope 0 otherwise
Now, since theith letter is equally likely to be put in any of theN envelopes, it follows that
P{Xi =1} =P{ith letter is in its proper envelope} =1/N and so
E[Xi] =1P{Xi =1} +0P{Xi =0} =1/N Hence, from Equation 4.5.2 we obtain that
E[X] =E[X1] + · · · +E[XN] = 1
N N =1
Hence, no matter how many letters there are, on the average, exactly one of the letters will be in its own envelope. ■
EXAMPLE 4.5g Suppose there are 20 different types of coupons and suppose that each time one obtains a coupon it is equally likely to be any one of the types. Compute the expected number of different types that are contained in a set for 10 coupons.
SOLUTION LetX denote the number of different types in the set of 10 coupons. We computeE[X]by using the representation
X =X1+ · · · +X20 where
Xi =
1 if at least one typeicoupon is contained in the set of 10 0 otherwise
Now
E[Xi] =P{Xi =1}
=P{at least one typeicoupon is in the set of 10}
=1−P{no typeicoupons are contained in the set of 10}
=1−19
20
10
when the last equality follows since each of the 10 coupons will (independently) not be a typei with probability1920. Hence,
E[X] =E[X1] + · · · +E[X20] =20
"
1−19
20
10#
=8.025 ■
An important property of the mean arises when one must predict the value of a random variable. That is, suppose that the value of a random variable X is to be predicted. If we predict thatX will equalc, then the square of the “error” involved will be (X −c)2. We will now show that the average squared error is minimized when we predict that X will equal its meanµ. To see this, note that for any constantc
E[(X −c)2] =E[(X −µ+µ−c)2]
=E[(X −µ)2+2(µ−c)(X −µ)+(µ−c)2]
=E[(X −µ)2] +2(µ−c)E[X −µ] +(µ−c)2
=E[(X −µ)2] +(µ−c2) since E[X −µ] =E[X] −µ=0
≥E[(X −µ)2]
Hence, the best predictor of a random variable, in terms of minimizing its mean square error, is just its mean.
4.6 VARIANCE
Given a random variableX along with its probability distribution function, it would be extremely useful if we were able to summarize the essential properties of the mass function by certain suitably defined measures. One such measure would beE[X], the expected value ofX. However, whileE[X]yields the weighted average of the possible values ofX, it does not tell us anything about the variation, or spread, of these values. For instance, while the following random variablesW,Y, andZhaving probability mass functions determined by
W =0 with probability 1 Y =
−1 with probability 12 1 with probability 12 Z =
−100 with probability12 100 with probability12
all have the same expectation — namely, 0 — there is much greater spread in the possible values ofY than in those ofW (which is a constant) and in the possible values ofZ than in those ofY.
Because we expectX to take on values around its meanE[X], it would appear that a reasonable way of measuring the possible variation ofX would be to look at how far apartX would be from its mean on the average. One possible way to measure this would be to consider the quantityE[|X −µ|], whereµ = E[X], and[X −µ] represents the absolute value ofX −µ. However, it turns out to be mathematically inconvenient to deal with this quantity and so a more tractable quantity is usually considered — namely, the expectation of the square of the difference between X and its mean. We thus have the following definition.
Definition
IfX is a random variable with mean µ, then thevarianceof X, denoted by Var(X), is defined by
Var(X)=E[(X −µ)2] An alternative formula for Var(X) can be derived as follows:
Var(X)=E[(X −µ)2]
=E[X2−2µX +µ2]
=E[X2] −E[2µX] +E[µ2]
=E[X2] −2µE[X] +µ2
=E[X2] −µ2 That is,
Var(X)=E[X2] −(E[X])2 (4.6.1)
or, in words, the variance ofX is equal to the expected value of the square ofX minus the square of the expected value ofX. This is, in practice, often the easiest way to compute Var(X).
EXAMPLE 4.6a Compute Var(X) whenX represents the outcome when we roll a fair die.
SOLUTION SinceP{X =i} = 16,i=1, 2, 3, 4, 5, 6, we obtain
E[X2] = 6 i−1
i2P{X =i}
=121
6
+221
6
+321
6
+421
6
+521
6
+621
6
= 916
Hence, since it was shown in Example 4.4a that E[X] = 72, we obtain from Equation 4.6.1 that
Var(X)=E[X2] −(E[X])2
= 916 −7
2
2
= 3512 ■
EXAMPLE 4.6b Variance of an Indicator Random Variable.If, for some eventA, I =
1 if event Aoccurs
0 if event Adoes not occur then
Var(I)=E[I2] −(E[I])2
=E[I] −(E[I])2 sinceI2=I(as 12=1 and 02=0)
=E[I](1−E[I])
=P(A)[1−P(A)] sinceE[I] =P(A) from Example 4.4b ■ A useful identity concerning variances is that for any constantsaandb,
Var(aX +b)=a2Var(X) (4.6.2)
To prove Equation 4.6.2, letµ=E[X]and recall thatE[aX +b] =aµ+b. Thus, by the definition of variance, we have
Var(aX +b)=E[(aX +b−E[aX +b])2]
=E[(aX +b−aµ−b)2]
=E[(aX −aµ)2]
=E[a2(X −µ)2]
=a2E[(X −µ)2]
=aaVar(X)
Specifying particular values for a andb in Equation 4.6.2 leads to some interesting corollaries. For instance, by settinga=0 in Equation 4.6.2 we obtain that
Var(b)=0
That is, the variance of a constant is 0. (Is this intuitive?) Similarly, by settinga= 1 we obtain
Var(X +b)=Var(X)
That is, the variance of a constant plus a random variable is equal to the variance of the random variable. (Is this intuitive? Think about it.) Finally, settingb=0 yields
Var(aX)=a2Var(X) The quantity √
Var(X) is called thestandard deviation ofX. The standard deviation has the same units as does the mean.
REMARK
Analogous to the mean’s being the center of gravity of a distribution of mass, the variance represents, in the terminology of mechanics, the moment of inertia.