What is an error? It is a difference between a "truth" and an "approximation".
What is a standard error? It is a standard deviation of a sampling distribution.
What is a sampling distribution? It is a frequency distribution of a statistic for an infinite number of samples of the same size drawn at random from the same population.
How many different kinds of standard errors are there? Aye, there's the rub.
Read on.
The standard error of measurement
The standard error of measurement is the standard deviation of a distribution of a person's or an object's obtained measurements around its "true score" (what it
"should have gotten"). The obtained measurements are those that were actually obtained or could have been obtained by applying a measuring instrument an infinite (or at least a very large) number of times. For example, if a person's true height (only God knows that) is 69 inches and we were to measure his(her) height a very large number of times, the obtained measurements might be something like the following: 68.25, 70.00, 69.50, 70.00, 68.75, 68.25. 69.00, 68.75, 69.75, 69.25,... The standard error of measurement provides an indication of the reliability (consistency) of the measuring instrument.
The formula for the standard error of measurement is σ√ 1 - ρ, where σ is the standard deviation of the obtained measurements and ρ is the reliability of the measuring instrument.
The standard error of prediction (aka the standard error of estimate) The standard error of prediction is the standard deviation of a frequency
distribution of measurements on a variable Y around a value of Y that has been predicted from another variable X. If Y is a "gold standard" of some sort, then the standard error of prediction provides an indication of the instrument's criterion-related validity (relevance).
The formula for the standard error of prediction is σy√ 1 - ρxy2 , where σy is the standard deviation for the Y variable and ρxy is the correlation between X and Y.
The standard error of the mean
The standard error of the mean is the standard deviation of a frequency
distribution of sample means around a population mean for a very large number of samples all of the same size. The standard error of the mean provides an indication of the goodness of using a sample mean to estimate a population mean.
The formula for calculating the standard error of the mean is σ/√n , where σ is the standard deviation of the population and n is the sample size. Since we usually don't know the standard deviation of the population, we often use the sample standard deviation to estimate it.
The standard errors of other statistics
Every statistic has a sampling distribution. We can talk about the standard error of a proportion (a proportion is actually a special kind of mean), the standard error of a median, the standard error of the difference between two means, the standard error of a standard deviation (how's that for a tongue twister?), etc. But the above three kinds come up most often.
What can we do with them?
We can estimate, or test a hypothesis about, an individual person's "true score"
on an achievement test, for example. If he(she) has an obtained score of 75 and the standard error of measurement is 5, and if we can assume that obtained scores are normally distributed around true scores, we can "lay off" two standard errors to the left and two standard errors to the right of the 75 and say that we are 95% confident that his(her) true score is "covered" by the interval from 65 to 85. We can also use that interval to test the hypothesis that his(her) true score is 90. Since 90 is not in that interval, it would be rejected at the .05 level.
The standard error of prediction works the same way. Lay it off a couple of times around the Y that is predicted from X, using the regression of Y on X to get the predicted Y, and carry out either interval estimation or hypothesis testing.
The standard error of the mean also works the same way. Lay it off a couple of times around the sample mean and make some inference regarding the mean of the population from which the sample has been randomly drawn.
So what is the problem?
The principal problem is that people are always confusing standard errors with
"ordinary" standard deviations, and standard errors of one kind with standard errors of another kind. Here are examples of some of the confusions:
1. When reporting summary descriptive statistics for a sample, some people report the mean plus or minus the standard error of the mean rather than the mean plus or minus the standard deviation. Wrong. The standard error of a mean is not a descriptive statistic.
2. Some people think that the concept of a standard error refers only to the mean. Also wrong.
3. Some of those same people think a standard error is a statistic. No, it is a parameter, which admittedly is usually estimated by a statistic, but that doesn't make it a statistic.
4. The worst offenders of lumping standard errors under descriptive statistics are the authors of many textbooks and the developers of statistical "packages" for computers, such as Excel, Minitab, SPSS, and SAS. For all of those, and for some other packages, if you input a set of data and ask for basic descriptive statistics you get, among the appropriate statistics, the standard error of the mean.
5. [A variation of #1] If the sample mean plus or minus the sample standard deviation is specified in a research report, readers of the report are likely to confuse that with a confidence interval around the sample mean, since
confidence intervals often take the form of a ± b, where a is the statistic and b is its standard error or some multiple of its standard error.
So what should we do about this?
We should ask authors, reviewers, and editors of manuscripts submitted for publication in scientific journals to be more careful about their uses of the term
"standard error". We should also write to officials at Excel, Minitab, SPSS, SAS, and other organizations that have statistical routines for different kinds of
standard errors, and ask them to get things right. While we're at it, it would be a good idea to ask them to default to n rather than n-1 when calculating a variance or a standard deviation.
CHAPTER 25: IN (PARTIAL) SUPPORT OF NULL HYPOTHESIS