Show
Twins from Different UniversesPhoto by Martin Sanchez on UnsplashStandard Deviation and Standard Error are two statistical concepts that often cause confusion. Do they have the same interpretations or they are meant to represent something totally different? We’ll discuss more in this post. What is Standard Deviation (SD)?The standard deviation measures the variability (aka, the spread) of data points around the mean in a given dataset. In other words, it tells us, on average, how far each data point is away from the mean. Population Standard DeviationIn the real world, we’re interested in estimating a certain characteristic in a population. Standard deviation is anexample of these characteristics. When you have ALL the data points from a population, you can compute the TRUE value of the population standard deviation using the following formula. Image by authorSample Standard DeviationOftentimes, it is difficult to collect all the data points from the population due to time, financial, or technical limitations. For example, if we would like to compute the TRUE standard deviation of household income in Los Angeles, we would need to get income from all the households in Los Angeles, which is almost impossible to do. Instead, we can collect random samples from the population and make inferences about the population standard deviation using Sample Standard Deviation. The formula for sample standard deviation is Image by authorWhy use n-1 for sample standard deviation?You will notice that we are using the sample mean (x̄) instead of the population mean (μ) for the sample standard deviation because we don’t know anything about the population mean. x̄ is a reasonable estimate for μ. Therefore, any value X in the sample dataset would be closer to x̄ than to μ. The numerator in the sample standard deviation would get artificially smaller than it is supposed to be. As a result, the sample standard deviation would be underestimated. To correct this bias in the sample standard deviation, we would use “n-1” instead of “n” (aka, Bessel’s correction) for sample standard deviation. Using n-1 would make the sample standard deviation larger than otherwise using n. Therefore, we have a less biased estimate of the population standard deviation, giving us a conservative estimate of variability. What is Standard Error (SE)?Before we discuss the Standard Error, let’s first get familiar with the concepts of Sample Distribution and Sampling Distribution. Sample Distribution vs Sampling DistributionThe sample distribution is simply the data distribution of the sample which is randomly taken from the population. For example, we ask 100 random people in Los Angeles what their incomes are. The sample distribution describes the ACTUAL income distribution in these 100 people. But what is Sampling Distribution? The sampling distribution is the distribution of the sample statistic (e.g., the sample mean, sample variance, sample standard deviation, and sample proportion) over many samples drawn from the same population (i.e., repeated sampling). For example, we ask 100 random people in Los Angeles what their incomes are. Then compute the average income. We repeat this 1000 times, then we have 1000 different average incomes. The distribution of these 1000 average incomes is called the sampling distribution.
How to interpret Standard Error (SE)?The Standard Error measures how far the sample statistic (e.g., sample mean) is likely to be from the true population statistic (e.g., the population mean). Why do we need Standard Error (SE)?Typically you might want to construct confidence intervals when we try to make statistical inferences, and it is more informative to assign a probability to construct a confidence interval that contains the mean.
How to compute Standard Error (SE)?We typically use the following formula to compute the standard error. I will discuss how to derive this formula in the next sections. Image by authorWhat are the examples of Standard Error?Standard Error can be applied to various types of statistics. Some popular examples are
What is the Standard Error of the Mean (SEM)?
Technically, the standard error of the mean is computed as the standard deviation of the sample mean. Image by authorHypothetically, we can compute the standard error under repeated samples using the following steps:
Thanks to Central Limit Theorem (CLT), we don’t need to consider the Sampling Distribution under repeated samples. Instead, the sampling distribution of the sample means can be estimated from just ONE random sample.
How to derive the formula for SEM?Image by authorTherefore, Image by authorIn most cases, the standard deviation of the population data is unknown. We will estimate it using the standard deviation of the sample data (sample standard deviation). Therefore, Image by authorWhat is the Standard Error of the Proportion(SEP)?
The standard error of the proportion is computed as the standard deviation of the sample proportions. You will notice that in each sample data, we only have data either 1 or 0. Each value follows a Bernouilli distribution. The computed sample proportions are no longer binary values. Instead, they could be any value between 0 and 1.
How to derive the formula for SEP?Image by authorSimilar to SEM, Image by authorImage by authorWe can estimate σ using the sample standard deviation √p(1-p) (i.e., the standard deviation of a Bernouilli distribution) Image by authorConclusion:Standard Deviation and Standard Error are similar concepts that both are used to measure variability. Standard Deviation indicates how the sample data values are different from the mean in the sample distribution. Standard Error indicates how the sample data statistics are different from the population statistic in the sampling distribution. How does the standard error of the mean compare to the standard deviation of the population?What's the difference between standard error and standard deviation? Standard error and standard deviation are both measures of variability. The standard deviation reflects variability within a sample, while the standard error estimates the variability across samples of a population.
What is the relationship between standard error and standard deviation?The standard error of the sample mean depends on both the standard deviation and the sample size, by the simple relation SE = SD/√(sample size).
What is the difference between standard error mean and standard deviation?Standard deviation measures the variability from specific data points to the mean. Standard error of the mean measures the precision of the sample mean to the population mean that it is meant to estimate.
How is the standard deviation related to the standard error and the sample size?Standard Error and Sample Size
The standard error of a statistic corresponds with the standard deviation of a parameter. Since it is nearly impossible to know the population distribution in most cases, we can estimate the standard deviation of a parameter by calculating the standard error of a sampling distribution.
|