Confidence interval

Video Mean,σ confidence interval calculator Proportion confidence interval calculator

When using the sample data we know the sample's statistic but we don't know the true value of the population's measures.
Instead, we may treat the population's measure as a random variable.
The confidence interval is the range that is likely to contain the true value with a probability of the confidence level. The range includes the hedges, this is relevant for discrete statistics.

Mean confidence interval (go to calculator)

The mean's confidence interval is based on the sample average.
When you know the population's standard deviation (σ) you should use the normal distribution.
$$\bar{x}\pm Z_{1-\frac{\alpha}{2}}\frac{\sigma}{\sqrt{n}}$$ When you don't know the population's standard deviation you should use the sample standard deviation (S) and the t distribution.
$$\bar{x}\pm T_{(1-\frac{\alpha}{2},n-1)}\frac{S}{\sqrt{n}}$$

Standard deviation confidence interval (go to calculator)

The population's confidence interval is based on the sample standard deivation.
The following statistic distribute Χ2(n-1) $$\frac{(n-1)S^2}{\sigma^2} \sim \chi^2_{(n-1)}$$ You may extract the σ based on Χ2 α/2 percentile and 1-α/2 percentile. $$\frac{(n-1)S^2}{\chi^2_{(1-\frac{\alpha}{2},n-1)}}\le \sigma^2 \le \frac{(n-1)S^2}{\chi^2_{(\frac{\alpha}{2},n-1)}}$$

Proportion confidence interval (go to calculator)

The population's confidence interval is based on the sample proportion.
You may expect that the exact confidence interval (Clopper–Pearson) using the binomial exact model and not approximation will be the best method,
but it appears that the Wilson score interval is the recommended method.
The Clopper–Pearson results in too wide confidence interval with actual confidence level which is bigger than the required confidence level.
And the normal approximation results in too narrow confidence interval with actual confidence level which is smaller than the required confidence level.

The following chart was created with R using the binom.confint (conf.level =0.95) function and the following methods:
"wilson" - the Wilson score interval
"exact" - the Clopper–Pearson interval.
"asymptotic" - the normal approximation.
We would expect the actual confidence level to be 0.95. As you can see the "Wilson" method supports an actual confidence level around 0.95.
compare proportion confidence interval methods
The "Wilson score interval with continuity correction" ("prop.test") seems to have similar results as the "exact" method. Following similar R simulation with "prop.test"
compare proportion confidence interval methods2

Clopper–Pearson (exact)

The Clopper–Pearson is the exact calculation base on the binomial distribution and calculated with the beta distribution. $$Left=B(\frac{\alpha}{2},x,n-x+1)\\ Right=B(1-\frac{\alpha}{2},x+1,n-x) $$

Normal approximation

For large enough sample size the sample proportion distributes normaly: $$\hat{p}=\frac{successes}{n} \sim N(\hat{p},\sqrt{\frac{\hat{p}(1-\hat{p})}{n}})$$ Following the confidence interval formula: $$\hat{p}\pm Z_{\alpha/2} \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$$ The normal approximation doesn't support good result for edge proportions, near 0 or 1.

Wilson score interval

The Wilson score interval support better results than the normal approximation interval, especially for small sample sizes and for edge proportions, near 0 or 1.
Surprisingly is also support a better result than the Clopper–Pearson interval (exact) $$\frac{\hat{p}+\frac{Z^2}{2n}}{1+\frac{Z^2}{n}}\pm \frac{Z}{1+\frac{Z^2}{n}} \sqrt{\frac{\hat{p}(1-\hat{p}}{n}+\frac{Z^2}{4n^2}}$$

Wilson score interval with continuity correction

The Wilson score interval with continuity correction support similar results as the exact test, but the Wilson score interval seem to be better that the exact test, so it is better without the continuity correction. $$ Left=\max[0,\frac{2n\hat{p}+z^2- (z \sqrt{z^2-\frac{1}{n}+4n\hat{p}(1-\hat{p})+(4\hat{p}-2)}+1)}{2(n+z^2)}]\\ Right=\min[1,\frac{2n\hat{p}+z^2+ (z \sqrt{z^2-\frac{1}{n}+4n\hat{p}(1-\hat{p})-(4\hat{p}-2)}+1)}{2(n+z^2)}]$$
Reference
A. Agresti and B.A. Coull, Approximate is better than "exact" for interval estimation of binomial proportions, American Statistician, 52:119--126, 1998.