T-Test

A T-test is not one test, but a group of tests which constitutes of all statistical tests which distribute as T Distribution (Student’s).

We usually use the T test(s) to compare sample average (Mean) to the known Mean, or to compare between the averages of two groups when we don’t know the standard deviation

When the sample is more than 30 you should still use the T Distribution, but using Normal distribution instead will support similar results. Student's T distribution is an artificial distribution used for a normally distributed population when we don't know the population's standard deviation or when the sample size is too small.

T distribution looks similar to the normal distribution but lower in the middle and with thicker tails. This makes it more realistic than Normal distribution, as for small size the average and standard deviation estimation are less accurate

The shape depends on the degrees of freedom which is usually number of independent observations minus one (n-1). The higher the degree of freedom the more it resembles the normal distribution.
T Distribution

One sample T-test (Calculator)

We use this test to check if the known Mean is statistically correct, based on a sample average and sample standard deviation.
The null hypothesis assume that the known mean is correct. The statistical decision will be based on the difference between the know Mean and the sample

Example: A farmer calculated last year the average weight of apples in his orchard (μ0) is 17 kg, based on a big sample.
This year he checks a small sample of apples and the sample average x is 18 kg.
Did the average weight of apples change over the past year?

Assumptions

Required sample data

Calculated based on a random sample from the entire population

Test statistic

t statistic

Two Sample T-test - Equal Variance (Pooled-Variance) (Calculator)

We use this test to check if the Mean of group1 is the same as the Mean of group2, or the known difference between the groups is correct, and the standard deviation is identical for the groups

The null hypothesis assume that the known difference between the groups is correct. When the known difference is zero, the null hypothesis assume the Means of the groups are identical.

Assumptions

Required sample data

Calculated based on random sample from the entire population

Test statistic

t statistic
DF (Degrees of Freedom) = n1 + n2 - 2

Two Sample T-test (Welch's T-test) (Calculator)

We use this test to check if the Mean of group1 is the same as the Mean of group2, or the known difference between the groups is correct, while the standard deviation is NOT identical for the two groups.

The assumptions and required sample data of the Welch’s T-test are similar to the Pooled-Variance T-test with one exception - this time we assume that the standard deviation of the groups is not the same

Assumptions

Required sample data

A few statistical inputs are calculated based on random sample from the entire population:

Test statistic

t statistic

How do you know if the standard deviation of the groups is the same?

If you have preliminary knowledge, it is easy decision, but what happened if you don’t?

Using the Welch’s T test (unequal variances) with equal variance across samples will support reasonable results with relatively minor differences from the correct Pooled-Variance T test (equal variances)
When using Pooled-Variance T test (equal variances) with unequal variances across samples it will not support good results (unless using equal sample sizes)

The common practice was to run a test to compare the standard deviation, of the groups and then decide which t test to run.

This method is not so recommended, as in case of type2 error for the first step (failing to reject the null hypothesis) we assume that the standard deviations are equal while actually it is not. In this case we will run the Pool-Variance T test instead of the unequal variances.

Conclusion

If you don’t know if the standard deviations are equal you should run the Welch’s T test (unequal variances)

Example Table: Error when using incorrect test Pooled-Variance T test via Welch’s T Test

t statistic

* P-Val Error = 1 - (Correct P-Val / Incorrect P-Val)

Paired T-Test (Calculator)

In paired samples we compare the results of the same items in two different conditions.
For example before treatment and after treatment. ie: to test a new cholesterol pill, an experiment is performed and results are collected before they took the pill and several days after. Unlike a regular T-test where there are two groups of people – one who took the pill and one who didn’t, this test constitutes of the same group of people both before and after taking the pill. This test is more powerful than regular T-test since we use the same “lab rat” for both samples, instead of different lab rats for each sample, thus limiting noise.

The null hypothesis assume that the known difference between the groups is correct. When the known difference is zero, the null hypothesis assume the Means of the groups are identical

Degrees of freedom equal number of items minus one (or number of observation divided by 2 minus 1)

Assumptions

Required sample data

Test statistic

t statistic
DF (Degrees of Freedom) = n – 1

References: