Cohen's d and Effect Size

In my previous posting I explained the idea of significance testing. A statistically significant result does not necessarily mean that the result is practically significant. The “effect size” usually gives an indication of whether something is practically significant.


There are a couple of different ways of calculating an effect size.

r which is the correlation coefficient or R² which is the coefficient of determination
Eta squared ή²

Cohen’s d

This time, I will focus on Cohen’s d.

If you did a t-test, it’s usually a good idea to calculate cohen’s d.

Cohen's d is an appropriate effect size for the comparison between two means. It indicates the standardized difference between two means, and expresses this difference in standard deviation units. The formula for calculating d when you did a paired sample t test is:

Cohen’s d = Mean difference

                 Standard deviation

If you have two separate groups (in other words you conducted an independent sample t test), you use the pooled standard deviation  instead of the standard deviation.

If Cohen’s d is bigger than 1, the difference between the two means is larger than one standard deviation, anything larger than 2 means that the difference is larger than two standard deviations. It is seldom that we get such big effect sizes with the kinds of programmes that I evaluate, so the following rule of thumb applies:

A d value between 0 to 0.3 is a small effect size, if it is between 0.3 and 0.6 it is a moderate effect size, and an effect size bigger than 0.6 is a large effect size.



Here is an example:

Kids wrote a grade 12 exam, then completed a programme that provides additional compensatory education, and then they rewrite the grade 12 exam. Below is a table that compares the Maths mark prior to the programme, to the Maths mark after the programme.

The result is statistically significant (see the last column, p < .000). The learners' results, on average, improved with about 9.9% (Mean difference is indicated in the “mean” column. Usually such a result is indicated as follow:

t (54) = 6.852; p <  .000

To calculate Cohen’s d, we divide the mean difference by the standard deviation

d = mean difference/ standard deviation = 9.98148 / 10.70442 = 0.932

0.932 is larger than 0.6 so this can be classified as a large difference. In fact it is close to 1, which means that this programme probably helped the learners, on average, to improve their marks with about 1 standard deviation. That is amazing!