Dunning-Kruger Effect and Evaluation

Justin Kruger and David Dunning published a paper in the Journal of Personality and Social Psychology (1999, Vol 77, No.6, 1121 -1134) and the term "Dunning Kruger effect" was coined.  This is the abstract:

People tend to hold overly favorable views of their abilities in many social and intellectual domains. The authors suggest that this overestimation occurs, in part, because people who are unskilled in these domains suffer a dual burden: Not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the metacognitive ability to realize it. Across 4 studies, the authors found that participants scoring in the bottom quartile on tests of humor, grammar, and logic grossly overestimated their test performance and ability. Although their test scores put them in the 12th percentile, they estimated themselves to be in the 62nd. Several analyses linked this miscalibration to deficits in metacognitive skill, or the capacity to distinguish accuracy from error. Paradoxically, improving the skills of participants, and thus increasing their metacognitive competence, helped them recognize the limitations of their abilities.

Errol Morris described how the following sad story about a guy called McArthur Wheeler, inspired Dunning's scientific inquiry:

Wheeler had walked into two Pittsburgh banks and attempted to rob them in broad daylight.  What made the case peculiar is that he made no visible attempt at disguise.  The surveillance tapes were key to his arrest.  There he is with a gun, standing in front of a teller demanding money.  Yet, when arrested, Wheeler was completely disbelieving.  “But I wore the juice,” he said.  Apparently, he was under the deeply misguided impression that rubbing one’s face with lemon juice rendered it invisible to video cameras. If Wheeler was too stupid to be a bank robber, perhaps he was also too stupid to know that he was too stupid to be a bank robber — that is, his stupidity protected him from an awareness of his own stupidity.

What does this have to do with evaluators? All I suggest is that you should think a little about the Dunning-Kruger effect next time you ask people to rate their own competence level in a survey. You would not want to design such a survey without knowing that it is not a very smart thing to do, right?

Alternatively you might want to read an earlier post I did about it here.