*This post was authored by Ryne Sherman and Brandon Ferrell.
A recently published study suggests that some of the most common personality assessments (i.e., one’s based on the Big 5) don’t work in other countries. The study was published in a prestigious journal (Science Advances, impact factor > 12), and it has already gained prominent media attention. One outlet said that these personality tests don’t hold up around the world. NPR said that personality tests don’t reveal the real you. Reading these articles might make you conclude that personality assessments just can’t be used in other countries. Fortunately, despite what the economists who contributed to the article and the journalists who are covering it might have you believe, such a conclusion is just wrong. In what follows, we show you why.
Comparing Assessments
Across Borders and Languages
If
you had a rod that measures 1 meter in Australia, but 2 meters in Kenya, you
have a big problem. Clearly, the term “1 meter” doesn’t mean the same thing in
different locations or different languages. As a principle of measurement, you
want to be sure that whatever you are measuring in one location (or one
language) is the same thing that you are measuring in another. In terms of
personality assessment, comparing countries (or languages) absolutely requires
that the assessments are used in the same fashion across countries and
languages. Psychologists use a metric called the congruence coefficient to determine the degree
to which instruments are measuring the same thing. Scores on the metric can
range from -1.00 to +1.00, with higher scores indicating greater similarity. The
accepted standard for declaring the instruments as similar is a congruence
coefficient > .84. The recently published study found average congruence
coefficients of .73 and .71 in survey data gathered in so-called non-WEIRD countries (e.g., Kenya, Philippines,
Colombia, etc.).
Read More »