Testing Maximum Performance

This is an interesting article by Jonah Lehrer of the Wall Street Journal about the limits of standardized testing.

https://www.wsj.com/articles/SB10001424052748704471904576230931647955902

Lehrer discusses the results of a study from the 1980s in which psychologist Paul Sackett attempted to measure the speed of supermarket cashiers.  A short “check-out test” was developed which involved scanning a small number of items.  The test was administered, and resulted in a list of the fastest cashiers.

What is interesting is that when Sackett compared the results of the test with long-term data collected by the electronic scanning systems, there was a surprisingly weak correlation between the results of the speed test with the data from regular usage.  That is to say, there was no real connection between being fast on the test and being a fast on a day-to-day basis.

Sackett’s misconception, and perhaps one held by many, is that there is a natural correlation between maximum performance (that on a short test) and typical performance (that is, under normal, day-to-day circumstances).   Tests like the SAT, the GRE, and other high stakes tests, are tests of maximum performance.  Our educational system relies on these  more and more, but are we sure they measure what we assume they measure?

Lehrer points out that individual success is determined more by character traits like perseverance and self-control,  but of course, it’s hard to capture that in a timed, multiple choice exam.