Preliminary Thoughts on TIMSS/PIRLS 2012

I had been meaning to post about this for a long time. Better late than never, I suppose.

The TIMSS and PIRLS are international assessments of academic ability in math, science and literacy that are conducted once every four years. They are similar to the PISA tests, although the latter are less purely academically focused and more a test of pure IQ.

Here are the results of TIMSS/PIRLS (h/t North Asian). And here are the results of PISA from 2009 for comparison.

As can be expected, they are highly correlated (r > 0.8 to be precise). This however makes the few differences all the more interesting. The gap between the East Asian countries and European countries, though substantial in PISA, is significantly greater in TIMSS/PIRLS. And most strikingly, both Russia and Israel go from being laggards in the OECD group to being at the forefront of the class.

  Math (PISA) Math (TIMSS)
Korea 539 613
Sweden 494 484
Russia 468 539
Israel 447 516

From performing more poorly than Turkey in the PISA reading test, Russia soars to take second global position in the PIRLS.

  Reading (PISA) Reading (PIRLS)
HK 533 571
Sweden 497 542
Russia 459 568
Israel 474 541

Meanwhile, some European countries, especially Sweden and Norway, plummet quite substantially.

What explains all this?

There are two possibilities. First, the TIMSS/PIRLS tests may have poorer samples than the PISA. For instance, we know from the latter that Moscow has a 10-point IQ lead over the rest of the country. If Muscovite pupils are over-sampled, then it’s quite feasible for the consequent result to be closer to say Hong Kong or Korea than to Greece or Turkey.

However, a second possibility is that the PISA-TIMSS/PIRLS gap is a proxy for differences in the quality of educational systems. It is more feasible to prepare for the TIMSS/PIRLS than it is for PISA, which is closer to an IQ test and is, as such, more difficult to improve through policy interventions. It is nowadays fashionable to lambast the ex-Soviet and East Asian school systems for “rote learning,” “stifling creativity,” and whatnot. However, the data shows that under these systems, pupils perform well above the levels they “should” as indicated by their underlying IQ levels. Meanwhile, in places where “creativity” and “self-expression” are given full bloom, where science lessons focus on the evils of plastic bags in between sermons on LGBT appreciation and the progressiveness of Islamic civilization, academic performance is somewhat less than what might expect based on the local students’ apparent IQ levels.

This all makes sense, I suppose. To be truly “creative” you first have to acquire a ton of skills and knowledge via the old method of applied hard work. Without that, “creativity” simply boils down to a sea of PoMo-waffling curmudgeons and MacBook-toting hipsters. And whoever needs that?

If you like the words I write, and want me to write more of them, consider donating or supporting me on Patreon.