Preliminary Thoughts on TIMSS/PIRLS 2012

I had been meaning to post about this for a long time. Better late than never, I suppose.

The TIMSS and PIRLS are international assessments of academic ability in math, science and literacy that are conducted once every four years. They are similar to the PISA tests, although the latter are less purely academically focused and more a test of pure IQ.

Here are the results of TIMSS/PIRLS (h/t North Asian). And here are the results of PISA from 2009 for comparison.

As can be expected, they are highly correlated (r > 0.8 to be precise). This however makes the few differences all the more interesting. The gap between the East Asian countries and European countries, though substantial in PISA, is significantly greater in TIMSS/PIRLS. And most strikingly, both Russia and Israel go from being laggards in the OECD group to being at the forefront of the class.

  Math (PISA) Math (TIMSS)
Korea 539 613
Sweden 494 484
Russia 468 539
Israel 447 516

From performing more poorly than Turkey in the PISA reading test, Russia soars to take second global position in the PIRLS.

  Reading (PISA) Reading (PIRLS)
HK 533 571
Sweden 497 542
Russia 459 568
Israel 474 541

Meanwhile, some European countries, especially Sweden and Norway, plummet quite substantially.

What explains all this?

There are two possibilities. First, the TIMSS/PIRLS tests may have poorer samples than the PISA. For instance, we know from the latter that Moscow has a 10-point IQ lead over the rest of the country. If Muscovite pupils are over-sampled, then it’s quite feasible for the consequent result to be closer to say Hong Kong or Korea than to Greece or Turkey.

However, a second possibility is that the PISA-TIMSS/PIRLS gap is a proxy for differences in the quality of educational systems. It is more feasible to prepare for the TIMSS/PIRLS than it is for PISA, which is closer to an IQ test and is, as such, more difficult to improve through policy interventions. It is nowadays fashionable to lambast the ex-Soviet and East Asian school systems for “rote learning,” “stifling creativity,” and whatnot. However, the data shows that under these systems, pupils perform well above the levels they “should” as indicated by their underlying IQ levels. Meanwhile, in places where “creativity” and “self-expression” are given full bloom, where science lessons focus on the evils of plastic bags in between sermons on LGBT appreciation and the progressiveness of Islamic civilization, academic performance is somewhat less than what might expect based on the local students’ apparent IQ levels.

This all makes sense, I suppose. To be truly “creative” you first have to acquire a ton of skills and knowledge via the old method of applied hard work. Without that, “creativity” simply boils down to a sea of PoMo-waffling curmudgeons and MacBook-toting hipsters. And whoever needs that?

Minorities’ Cognitive Performance In The UK

Here is data from the Cognitive Abilities Test for UK students in 2009/10 via Ambiguous.

Some interesting things to take away here:

(1) The sample is very large. Verbal IQ has the highest correlation with academic performance in most subjects, followed by Quantitative IQ, and then Non-Verbal Reasoning (recognizing patterns and such, I imagine).

(2) Indians do almost as well as Whites, although the structure of their cognitive abilities are a bit different: About 4 points lower than Whites in Verbal, but almost 2 points better in Quantitative. As rec1man said, “The Patels and Sikhs are Upper-Shudra / Vaishya and this is 80% of the diaspora in UK.” So this is highly encouraging for India’s eventual prospects; in indicates that the broad middle can in principle build a reasonably wealthy, middle-class society.

(3) The Pakistanis and Bangladeshis also don’t do too badly – certainly better than I would have expected (I visited a Pakistani school once in the UK and it was horrific).

(4) This might imply we are actually looking at the following average-IQ groups in India: Lower 40% – 93; Middle 40% – 99; Top 20% – 105, for an overall average of 98 (once Flynn Effect is done with them). That’s better than Greece today and certainly good enough  to have a developed society. But there’s tons of challenges: Malnutrition, slums, poor education, widespread vegetarianism (both voluntary and involuntary – due to poverty) that have to be sorted out for India to perform to its potential.

[Read more...]

Here Be Dragons: World Map Of Women’s Studies Programs

One notices a remarkable correlation between this, and the perceived attitudes of local women and their obesity rates.

(The map above was made by RVF commentator “durangotang” based on the geographic data here).

Analysis Of Russia’s PISA 2009 Results

A few months ago I posted a table and map of Russian IQ’s as derived from regional PISA performance. Those figures are based on Jarkko Hautamäki’s slideshow comparing regional PISA performance in Finland and Russia.

That material is a bit inadequate because, as had been my custom up that point, I was only making IQ estimates based on the Math and Science components of the PISA tests, and avoiding Reading to maintain reverse compatibility with my (now disused, in favor of just IQ) Human Capital Index. In light of some realizations that verbal IQ is no less important than numerical, I have updated the figures to include the verbal component as well. This doesn’t create any radical changes – the overall IQ only drops by 0.3 points – so I reuse the same map.

(Note that the legend on the map isn’t converted to IQ. “PISA scores, mean 500, SD 100, have to be transformed into IQ values, mean 100, SD 15, by adding or subtracting the deviation from the mean in the relationship 100 : 15 = 6,67.”)


There are any numbers of comments one can make, but I will confine myself to the most important ones:

(1) In some regions, margins of error are high, as samples were low. Nonetheless, it is still possible to identify some concrete patterns. The overall estimate is very accurate because the sample was N=5,308 and representatively distributed across the country.

(2) Moscow pupils performed very well, at the level of the highest scoring OECD countries like Finland, Taiwan, and Korea. This is especially impressive considering the significant numbers of immigrants in that city from the North Caucasus and Central Asia, who come from poorly-scoring countries and rarely have good Russian. This is surely the result of a century of attracting Russia’s (the USSR’s) cognitive elite.

(3) St.-Petersburg and Tyumen oblast performed above the OECD average, while a few other regions performed at or only slightly below the OECD average.

(4) Among ethnic Russian republics, Siberian regions performed well, while the Urals and southern regions performed badly.

[Read more...]

Is The Ukrainian Children Learning?

According to a recent Vzglyad article by Olga Gritsenko titled Universal Stupefaction, no they are not. Here are the cold raw facts:

  • Libraries stock 4% of books published in Ukraine, compared to 18% in Russia and 40% in the US and Canada.
  • The average Ukrainian spends $2.5 on books in one year, compared to $22 in Russia.
  • In 2010/11, the average Ukrainian spent just under 3 hours reading newspapers and journals per week, down 25% from 2007/08. The equivalent figure in Russia is 7 hours.
  • In fairness, their universities are rated higher than Russia’s (as well as Poland’s and the Czech Republic’s) by an outfit called Universitas 21.

Obvious counter-objections don’t explain these shortcomings. Russia has a higher Internet penetration, but nonetheless Russians read a lot more books and newspapers. Nor can a nearly tenfold difference in per capita book sales be purely or even mostly a reflection of lower book prices in the Ukraine.

That said, in a sense these statistics aren’t surprising. According to international student assessments, the level of human capital in Ukraine appears to be similar to the lowest ranked ethnic Russian provinces in Russia. This does not bode well for Ukraine’s future economic growth, given the tight interrelationship between human capital and development, and might go some way to explaining the already big – and growing – prosperity gap with its Moskali neighbors.

Italy, A Showcase For Why Human Capital Matters

One of the key criticisms of my last post on the tight connections between (educational) human capital and economic performance is that correlation need not imply causation. An alternate (and PC-compliant) explanation is that “you get the education system you could afford, and the level of human capital in the kids is mostly determined by the efficiency of the schooling system.” Is there any evidence to support this argument?


As a country with a long history of normal capitalist development, and with no regions enjoying particularly lucrative resource windfalls, it is not a surprise to see a close exponential correlation between my Human Capital Index (see methodology here) and Italy’s GDP (PPP, in US dollars) per capita (R2=0.7302).

[Read more...]