Why school league tables can’t answer your crucial question

You’ll remember that I found a good guide to how to win big on the UK lottery in my breakfast reading the other day. Well that same journal also helps us to understand what can be done with school league tables and, more importantly what can’t. For those of you who may have been thinking about moving house to get into a specific catchment area, read on.

UK school performance league tables are (currently) based on the performance of students in ‘National Curriculum Assessments’ (SATs) at various stages as well as exam results. These tables are therefore rather obviously a snapshot of current performance and, where they are available, give some information on trends in performance. But trends in what? In their paper The limitations of using school league tables to inform school choice, George Leckie and Harvey Goldstein explain.

The first thing Leckie and Goldstein point out is that it is very difficult to unravel the effects of the school (good teaching etc) from the effects of ‘who attends that school’. So leafy suburbia schools appear to do well because their pupils are generally more able. But in fact schools in deprived areas of lower general educational attainment may in fact be making more of a difference to their pupils. But you can’t tell that from the basic ‘performance tables’, you have to look instead at the ‘Contextual Value Added Measure‘ which tries to work out how well children have progressed from Key Stage 2 relative to other children like them. This helps but it is not the end of the story.

Leckie and Goldstein use a statistical method to analyse just how much effect schooling has on GCSE scores compared to parental background, gender, age within cohort (i.e. young for year), special educational needs, neighborhood deprivation etc. What they find is that schooling accounts for only about 10% of pupil’s progress from Key Stage 2 to GCSEs. Now this is not exactly 10% in any school. Using their statistical model they calculate that some schools make a larger difference than others. But the problem is that because these effects are not precise, it is really not possible to distinguish (statistically) between 47% of schools.

But the really good bit is that Leckie and Goldstein have repeated the analysis for each year between 2002 and 2007 and discovered that the ‘school effect’ for each school does not stay constant over time. There are many with high school effects in 2007 but who had low in 2002 and vice versa. So? So you can’t assume that current school performance is a good predictor of  school performance in 5 (or 7) years time. And if you are choosing a school for an 11 year old that is exactly what your are trying to predict.

As a parting shot their final calculation is to see if they can distinguish between the ‘effects’ of pairs of schools. After all as a parent you are not trying to tell if the school you are looking at is different from all schools, just is it ‘better’ than the other choices around you. Their conclusion is that when trying to predict future performance, mostly you can’t.

So don’t get hung up on school league tables – they may tell you a bit about what is happening now but they are generally incapable of telling you much about the future.

And for the record, the FT got it spot on.

---

Article access:

If you are not a subscriber to the journal you may be able to find a PDF of an earlier draft of the article via google scholar.