Lies, damned lies and statistics

Most people would regard publishing as lacking cultural diversity – and they would be right, according to a survey into ethnicity in the industry. […] Only 13% of respondents to the survey belonged to Asian, black, Chinese or other minority ethnic groups…
Guardian Review , 13 March 2004

Accurate statistics are just facts, and as such, they don’t lie. Nevertheless, the bad reputation they have as being the source of the darkest deceptions is not entirely unfounded. For the very indisputability of a statistic can transfix us, leaving us blind to the unproven fact it is supposed to demonstrate.

This little statistic about cultural diversity in publishing is a wonderful example of how a statistic by itself rarely tells you anything. The writer does obey the first rule of good statistical reporting, by making clear that this all “according to a survey”. We need to be reminded that statistics are only as good as the data used to generate them, and so just because one survey said this many people think or do that, it ain’t necessarily so.

But having made that caveat plain, the rest of the report is a disaster. It takes the statistic to support the view that publishing “lacks” culturally diversity. The clear implication is that the industry is not as diverse as it should be because “only” 13% of respondents belong to minority ethnic groups. But what percentage would be high enough to indicate sufficient cultural diversity? We are given no benchmark or comparative figure to contextualise the statistic and enable us to give it a fair interpretation.

The most obviously relevant statistic would be the actual proportion of the population as a whole which belongs to a minority ethnic group. If the proportion of people working in publishing is less than this, then we would have grounds for saying that it is not culturally diverse enough. In fact, according to the most authoritative statistic, the last UK census in 2001 , ethnic minorities make up 7.1% of the population. In other words, publishing contains, proportionately, almost twice as many people from ethnic minorities than the population as a whole. This leads us to exactly the opposite conclusion to the one implied by the report: publishing is more culturally diverse than we would expect it to be.

This is typical of what makes so much reporting of statistics misleading: they are presented as though they speak for themselves when comparators and interpretations are indispensable. Consider the number of people in the UK who leave school with no qualifications. The opposition Conservative party points out on its website that 30,000 children leave school without a GCSE (the standard qualification taken at age 16). We are presumably supposed to think this is scandalously high. However, the Department for Education and Skills can also report that “the number of children who leave school without qualifications has decreased for the seventh year running. Now nearly 95% leave school with a qualification.”

But even that is not game, set and match to the government. For it is possible that the number of children leaving school without a qualification has decreased because the number of school leavers as a whole has decreased, for demographic reasons. If that is the case, the 95% figure may not have changed, or may even have come down. Furthermore, even if there has been an improvement, how are we to judge if it has been a good enough one?

It is perhaps surprising that so many statistics sound good or bad to us, even though we have no knowledge at all of the context and comparisons that would enable us to say if they are truly good or bad. Whatever the explanation, innumeracy itself is not entirely to blame. You might be perfectly able to do the maths and still misinterpret what the statistics actually mean.

Comments are closed.