Just over a week ago, the OECD released its two-yearly economic report card for New Zealand.

At first glance, New Zealand looks picturesque according to the 2017 Economic Survey. The report points to our strong economic growth supported by our thriving tourism sector, strong migration policy settings, and robust monetary policy.

But the Survey also does a solid job pointing out areas that require urgent attention. It notes that productivity has slowed, the housing crisis has worsened, and our remoteness continues to limit our ability to trade internationally.

The OECD also calls New Zealand to action over another persistent pain point. In fact it is one that was also highlighted in the 2015 Survey. And that is our dismal competency in basic numeracy.

Basic numeracy and literacy skills have become increasingly crucial as the trend towards high-skilled and highly-automated jobs continues. Research also shows that differences in these abilities are important predictors of countries’ long-term economic performance.

Improving the maths ability of primary and secondary school students matters. A lot.

To lift numeracy standards, the OECD recommends supporting teachers’ maths pedagogy, using evidence-based teaching, and raising entry requirements into teacher training programmes as well as the quality of that training.

There is cross-sector agreement that a problem exists. The Education Council has put out a consultation document in which it considers higher entry standards into teaching programmes.

The Government has too redesigned the teacher professional development framework to include maths teaching as a core area. It also supports Teach First NZ, a relatively new teacher-training programme that fast tracks top maths, science, and technology graduates into a teaching career.

Clearly New Zealand is not short of solutions to the maths equation. But herein lies the problem. How do we know that these solutions will indeed improve our dire situation?

Albert Einstein once said “If I had one hour to save the world, I would spend fifty-five minutes defining the problem and only five minutes finding the solution.” And he makes much sense.

What if we reframed our thinking and start treating poor maths performance as a symptom of other problems rather than the problem in and of itself?

Shifting our focus from solution to problem definition could for example show that teacher numeracy skills are on par with other graduates. Such an insight could mean that raising entry standards into teacher training may not to solve the maths concern. But we just don’t know.

We need to know what we know about the problem and what we don’t know.

Starting with what we know: Over the past twenty years New Zealand’s performance in international benchmarking tests has been consistently below the OECD average. We have one of the highest proportions of students unable to answer basic maths questions and the proportion has grown for since 1995.

In 2011, our 9-year-olds were bottom equal among developing nations in the science and maths test. In 2013, our own national monitoring study found that less than half of our 12-year-olds were working at the expected level in maths.

Many point to anecdotes of poor teacher quality in primary schools as the reason. Anecdotes can be powerful, but if we want to make lasting changes, a few war stories should only encourage us to dig deeper for evidence.

We are aware of the unintended consequences that can result from assuming the wrong problem.

In 2001, the government implemented the Numeracy Project to pioneer a new way of teaching maths and solve the maths issue once and for all. Instead the project cost $70 million yet our eight and twelve year olds are no better at maths than they were a decade ago when compared internationally. The programme that was meant to solve what was thought to be the problem is now widely blamed for worsening matters.

But there is plenty we do not know.

To begin, we actually don’t know how numerate our primary school teachers are. We do not know if to be a good maths teacher you need to be good at maths. We also do not know if being good at maths necessarily means you will be good at teaching maths.

In fact the Education Council says that while it has anecdotal information that some teachers do not have adequate numeracy or literacy levels to teach, there is limited evidence about how widespread the issue is.

What if we found that those who scored relatively low in numeracy when they were 15 years of age were also more likely to choose teaching as a career compared to those who scored higher in maths?

Our qualifications framework allows students to gain numeracy credits in subjects other than pure maths. Could it be possible that those who choose to get into primary school teaching generally gain their numeracy credits through non-traditional maths subjects?

Luckily, these are questions that we can start to unpick. And there is a vault with information that can help piece things together.

The Integrated Database Infrastructure hosted by New Zealand Statistics enables vetted researchers to link anonymised data on every individual in New Zealand, with their subject choices in high school, their results in those subjects, and their post-secondary pathways and outcomes.

It is difficult to improve anything if you do not know why it is the way it is in the first place. We need to separate the root problem from the symptoms. If we do not, it is only a matter of another two years and the numeracy crisis will resurface in the OECD’s report card.