Our maths performance results on TIMSS and National Standards

Interest.co.nz
5 December, 2016

From ‘that’ NCEA maths exam that left students in tears, the confusion around exams during last month’s earthquake, to the seclusion of special needs students, it seems the education sector rarely gets a break. And it looks like that will not change anytime soon with the latest news in education. The most recent international benchmarking test results tell, once again, a doom story for maths performance.

The Trends in International Mathematics and Science Study 2015 (TIMSS) assesses Year 5 and 9 students’ proficiency in these two subjects every four years. The latest test was taken by about 600,000 students in around sixty countries, of which 10% of Year 5 and 9 New Zealand students took part. The previous 2010 results had revealed that by international benchmarks our maths performance had continued to drop since 2002.

The latest results too bear bad news; despite reform efforts in the past decade to lift achievement, New Zealand’s performance in maths at both levels has remained stagnant. We are no longer declining but neither are we inching our way up. It is unlikely that a student can get away with saying ‘I am just not good at reading or even writing’ but it is common and almost accepted to hear one say ‘I am just not good at maths’. And a pessimist might argue there is a point in the latter because no matter which way you slice it, the maths story leaves little to celebrate. Some key outcomes are worth highlighting here:

Relative to past performance, there was no significant change between 2010 and 2014 results for Year 5 and Year 9 students.
With regards to progress, there was no significant change between 2010 Year 5 students and 2014 Year 9 students. The four-year administration makes TIMSS performance a good indicator of how the middle primary school cohort have progressed four years-on. Although not exactly the same students, the Year 5 student cohort assessed in TIMSS in 2010 formed the Year 9 TIMSS cohort in the latest round.
Average New Zealand performance across both levels puts the country below Australia, England, and the United States. Year 5 students’ performance was similar to students from France and Turkey while Year 9 performance was similar Sweden, Italy, and Malta.
In fact, during the 20 years of the test, we have been consistently on par with or below the international centre point. So our average performance isn’t even average.

But there is another clincher; the gulf that exists between what international assessments of curriculum knowledge show, and what our national maths assessments show. The first tells us we have a great long way to go while the other tells a story of success.

On the one hand, the international test showed that despite test questions being aligned to the curriculum of each country, the average student answered just under half of those questions correctly. Worse, by the end of 2014 less than half of the students were working at the desired level of in the New Zealand Curriculum. This was true for both Year 5 and 9 students. One would rationally think then that domestic assessments would reflect a similar proportion of students meeting curriculum level targets. Looking at the numbers, this does not appear to be the case.

On the other hand, domestic assessments of performance show that, by the end 2014 - the same time that a random sample of Year 5 students took the TIMSS test - 73% of Year 5s were judged to be achieving ‘at’ or ‘above’ their year level standards in maths. Almost 70% of Year 8 students were judged similarly. A sample from the latter group sat the Year 9 international test the following year.

It is difficult to understand how tests developed to assess a similar level of knowledge place the same cohorts at different levels of the achievement ruler. National Standards were introduced in 2010 for Years 1 to 8 students to get a more accurate assessment of the progress that each child had made against the curriculum. This progress is assessed through subjective overall teacher judgements and teachers can decide which assessments to use to determine whether each of their students is below, at, or above National Standards.

Indeed, TIMSS surveyed teachers’ assessment methods and found that more Year 5 students had teachers who emphasised the evaluation of students’ ongoing work vis-a-vis nationally available tests. The reverse was true internationally, as many more teachers placed major emphasis on classroom tests. But are teachers’ judgements here perhaps too optimistic or too subjective?

There have been attempts to moderate teacher judgments. For example, the Process and Consistency tool (PaCT) was developed to give teachers confidence in their assessments on National Standards, and to ensure their interim and end-of-year Overall Teacher Judgments are based on valid information, consistent with those of other teachers in the same year levels, and also their previous judgments. Given the contradiction in international assessments when compared to domestic assessments of how many students have a grasp of maths skills, it is a pity that some schools have resisted using the tool. In any case, the paradoxical equation may be worthy of a solution.

Stay in the loop: Subscribe to updates