New Zealand is world leading in many aspects, most notably for Sir Edmund Hillary’s triumph on Mt Everest, Ernest Rutherford’s breakthrough in nuclear physics, and women’s suffrage.

We can also be proud of leading the world in integrated data, a process that combines data from different sources and displays results in a unified view to users.

Statistics New Zealand (SNZ) in 2011 started integrating data from all government agencies in New Zealand in what is now called the Integrated Data Infrastructure (IDI).

Today, the IDI is New Zealand’s largest research database. It contains data on more than 5 million Kiwis covering information from education, health, tax and income, social services, housing and many other aspects.

Researchers use this data to study populations and investigate the impact of social services and programmes on people’s lives. Insights gained from the IDI are then used to inform policy, including health interventions, social services and education reform. Projects in the IDI are strictly limited to studies that will benefit the public and have no commercial gain.

Integrated data – and the IDI – is unique because data on individuals is de-identified while also linkable across different ministries, social surveys and censuses. This means Joe’s income can be linked to his education, his health, and his household during the last census. It also means Joe’s data is completely confidential; researchers can only see a unique ID number, never Joe’s name or address.

Because of the sensitive and personal data in the IDI, SNZ highly prioritises confidentiality. To safeguard against any misuse of data, only qualified and vetted individuals and organisations are granted access to the data. Access is granted on a project-by-project basis and only to the specific databases the project requires. Research is typically conducted by government agencies, universities and think tanks such as The New Zealand Initiative.

The Initiative was privileged to be granted access to the IDI this year to conduct research for our upcoming report on education evaluation.

This project aims to find out how much value a school adds to its students after adjusting for each student’s background characteristics. By creating New Zealand’s first contextualised value-added model, we will determine how much variation there is in secondary school quality in New Zealand. We are also examining the factors that contribute to a student’s success at school and later in life.

We look forward to sharing our findings in 2019 made possible only due to New Zealand’s world leading data resource.