The Government agency that holds a mirror up to society

All Blacks hooker Dane Coles is still in doubt for  next weekend’s test against the Lions because of the concussion he suffered during a game back in May.

That lay off is part of rugby’s new tough concussion regime; a situation that was forced on the sport by the recognition that concussion suffered as a player might result in dementia in later life.

And now, perhaps surprisingly, the Statistics Department, or Stats NZ as it now likes to be known, has got in on the act.

The Rugby Union has linked into Stats database to compare rates of dementia among top-level players in New Zealand between 1950 and 1970 with the rates of those who did not play rugby.

It’s something of a show piece for Stats’ “Integrated Data Infrastructure” — or IDI — which is now becoming the heart of what the organisation does.

Using the IDI the Rugby Union will be able to access a vast amount of “person-centred” data that Stats is accumulating from other Government departments and more recently from non-governmental organisations (like the Rugby Union).

When Stats gets the data it “strips out” its identifying aspects such as names and addresses and then by making it possible to link it all together it becomes possible to draw up complex matrices which offer a new dimensional way of looking at New Zealand life.

Government Statistician Liz MacPherson is driving the move to open Stats up but all the time she has to be conscious of keeping sensitive data “safe” and private so that individuals’ privacy is not compromised.

“We now have this very potent set of data which can allow you to look at the impact of life experiences on life outcomes for the first time,” she says.

“It can allow you to look at complex overlapping areas that simply weren’t able to do before.

“So it’s very powerful so you have to keep it safe and the way we keep it safe is that we focus on ensuring the people who are using it are safe.

“So they are researchers who have got good bona fides who know how to use this stuff.

“We make sure that the projects are safe so that they need to be in the public interest.”

There are other controls around how and where the data is accessed — but the overall objective is to maintain what is called the “social license” which simply means that agencies can only get help from the public if the public agrees with what they are doing and trust them.

But as the Rugby Union example shows, it can be a two-way street, where agencies contribute data to the overall database or, they can just “clip” their data on temporarily while they conduct research using the system.

This places the IDI system at the heart of the Government’s social investment programme.

“If you think about it from an NGO perspective and you are being asked by the Government to demonstrate that you are achieving good outcomes for the money that

you are being given then this is an ideal way of doing it.”

But the system also has applications that extend beyond the social services sector.

As a result of work done by Treasury and Stats, there is now an app on the Career Services website which allows students to input their course to find out their employment chances when they graduate.

Stats, however, knew more than what was in the app. They knew which institutions were producing employable graduates (and potentially more controversially, which were not) but they were unable to divulge that information because of restrictions in their Act.

However, after negotiations with the Ministry of Education and the institutions, most have now agreed to allow the information to be made public.

Again, there is a very fine line between privacy (and confidentiality and public knowledge.

This year the institutions are getting to see their results and next year they will go into the app.

The implications of this for the tertiary sector are huge.

Inevitably the data will be seen as another move along the road to creating an education “market” driven by pragmatic considerations like employability rather than providing a full education and for some institutions, which are failing to deliver employable graduates, the results could see a downturn in student numbers and therefore funding.

But issues like this involving privacy point to one of the difficulties Stats faces with the IDI.

Social Investment Agency Minister Amy Adams has complained that privacy restrictions make it difficult to direct aid and interventions for some clients of the welfare sector who have been identified from the data as likely to need it.

MacPherson acknowledges this limitation.

“The IDI is very good for doing things like looking for patterns or trends.

“It’s very very good at telling you the effectiveness of a programme.”

What it can’t do — because the data is “de-identified”  — is point social workers in a specific direction.

MacPherson suggests that there are other ways of achieving this. In a sense she recognises that statistics can go only so far; sooner or later human inter-actions have to take place.

“A lot of this comes down to social workers working with their clients to basically gain consent,” she says.

This was an issue among Maori at last year’s Data Hui, and iwi groups agreed that if Maori had the reasons for the way the data was going to used explained to them they were more likely to co-operate.

The IDI is world leading and is attracting international attention.

MacPherson hopes that Stats may be able to monetise the software.

Already there are inquiries from Britain, Australia and just recently a delegation from Singapore came to look at how it works.

But the IDI is not all that is changing at Stats.

The organisation began by using so-called “administrative data”; official Government statistics from activities like births and deaths registrations or customs duties. Then it moved on to survey data like the consumers’ price index where researchers fan out and manually check prices in shops.

But now the web has made real-time data possible.

So the CPI is increasingly deriving its raw data from ads on websites.

New Zealand is recognised as a world leader in this field, and a young Stats staffer is chairing a committee for the UN looking at real time data.

What this all adds up to is the increasing centrality of Stats’ role within the Government sector.

And it’s a role MacPherson regards as logical and inevitable.

“Ultimately our goal, and it is a goal we take very seriously, is to hold a mirror up to society and effectively say this is where we were, this is where we are, and this is where we are going,” she says.