top of page

Subscribe Form

Thanks for submitting!

Search

ERIC CRAMPTON: The stakes are too high for this much stupidity

For context and further information this column is followed by an opinion expressed by Professor of Philosophy Tim Dare at the time when National Minister Anne Tolley pulled the pin on proceeding with similar work in New Zealand.


The New Zealand Economics Association annual meetings are a great way of keeping abreast of what the country’s economists are working on. And sometimes they’re downright depressing. At last week’s meetings, Auckland University of Technology’s Professor Rhema Vaithianathan’s keynote explained what she’s been up to over the past decade. Her team has been helping American child protection services to do a better job protecting kids. Child protection work is grim. Officials balance two terrible kinds of errors. Over-zealousness means a lot of families will be put through a painful wringer unnecessarily. But under-intervention means some kids who could have been helped will wind up abused, hospitalised, or killed. Unless you can find a way of reducing both types of errors. And Prof Vaithianathan’s team found a good one. Child protection workers have a mountain of administrative data for making decisions on whether to intervene in response to a call, but only about ten minutes to make each decision – then on to the next case. It is impossible to regularly make good decisions faced with that much complexity and that little time. Prof Vaithianathan’s team reduced complexity by turning data into a predictive score laying out the risk each case posed, to help child protection workers make the right call. They started the U.S. work in Allegheny County, Pennsylvania, because Allegheny wanted to use data to make better decisions. The programme’s success inspired others to try it out. And a later randomised control trial showed that the system reduced child hospitalisation by a third. It also reduced the bias that case workers otherwise bring with them in making assessments. The risk scored meant more high-risk white families received help and fewer low-risk black families had to deal with child protection services. It’s a great story. The depressing part? The work started here in New Zealand. It was killed by Anne Tolley as Minister, who described it as experimenting on kids. And the subsequent Labour government showed even less interest in data-based approaches. One third fewer hospitalisations for children in risky families. But not here. In America instead. Thanks to Kiwi researchers, who were chased away from doing the work here. An innovative American county can try something new and let others follow. New Zealand’s centralisation means a single bad Ministerial decision can cause a lot of harm for a very long time. Far better policy, and outcomes, are possible. Even here. But voters have to demand it.


Eric Crampton is the Chief Economist at The New Zealand Initiative. This piece was published in the NZ Initiative’s newsletter, Insights, 7 July 2023


PROFESSOR TIM DARE: Anne Tolley's 'lab rats' call inflammatory political rhetoric


A minister sees a briefing paper with a proposal to test a computer model designed to identify children at risk of maltreatment. She reacts strongly.


"Not on my watch!" she writes in the margin, "these children are not lab-rats". The study is shelved.


The media obtain the briefing paper, complete with the marginalia, and publicise it.


The Opposition seize on the lab-rats cry and use it in the House against the Minister of Social Development and the ministry.


Should we feel relieved? Have we averted another unfortunate experiment? No.

The minister's reaction, and the media and Opposition response to it should make us feel uneasy.


The problem is not the shelving of the study – though that was a mistake too – rather it is the chilling effect of the knee-jerk and political response to an attempt to produce evidence for important social policy.


Science collided with politics, and politics won.


The study was an observational study intended to test the accuracy of a computer model designed to identify children at risk of maltreatment.


It would have seen all children born in 2015 rated for risk. Data, which would have been collected over the next two years in any event, would have been analysed to see if the computer model had accurately identified those who were in fact been maltreated over that period.


The study did not involve any new or untried interventions: that is what it means to say it was an observational study.


During the period of the study, the children being observed would have received exactly the same services they would have received if there had been no study.


In the ministry's words, the study "would not have displaced standard response. Agencies such as Child Youth and Family and the Police would have, at all times, continued to act on notifications in relation to vulnerable children, as they do now. All children would have received the full range of support available from agencies".


The study did not propose to get the children or their families to do anything they would not otherwise have done.


It did not involve the withdrawal of any services.


Any notification that a child was at risk – notifications generated just as they are now, and independently of the computer model being tested – were to have been responded to exactly as they are now.


The 'observations' were entirely in the background: producing an automated risk ranking from data already held, and looking through records of substantiated abuse at a future date.


The study posed no risk to the children: it did not involve any intervention.


Now, of course, observational studies can raise significant ethical issues.


It is probably impossible to obtain consent from everyone in a study of this size; we know that the cases of verified abuse – the ones we would have looked for in the computer predictions – will not include all cases of actual abuse, and we might be concerned that it's easier to hide abuse if your GP thinks you're a good, hard-working, decent couple than it is if you're a beneficiary.


Those are the sorts of issues that would have been discussed by an independent ethical review of the research.


Perhaps such a review would have found the problems insurmountable and refused to allow it to go ahead.


But none of this warrants the knee-jerk dismissal of the research in a 10-word comment in the margins of a briefing paper.


The suggestion that the study would treat children as lab-rats is inflammatory and misleading, but rhetoric won the day.


The failure of anyone to take the trouble to try to substantiate the minister's initial concerns, rather than latching on to misleading rhetoric, bodes ill for the possibility of anything like a genuine research-driven approach.


That is something we should all be worried about.


Child maltreatment is an appalling and recalcitrant problem and New Zealand's record is particularly poor.


We will make progress in these areas only if we seek good evidence-based policy.


We must not stumble along relying on experience and anecdote: it leads us astray.


We owe the kids a better, more informed, approach.


Tim Dare is a professor of philosophy at The University of Auckland. In 2012 he was contracted by the Ministry of Social Development to provide an ethical analysis of the predictive risk modelling tool this study was intended to test.


3,376 views61 comments

61 Comments


justise xtra
justise xtra
Jul 10, 2023

Anne Tolley Queen of School Mergers, Killer of Teacher Degrees.

Has all her children grown up and left school?

Gosh (for a better word)

Can we just get over the countless Maori Courts those cost me money.

If these people and their donorship to humanity is to go by it is far from me to make sure they don't stay at home with me.

The fact that the New Justice Court or the old one in this case of 6 years in the making but trailing off my entrails for forever, just screwed with an Apostollie Act on both my Grand parents and they have uplifted them saying they were my family which are clearly not because I am the…

Like

Wasn't the Bill English "targetted welfare" utilising data to identify at risk families?

Like

Aaron Shanahan
Aaron Shanahan
Jul 10, 2023

No, no, no. And bloody well no.

Their is no science, no computer controlled system that will ever replace any parent, and the continual advancement of the taking away from me, a good dad, yes, with shortcomings like working to hard for my children's welfare leaves me cold.

I'm straight up now. Fuck this.it .makes me so fucking pissed off.

Don't look at the kids.

Fucking christ almighty.

Look at the problem that creates this cesspool, this collective mediocre attitude towards parenting. And I'm steamed.

Look at the way of life a lot of kids have to endure.

It's not their fault, it's the lazy, indolent lifestyle of those that breed simply for cash benefits fault and .lets be absolute…


Like

Robert Mann
Robert Mann
Jul 10, 2023

Where do we find the promised piece by Prof T Dare?


Like
gusimondo
gusimondo
Jul 10, 2023
Replying to

Had a nice wee snooze there Robert? 😎

Like

Right on Ron " to thick to understand " 100% correct applies to all parties. I have a simple methodology. Would you employ any of them. A resounding NO. I can't vote for any of them. I know that is copping out but at least you can't blame me for voting for A, B, C etc etc.

Like
bottom of page