In 2016, researchers discovered that the Northern Territory’s efforts to improve school attendance by making welfare payments conditional on getting kids to show up had no effect on attendance.

Loading

In Dubbo, other researchers found that if you made a literacy test more culturally relevant by changing a story about lighthouses to one about the dish-shaped telescope in Parkes, you halved the gap between the scores of Indigenous and non-Indigenous kids.

In NSW, researchers found that giving early childhood staff a half-year professional development program boosted the achievement of their kids, especially their literacy.

Yet more researchers – in Brisbane, Perth and Sydney – found that, despite the students’ involvement in designing the Health4Life program, it had no effect on alcohol use, smoking, screen time, physical inactivity, poor diet or poor sleep.

What all these research efforts had in common was that they evaluated these programs using RCTs – randomised controlled trials. This involves using the toss of a coin to divide similar participants in the trial into two groups. One group gets the treatment and the other “control” group doesn’t. You then compare the two, confident that any differences between them have been caused by your intervention.

Loading

Point is, this is a far more rigorous way of judging whether government spending programs achieve the benefits you were hoping for, rather than just doing a pilot program and deciding whether it seems to have worked.

But these four careful trials are the exception, not the rule. A study by the Committee for Economic Development of Australia examined a sample of 20 federal government programs worth more than $200 billion. It found that 95 per cent of them hadn’t been properly evaluated. The committee’s examination of state and territory government evaluations reported similar results.

“The problems with evaluation start from the outset of program and policy design,” it said. Across the board, the committee estimated that fewer than 1.5 per cent of government evaluations use a randomised design.

Similarly, a Productivity Commission report in 2020 into the evaluation of Indigenous programs concluded that “both the quality and usefulness of evaluations of policies and programs … are lacking”.

Loading

“Evaluation is often an afterthought rather than built into design,” it found.

This is in marked contrast to the medical profession, where controlled trials are standard in the evaluation of medical operations. These have demonstrated that the treatments preferred by experts were often worse for patients.

For instance, radical mastectomies for breast cancer disfigured 500,000 women while doing nothing to increase their odds of survival. Many treatments found to be harmful had been supported by expert opinion and low-quality before-and-after studies.

If you can feel a commercial message coming on, you’re right. Dr Andrew Leigh, former economics professor and now Assistant Minister for Treasury and many other bits and bobs, has been championing the use of randomised controlled trials in government program evaluation for years.

Loading

And last year the Albanese government set up within Treasury the Australian Centre for Evaluation, with Leigh responsible. It aims to expand the quality and quantity of program evaluation in co-operation with other government departments. Its leader, Eleanor Williams, has a modest budget and a staff of more than a dozen. A key principle is that high-quality evaluation of a program’s impact needs to be built into the design of the program from the get-go. The centre will also collaborate with evaluation researchers outside government.

And now the Paul Ramsay Foundation, Australia’s largest charitable foundation, is providing a $2.1 million round of grants for people to run randomised trials on important social problems. The centre, which has been given access to a wealth of “administrative data” – statistical information collected by government departments – will make this available to academics and others receiving grants.

I think this is all to the good. And about time. Econocrats went for decades supporting the push for smaller government, which led to the privatising of many government-owned businesses (including a national electricity market now dominated by three big companies) and much outsourcing of government services to private businesses – which, as should have been expected, have proved highly efficient at increasing their profits.

Great. What we could use now is a lot more attention to achieving better government.

Ross Gittins is the economics editor.

The Business Briefing newsletter delivers major stories, exclusive coverage and expert opinion. Sign up to get it every weekday morning.