I noted a few problems in Treasury’s staffing in my column a fortnight ago. Among staff whose qualifications are known working as analysts, senior analysts, principal advisors and senior managers, economists are in the minority. It would be ridiculous for every analyst at Treasury to be an economist, but Treasury’s core analytical work requires having deep economic expertise. 

Hire your way out?
Recent hiring shows little evidence that Treasury’s management sees diminishing economic capabilities as a problem. 

Treasury hired nine graduate analysts in 2018. One had an undergraduate qualification in economics and one had a Masters. The rest had training in disciplines like sociology, German studies, anthropology, literary translation studies, and public policy. Of the fifteen hired in 2019’s cohort, ten had some background in economics, but most had only undergraduate qualifications, often in double-degrees. Double majors are excellent and well-rounded, but do not bring the depth of economic expertise that Treasury needs. Only four of the fifteen had at least Honours-level training in economics or finance. 

If Treasury’s senior management does not see this as a problem, others should. 

Staff see the problem
Internal staff surveys at Treasury note the problem of declining capabilities, although this requires caveat. Those results are not publicly available and I rely on the reports of others. I am reliably informed that, among surveyed staff asked about Treasury’s critical capabilities in economics and finance, those seeing diminished capabilities outnumbered those reporting improving capabilities by a ratio of 2:1 – though a plurality sees no change. 

I am also reliably informed that staff comments included statements like, “The Treasury has been getting rid of economic and financial experts”; that economics expertise is “weak” and that “This is our core work and should be prioritised”; and, that “Our deep economic and financial expertise is very thin.” 

And I understand other comments warned that technical expertise has declined, that technical expertise is no longer highly respected in Treasury, and that some teams in Treasury do not view analytical and quantitative skills as being important in policy work. 

The internal survey will not include the views of those economists who have already left Treasury. It is not hard to find former Treasury staff providing similar critique. 

Stakeholders see the problem
Every two years, Treasury undertakes an external stakeholder survey. The results of that survey for 2011, 2013, and 2015 are on Treasury’s website. The 2015 survey was released only two months after its completion. I was a respondent to the survey in August 2017, but the results of that survey are not yet on Treasury’s website.

I have requested the survey as well as Treasury’s reasons for failing to follow prior years’ practice in putting it up on Treasury’s website. But I have also informally seen parts of it, as these things have a way of floating around town.

And I think I know why Treasury has not released it.

The summary table shows every single measure of stakeholder satisfaction has declined since 2015.
 

Results in prior years were worse on some measures than in 2015, so it would be a mistake to call this an across-the-board longer-term downward trend. But for the key issues related to economic capabilities, decline is evident – and especially from 2015 onward. Perceptions that Treasury staff are well-informed ranged from 74% to 76% from 2011 through 2015 before dropping to 66%. Confidence that staff do a good job ranged from 74% to 77% from 2011 through 2015 before dropping to 68%. And the only measure on which 2017’s result was not the lowest was agreement that “I can offer the Treasury insights” – a measure of ambiguous interpretation.

I also understand the sharpest drop in satisfaction with Treasury was among those who generally interact with Treasury about economics. The 2015 survey showed that 70% of stakeholders interacting with Treasury about economics, macroeconomics, or fiscal projections were satisfied with that interaction. I understand that the current survey has that figure below 50%.

It would be very nice to have the full survey to know whether these results are representative. That Treasury has chosen not to follow prior practice in putting the survey results up on the website might suggest an answer.

Quality of policy advice has been dropping
Finally, Treasury’s annual reports show that the assessed quality of Treasury policy advice has been declining.

Declining scores through 2014/15 were accompanied by critique of poorly scoring papers for things like ease of reading, lack of detail on how conclusions were reached, and limited discussion of alternatives. Lack of depth was not noted.   

The assessment measure changed in 2015/16, making scores less directly comparable. But that year’s average assessed score of 6.2 was also well below the target of 7. Weaker reports were faulted for lack of clarity about the frame of analysis and for adding little value for the decision-maker.

In 2016/17, the target score increased to 7.5 while the average score dropped to 5.9. Where prior reports included more extensive discussion, this one provided only a brief footnote suggesting greater focus on quality of advice is needed and that papers need to make clear the reason for and action from the advice.

Though the report does not point it out, a strong economic framework will typically ground the reason for action either in a demonstrated market failure or in a reasonable problem definition, then proceed to explain the interventions that might address the identified issue.

Good, broad economics requires depth
In response to my earlier column, Secretary Makhlouf’s newsletter to Treasury staff argued that my critique “represents a strain of thinking which narrows the role of the Treasury and what good economics is actually about.”

That is far from the case.

Good economics covers a very broad domain. Handling that breadth of scope requires substantial depth of training. The decline in the assessed quality of Treasury policy advice coupled with what I understand of Treasury’s internal staff survey and what I have seen of the external stakeholder survey suggest that hiring non-economists to achieve that breadth has failed – albeit with the caveat that there could be more positive results in other parts of the surveys.

We might ask what a private sector Board would do if faced with similar results, and whether the government’s performance review framework for Chief Executives is doing the job it needs to do.

Fix the problem
Treasury’s problem is not just that it does not have enough economists. Its problem is that its senior leadership was able to convince itself that any of this was a good idea to begin with and seems not to have changed tack since then.

Those excellent analysts who remain at Treasury deserve better. And so do the rest of us.

When the next economic crisis comes, as it inevitably will, the government will likely have to make major decisions, some of them in the space of hours. At stake will be the jobs of thousands and billions of dollars of value across the economy.

As the lead economic advisor, Treasury will be on the front line for advice. The depth of its expertise in economics and finance will strongly influence the quality of the government’s response.

It is a problem that Minister Robertson must take seriously if Treasury does not.