The largest fall in monthly retail spending in over a year has been blamed on a drop in consumer confidence following the budget.

Labor has turned its sights on Treasury boss Martin Parkinson for being out of touch with the real world, as retail spending shrank in response to the government’s May budget.

In a speech defending the government’s budget measures earlier this week, Dr Parkinson appeared to take a swipe at Labor, saying it was one thing to argue for fairness but another to invoke notions of fairness to oppose all reform.

Opposition spokesman for Trade and Investment Jim Chalmers does not think Dr Parkinson fully understood the unfairness of the budget.

“The Treasury secretary wouldn’t mix with the kind of people who are affected by this budget, people that are asked to do the heaviest lifting,” Dr Chalmers told Sky News.

Dr Chalmers was a key staffer to former Labor treasurer Wayne Swan, whose government appointed Dr Parkinson to the top Treasury job.

Consumer confidence fell sharply in the weeks running up to and after Treasurer Joe Hockey’s first budget and is still down nine per cent since late April, according to one survey.

However, Mr Hockey remains determined to get his budget passed by parliament, saying the fairest thing the government can do is prepare Australia for the future.

“It may take more than one occasion to get things through,” he told a conference in Melbourne.

“But we will not waver in our determination to get the best policies through the parliament to deliver the growth and jobs agenda that Australia sorely needs.”

ANZ Research economist Savita Singh said the deterioration in confidence and unseasonably warm weather during May seems to have been reflected in the latest retail spending figures.

Retail spending fell 0.5 per cent during May to just under $23 billion with large declines recorded in department stores and clothing and footwear retailing.

Retailers said the largest decline since March 2013 was disappointing but not a surprise given the difficult measures contained in the budget.

But Reserve Bank governor Glenn Stevens did not expect the budget to have a lasting impact on confidence.

He also told a conference in Hobart that while the central bank had “ammunition” on interest rates, he said previous cuts were still working their way through.

The RBA’s cash rate has been unchanged at an all time low of 2.5 per cent since last August.

“People may react by thinking that, if the bank is not thinking about easing, then it must be thinking about tightening,” he told a conference in Hobart.

He was not contemplating lifting interest rates.

“In fact … we might be on the brink of sitting still for some time.”