Somerset leader Chris Clarke said: 'The minister's announcement indicates the battle is nearly over and we are about to win it.
'I am delighted the government is going to look at the whole thing again. The commission's proposals cannot be justified and the public has made clear they do not want any change to the existing arrangements.'
But local MP Mark Robinson (Somerton and Frome), who supports the county's case, warned councillors not to get carried away by Mr Baldry's announcement.
Mr Baldry wants the commission to examine the cost implications of introducing three unitaries in Somerset without splitting Mendip.
For North Yorkshire and North Humberside the DoE wants more detail on the costs of three options for north of the Humber.
'The estimates of potential costs and savings are important considerations. We feel further information will help all concerned to assess which option is likely to be most cost-effective,' Mr Baldry said.
The commission expects to provide the data within a month.
Somerset has decided not to appeal against the High Court's rejection of its judicial review of the commission's conduct.
'We had advice from our four Tory MPs that the best course was to leave it with the secretary of state,' said chief executive Brian Tanner.
Lawyers had warned the council its chances of winning an appeal were slim.
Avon and Cleveland CCs, its partners in the original case, were due to make their decisions last night. The High Court has yet to decide if it will grant leave to appeal.