The move means that two Liberal Democrats, the council's deputy leader Cllr Joan Stocker and Cllr Ian Stewart, have left the cabinet with immediate effect.
Last month, Cllr Stocker's responsibility for finance was removed by the leader, Tim Stoddard (Con), following a report which was critical of the council's financial reporting.
There had been talks between the two senior members but a resolution was not considered possible.
In a statement, the Liberal Democrats said: "Effective working of the Partnership relies on there being trust, respect and confidence between the two Groups.
"The view of the Liberal Democrat Group is that, because of the recent unilateral actions by the Conservative Leadership, these no longer exist and are not likely to in the near future."
Conservative cabinet members have redistributed the pair's portfolio responsibilities amongst themselves.
The cabinet will meet as planned on September 1 but any changes, such as a new deputy leader, are subject to a full council meeting.
The next full council meeting is scheduled for September 10.