The move means Labour has no overall control, though it remains the largest party, as the Conservatives and the breakaway group want 'collective decision making'.
The council's political composition is now: 23 Labour, 22 Conservatives, seven Labour rebels, one independent and two vacant seats.
The leader of the breakaway group, John Fillis, said: 'The year before we had record increases in councillors' allowances set by a Labour administration. We're a small council and the allowances totalled£330,000.
'This year's budget increased council tax by 15%, rents by 5% and cut community services. Quite clearly a difficult decision all councils have faced, but you don't go spending a lot of money on yourself and then cutting everything else.
'The Labour group was being very dictatorial. The whip was being used to prevent us representing the local community.'
The split was nothing to do with Labour policy nationally, he added, though all seven councillors have resigned or are waiting to be expelled from the party.
Labour group leader Alan Bullen said the seven were 'incoherent, vain and egotistical'. He added: 'They have given about 50 different reasons. Every one of these seven voted for these allowances and the leader John Fillis of course enjoyed an even more generous scheme from the county council.'
He said they were refusing to acknowledge they had formed a coalition with the Tories in order to prevent Labour forming an official opposition. Shortly before the split, the Tories and breakaway group voted against a cabinet and leader.
Tory group leader Geoff Roberts agreed there was an 'alliance', but denied it was a coalition.
The regional Labour office was unavailable for comment.