Mr Gray, who is currently the party’s finance spokesman at Holyrood, secured 57.8 % of the final vote in a run off against Scottish deputy leader Cathy Jamieson. Johann Lamont MSP is Labour’s new deputy leader in Scotland.
The third leadership candidate Andy Kerr was eliminated following a first ballot, and his votes were redistributed.
Mr Gray was a teacher and campaigns director for Oxfam Scotland before being elected as MSP for East Lothian. He held four ministerial portfolios in Labour’s Scottish administration, including Enterprise, Transport and Lifelong Learning.
He was also a Special Adviser for four years to Chancellor of the Exchequer Alistair Darling when he was Secretary of State for Scotland.
Speaking after the announcement in Glasgow, Mr Gray said: "To be given the chance to serve this movement as leader of Labour in the Scottish Parliament is a precious privilege.
"But the real prize we now seek as party is the opportunity to serve Scotland and make our country all that it can be for every one of its citizens."