Baroness Uddin, the first Muslim woman in the House of Lords, will chair the taskforce that will look into why only 168 out of 20,000 councillors are black or Asian people, despite them making up more than 5% of the population.
Ms Harman said: "Empowering people from black and Asian backgrounds in public life is a key priority for government. They are a force for good within their communities, and in building bridges between communities. Their contribution must be better recognised and supported. And we need our local councils to better reflect the local community."
To be fully representative the numbers of people from ethnic minority groups needs to increase more than five-fold. According to the government, at the current rate of increase this could take more than 130 years.
The taskforce proposes to take "practical action", including mentoring and holding outreach events in community centres to promote the role of councillors.
It also proposes working with organisations such as the Local Government Association, Operation Black Vote and the Fawcett Society as well as local councils, political parties and organisations like the Improvement & Development Agency.