The government “overstated” the success of the Troubled Families programme and has been “evasive” about delays to its evaluation of the project, MPs say in a report.
The public accounts committee’s report on the government’s flagship programme, which aimed to “turn around” the lives of 120,000 families with complex needs, referred to an evaluation by Ecorys on behalf of the Department for Communities & Local Government, published in October. That report found no evidence that the programme had “any significant or systemic impact” on the lives of the families involved.
The committee’s report, published today, also said that the DCLG should have published an evaluation of the programme in late 2015 and that the delay until a year later was “unacceptable”.
Then when PAC requested the evaluation reports on 5 October 2016 the DCLG “did not provide them to us until they were published on 17 October, just two days before the evidence session,” the committee said. The report added: “These delays and obfuscation have given a bad impression about the Department’s willingness to be open.”
Committee chair Meg Hillier (Lab) said in a statement that the report was “far more serious” than a “slap on the wrist about Whitehall bureaucracy”.
She added: “It is particularly important with a new initiative that there is transparency so that the government can learn and adapt the programme. The [DCLG] has undermined any achievements the government might legitimately claim for its overall work in this area.”
However, Ms Hillier added: “The DCLG has now committed to providing parliament with an annual report on progress with the Troubled Families programme, starting in March next year. For this to be meaningful the government must be far clearer about the benefits that can be directly attributed to the public investment in it.”
PAC’s report said that the DCLG’s claim that 116,654 families had been “turned around” by the programme – 99% of the original number of families eligible – was “misleading” as it was only indicative of “short-term outcomes” and not “sustainable change” to people’s lives.
“The use of this term suggested that long-term social problems could be fixed within a few months or years, and risks undermining the entire concept of this work,” the committee said.
PAC acknowledged there was some evidence of good practice resulting from the programme, but added that the evaluation “could not directly attribute improvement in a range of outcomes related to employment, crime and health” to the programme.
The programme’s use of a payment-by-results system also gave cause for concern. The committee noted the DCLG’s evaluation of the programme highlighted problems with payments, as these incentivised councils to move families through the programme quickly to draw down funding “without providing the support necessary to tackle deep-rooted problems”.
The committee also said the DCLG’s claim that the programme had saved £1.2bn was “an overestimate” because it did not take into account the delivery costs of the programme.
It added that the DCLG has acknowledged the £1.2bn figure did not reflect net savings directly attributable to the programme and is looking for a better way to estimate the benefits.
The committee made a number of recommendations to DCLG:
1) The DCLG should ensure that its reports to parliament and the public on the progress of the Troubled Families programme contain the evidence necessary to conclude meaningfully on the programme’s progress. Furthermore, to support effective parliamentary scrutiny, the DCLG must publish evaluations promptly and supply the committee with timely information.
2) The DCLG must develop a more meaningful longitudinal methodology for evaluating the impact and quality of the Troubled Families programme at both a national and local authority level. This should include an effective annual reporting system.
3) The DCLG should review its payment by results mechanism for Troubled Families to ensure that this is not resulting in local authorities diluting the quality of the support they provide to these families. The DCLG needs to track what ongoing support is provided to determine what works.
4) The DCLG should ensure that the terminology it uses to communicate the achievements of the Troubled Families programme gives an accurate depiction of how disadvantaged families make progress.
5) The DCLG needs to be able to make more accurate claims about any savings that can be attributed to the Troubled Families programme and, with local areas’ input, develop better ways of measuring whether the programme is achieving value for money.
Responding to the report, a DCLG spokesman said: “As the PAC report recognises, the Troubled Families programme enabled local authorities to expand and transform the way local services work with families.
“But of course, there will always be lessons to learn and we have already made significant improvements to the second stage of the programme.
“We will look carefully at the evidence to find out how we can improve the programme further to help some of the most vulnerable people in our society.”