Value SOLACE's Approach and Position.
(Background Note: This Report has been approved by both HOG and the EC. Comment and views are invited. Current revise of this document is 14/5/01)
There are mixed messages being reported in relation to Best Value, including Best Value Inspection. An early Survey of the Society's Members relating to the BV Inspectorate has shown a balance of concern, alongside a recognition of some positives and strengths. A similar LGA Survey appears to have had a more positive balance. The issues need to be set in the wider policy context of Best Value and of the many other aspects for improvement and change in Local Government. It is timely for the Society to articulate a position, based on the views and experiences of the Membership, and use that as the framework through which it contributes to policy ideas and development in this critical area.
Broad Approach: The Society is strongly committed to improvement and change in Local Government. That process is at its best when it is self-directed by Local Government in the interest of the citizens and communities it serves. Much has already been done by Local Government itself, but much remains to be done. Best Value and Inspection can play a vital part in that, a part which is developing rapidly and which needs to be shaped by experience on the ground.
Current Experiences: A number of Authorities are reporting early problems with Best Value Inspections and with other aspects of the BV regime, alongside a number of positives. These include concerns about:
* The quality of Inspectors and Inspections
* Inspection 'congestion' and delay
* Excessive Inspection
* The way BV has been a de-motivating experience for many
* The need for better follow up and support
* The impact on small authorities in terms of capacity and of the overhead of Inspection
* Insufficient linkage between various aspects of the overall Inspection and Audit Regimes, and with service improvement
Positives: Having said that, survey work and general experience has highlighted a number of positives including the part that Best Value Inspection is playing in:
* Providing stimulus to change;
* Highlighting poor performance in some places;
* Giving a focus to likelihood of improvement, as well as the current level of service;
* Contributing to debate and reflection about the self-improvement agenda.
Context: There is an important context in understanding the extent to which current problems are likely to be endemic or may be resolved as the Inspection process develops and matures, and as the BV regime itself evolves in the light of the rapidly growing body of experience and of the wider policy environment (eg local PSAs, etc).
Positive trends include:
* Movement within the Audit Commission to lower costs, to focus on fewer reviews and to strengthen the emphasis on an Authority's capacity to improve;
* The relatively heavy investment in Inspector training, and a growth of competence and comparison as Best Value Inspection gains momentum;
* Some steps towards linking up Inspectorates and some of the associated machinery.
Possible Further Steps: There are a number of specific steps which might be contemplated, and which should be debated and tested. They include:
a) A stronger concentration on developing planning for the various inspectorate processes and from the Peer Review concept. We believe there should be a concentration on the capacity of an authority to achieve self-improvement alongside a programme of fewer, broader, best value reviews. The Audit Commission could retain the right to conduct a forensic investigation and inspect and assess local authority performance, perhaps on an annual basis. Adding a Chief Executive from another, similar sized, authority to act as a 'reality check' for the inspectors, would enhance this process. This should not be used to negate difficult findings but would enable inspectors to put their findings in a wider context.
b) Those authorities with in-built challenge and with demonstrable self-improvement capacity should not be subjected to the same rigour/number of inspections [a light touch] as those that are obviously failing to improve. There should be a downscaling of external intervention for authorities/services performing effectively, leaving them to account to their local communities.
c) A Review of the Statutory Guidance, even (perhaps especially) at this early stage could be helpful. The BV Guidance and the Community Strategies Guidance should be reviewed, both to connect them up better and also to place change and improvement as more central to both. The connections with Political Executives and with related policy areas such as Neighbourhood Renewal also need to be thought through.
d) Greater integration is needed of targets and of the outcomes aspired to. Thus, the outcomes expected at local level from the Neighbourhood Renewal Strategy (Communities First in Wales) could be connected into Local Authority Community Strategies, and into the BVPP, and the targets in the local PSAs (Policy Agreements in Wales) could be integrated into the BVPP as 'local indicators'. Authorities could be encouraged to absorb the voluntary 'Quality of Life' indicators into their Community Strategies and Neighbourhood Renewal work. Further work should be undertaken to establish and fully stabilise a core set of Performance Indicators.
e) Aspects of the Inspection/Audit process itself could be further developed, although it should be acknowledged that the learning/development curve for the Inspectorate is a steep one. Strengthened communication with Authorities would be beneficial, especially at an early stage when Review Programmes are being shaped and settled. Whether or not Authorities with problems at BVPP stage are 'named', full and clear engagement at a senior political and managerial level is essential sothat difficult messages are fully understood and, where appropriate, subject to test and challenge. Inspectors need to be involved 'upstream' when the BVPP is being finalised, and they need to be playing a part during Reviews, rather than only after they are completed, helping to identify relevant best practice and learning experiences from elsewhere, and advising on process where appropriate. Ideally, the Inspector training programmes should be available to Authorities (probably separately from the Inspectors) so there is faster and clearer understanding of the process and its rationale, and maximum transparency about how judgements are made.
e) At national level, discussion and debate about the self improvement agenda for local government and public services needs to be strengthened and extended, and the experiences of SOLACE members and others at the front line need to be drawn on fully. Best Value needs to seen in context as a part of the wider modernising agenda. The over-riding context is, or ought to be, of a shared commitment to creating excellence in public service delivery, to supporting and encouraging innovation, to enhancing the capacity and the will for self improvement, and to maximising value for citizens and service users. All relevant bodies need the opportunity to contribute to realising and delivering that shared commitment.
14th May 2001
* see LGCnetfor Local Government Chronicle's story 'SOLACE PROMPTS BEST VALUE INSPECTION RETHINK'
BV INSPECTORATE - COMMENTS
Colleagues will recall that I asked SOLACE members to reflect upon their experiences of the best value inspectorate. Here are a selection of their comments from which I have removed specific references to individual authorities. I am committed to sharing this with Wendy Thomson before we go public with any of it.
First we have some very positive comments:
1)'As regards BV we went into the inspection process critical of the bureaucracy & negative about the likely benefit. However the reporting back was good enough & sufficiently analytical to win us over. The fact that we have provisionally been awarded 2 stars for our Waste, recycling & street cleaning services may have seduced us but they did a good job.'
2)'Our only Inspection so far has been a good experience with the Inspectorate generally. The team were knowledgeable and experienced in terms of local government and the subject matter (would you believe, Caravan Camps). In point of fact, they were better qualified than I was to look at caravan camps and the leisure industry - so why did I have to do the review then be told by them what the shortcomings were?
Relationships, both with the two Inspectors and the Lead Inspector is cordial - although there was a bit of friction when it became evident we were not going to completely finish the review before they wanted to come and inspect, but we reached a compromise.'
3)'It was a Housing Inspection - good process, and competent Inspector. Key disagreement was that they used 1999/2000 data to judge the service as 'one star', even though 2000/2001 data was greatly improved. But then they would not accept 2000/2001 data as cast iron proof that the service was going to improve. We felt this was not fair. But no evidence of 'red-guardism' or lack of understanding. There are however questions over consistency of approach'
4)'You asked for experiences both good and bad. I know other Authorities in this area have had negative experiences particularly qualitative in nature but ours has been positive.
On one of our first year reviews the Inspectorate could have gone ahead with the Inspection and produced a damning report. That they postponed the inspection to help us get our act better together is an accurate reflection of the excellent relationship between this Council and the Inspectorate.
The relationship itself is very open honest and constructive and is helping us enormously to meet and exploit the challenges and potential of Best Value.'
5)'We have had one inspection so far and we were impressed with the quality of the work that was done and have been happy with the inspection process up to the interim challenge - despite it being quite critical.
Other councils have criticised the inspectors when they have seen the final report which sometimes has given a more negative slant in comparison with the feedback at interim challenge. We will wait to see how our final report compares with our expectations.'
After these five, most comments become more negative:
6)'We are awaiting a report on a 'Community Leadership' inspection. The evidence from the 'challenge' feedback and subsequent discussion suggests that they are way out of their depth. Happy to discuss further in confidence and to add to your general picture but don't quote me at this stage please.'
7)'We have been having fun with the Inspectorate. We have taken issue with the description of the process as open and sharing of thinking and conclusions as the inspection goes on. Our experience of our first was conflicting pressures on inspectors doing more than one inspection, short deadlines for asking for info and particularly meetings with staff, Members contractors and public and also understanding their deliberation processes to ensure their understanding of the council was also ours. Interestingly our experience didn't correspond with the description of the likely process given to us this week by our lead inspector!'
8)'The quality of the work by the teams who have been involved has been high, but we have had some rather complex discussions over the contexts in which the significant BV reviews have taken place. Our experience is that the BV reviews which are leading to major changes in practice have complex histories and lead on to complex implementation processes. The BV Inspection process is designed around a concept of clean, self-contained BV reviews, and there is thus a difficulty when life is not so simple.
An example is a review of certain management arrangements. The issue of whether to outsource/create a Trust/keep in-house has been live for approx 4 years here. We have carried out a best value review, but the inspectorate has found the existence of prior discussion with Members difficult to deal with. They are also worried because some of the decisions required by members are staged as further work follows the decisions in principle to create a trust. They are not sure how to comment on 'likelihood of improvement' given lack of a single member decision point.
On a rather more specific point - the need to respond to draft findings 'within 5 working days' is unfortunate if a Council is engaging its Members fully in the inspection process.'
9)'The key issues for me on inspection are:
* The process is not transparent
* There is no model or definition of what makes the difference between categories and so the process appears arbitrary
* The workload is too great for the numbers of inspectors to do well
* The quality of inspectors is in inverse proportion to their arrogance
* There is no conception that an inspector is capable of mistakes'
10)'Our tourism service was inspected. We have been working with a neighbouring authority to re-brand our two areas. This involved£300k of European money, genuine partnership arrangements, and a massive change of culture particularly from the small B&B proprietor. We now have a single brochure, internet site, common quality rating etc for the whole area, and the venture has won the White Rose award for tourism, and is cited by the LGA and others as a model of its kind. Imagine our surprise when the Inspectorate's verdict was 'fair service' and 'unlikely to improve'.
Issues for us were:
* No one on the inspectorate team had any knowledge of or experience in tourism. This showed!
* They relied heavily on anecdotal evidence from the few people they spoke to and failed to make a sensible judgement about how that should be balanced against 'hard' evidence.
* We were told that 'resources are not an issue' in making judgements. In other words the size of Council, its establishment and financial situation are irrelevant. This is plain daft.
* The four-way scoring box is crude and unhelpful. The fact is we were nudging 'good' not 'poor' but nobody knows this when the results are published. And the language is loaded and damaging. This is particularly so with a market sensitive service like tourism where words like 'fair' and 'unlikely' get published.
* The analysis of improvement is solely based on the review process followed. No account is taken of any historical continuum of improvement, which can be demonstrated. This seems illogical.
* Inspectors identified nothing, which our own action plan had not already flagged up. If we had done this and said how we were going to address the issues why were we unlikely to improve?
* At times the inspectors were economical with the truth. For example, they told us their presentation had been viewed by colleagues for 'moderation' purposes then they told us it hadn't.
We naturally took these and other issues up with the Lead Inspector who subsequently gave me an apology for the way in which the review had been carried out. We produced a 'line by line' rebuttal of the final draft report, which led almost to its complete rewriting - except for the two judgements. I think these two may have gone except that the inspectorate would have lost too much face as a result. Delivering 'findings' without conclusions Ombudsman style might be a better approach, since this would enable the inspectorate to rethink after the Council's response.
Obviously we have now moved on, but the net result is that we will be delivering our action plan in exactly the same way. We have improved our review process and we told the inspectors that we recognised its shortcomings and had made changes. The biggest problem has been rebuilding a totally demoralised staff who, mainly because of the way the review was carried out, have lost all faith in the concept of best value and the inspectorate.'
11)'Lack of evidence-base in our internal audit BV review. Opinions rather than evidence lead to qualification that an otherwise '2 star' service did not have the corporate capacity to improve. This was based on a 'snapshot' inspection (as they always are) in August immediately prior to the interim management arrangements, which the Council had put in place to cover the period in which it did not have a CE and one Head of Service. Despite strenuous efforts to point out that the lead inspectors 'opinion', for which he could produce not a shred of evidence, was completely contrary to the District Auditor's view that the Council was poised to move forward despite its temporary difficulties, the Lead Inspector dug his heals in and declined a proper dialogue. Fortunately it was the Ad's view, which found its way into an extremely positive Management Letter at the end of November - so perhaps no great harm done. But it was an irritant and confirmation for me that the Inspectorate is less likely to be the 'critical friend' than adopting the adversarial style of OFSTED.'
12)'We are currently experiencing a turbulent and most unprofessional approach with the Housing BVI. We have had such subjective interpretation of comments made by Housing Officers to the BVIs, plus bad practice too lengthy to put in writing.
Happy to discuss by phone if you wish.'
13)'We have had one inspection so far - our BV Waste Review - in which we produced an implementation programme of a very radical nature to hit government targets and member aspirations. From the first day of the inspection, staff were complaining about the 'prejudice' of the inspectors who appeared to conclude quite early on that our service was unlikely to improve. My session with them on the final day was very unsatisfactory. We have re-structured to face the challenge of BV, well - being and political change in a quite radical way. We are doing LSVT. To be told by 2 'early retired' minor chief officers that we are not capable of delivering change is very irritating.
We are continuing to argue and the final report remains to be published. In the meantime, we are on the verge of completing an agreement, which will up our re-cycling % from 8% to 25%.
We were led to believe that the BV Inspectorate would not seek to acquire an OFSTED like reputation but from what we have seen so far we are not too impressed.'
14)'I could not resist sharing with you our experience of our first Best Value inspection!
The inspectors are reviewing our waste PFI contract process - the contract is worth£1 billion over 25 years - and the whole process is taking 3-4 years to go through. It is, of course, related to a very difficult and contentious area of service delivery, made a little more so, politically at least, because we are undertaking the process jointly with a neighbouring authority.
We are not yet able to comment on the quality of the final inspection process as we await our interim challenge session but the experience to date has not been a totally happy one. The customer care aspects of the inspection have been lacking to say the least, with Members and senior officers alike being summoned to meetings at short notice with little recognition of the other pressures on their diaries. At the first briefing meeting, the Best Value inspectors showed relatively little understanding of the sensitivities surrounding the area of service and the level of contract expertise required in the project they were reviewing. They brought smiles (or frowns) to the faces of those in the room when they explained that their approach to testing customer satisfaction would be to visit household recycling centres during the week (they are, of course, mainly used at weekends!) and ask members of the public what they thought about the waste PFI contract process! The inspection has now finished on site.
In summary, I cannot say that it has been a process that has increased the credibility of the Best Value inspection process with either officers or Members. It is an area I discussed in some detail with our Lead Regional Inspector, prior to the inspection, and one that I will be taking up with both him and the inspectors subsequently. Without wishing to strike too negative a note, this was their first inspection here and clearly there are lessons that must be learnt on both sides!'
15)'This week we have had our first inspections' interim challenge report. I was fairly happy with the approach so far. However I continue to be concerned about the inspectorates' push that authorities do a smaller number of large/crosscutting reviews and tends to think that this is more to do with administrative convenience than empirical evidence that larger reviews are better. It would be interesting to see an analysis of savings and changes in performance, correlated with size of review, to back up their assertions (or have I missed something?).
Also I thought that in terms of the recent consultation document on fees for inspection, it was a shame that authorities can only reduce audit costs by doing larger reviews. It seems to me that if a review is done thoroughly so that less time is spent on site by inspectors, a rebate ought to be due.'
16)'Our experience with the BV Inspectors has been appalling. They promised us help and they have been destructive and worse than our Auditors - and that is saying something. We are likely to refute their findings. We tackled our most difficult area first - the procurement and management of our 160 or so contracts (incidentally, our proportion of Members to staff is 1:4 - population 100,000). They were disorganised, no concept of imagination or innovation and rather aggressive.'
17)'My concerns can be summarised as:
Lack of clarity. The basic problem with BV is epitomised by Wendy Thompson's own comments that authorities ought to have one or two strategic BVs a year. Such a comment suggests that a 5 year programme to review all services (as per legislation) is not feasible. We need a clear statement that it is up to each authority to choose say 'no more than 3 BV's a year' and for that choice to be guided by local decision as opposed to lobby groups, DA, or Government Departments.
Quality of inspectors. I have heard external consultants refer to their BVs reviews in terms that regard it as another source of cash, and seem quite unable to comprehend that we work to political direction and electorally determined mandates. Also it would help if the Inspectors were up to speed.
Single issue focus. Reviews we have had seem to ignore context and over focus on the issue under examination. Having worked as an inspector myself in the past, the element of enabling and encouraging seems to have been lost in a criticise and convict approach'
18)'The main issues are:
* insufficient BVI resources to match the programme
* insufficient notice for preparation and delivery of documentation pre-review
* findings not obviously supported by report - or sometimes fact
* no feedback on comments made on presentation or draft report
* slow response
* late or incomplete reports without revised timescale
* doubts about adequate level of technical input
* the cost and ??VFM
All resulting in great difficulty in maintaining credibility for the process or enthusiasm for BV in general!
Hope this gives a flavour - happy to lend any support to help.
19)'You may wish to add us to your list of authorities who have experienced difficulty with the Best Value Inspectorate. In our case, a review was carried out of our Best Value preparations and progress rather than of a specific service. The results as presented at the 'Interim Challenge' session were grossly inaccurate; based on flawed or insufficient (or no) evidence; and delivered in a confrontational, aggressive manner; and overall demonstrated a lack of professionalism on the part of the review team. This has consequently left Members and officers angry, demoralised and frankly disillusioned with the best value process.
We are now engaged in delivering evidence to challenge the Inspectors' interim verdict. Hard evidence is required from us; soft 'evidence'; flawed indicators; and limited knowledge appear to suffice for the Inspectorate.
Our longer-term task will be to rebuild respect for the best value process within the organisation. I will advise you further as to whether we have any success in persuading the Inspectors to a more balanced conclusion about this authority.'
20)'I suggest your report registers our concern about the costs of BV inspection and the flat rates which are being charged. The Inspectorate must have substantial overheads!'
21)'Our worry is the fee level, particularly for this year, where we have paid the full£70,000 and have received one inspection. This seems a little excessive (or minimal, if you look at it from the other viewpoint).
Wendy needs to know just how much irritation this is causing with members, particularly at a time when we are scrimping on every last penny and making cuts to voluntary sector grants, etc. We have still not received the report from an inspection which took place in October of last year, and the combination of perceived cost, volume of work and lack of follow through from the Inspectorate is not proving to be a winning combination!
Sorry to whinge at you, but thought you might like to have some concrete examples of concerns.'
22)'The following may be of interest to you.
We are about to review our financial and accountancy services and have invited our external auditors, to sit on the review team. They were keen to do so but have been told by the Audit Commission they cannot. Apparently, the Audit Commission insists on a 'buffer' zone before and after their review process, which disbars external auditors from participating. The line is that they would not wish one set of external inspectors saying the service is good when another set may disagree.
This approach is daft. The aim is, surely, to improve services for the overall benefit of the local authority and its customers. Our external auditors are an important stakeholder in this particular area - and in others. We took benefit from their challenge, experience, and knowledge to put in place a continuous improvement plan which could deliver benefits over the next 5 years. The opportunity afforded by having that stakeholder included within the process is now not available.
Local authorities are expected to face challenge from others having a different opinion to them. Why is the Audit Commission being so precious?'
23)'Five principal concerns (so far!) about BV inspections.
1. The practice of inspecting the service, rather than just the council's own BV review, with all the emphasis on the 'star' rating rather than the 'improvability' rating. (The Audit Commission press release on our first inspection mentioned the former in the first line, and the latter not until the penultimate sentence on page 2.)
2. We have just reviewed a service we knew was not good. Staff morale fell. We are now addressing that. Inspectorate due next month; they will inevitably end up reopening the wounds that have just begun to heal, and will tell us (and the world) that - surprise, surprise - this service was not good.
3. The use of 'reality checks' that are based on sample sizes of dubious statistical significance.
4. Councils that followed DETR/Audit Commission guidance to review their least good services first are now finding (of course) that the early inspection reports are all more or less unfavourable. There are serious results for collective morale, and also for party politics (particularly in a council like ours with a majority of one and local elections due in May 2002) - especially when the emphasis is on the poor star rating, not on a well-conducted review.
5. An individual housing inspection, the conduct of which left us appalled in about a dozen ways. I've discussed it at length with our regional BV leader, and don't see the point in raising it at a higher level. But it has seriously set back my attempts to harness BV as a force for good in the council.'
This snapshot was taken before many inspections had taken place and certainly before many judgements have been delivered by the Inspectorate. We had many more comments but not that add greater clarity to the picture. These seem representative.
It is hard to draw any conclusions but an overall picture of poor quality control within the inspectorate does emerge.