The Islington ‘Doing What Counts: Measuring What Matters’ Evaluation Report

The project intended to improve the impact of direct social work practice with children and families referred for a statutory assessment of need (CIN) in the London Borough of Islington. It aimed to improve outcomes for these children and families, indicated by reductions in the need for extended or repeated periods of social work intervention. The project was designed and delivered by an innovative partnership between senior leaders in the London Borough of Islington Targeted and Specialist Children and Families Service and an embedded research team, based in the Tilda Goldberg Centre (TGC), University of Bedfordshire. A novel Motivational Social Work (MSW) practice methodology was developed and implemented as the core offer for CIN cases. Where need and risk were complex, an enhanced offer added in-house multi-professional team support to the core MSW intervention. The TGC research team aimed to evaluate the core and enhanced MSW, and the planned improvements in practice conditions. Practice evaluation findings were fed back to practitioners (through a coaching relationship) and managers (in practice reports for the team and service) in order to form the basis of a new model of continuous practice improvement. In the final stage of the project, it was intended that the DWC and MWM methodologies, and their funding, would become mainstreamed in the London Borough of Islington with the development of practice system capability and demonstration of positive impact on child outcomes. Wider dissemination across local authorities in England could follow.

Overview of the evaluation
The role of the external evaluation was to validate and enhance the findings of the TGC embedded research team during the first 15 months of project implementation, to July 2016. The evaluation questions were:

1. Was Motivational Social Work confirmed as a cost effective method of achieving child outcomes as expected? (Were Islington social workers and managers doing what counts?)
2. Was the Measuring what Matters model of practice improvement and performance management implemented successfully?
3. Were project assumptions about the practice system conditions necessary for successful implementation of DWC:MWM confirmed in the light of experience?
4. Was project capacity for learning developed sufficiently to enable initial findings on design and implementation to inform further improvements in the practice system?

A 2-stage mixed methods, collaborative approach to data collection and analysis was agreed between the London Borough of Islington project leads, TGC and the University of Sussex research team.1 The 3 data sources included:

• all relevant TGC and London Borough of Islington project documentation
• a sample of 50 CIN cases extracted purposively from the whole population of 281 cases which had been identified for tracking and MSW practice evaluation by TGC embedded researchers. Interviews were conducted with 27 of the 34 social workers (80%) holding responsibility for these 50 cases
• interviews with Deputy Team Managers (DTMs) (11 out of 14) and Team Managers (TMs) (7 out of 7) responsible for supervision and immediate line management of the social workers, Operational Managers (3) and Heads of Service (2)

Key findings
• parents reported positively about the quality of practice, and the Motivational Social Work methodology was shown to improve practitioner skill and confidence. However, there is as yet insufficient evidence to demonstrate improved outcomes for children
• novel models of practice evaluation, improvement and performance management, which focus attention on quality and impact, benefit significantly in their design, implementation and refinement when embedded researchers work alongside local authority project and service leads
• whole service re-design consumes significant amounts of time and money without the guarantee of demonstrable returns on investment in the short and medium term