April 18, 2018

How to make what works work: initial lessons from process evaluations

Dr Howard White

The new Centre for Homelessness Impact will provide evidence on what works to improve the lives of those affected by homelessness plus people and families at risk of homelessness. As a first step in this journey, the Centre has joined forces with the Campbell Collaboration and Heriot Watt University to produce a map of evidence of what works.

But as well as needing to know what works, we also need to know how to make it work. Previously successful interventions may fail if poorly implemented. It is common for programmes which attain  successful pilots to fail to achieve the same impact once taken to scale. So, as well as mapping experimental and non-experimental studies which tell us what works, we are mapping data from process evaluations on what implementations issues arise:  

The effectiveness map will show impact evaluations – studies using counterfactual analysis to assess what difference a policy, project or programme makes. 
The implementation map will show process evaluations: studies which capture stakeholder perceptions on programme implementation.

We are just starting our journey on the initial coding for the process evaluation map. Whilst the map shows issues which arise rather than an analysis of those issues, some clear impressions are emerging.

People who are sleeping rough or who experience protracted and repeated homelessness, often face multiple challenges including substance abuse, mental health issues and holding down a job. Service delivery thus often requires successful coordination between multiple agencies.

An example of the need for information sharing comes from a city in which seven organisations provide a free breakfast – but no one is providing lunch or dinner. Information also needs to be shared between partners in service delivery – such as the timing of discharge from prison or mental health institutions risking discharge to the street without adequate support.

And information needs to be shared with people who are homeless. As one man put it: “I spent three and half months sleeping in a field because I didn’t know where to get help”. Information needs to be accessible, understandable and actionable: who can provide what help, when, where and how can you contact them.

Such information is important and the advertised services need to be available and acceptable. The most common problem of availability is the simple lack of affordable housing. Perhaps unsurprisingly, one way to deal with homelessness is to build more houses and ensure that a sufficient number are accessible to people affected by homelessness. But acceptability also matters. People may refuse housing too far from their job or children’s school – or which is in a bad neighbourhood.

How people affected by homelessness are treated affects acceptability. It is clear that there are thousands of dedicated workers who have empathy for their homeless clients and treat them with respect. But there are also those without such empathy or who are judgemental. People may well not come back again if treated that way. These issues are ones which can be successfully dealt with. For example, checklists are an effective way of making sure procedures are followed, such as information sharing. They also empower junior staff to request action from their seniors when necessary.

Over the coming year the new Centre for Homelessness Impact will be publishing a range of resources to support the use of rigorous evidence on what works – and how to ensure it actually will work going forward.

More updates
Back to all updates
manage cookies