What counts as good evidence?
We want to make it easier for all those working to tackle homelessness - including commissioners, investors, policy makers, and providers – to know which solutions to use, and what difference they will make. At the heart of these activities is a focus on robust evidence.
Yet, we know that evidence is complicated, contested, and far from straightforward. We know that even where evidence does exist, it is not consistently acted upon. We also know that evidence is only ever one part of the decision making process, and needs to compete with other pressures, from political will to cost. And even when evidence does exist, it is often fraught with tension and confusion. Who decides “what works”? How do you compare different types of research from different sources? How do you understand the quality of evidence? How confident can we be in what the evidence tells us?
This is where standards of evidence come in. They will provide a vital infrastructure to trial and assess what works in the homelessness field. The evidence standards will help us gain a deeper understanding of what is working, when, for whom and under what conditions. They provide a common language for talking about different interventions – and help highlight where claims behind an intervention’s impact is strong, but the evidence is weak.
Who’s doing it well?
Standards of evidence are becoming increasingly common across social policy over the past decade so there’s lots for us to learn. In 2015, the UK Government announced the What Works Network, evidence centres which provide guidance on the most cost-efficient, useful services. Most of the What Works Centres use standards of evidence, for example, NICE and the What Works Centre for Wellbeing using the Grade of Recommendations Assessment, Development and Evaluation (GRADE) framework. The Education Endowment Foundation developed its own Teaching and Learning Toolkit, supported by a padlock system to show how likely it is to get the same results from the intervention if it was used again. The What Works Centre for Local Economic Growth uses the Maryland Scientific Methods Scale, whilst the What Works Centre for Crime Reduction uses the EMMIE framework. There is also the framework for organisations working in the homelessness sector developed by the Canadian Homeless Hub.
With so much activity underway, there’s no need for us to reinvent the wheel and create standards from scratch. Instead, we’re going to draw upon this experience and expertise, and use it as the starting point for co-creating the standards of evidence for the Centre for Homelessness Impact.
What will be important for us?
In short, the standards will need to suit the aims of the Centre and be appropriate for the homelessness sector. Significantly, our purpose won’t be simply or mainly to use standards of evidence to endorse what are considered to be best practices and proven interventions. Instead, they will serve an developmental purpose - aimed at improving both practices and the available evidence.
As such the Centre’s goal is to encourage progress through some assessed stages on an evidence journey. There is no natural end to this journey: all evidence is partial, provisional and contingent, and thus needs to be used as part of an ongoing process of evaluation, learning, adaptation and innovation.
Over the coming months, we will be developing our programme of work, with the standards of evidence underpinning all that we do. We’d love to hear from others in the field. Do you have experience in working with and developing useful standards of evidence? Have you faced any barriers in enabling others to adopt and use standards? Please get in touch with us via firstname.lastname@example.org or tweet us at @homelessimpact. To follow our progress, you can sign up to our newsletter.