Impact Forums 2019: Opportunities for Data
In August 1854, London’s Soho was stricken with a severe outbreak of cholera, a gastrointestinal infection caused by the bacterium Vibrio cholerae. Thousands fell ill and at least 600 people died. But those who survived owed their lives to a local doctor named John Snow.
At the time, ‘miasma theory’ held that disease was spread among the population by a poisonous bad air emitted by rotting matter. Snow believed otherwise. His ‘germ theory’ suggested that it was in fact sewage dumped into rivers and cesspools near wells that was contaminating water supplies and causing deadly cholera outbreaks.
From his practice in 54 Frith street, Snow began investigating germ theory at the height of the 1854 outbreak. He interviewed local residents and quickly started to suspect that the source of the outbreak was the public water pump on Soho’s Broad Street. Using data from local hospital and public records and information from local residents about where they sourced their water, Snow created a dot map to illustrate the cluster of cases around the pump, writing:
“Within 250 yards of the spot where Cambridge Street joins Broad Street there were upwards of 500 fatal attacks of cholera in 10 days… As soon as I became acquainted with the situation and extent of this irruption of cholera, I suspected some contamination of the water of the much-frequented street-pump in Broad Street.”
Snow had the handle of the pump removed by local authorities and soon after, the outbreak ceased.
While homelessness is not an issue with a single cause or solution, Snow’s approach to formulating a hypothesis and gathering data has plenty to teach us about freeing ourselves from our own miasma theories. As Welsh Minister for Housing and Local Government Julie James AM said at our Cardiff Impact Forum, “We don’t know how much homelessness is to do with welfare reform, family breakdown or what the direct causation is.” Gathering the right data is key to addressing this dearth of understanding and ending homelessness for good.
Currently, we lack the tools to analyse what the myriad relevant datasets collectively tell us about the state of homelessness across the UK. While statutory homelessness and rough sleeping data are vital, they do not provide a rich enough picture of the number of people currently experiencing or at risk of homelessness around the country.
If we want to improve our understanding of the issue and whether our efforts are leading to real lasting change we have to look at the whole gamut of local and national data to fully understand individual journeys into homelessness and wider system factors. This is at the heart of our ongoing work with the ONS and the reason we developed SHARE.
“All systems need to be at the table — health, transport, social care — to ensure that homelessness policy is fit for purpose,” said Beth Blauer of Johns Hopkins Center for Government Excellence at our London Impact Forum. Linking disparate public data sources will enable us to map individual journeys and find patterns so we don’t simply see homelessness as an outcome.
Data quality and harmonisation
In order to do this we must first be certain that we identify all relevant data sources and highlight important gaps. Our hope is that the Beta version of the SHARE homelessness indicators dashboard, developed with the ONS and scheduled for release in October, will help improve the overall utility of existing datasets.
Also, many programmes at the local, national and UK level have minimal information to use for project management and even fewer staff with the skills required to use it effectively. We need to figure out what data is needed to support evaluation, research and development, and project management, and advocate for collecting it.
We also believe that harmonising definitions of homelessness statistics across the UK could make a huge difference. Harmonisation matters because it would provide us with a comparable view of data from the different UK nations, accelerating learning.
The Government Statistical Service (GSS) has recently published a feasibility study into harmonising definitions of homelessness for UK official statistics, concluding that it would currently be too complex to harmonise definitions of homelessness across all the UK’s administrations, “because homelessness and housing are devolved matters. Homelessness data is often collected through administrative systems which were built using definitions based on each country’s legislation, and so data are not currently comparable.”
For the time being, GSS makes three recommendations:
1. Develop more comprehensive guidance on the processes and definitions of homelessness used in each country’s statistical publications, which is consistent across publications.
2. Create a separate, user-friendly paper on UK comparability of homelessness statistics which will include a conceptual framework.
3. In the long-term, it is important that across the Government Statistical Service we consider what are the big questions around homelessness in the UK and whether some of that requires coordinated working across each UK country, and to consider how data can be collated to answer those questions.
From our perspective at CHI we believe that data harmonisation is both possible and desirable. We look forward to contributing to this important debate over coming months. Get in touch if you’d like to learn more.
Better data is of limited value unless it is used. Rapid advances in technology and data tools have created new opportunities to understand pressing challenges and the impact of investments more quickly and at a lower cost. The revolution in big data, analytics, and rapid-cycle evaluation that is currently benefiting other sectors could equally help transform how we tackle homelessness.
Continuous improvement in homelessness requires lots of types of information, including data on how well a programme reaches its target population, whether the needs of that population are changing, whether interventions are effectively implemented, and whether outcomes are moving as expected.
In England, MHCLG is already making impressive progress in this area by linking datasets that were previously unavailable for analysis. In March 2019, MHCLG published an evaluation of the Troubled Families’ Programme, an initiative that has sought to transform the lives of 400,000 families with multiple, persistent and often severe problems across six headline issues: worklessness and financial exclusion; school absence; mental and physical health problems; children needing help; domestic violence and abuse; crime and anti-social behaviour.
The evaluation used administrative data from local authorities and government departments to measure outcomes on a scale not attempted before. The data matching provided MHCLG with information on offending, school attendance and attainment, children’s social care, and benefits take-up and employment. The result is a very large dataset with over a million cases and over 3,000 variables.
Studies like these are the exception rather than the rule at the moment. This needs to change.
To achieve step change in homelessnes, sufficient resources need to be dedicated to building or using data and evidence. Currently there are limited resources for evaluation and other evidence-building activities, which too often are seen as ‘extras’.
Simply having access to this diversity of data won’t be enough to bring about lasting change in the field. As Sheffield Hallam University’s Kesia Reeve said at our London Impact Forum, “We need to bring more data and analytic skills to the frontline. Nobody needs this more than we do, but practitioners haven’t been nourished with these skills.” Upskilling people on the frontline to use data effectively is as pressing a challenge as gathering the relevant data itself.
We also need to get better at investing in resources based on evidence and data, steering funds toward programmes that improve outcomes, and increasing investments where evidence of effectiveness is clear. Perhaps even more importantly, there needs to be a widespread commitment to redirecting resources away from interventions or initiatives that consistently fail to achieve desired results.
None of these developments are without their challenges. There is an inherent tension between using data for accountability and using it for improvement. When there is a risk of being defunded for showing weaknesses, no one is going to speak candidly about the need to improve.
What is certain is that there are fewer and fewer limits to what data our field has access to; data that will facilitate a deeper understanding of the system we’re trying to change. We must seize the opportunity.