Our Evidence and Gap Maps bring together evidence on homelessness interventions from around the world to highlight where evidence does or doesn’t exist on what works and why they work or not. This helps target research investments faster and in a more strategic, impactful way.

The Effectiveness Map, or ‘what works’ map - captures impact evaluations and effectiveness reviews. The Implementation Issues or ‘why things work or not’ map focuses on the factors that impact on the successful implementation of homelessness interventions. Together the two maps capture around 600 studies on interventions – the largest resource of its type anywhere in the world.

These EGMs have been created with our partners at The Campbell Collaboration and Heriot-Watt University. Both will be updated at regular intervals. The Effectiveness map has now been categorised to show how much confidence we can have in the findings: high, medium and low confidence.

Evidence Standards

The Centre is applying standards of evidence to each of our tools and maps, which have been developed with our partners The Campbell Collaboration.

‍

Standards of Evidence for the Effectiveness Map

‍

Each study in the map has been rated as high, medium or low for ‘confidence in study findings’. For systematic reviews in the map this rating was made using the revised version of ‘A MeaSurement Tool to Assess systematic Reviews’ (AMSTAR 2). The rating of primary studies was made using a critical appraisal tool based on various approaches to risk of bias assessment.

The two tools – AMSTAR 2 and the primary study critical appraisal tool – assess a range of items regarding study design and reporting. Some of these items are designated as ‘critical’. The overall rating for the study is the lowest rating or any critical item.

Primary studies

Critical items

Study design: We have high confidence in study findings from studies designs best able to detect causal effects such as randomized control trials.

Attrition: High levels of attrition, especially differential attrition between the treatment and comparison groups, reduce the confidence we can have in study findings.

Outcome measure: For the study findings to be usable and meaningful there should be a clear description of the outcome measures, preferably using existing, validated approaches.

Baseline balance: We can have less confidence in study findings if there were significant differences between the treatment and comparison groups at baseline.

Other items

(assessed but not affecting overall rating):

Blinding: The absence of blinding of participants and researchers can bias study findings. This may be so even though blinding is not possible.

Power calculations: Power calculations help determine the sample size required. Without such calculations there is a risk of underpowered studies and so a high likelihood of not correctly identifying effective programmes.

Description of intervention: A clear description of the intervention is necessary to be clear what is being evaluated, so that effectiveness is not assigned to similar, but different, interventions.

‍

Systematic Reviews

‍Critical items

‍

Protocol registered before commencement of the review

Adequacy of the literature search.Justification for excluding individual studies

Risk of bias from individual studies being included in the review

Appropriateness of meta-analytical methods

Consideration of risk of bias when interpreting the results of the review

Assessment of presence and likely impact of publication bias

Other items

‍

PICOS in inclusion criteria

Rationale for included study designs

Duplicate screening

Duplicate data extraction

Adequate description of included studies

Report sources of funding

Risk of bias assessment for meta-analysis

Analysis of heterogeneity

Report conflicts of interest

‍

For a complete description of the coding tools click here for the primary study critical appraisal tool and here for AMSTAR 2.

‍

Evidence and Gap Maps

See the big picture of evidence on homelessness interventions – what do we know, and what don’t we know?

ABOUT THE Evidence & Gap Maps

Our Evidence and Gap Maps bring together evidence on homelessness interventions from around the world to highlight where evidence does or doesn’t exist on what works and why they work or not. This helps target research investments faster and in a more strategic, impactful way.

Evidence Standards

Standards of Evidence for the Effectiveness Map

Primary studies

Critical items

Other items

Systematic Reviews

‍Critical items

Other items