Causal Validity Considerations for Including High Quality Non-Experimental Evidence in Systematic Reviews

Federally funded systematic reviews of research evidence play a central role in efforts to base policy decisions on evidence. These evidence reviews seek to assist decision makers by rating the quality of and summarizing the findings from research evidence. Historically, evidence reviews have reserved the highest ratings of quality for studies that employ experimental designs, namely randomized control trials (RCTs). The RCT is considered the “gold standard” of research evidence because randomization ensures that the only thing that could cause a difference in outcomes between the treatment and control groups is the intervention program. However, not all intervention programs can be evaluated using an RCT. To develop an evidence base for those programs, non-experimental study designs may need to be used.