Adapting evidence-based programs to diverse populations and settings
By Caroline Kuo, DPhil, MPhil
Associate Professor, Brown University School of Public Health
Honorary Associate Professor, University of Cape Town
As we work to close disparities in substance use prevention and treatment, we need to turn to the evidence-base and choose interventions that not only work, but are likely to have the largest impact upon the health behaviors we are trying to address. However, interventions are designed and tested in a particular population and context. The transportability of evidence-based models is not always appropriate or guaranteed. Culture, language, poverty, health systems differences, and other social and structural determinants of health may adversely affect engagement in, and response to evidence-based intervention models that are applied to new populations and settings unless the interventions are adapted. Such adaptations to existing evidence-based programs should not alter the core components of the intervention in order to maintain the intervention’s efficacy and effectiveness. Ideally, adaptation procedures strike a delicate balance between the idea of interventions being “universally applicable” to all contexts with the notion that “custom tailoring” is necessary to address inevitable differences in populations, settings, and cultures.
The following six sequential steps can guide the process of assessing, and if necessary, adapting existing interventions to new populations and settings:
Step 1: Evaluate – What is the evidence for potentially promising existing intervention models that address the target outcome(s)? Review the evidence on existing psychological intervention models in order to identify a candidate intervention for adaptation. Such a review can be conducted by searching for existing systematic reviews or meta-analyses or conducting one’s own review. Narrowing down choices to one candidate intervention can be accomplished by examining the depth of empirical evidence. In choosing the final candidate model, it is helpful to evaluate evidence around efficacy (effects of treatment under controlled trial settings focused on high internal validity) as well as effectiveness (effects of treatment in real world settings focused on high external validity). If possible, choose a candidate intervention that has been tested in multiple populations and adapted successfully to diverse populations and settings. This may indicate promise in adaptation potential to the new target population and setting.
Step 2: Retain core components – What are the core components of the candidate intervention that should be retained to ensure fidelity? Identify and retain core components of the candidate intervention that will be used for adaptation. Core components are essential intervention elements thought to impact the target outcome(s). Core components typically align with the theoretical origins of the intervention, including the mechanisms of change thought to alter outcomes. These components could be comprised of functions (e.g., education to create an understanding of the illness) and activities (e.g., frequently practicing coping skills will increase self-efficacy and result in improved outcomes). Identification of core components of the candidate intervention can be accomplished by contacting the original intervention designer, reviewing published literature, and if available, consulting the intervention manual.
Step 3: Operator changers – What unique aspects related to the target population and settings are likely to negatively impact target outcomes if the intervention were to be delivered in an un-adapted format? Identify these aspects in order to balance fidelity with issues of fit. The process of assessing adaptation needs should be data-driven to safeguard against “improvised” drift that could compromise fidelity or result in unwarranted changes to ensure fit. Assessing adaptation needs should be data-driven and might affect the following aspects of the original intervention model: 1) changes to intervention materials including curriculum content, role-play, vignettes, examples, goals, and methods to increase salience with the target population; 2) integration of principles or metaphors unique to the study population which might affect the original interventions theory of change; and 3) changes to delivery modality including who delivers the intervention and the concomitant training and/or supervision needed for interventionists to implement with integrity in the particular context.
Step 4: Generate a list of valid and reliable measurement tools – What are psychometrically valid tools for assessing outcomes with the target population? Generate a list of these tools to screen and evaluate target outcomes for the adapted intervention. These measures will help to verify that adaptation changes have retained the core components of the intervention.
Step 5: Research acceptability and feasibility of the adapted intervention – Do initial adaptations have salience with the target population? Research the acceptability and feasibility of the adapted intervention with the target population and with service providers operating in the target health system as an initial step in preparation for either efficacy testing or effectiveness testing of the adapted intervention. Assessing initial salience with the target population can be conducted using a number of approaches. Interviews or focus groups can be used to gather feedback on the acceptability of the intervention by the target population and their social ecosystem, alliance with interventionists, and satisfaction with the intervention materials and format. These assessments of salience should also investigate whether principles of the intervention (Step 2) need to be achieved through population- and context-specific approaches.
Step 6: Assess model efficacy – What is the efficacy or effectiveness of the adapted intervention on the desired outcome? This final step assesses model efficacy or effectiveness upon target outcomes. The principles of efficacy testing include randomization, comparison with a control condition, and adequate power to assess effect sizes. If the goal of the adaptation process was to prepare for effectiveness testing, three commonly used research designs can be considered: 1) a study design that is situated closer towards the effectiveness end of the spectrum where the main goal is to test the impact of the intervention on outcomes, often under less controlled conditions, while gathering information on implementation strategies; 2) a hybrid model that simultaneously examines the effectiveness of the intervention and implementation strategies; and 3) a study design that is situated closer towards the implementation end of the spectrum where the main goal is to test implementation strategies including adoption and fidelity to the intervention with the secondary goal of gathering information on the effectiveness of the intervention.
In summary, this conceptual model can provide a flexible guide to the process of transporting existing empirically-based intervention models to diverse contexts. Such a model might lend crucial scientific rigor to the process of transporting empirically-supported models from one context to another, and in so doing, help to broaden and diversify the use of the existing evidence-base to help close gaps in care.