Volume 1, Issue 2

Response 1: The Value of Evidence-Based Practices in Comparison to “Treatment as Usual”: The Need for Meaningful Data
Hannah KnudsenBy Hannah K. Knudsen

“…[The] dominance and influence of Federal and state governments on the substance use disorder treatment field…creates the perceived need for control and accountability. The use of evidence-based practices in substance use disorder treatment helps meet this need.”

The adoption and implementation of evidence based practices (EBPs) has become an increasingly significant focus of a variety of groups including health services researchers, policymakers, and stakeholders in the substance use disorder treatment field. Generally, discussions of EBP adoption are centered on EBPs as a method of quality improvement, which is certainly a worthy goal. Adoption and implementation of EBPs also legitimizes the investment of basic research dollars in the processes of developing and testing EBPs’ effectiveness. Without adoption, those EBPs end up “on the shelf” and the investment in the research yields little in terms of societal benefit.

As Dr. Roman describes in his essay, the adoption of EBPs is increasingly talked about as a proxy for treatment quality and as a way to meet demands for accountability made by funders of addiction treatment. With some funding agencies linking reimbursement policies to the adoption of EBPs, such adoption is a means to ensure continued resource acquisition and likely has symbolic value as well.

But beyond those issues, for individual treatment organizations, what does EBP adoption accomplish, particularly for the clients that they serve?  Underlying much of the rhetoric about EBPs is the assumption that existing practice is not sufficient or is not effective, which may or may not be true in any particular treatment organization. For treatment organizations to make rational choices about the value of EBPs for their agencies, meaningful data about the effectiveness of their organization’s current practices or “treatment as usual” is critical. This is not data collection for its own sake, where data is simply collected and submitted to a regulatory authority without there ever being feedback about what the data actually revealed. Rather, it is about the collection of data in a timely fashion which is then analyzed and interpreted so as to be meaningful to organizational decision-makers.

Before data can be collected, however, treatment organizations must have a clear sense of what they consider to be the measures of quality that are relevant for their organization. Organizational decision-makers and staff within treatment agencies must reach a consensus about what the goals of their treatment protocols actually are. While long-term abstinence is a lofty ideal, is it more reasonable for agencies to focus on clinical progress during a treatment episode? Such a focus on progress during treatment has been gaining greater attention as a better metric for judging the effectiveness of treatment rather than focusing on outcomes long after clients have exited active treatment. (Dr. Tom McLellan and colleagues have written about this aspect of the chronic care model—see Resources for additional reading.)

Whatever an agency decides its goals of treatment are, the next step then is to define in very concrete terms how to measure those goals.  It may be measures that track days of retention in treatment, since study after study has shown that greater retention improves long-term outcome.  It might also include measures that address the number of days of abstinence during treatment or the number of treatment sessions attended. The measures developed by the Washington Circle might be useful in this regard (Garnick et al., in press).

The careful collection, analysis, and interpretation of data—particularly when those efforts are tailored to an organization’s mission and goals—then can allow agencies to gain a sense of: “How effective is treatment as usual?” The answer to that question is critical because it can inform the assessment of how much value might be gained from adopting (or not adopting) a particular EBP. For treatment programs that are accomplishing their goals in terms of delivering high-quality treatment services, adoption of EBPs may yield only modest benefits. However, for those programs that identify gaps in quality, the EBP adoption process may result in far greater gains for their clients.

Having a data management system that produces meaningful information is not only important for adoption decisions. It is the key to answering the question about whether the adoption and implementation of an EBP actually makes a difference for the organization’s clients. Knowing the baselines for these kinds of performance indicators can help decision-makers evaluate whether there are good reasons to continue to invest in the implementation of an EBP. Deciding to adopt an EBP certainly does not guarantee that it becomes a part of routine practice—that requires investment of resources into the implementation process, whether it’s investing dollars and time in on-going staff training or designing strategies for clinical supervision. Meaningful data about whether an EBP is helping an organization to reach its goals is needed to justify its implementation over the long-term.

Continue reading Response 2: The States are the Key Factor
in Successful EBP Adoption and Technology Transfer

Resources:

Garnick, D. W., Lee, M. T., Horgan, C. M., Acevedo, A., and Washington Circle Public Sector Workgroup. (in press). Adapting Washington Circle performance measures for public sector substance abuse treatment systems.

McLellan, A. T., Chalk, M., & Bartlett, J. (2007). Outcomes, performance, and quality—what is the difference? Journal of Substance Abuse Treatment, 32, 331-340.

McLellan, A. T., McKay, J. R., Forman, R., Cacciola, J., & Kemp, J. (2005). Reconsidering the evaluation of addiction treatment: From retrospective follow-up to concurrent recovery monitoring. Addiction, 100, 447-458.



Return to Top | Back to Home