Implementation of Evidence-Based Practices

Evidence-Based Training Initiative

Upon completion of this learning module you will have:

• Identified the concepts of evidence, the general evidence-based practice process, and the process of implementing evidence-based practices
• Identified key decision points in the process of implementing evidence-based practices
• Applied knowledge to better navigate potential challenges in real-life implementation efforts

Evidence-based behavioral practice: making decisions about how to promote healthful behaviors by integrating the best available evidence with practitioner expertise and other resources, and with the characteristics, state, needs, values and preferences of those who will be affected.

How exactly do practitioners and their institutions or organizations go about implementing evidence-based practices?

What can be done to ensure they have access to information about evidence-based interventions?

What can be done to assure they possess the knowledge, skills, and means to implement the interventions successfully?

The answers to these questions can be found in implementation science.

Questions and barriers to implementation may arise at multiple levels of healthcare delivery: the patient level, the provider team or group level, the organizational level, or the market/policy level.

Upon completion of this learning module, you will have:

• Identified the concepts of evidence, the general evidence-based practice process, and the process of implementing evidence-based practices
• Identified key decision points in the process of implementing evidence-based practices
• Applied knowledge to better navigate potential challenges in real-life implementation efforts

Implementing Interventions

Take a moment to review what an evidence-based practice is and what considerations go into effectively implementing an intervention.

• For a practice or intervention to be defined as evidence-based, it must undergo sufficient scientific evaluation to be proven effective (e.g., intervention is considered valid or ―proven‖ because it is strongly linked to a desired health outcome).
• The five steps of evidence-based practice are used to integrate evidence with available resources and patient or population preferences to make decisions about promoting healthful behaviors.



For more information, or to review more about the steps of the EBBP process, try the EBBP Process Module at http://www.ebbp.org/training.html

While an expert panel outside your organization might rate an intervention as having a ―gold standard level of evidence supporting its use, stakeholders within your organization may have an entirely different perception of that same evidence.

• The organization's perception of the evidence will largely determine how well it is received and implemented.
• The consideration of patient needs and resources must be integral to any implementation that seeks to improve patient and/or community outcomes.(1)
• How an intervention‘s implementation is rewarded, supported, and accepted within an organization is also important in determining success or failure.(2)
1. Committee on Quality of Health Care in America & Institute of Medicine (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, D.C.: National Academies Press.
2. Klein K.J., & Sorra, J.S. (1996) The challenge of innovation implementation. The Academy of Management Review 21(4)1055-1080.

The Implementation Process

The implementation process is very similar to the EBBP process, with activities that can be revisited, expanded, refined, and re-evaluated throughout the course of the process. These essential activities(1) are:

• The Consolidated Framework for Implementation Research (CFIR) provides a pragmatic structure for implementation that practitioners can use to:
1. Plan a course of action
2. Engage appropriate individuals and organizations in the implementation and use of the intervention
3. Execute the implementation according to plan
4. Evaluate implementation efforts

1. Damschroder, L.J., Aron, D.C., Keith, R.E., Kirsh, S.R., Alexander, J.A., & Lowery, J.C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science 4(1), 50.

The first step of implementation, Plan, is conceptually very similar to the first three steps of EBBP: Ask, Acquire, and Appraise.

Implementation Steps

Evidence-based interventions begin with three key steps:

• Ask important questions about the care of individuals, communities, or populations.
• Acquire the best available evidence regarding the question.

Critically appraise the evidence for validity and applicability to the problem at hand.

Similar to the first three steps of EBBP, the implementation process involves planning a course of action to promote effective implementation. During the Plan activity, you should:

• Ask important questions about the goals for the intervention and how involved individuals and organizations will be impacted.
• Acquire evidence on what interventions have worked in systems or settings similar to yours and gather information on how they were implemented.

Appraise those implementation strategies to determine what aspects will work in your setting and what elements may need to be adapted.

When conducting a critical appraisal of an implementation strategy, there are certain elements you should be sure to review:

• How were the needs and perspectives of all stakeholders considered?
• How were implementation strategies tailored for appropriate subgroups (e.g., delineated by professional, demographic, cultural, and organizational attributes) and what methods were developed to educate those subgroups on implementation strategies?
• What were the communication channels and how were they identified?

How rigorous were the monitoring and evaluation methods used to track progress toward goals and milestones?

During the Apply step of EBBP, evidence is applied by engaging in collaborative health decision-making.

With implementation, you are engaging the communities, institutions, organizations, etc., that will be impacted by the process of implementing the intervention.

• This is done by attracting and involving appropriate individuals in the implementation process through a combined strategy of social marketing, education, role modeling, training, and other similar activities.(1)
• Planning should continue during this step.
• During the engage step, implementation leaders may be carefully selected or allowed to rise naturally.

1. 1. Damschroder, L.J., Aron, D.C., Keith, R.E., Kirsh, S.R., Alexander, J.A., & Lowery, J.C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4(1), 50.

Just as certain factors need to be taken into account during the Apply step for EBBP, there are factors that can affect the execution of an intervention‘s implementation. These factors include:

• the degree to which implementation efforts follow the planned course of action
• the number of individuals or groups (and associated resources) impacted by the implementation process
• the timeliness of task (social marketing, training, organizational restructuring, etc.) completion
• the values, preferences, and engagement of involved individuals and organizations.

The final step of evidence-based behavioral practice is Analyze and Adjust, in which intervention outcomes are assessed and certain aspects are adjusted. Evaluating the implementation process has the same objective.

• Evaluation can include quantitative feedback such as reports and graphs, as well as qualitative feedback and anecdotal stories of success.

Results are measured against objectives developed in the planning stages that are specific, measurable, attainable, relevant, and timely (SMART).

The Fifth Domain

The four essential activities of implementation were developed by Laura J. Damschroder and colleagues as the fifth domain in The Consolidated Framework for Implementation Research (CFIR). The CFIR is a comprehensive framework that includes common constructs from a broad array of published implementation theories.

Constructs are organized into five major domains: intervention characteristics, outer setting, inner setting, characteristics of individuals involved, and the process of implementation. While this course will focus mainly on the fifth domain, the process of implementation, it is important that you are familiar with all five.

Things to consider when examining the characteristics of an intervention:

• What are the stakeholders‘ perceptions of the intervention‘s source (who developed it, where, and why) and the quality and validity of the evidence supporting the intervention?
• To what degree can the intervention be adapted, tailored, refined, or reinvented to meet specific organizational needs?
• What is the stakeholders‘ perception of the advantage of implementing the intervention versus an alternative solution?
• How complex would this implementation be, based on the duration, scope, disruptiveness, intricacy, and number of steps required?
• Do we have the ability to pilot test the intervention on a small scale and reverse course if warranted? (trialability)

Cost of Implementing an Intervention

How much will the intervention cost to implement?

Primary

• When considering the characteristics of an intervention, you will want to take into account how well patient or community needs, as well as barriers and facilitators to meet those needs, are known and prioritized by your organization
• Barriers and facilitators that are external to your organization are considered part of the outer setting
• Outer setting: The economic, political, and social context the organization conducting the intervention resides in.

Node 1: Patient needs and resources

• The consideration of a client or community‘s needs and resources is integral to any implementation designed to improve outcomes. If the client or community resists an intervention, successful implementation of that intervention is extremely unlikely.
• The Practical, Robust Implementation and Sustainability Model (PRISM) outlines six elements that can help ensure patients are at the center of the design-making process:
• Patient choices are provided
• Patient barriers are addressed
• Transition between program elements is seamless
• Complexity and costs are minimized
• Patients have high satisfaction with service and degree of access and receive feedback

Node 2: Networking and peer pressure

• Networking with people or groups outside of an organization helps to promote information sharing.
• Organizations that are networked with other external groups or organizations are more likely to implement new interventions quickly due to the sharing of lessons learned and best practices.
• If there is a lot of support for the new practice, the pressure to implement the intervention can be especially strong for late-adopting organizations.

Node 3: External policies and incentives

• External policies and incentives to spread the implementation of an intervention can include:
1. Policy and regulations (governmental or other central entity)
2. External mandates
3. Pay-for-performance

Public or benchmark reporting

Primary

An organization‘s readiness to implement an intervention can be determined by several factors, such as its structural characteristics and internal networks.

Node 1: Structural characteristics

• The structural characteristics include the age, maturity, and size of an organization.
• The number of units or departments within an organization often represents the diversity of knowledge in that organization.
• The longer members of a team are able to remain with that team, the more stable the team will be, and the more likely that the implementation will be successful.

Node 2: Networks and communications

• Relationships between individuals within an organization may be more important than individual attributes.
• The clear communication of the mission and goals of an intervention can lead to cohesion between staff and encourage collaboration and open feedback among peers. This will increase the chances of effective implementation.

Node 3: Culture and implementation climate

• The CFIR views the culture of an organization as relatively stable, socially constructed, and subconscious, while climate is viewed as the localized and more tangible manifestation of the overarching culture. Six sub-constructs contribute to a positive implementation climate for an intervention:
1. Tension for change: the degree to which the current situation is perceived as intolerable or needing change
2. Compatibility: implementation is likely to be more effective when individuals feel the meaning they‘ve attached to an intervention aligns with the meaning communicated by upper management
3. Relative priority: individual‘s shared perception within the organization of the implementation‘s importance
4. Organizational incentives and rewards: these can include promotions, raises in salary, or increased stature or respect
5. Goals and feedback: degree to which goals and feedback are clearly communicated and acted upon
6. Learning climate: team members need to feel they are essential partners in the change process and that they have sufficient time to implement and evaluate interventions

Primary:

―People are not passive recipients of innovations. Rather they seek innovations, experiment with them, evaluate them, find—or fail to find—meaning in them, develop positive or negative feelings about them, challenge them, complain about them, ‗work around‘ them, gain experience with them, modify them to fit particular tasks, and try to improve or redesign them – often through dialogue with other users.

Greenhalgh, T., et al. (2004). Diffusion of innovations in service organizations: systemic review and recommendations. The Milbank Quarterly, 82(4), 581–629.

Node 1: Knowledge and beliefs about the intervention

• It is important to consider an individual‘s attitude toward the intervention and the value he or she places on it.
• Often, this attitude can be affected (positively or negatively) by the subjective opinions obtained from peers.
• An individual‘s familiarity with the facts, truths, and principles related to the intervention can also play a part in how he or she perceives it.

Node 2: Self-efficacy and stage of change

• If an individual is confident in his or her capabilities to achieve implementation goals, he or she will be more likely to make the decision to embrace the intervention and exhibit committed use, even in the face of obstacles.
• Different models have been used to characterize the stages an individual can go through as he or she progresses toward the skilled, enthusiastic, and sustained use of an intervention. The specific stages used will depend on the underlying model of the study.
• While Stages of Change characteristics are necessarily measured at the individual level, these measures may be appropriately aggregated to team, unit, or service levels in analyses(1), as organizations also go through Stages of Change in their readiness to adopt an intervention.
1. 1. Damschroder, L.J., Aron, D.C., Keith, R.E., Kirsh, S.R., Alexander, J.A., & Lowery, J.C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4(1), 50.

Node 3: Individual identification with organization

How individuals perceive the organization and their commitment to the organization may affect their willingness to fully engage in implementation efforts or use an intervention.

This course will focus mainly on the fifth domain of the CFIR, the process of implementation:

• Plan a course of action
• Engage appropriate individuals and organizations in the implementation and use of the intervention
• Execute the implementation according to plan
• Evaluate implementation efforts

The four essential activities of the implementation process are:

• Plan: establishing the methods of behavior change and tasks for implementing an intervention— these need to be developed in advance of actual implementation
• Engage: attracting and involving key stakeholders in the implementation of an intervention
• Execute: overcoming barriers and adapting strategies to accomplish the implementation according to plan

Evaluate: gathering and analyzing quantitative and qualitative feedback about the progress and quality of implementation

Overview

Primary

Take a few moments to look at the genesis of the Project Educare program:

• Recently, your team implemented a series of in-person trainings for clinical mental health counselors.
• You and your team presented your experiences and findings from this implementation at several conferences about evidence-based practices.
• You encountered strong desires among many clinicians to learn how to implement evidence-based practice, while others expressed a strong sentiment against EBP.
• Some people questioned the feasibility of providing training to clinicians in both urban and rural settings, given time and budget constraints.
• You recognized a clear need to develop a training opportunity that would reach a larger audience.

Node 1: The Idea

• You need to develop a training course that can teach clinical mental health counselors how to implement evidence-based practices for the treatment of children and adolescents, as well as clear up common misconceptions about EBP.
• The content for the training will be based on the content developed in cooperation with three local universities for a series of in-person trainings.
• This training will need to reach clinicians in both urban and rural settings.
• The available time and cost of the training will need to be considered in order to ensure the training is widely available to clinicians throughout the state.

Node 2: The Funding

• You and your team receive grant funding from the State Division of Mental Health's Child and Adolescent Unit to pursue your initiative
• This is not a large grant. Though you feel confident that it will be enough to implement the intervention, your team will be operating on a very tight budget.

Node 3: The Current Team

• You: PhD, licensed clinical professional counselor, assistant professor, director of Project Educare.initiative
• Five doctoral-level counselors/educators in training.
• Three master's students working within the clinical mental health preparation specialty.

Planning

As you begin planning the implementation of Project Educare, you meet with your team to discuss possible options for developing a training course that can reach clinical mental health counselors throughout the state and educate them on how to implement evidence-based practices for the treatment of children and adolescents.

Surveys from the in-person trainings along with follow-up communications with some participants indicated that clinicians had more favorable attitudes toward EBP and were implementing EBP practices with more confidence after attending the trainings. However, conducting similar in-person trainings statewide is simply not feasible, given the size of your team and budget.

Each of your team members has an opinion about how the in-person trainings can be adapted for Project Educare.

• The team needs to find a way to bring the training to participants instead of making them come to the training.
• The team could possibly write a book that those who are unable to attend the in-person trainings could buy online.

Your team has come up with a few options for how to present the curriculum from the in-person training to a statewide audience.

Help your team choose the option that best meets your initial intervention goals by reviewing each one and considering the pros and cons.

Primary

Implementation leaders can help generate support and encourage uptake of an intervention, as well as provide valuable input. These individuals can positively influence the opinions of their colleagues and peers, serve as role models, and actively champion the intervention.

You and your team have given some thought to what characteristics will distinguish the implementation leaders for this initiative, and you have chosen the following criteria:

• Subject matter expertise
• E-learning development expertise
• Stakeholder experience/influence
• Available time of the stakeholder
• Program budget impact

Node 1:Subject matter expertise

Experts on the content can help your team adapt the materials you have for the online format and ensure that the information remains clear and correct.

You and your team decide it would be beneficial to contact someone from the state's DMH Child and Adolescent task force on evidence-informed practices who can put you in contact with participating program directors, providers and parents.

Gathering input from the people who will be most impacted will help you generate a final course that will be meaningful to all members of your target audience.

Node 2:E-learning development expertise

Converting instructor led activities into an online format can be challenging. You will want someone with experience in this, as well as someone who can provide ideas for how to create a final product that is as interactive and engaging as the in-person training

If it's decided that discussion boards or video excerpts of expert trainers should be included, you will want someone who knows how best to integrate these types of elements into the overall online experience.

Node 3: Stakeholder experience/influence

Stakeholders with experience and influence will be able to exert that influence through authority, status, and credibility, encouraging other clinicians to participate by setting the right example.

You will want to look for people who are enthusiastic about the goals of the training and who ideally have formal or informal influence on the attitudes and beliefs of their colleagues. Those who have the ability to exert influence through either their authority and status or representativeness and credibility can be recruited as implementation leaders.

Another important factor to consider is the individual's leadership style. A leader with a more proactive approach will likely spend more time encouraging the implementation.

Node 4: Available time of the stakeholder

This project will require a varying amount of time and effort from your stakeholders. Those who you will look to as implementation leaders must have the availability to commit their time.

Node 5: Available time of the stakeholder

Some implementation leaders may be able to volunteer, while others may require reimbursement for their time. This factor should be taken into careful consideration, as your team is on a tight budget.

• You created an action plan after taking into consideration the needs, preferences, and resources of your stakeholders.
• You then engaged appropriate implementation leaders to help you address the items on your action plan.
• Now it is time for the third activity of the implementation process: Execute.
• Your team conducts an extensive review of the literature on the evidence-based practices for disorders such as depression, anxiety, and substance abuse.
• They separate these topics into course sections, and prepare themes and titles
• Finally, your team goes about creating the courses and scheduling and completing multiple reviews.

Planning

• You will again practice the implementation process by playing the role of implementation leader for the Evidence-Based Training Initiative
• You will complete simulated activities based on the development of the real Evidence-Based Training Initiative (EBTI) program.
• For the purposes of this course, certain liberties have been taken. The actual implementation process and strategy of the EBTI program varies from what is depicted here.

Primary

• There is a need for more clinician training on evidence-based practices at state agencies in the child and adolescent mental health system.
• Funding has been secured for a high impact, in-person training program designed to encourage clinicians to implement evidence-based practices with their patients.

Node 1: Concept

• The State Child and Adolescent Mental Health System would like to infuse concepts of science and evidence-informed practice in the state's community based children's mental health system.
• The goal is to design a high impact, in-person training program which will encourage agencies and clinicians to utilize and promote evidence-based patient interventions.

Node 2: Funding

• This initiative has received $420,000 in funding from the Children‘s Mental Health Partnership Appropriation.
• As part of this funding, the Department of Human Services Division of Mental Health has agreed to issue fellowships of up to $18,000 to participating providers to help cover the cost of travel and to buy clinician time to attend the trainings.

Node 3: Current Team

• You: the Associate Deputy Clinical Director of the Department of Human Services, Division of Mental Health, Child and Adolescent Services.
• Six parent representatives, who volunteered or were asked to join based on their knowledge of the state's Child and Adolescent Mental Health System.
• Seven university professors, who volunteered or were asked to join based on their interest in the initiative or their significant knowledge of child and adolescent mental health treatment.
• Ten Child and Adolescent Clinical Directors from community mental health agencies.

An important part of planning an intervention is reviewing the available evidence. Your team has discovered several studies, reviews, and articles that seem pertinent to your project to develop a high impact, in-person training program.

Based on the research you've found, your team has developed some basic options for the Evidence-Based Training Initiative. Take a moment to review the chosen options shown on the screen.

Basic options for the Evidence-Based Training Initiative (EBTI):

• Selected cohort agencies will receive training in Chorpita's Common Elements approach.
• The training will run for 12 months.
• Two expert trainers will lead the sessions.

Once initial training is complete, each mental health agency will assign a supervisor to meet with other agency providers every two weeks about implementing evidence-based practices with their patients.

Primary

Your team has also decided to use the Fixsen implementation model as a basis for EBTI. You meet with your team to discuss Fixsen and the evidence-based practices you want to teach.

Node 1: Carlos, a university professor

―There are core implementation components we will want to focus on as we plan and carry out our implementation efforts. The first few I found in the Fixsen implementation model are staff selection and pre-service and in-service training. As we select the agencies we are going to work with, we‘ll need to develop a process to determine how many personnel they will commit and what sort of experience those participants should have.

Node 2: Gary, a community mental health provider

―Fixsen also recommends that we provide ongoing consultation and coaching. We will need to determine what the consultation will look like, who will be responsible for it, and how it should happen. How are we going to select the experts who will conduct this training and provide continuing support

Node 3: Alexis, a parent representative

―We need to consider how we will evaluate our progress and the quality of our implementation efforts as well. How do we want to assess the use and outcomes of the evidence-based practices we'll be teaching during the training and reinforcing during the consultation and coaching process?

Primary

You and your team have chosen the following criteria for selecting participating agencies:

• Agency influence
• Experience level of participating clinicians
• Availability of participating clinicians
• Budget impact
• Commitment to further dissemination

Node 1

Agency influence

Agencies with influence will be in a better position to lead other organizations by example, and may be able to disseminate the training further than agencies with little influence.

Node 2

Participants with more experience treating children and adolescents will be in a better position to understand, apply, and teach others than clinicians with comparatively little experience in the field.

Because evidence-based practices will be further disseminated throughout the agencies by those trained, it is important that the participants be dedicated and have the necessary background to conduct the training appropriately.

Node 3

Availability of participating clinicians

This project will require a considerable time commitment, and both the trainers and participating clinicians must have the availability to attend the trainings.

Participating clinicians who will act as supervisors will need to be available to meet with their colleagues every two weeks about implementing evidence-based practices with their patients and to lay the groundwork for training new staffers in the future.

Node 4

Budget impact

Consider whether the agency qualifies for a fellowship to cover expenses and whether they have the budget to support any additional expenses, like compensating clinicians for their time or upgrading videoconferencing equipment to ensure they can participate in all segments of the training.

Node 5

Commitment to further dissemination

Agencies that currently partner or plan to partner with a training university will be able to disseminate the training to students, broadening the intervention‘s reach.

Participating agencies should also plan to disseminate the training to additional staff once the participating clinician or clinicians are sufficiently trained, to ensure that the message of EBTI reaches as many clinicians as possible.

Plan

• In the Plan phase, you met with your team of researchers, providers, and parents to develop your implementation approach.
• Based on these discussions, and your team‘s research, you developed an action plan.

Engage

• You carefully selected agencies that were best suited for participation in the program and determined how best to engage with the participating clinicians.
• Collaborating with the participants allowed you to identify potential challenges and to address them by revisiting and revising the action plan.

Execute

• You were challenged to select the best strategy for conducting the training.

Evaluate

• Once the first cohort completed the program, you applied an evaluation method that collected useful data for study.
• Using information gathered in evaluation and an analysis of the outcomes, you were able to identify new issues and to adapt the implementation strategy for future cohorts