What is Fidelity?
In research the “fidelity” problem is known as Type III error. To use plain, everyday language, this basically means that before we determine that a service or intervention is ineffective, we should make sure it was implemented or completed as directed, by qualified professionals and/or according to the required protocols. Indeed, the fidelity question is one that has been neglected for too long and is one that is at the core of the accountability movement and polices.
Fidelity can be described (in general) as composed by three elements (Lee et al., 2008):
- Exposure: the quantity, frequency, number of strategies recommended, evidence-based services available, accessed and/or used.
- Adherence: the degree of conformity of the action and strategies used to the standard guidelines (i.e., are all activities delivered as designed?).
- Quality of Delivery: the level of effort, commitment, and preparedness of the participants (including the degree to which attitudes and behaviors of participants are consonant with the goals and expectations of the program).
Why is it Important?
These three basic elements are said to be predictive of the responsiveness of the participants to the care processes as well as raising the likelihood for intervention to reach its highest effect. Most importantly, however, “fidelity” should also result in heightened awareness of the diversity of the clients’ needs, and this should lead to a greater level of program differentiation (i.e., uniqueness, personalization of the care strategies, activities, etc.). Fidelity is also critical for the providers’ delivery system in identifying the adequacy of their infrastructure to support and sustain the implementation effort (and this is not limited to High Fidelity Wraparound).
How is it Assessed?
Fidelity can be said to be in its infancy and efforts to study it have just begun. In general there have been three methodological approaches in assessing fidelity:
- Direct Observation: A trained independent observer/auditor; coded-videotaping (also considered the golden standard in fidelity assessment).
- Indirect Observation: Self- or provider-driven questionnaires – completed by program participants manually or online.
- Written Manuals: A step-by-step script that is followed to the detail by the implementers and is then reviewed on a regular basis.
Of course, in many instances a combination of these and/or or all three can be used for all intents and purposes. In the case of High Fidelity Wraparound (HFW) and System of Care (SOC) efforts, we are using some tools developed at the University of Washington by Eric J. Bruns and associates who are the originators of the National Wraparound Initiative, a collaborative nationwide effort. This team has developed what is known as the Wraparound Fidelity Assessment System (WFAS), a multi-method approach of assessing the above listed wraparound elements.
What are the WFAS Measures?
The Washington University team has developed a set of four measures of which three have already been made operational and available online. They are described by Bruns and associates as follow:
- Wraparound Fidelity Index, version 4.0:
“The Wraparound Fidelity Index 4.0 (WFI-4) is a set of four interviews that measures the nature of the wraparound process that an individual family receives. The WFI-4 is completed through brief, confidential telephone or face-to-face interviews with four types of respondents: caregivers, youth (11 years of age or older), wraparound facilitators, and team members. It is important to gain the unique perspectives of all these informants to understand fully how wraparound is being implemented. A demographic form is also part of the WFI-4 battery.
The WFI-4 interviews are organized by the four phases of the wraparound process (Engagement and Team Preparation, Initial Planning, Implementation, and Transition). In addition, the 40 items of the WFI interview are keyed to the 10 principles of the wraparound process, with 4 items dedicated to each principle. In this way, the WFI-4 interviews are intended to assess both conformance to the wraparound practice model as well as adherence to the principles of wraparound in service delivery”.
Team Observation Measure:
“The Team Observation Measure (TOM) is employed by external evaluators to assess adherence to standards of high-quality wraparound during team meeting sessions. It consists of 20 items, with two items dedicated to each of the 10 principles of wraparound. Each item consists of 3-5 indicators of high-quality wraparound practice as expressed during a child and family team meeting. Working alone or in pairs, trained raters indicate whether or not each indicator was in evidence during the wraparound team meeting session. These ratings are translated into a score for each item as well as a total fidelity score for the session overall”.
Community Supports for Wraparound Inventory:
“The CSWI is a research and quality improvement tool intended to measure how well a local system supports the implementation of the wraparound process. The CSWI is based on the framework of Necessary Conditions described by Walker, Koroloff and Schutte (2003), and presents 40 community or system variables that ideally are in place in communities that aim to implement the wraparound process. The CSWI is somewhat unique from the other WFAS instruments in that it assesses the system context for wraparound as opposed to the fidelity to the practice model for an individual child and family.
The CSWI can be used in several ways. First, it results in a quantified assessment of community supports for wraparound across multiple domains, so that researchers can determine the impact of these conditions on fidelity and outcomes of the wraparound process. Second, it presents the level of support across multiple domains (such as funding, collaboration, and accountability) so that evaluators and stakeholders can understand the full context for wraparound implementation as part of their local evaluation projects. Third, items and domains are structured so that local groups can assess community supports for wraparound, respond to areas of strength and weakness, and monitor improvements over time”.
CSWI domains include:
- Community Partnership
- Collaborative Action
- Fiscal Policies & Sustainability
- Access to Supports & Services
- Human Resource Development & Support
- What is Expected by the Participants?
The degree to which the PA HFW initiative will be successful depends on the providers’ determination and commitment to make this paradigm shift. Increasingly, community-based agencies are being required both at the state and federal level to adhere to and use best practice and/or evidence-based practice. In a way this can be seen by all the participants as a golden opportunity to make those adaptations in both content and structure that the granting agencies (state and federal) are demanding from community-based organizations. When programs move and commit to quality, the families and children they serve become the primary beneficiaries of this change. How well fidelity is attained (regardless of how each provider might call the HFW process) will determine what impact the programs, services, and treatments will have on the families and children they serve. It is this foresight on the part of the participants that will determine the degree of success or failure of these renewed efforts to achieve and sustain a system of care philosophy. But of course, fidelity is not just difficult to assess but also to sustain without the proper support and quid pro quo.
What’s in the Foresight?
It is quite alright for you to ask how committing to this foresight will contribute to your organization as a whole, what benefit will the implementation bring, how would it improve organization longevity, what if any support (technical, practical, resources) will commitment to this foresight offer; all of these are legitimate questions that we will try to address beginning with providing you an outline of the Youth and Family Training Institute (YFTI) implementation plan and collaborative vision.
There are basically three major (interrelated) parts to the HFW implementation:
- Fidelity Assessment: this includes the completion of the WFAS tools.
- Readiness Assessment: this involves training and technical support for the agency and its staff in HFW to attain and maintain credentialing guidelines.
- Online Data-Evaluation Support: This involves the long-term plan to provide online access to data tools and/or expertise and consultation for all of the participants in the HFW.
The underlying philosophy of the YFTI is to support the youth and families in getting better at what they do best. It is also to strengthen the outcomes of their care and services through collaborative efforts.