A constructive Indian country response to the evidence-based program mandate.
Indian country (United States)
Evidence-based medicine (Practice)
Evidence-based medicine (Government finance)
State finance (Usage)
State finance (Health aspects)
Native Americans (Civil rights)
Native Americans (Laws, regulations and rules)
Walker, R. Dale
Bigelow, Douglas A.
|Publication:||Name: Journal of Psychoactive Drugs Publisher: Taylor & Francis Ltd. Audience: Academic Format: Magazine/Journal Subject: Health Copyright: COPYRIGHT 2011 Taylor & Francis Ltd. ISSN: 0279-1072|
|Issue:||Date: Oct-Dec, 2011 Source Volume: 43 Source Issue: 4|
|Topic:||Event Code: 930 Government regulation; 940 Government regulation (cont); 980 Legal issues & crime; 200 Management dynamics; 900 Government expenditures Advertising Code: 94 Legal/Government Regulation Computer Subject: Government regulation|
|Product:||Product Code: 9210120 Public Finance-State; 9210124 Expenditures-State Govt NAICS Code: 92113 Public Finance Activities|
|Geographic:||Geographic Scope: Oregon Geographic Code: 1U9OR Oregon|
Abstract--Over the last 20 years governmental mandates for
preferentially funding evidence-based "model" practices and
programs has become doctrine in some legislative bodies, federal
agencies, and state agencies. It was assumed that what works in small
sample, controlled settings would work in all community settings,
substantially improving safety, effectiveness, and value-for-money. The
evidence-based "model" programs mandate has imposed immutable
"'core components," fidelity testing, alien programming
and program developers, loss of familiar programs, and resource capacity
requirements upon tribes, while infringing upon their tribal sovereignty
and consultation rights. Tribal response in one state (Oregon) went
through three phases: shock and rejection; proposing an alternative
approach using criteria of cultural appropriateness, aspiring to
evaluability; and adopting logic modeling. The state heard and accepted
the argument that the tribal way of knowing is different and valid.
Currently, a state-authorized tribal logic model and a review panel
process are used to approve tribal best practices for state funding.
This constructive response to the evidence-based program mandate
elevates tribal practices in the funding and regulatory world,
facilitates continuing quality improvement and evaluation, while
ensuring that practices and programs remain based on local community
context and culture. This article provides details of a model that could
well serve tribes facing evidence-based model program mandates
throughout the country.
Keywords--American Indian, culture-based program, evidence-based program, model program, tribal best practice
Funding for tribal behavioral health programs (including substance abuse and mental health) now sometimes comes with a requirement that those programs be replications of programs drawn from a list of evidence-based, "model programs" (e.g., NREPP 2011 ; Center for the Study and Prevention of Violence 2011), implemented with the assistance of the "developer," and be subjected to "fidelity" testing to prove they are implemented as originally designed (Donaldson et al. 2009; Oregon Addictions and Mental Health Division 2011 ). The strong suggestion to use model programs is written into important government policies and mandates, including Requests for Applications. In this way, an organizational approach known as "remunerative coercion" (Etizioni 1968) is employed; this tactic is known as the "golden rule" in some government circles.
The problem with disseminating model programs has been identified for decades (Bachrach 1986; Campbell 1979; Suchman 1967) while the major importance of local context in the efficacy of programs has been identified (Pawson & Tilley 1997). Further, Chandler and Lalonde (2004) identified the fallacy of coercive dissemination to native peoples, and provided evidence of the efficacy of tribal knowledge and practices. Problems for tribes subjected to a model programs mandate include:
* Model programs are not always adaptable to Indian communities
* Developer training and fidelity testing are often alien processes, and rarely available to tribes
* Loss of familiar and accepted practices and programs can be traumatic for the tribe
* The very mandate is in conflict with principles of sovereignty, consultation, and government-to-government relations
Phased implementation, adaptation with the exception of "core components," or some other token mollification is usually offered by advocates and those mandating model programs, but the model programs mandate is a declaration of epistemological war with no serious attempt to implement "government-to-government consultation" or to acquire a culturally competent understanding of AI/AN ways of knowing as they apply to behavioral health. There is some recent exception to this since President Obama's 2009 Presidential Memorandum on Tribal Consultation. Exceptions aside, and however well intentioned, such government policy is oppressive, disempowering and a bad fit for Indian Country--a recipe for exacerbating, not ameliorating health disparities.
The Oregon Tribal Best Practices (TBP) initiative began with a state-legislated evidence-based model programs mandate (Oregon Revised Statute 669, enacted in 2003) that was problematic for Oregon's nine tribes. Fortunately, the response in Oregon was constructive engagement. Tribes were respected and empowered by an invitation to come forward with a culturally appropriate approach to identifying best practices. This leads us to believe that the TBP methodology may be a prototype for responding to model programs mandates elsewhere. Oregon's TBP initiative was based on several premises, and involved many processes which are discussed below.
Even with replication, randomized controlled/clinical trials are unable to encompass all the significant moderating and mediating variables of local context and culture. The effects of many moderating and mediating variables are better known to community practitioners in prevention and treatment programs. Even more challenging to scientific study are these "dynamically complex, emergent systems" features of behavioral prevention and treatment:
* Choice is a powerful factor in effectiveness of an intervention that the experimental scientist cannot randomize or control
* Self-healing belief guided by an expert healer, is a factor to which the experimental scientist can't "blind" or "double-blind" participants
* Guided development of unique interpersonal relationships is a key intervention, which cannot be "controlled" by an experimental scientist
* Complex webs of interrelated and reciprocal factors are not readily reducible to simple, linear models
Among these considerations, cultural context is particularly salient. Culture is a body of knowledge which includes: philosophies; belief about causes of problems and solutions; local innovation, trial and error; medicinal use of plants and minerals; healing procedures; oral transmission of knowledge; and community evaluation and acceptance of practices and programs. Understanding the ways of knowing underlying culture-based practices such as traditional healing, ceremony, storytelling and canoe journey, will further strengthen culture-based programs in Indian communities. Specifically, this deeper understanding would enable:
* Facilitation and development of new culture-based programs based on explicit understanding of culturally appropriate health practice principles and technologies
* Validation of proposed culture-based programs using culturally appropriate criteria and proofs
* Development of culturally appropriate observation, measurement, and tests
* A cross-walk to scientific evidence-based and service-based practices, with an understanding of the basic principles and technologies of each of the three ways of knowing
* Dissemination of culture-based programs, aided by specific, detailed description
* Improvement of culture-based programs by having detailed description, observable and testable items, and an understanding of underlying program theory
* Empowering Indigenous knowledge by creating an improved and expanded understanding of and a greater appreciation for culture-based programs both within and beyond Indigenous communities
THE TBP MODEL
Making TBP "Evaluable"
To be on equal footing with evidence-based model programs, culture-based programs must ultimately be scientifically evaluated. But first they must be "evaluable." To be "evaluable," means that program effectiveness can be tested because of explicit logic and identified variables, together with management commitment to use that information (Wholey et al. 2004). Programs have to be made evaluable to establish their status as EBPs. Using a fundamentally scholarly approach, academic program evaluators developed such notions as "theory-based" evaluation. In this model, program theories such as the theory of change, theory of action, or other logical system leads to a program's outcomes thus making the program evaluable. This process is a challenge to ways of knowing in which tradition and authority are important, but evaluability is a challenge to which the TPB initiative was determined to rise.
TBP within a Logic Model
Whether presented as a graphic, a matrix, a series of columns, or in text format, a logic model is just a number of boxes, sometimes with connectors, containing certain bits of information about a program. The TBP logic model is presented to tribes as a tillable form in a Word document, together with instructions and suggestions. There are 13 items on the form, each with one or more tillable boxes. Below is the list of items.
2. Brief description
3. Replications (other examples)
4. Culture-based evidence
6. Target population
7. Risk and protective factors
8. Tribal personnel
11. Optional elements
13. Contact persons
The TBP Logic Model: Personnel
The personnel providing prevention and treatment services are crucial to Indian Country, as everywhere else. The model programs mandate tends to focus on activities and materials as the "active ingredients" which tends to overshadow the arguably greater importance of the personnel. Enthusiasm, confidence, caring, compassion, vision for health, and health-related knowledge are characteristics of personnel that contribute to the outcomes of health prevention and treatment: interpersonal skills and attitudes and even charisma of the personnel are important ingredients of their therapeutic effectiveness--whether the practice is exorcism or internal medicine. Perceived success in dealing with problems leads to credibility (i.e., community acceptance and reliance on a practitioner). In some Indian communities, status as a practitioner is conferred by a community as the result of unique traits of an individual; and in some Indian communities, personnel capacity and credentialing are conferred by peers as a result of structured training and apprenticeship.
The TBP Logic Model: Activities
Specific, observable, and measurable culture-based prevention and treatment modalities or activities used by Indigenous peoples in Indigenous communities include:
* Guided individual experience, observation, and listening that leads to healthy attitudes, choices, and lifestyle
* Participating in culturally prescribed traditions, ceremonies, daily observances or rituals, which strengthens healthy connectedness
* Sharing stories, songs, and artwork that convey health-promoting lessons
* Oral instruction, modeling, and guided practice by respected healers emphasizes and entrenches lessons
* Values, principles, and lessons are embedded in and conveyed by everyday language, worldview, teachings, experience, thus locking in the rules and guidelines for healthy living
* Fun is an indispensible and efficacious aspect of activities for any youth program
The TBP Logic Model: Materials
Specific, observable, and measurable materials used in culture-based programs include:
* Settings: e.g., Isolated and challenging settings for Vision Quest; big water settings for Canoe Journey; and culturally significant buildings such as kivas, long houses, and sweat lodges
* Dress and Ornamentation: Regalia to indicate and emphasize messaging and meaning; masks to suggest unseen spirits; and ornaments to express esteem and confidence
* Symbols, Graphics, Pictures: e.g., Medicine Wheel, pictographs, and eagle feathers
* Sensory Substances: Sweet grass smoke and steam of sweat lodge ceremonies
* Food is a powerful material for attracting participation, conveying a sense of caring, and signaling community role and status, which is particularly efficacious in prevention and community mobilization programs
The TBP Logic Model: Immediate (Proximal) Outcomes
Indian Country prevention and treatment practices include activities, materials and personnel that have immediate (proximal) outcomes, as stated in the program's theory. Changes occur in the individual and in the social group, even before we get to distal changes which are of principle concern to program funders. These outcomes include:
* Restoring balance within the individual' s life and the community
* Identifying and strengthening the connections of individual to family, culture, community, natural world, and spiritual world
* Establishing the identity of the individual, and the community
* Creating meaning for individual and group life
* Creating a sense of power and hope
* Establishing self-esteem
* Fostering healthy community norms, adaptive capacity ("community competence") and sharing and mutual support ("social capital")
Much of the cause-effect knowledge underlying this descriptive material is supported by the accumulated body of scientific research. All of the proximal outcomes of these culturally-based program interventions are testable hypotheses because of specific, observable, measurable items in the TBP logic model together with explicit program theory.
TBP Initiative: Processes
The process leading to Tribal Best Practices (TBP) with its logic diagram, detailed program information, evaluability, and evaluations did not form magically. This TBP approach requires major accommodation of their ways of thinking by many in Indian Country. The TBP processes evolved from an initial stand-off scenario, through several phases, and it still has a ways to go before we get to the status of such evidence-based cultural programs as Project Venture, Zuni Life Skills, and Canoe Journey (NREPP 2011).
Oregon TBP: First Phase
When the state's legislative mandate for model programs came down, tribes adamantly maintained they knew what worked in their communities, based on implicit cultural and practice/service knowledge. A major step forward was made in the creation of the TBP initiative, which identified criteria for cultural appropriateness, required tribes to document their programs, and established a peer review panel to certify the programs as TBP based on those criteria (Oregon Addictions and Mental Health Division 2007; Cruz ND). The criteria of cultural appropriateness identified for evaluating an applicant TBP in a way analogous to the use of criteria in the larger Oregon "model programs" implementation were (Item 4 of the TBP Approval Form):
* Longevity of the practice/program
* Teachings which the practice/program is based on
* Values which are incorporated in the practice/ program
* Principles which are incorporated
* Elder's approval
* Community feedback/evaluation/acceptance
Oregon TBP: Second Phase
The next major step forward was acceptance by the peer review panel and tribal stakeholders of a logic model and explicit, detailed, observable, measurable information for boxes in the logic model for proposed TBP, together with a TBP form (a text style logic model) for eliciting that information, and a commitment to the standard of evaluability (Oregon Addictions and Mental Health Division 2010). The TBP form was used by tribal representatives, with coaching from panel members, to create a number of TBPs which are now being reviewed by the TBP panel. One program has had a pilot study and is now getting ready for a formal evaluation under a SAMHSA Collaborative for the Application of Prevention Technologies (CAPT) Service-to-Science grant.
Tribal Best Practices under Review
More than a dozen TBP programs have already been reviewed and approved, including such well-known protocols as local versions of: Native American Therapeutic Horse Program; Canoe Journey; Talking Circle; and Culture Camp. The following are currently under review: Storytelling; Basketball Against Alcohol and Drugs (BAAD); Elder-based Family Mediation; Cultural Sobriety Recognition Dinner; Powwow. To give a flavor of the kinds of TBP being established, here are some brief descriptions:
Horse Program. In partnership with horses, tribal youth, and families, this program improves attitudes, behavior, mood management, sense of responsibility, communication and relationship skills; regular individually mentored and small group sessions include equine care, ground work, and riding training sessions. Activities needed to get a horse program up and running are: buy-in; referrals; and a programmatic home. Powering this program is the enthusiasm of the people who care for the horses, in this case, rescued mustangs. "Teaching horsemanship" is the key set of activities: Horses are acutely sensitive to nonverbal communication; they provide immediate, obvious, honest feedback, and, therefore, are a teaching tool in a critically important area for persons with interpersonal difficulties. There are a series of modules encompassed by "teaching horsemanship": communication, including use of nonverbal cues; foot-care, including inspiring trust; achieving desired outcomes by changing approaches; mastering the physical aspects of riding; learning to care about a living being. Finally, "ceremonies" provide recognition and announce a commitment to a changed lifestyle.
Storytelling. Storytelling disseminates information related to prevention and treatment of unhealthy life conditions. Learning and retelling stories is used for reconfiguration of an individual or group life condition through the metaphysical meanings within traditional and personal storytelling. Storytelling is a way of learning and applying ideas about things that have happened or may happen. Storytelling uses the suspension of disbelief, an innate and fundamental way of teaching and learning. Storytelling is also part of a "cultural renaissance" in which culturally important stories, and the people who have that knowledge, are rediscovered and promoted. Story telling can become entry-level story reading in Home Visiting and other early development programs.
Sobriety-Recovery Recognition Dinner. This is a multigenerational community gathering to recognize and celebrate sobriety and recovery, a cultural activity supporting movement towards community wellness. Community members have an opportunity to speak and represent healthy role models in recovery and sobriety. Storytelling is also included as an important aspect of sharing history and culture. Community members and representatives from different community agencies have an opportunity to offer support to one another. Community resources and services can be shared to help community members feel that those agencies are an important part of the community. Community members cook for each other and include spiritual healing and supportive thoughts in the meal preparation. It is an opportunity to bridge the generations with the importance of sobriety and recovery.
Basketball Against Alcohol & Drugs (BAAD). This is a community-based basketball tournament that attracts tribal teams from the Pacific Northwest including: Oregon, Washington, and Idaho. Age-appropriate prevention education in alcohol, drug, tobacco, suicide, and other prevention subject areas is an important component of the tournament. Fun is an effective ingredient of this youth-oriented program.
LESSONS LEARNED ABOUT ADVANCING CULTURE-BASED PRACTICES
Some achievement toward consistency with the evidence-based programming philosophy, and incidentally, the SAMHSA Service-to-Science program, has been accomplished through the TBP project. The logic model approach with the TBP application form and accompanying suggestions and instructions has been incorporated. Tribal understanding of evaluability with goals, activities, and outcomes stated in specific, observable, and measurable terms has increased, as well as increased commitment to evaluation and improvement of services through the ownership and involvement of tribes in conceptualizing their Tribal Best Practices in terms that are both culturally appropriate and evaluable.
The approach used in the TBP initiative is labeled evidence-informed to distinguish it from the evidence-based model programs approach. Scientific evidence and its application are much more complicated, and subject to many more contingencies, than government policy-makers, model program list-makers, and advocates appreciate. Typically, model program advocates argue that "fidelity" to the original protocol eliminates anomalous outcomes. However, attempting to ignore the contingencies of local context and culture by insisting on "fidelity testing" is an ineffective approach to program improvement.
Because the randomized controlled/clinical trial (RCT) is imperfectly designed and imperfectly implemented, and because any sampling of personnel, subjects, and contextual conditions may be less than random, and because RCTs are inherently more or less unrepresentative of actual field conditions, therefore, no one or two or three RCTs yield sufficient evidence upon which one can depend to draw conclusions about what "works" for everyone, everywhere, and under all conditions. Therefore, studies must be replicated with different populations and different conditions to increase the generalizability of conclusions. Furthermore, results of those studies must be reviewed for "quality of evidence" based on methodological strength, and then combined by meta-analysis to entitle their recommendation as evidence-based programs by the Cochrane Commission and similar bodies. After such review, many programs that had been listed as "model programs" look less solidly "evidence-based."
Notwithstanding this reason for caution, many of the listed model programs vigorously promote themselves as "evidence-based, scientifically proven" on their websites and enthusiastically participate in imposition of the model programs mandate. Further, the purpose of the evidence-based program movement--to make programs safer, more beneficial and to deliver better value--is expected by some to deliver more than it can. The newly emerging "science of dissemination and implementation" seems to indicate that the impact of disseminated evidence-based programs is actually much less than one would expect--sometimes 10% or less; even negative impacts have been observed (Grimshaw & Eccles 2004; Oxman et al. 1995).
Evidence-informed, on the other hand, means incorporating efficacy, effectiveness, basic research, facts, principles, theory, and local and cultural context, as well as stakeholder preferences, into programming. The evidence-informed approach takes account of significant variation in the replication of research protocols, program implementation, and context. The Tribal Best Practices initiative is an example of how to meld the best of evidence-informed with culture-based practices to identify and improve best practices in Indian Country.
Bachrach, LL. 1986. An overview: Model programs for the schizophrenic patient. In: L.L. Bachrach (Ed). Leona Bachrach Speaks. New Directions for Mental Health Services #35. San Francisco, CA: Jossey-Bass.
Campbell, DT. 1979. Assessing the impact of planned social change. Evaluation and Program Planning 2: 67-90.
Center for the Study and Prevention of Violence. 2011. Blueprints for Violence Prevention. Available at: http://www.colorado.edu/cspv/ blueprints/.
Chandler, M. & Lalonde, C. 2004. Transferring whose knowledge? Exchanging whose best practices? On knowing about Indigenous knowledge and Aboriginal suicide. In: J. White: R Maxim & D. Beavon (Eds.) Aboriginal Policy Research: Setting the Agenda for Change, Vol. H. Toronto: Thompson Educational Publishing.
Cruz, C. No date. Tribal Best Practices. There are Many Pathways. Available at: http://www.oregon.gov/OHA/mentalhealth/ebp/nativeamerican/presentation.pdf
Donaldson, S.I.; Christie, C.A. & Mark, M.M. (Eds.) 2009. What Counts as Credible Evidence in Applied Research and Evaluation Practice ? Thousand Oaks, CA: SAGE Publications.
Etizioni, A. 1968. Organizational dimensions and their interrelationships: A theory of compliance. In: B.E Indik & F. K. Berrien (Eds.) People, Groups, and Organizations. New York: Teachers College.
Grimshaw, J.M. & Eccles, M.R 2004. Is evidence-based implementation of evidence-based care possible? Medical Journal of Australia 180 (Supplement): 80-81.
National Registry of Evidence-Based Programs and Practices (NREPP). 2011. Available at: http://www.nrepp.samhsa.gov/
Oregon Addictions & Mental Health Division. 2011. Evidence-Based Practices (EPB). Available at: http://www.oregon.gov/OHA/mental health/ebp/main.shtml
Oregon Addictions & Mental Health Division. 2010. EvidenceBased Programs Tribal Practice Approval Form, Mk V. Available at: http://www.oregon.gov/OHA/mentalhealth/ebp/main. shtml
Oregon Addictions and Mental Health Division. 2007. Position Paper on Native American Treatment Programs and Evidence-Based Practices. Available at: http://www.oregon.gov/OHA/mentalhealth/ ebp/native-american-trtmntpro-ebp.pdf
Oregon Revised Statutes. No date. Chapter 669. Available at: http://www. leg.state.or.us/03orlaws/sess0600.dir/0669ses.htm
Oxman, A.D.; Thomson, M.A.; Davis, D.A. & Haynes, R.B. 1995. No magic bullets: A systematic review of 102 trials of interventions to improve professional practice. Canadian Medical Association Journal 153 (10): 1423-31.
Pawson, R. & Tilley, N. 1997. Realistic. Evaluation. Twin Oaks, CA: Sage.
Suchman, E.A. 1967. Evaluative Research. New York: Russell Sage Foundation.
Wholey, J.S.; Harry, H.R & Newcomer, K.E. 2004. Handbook of Practical Evaluation. San Francisco: John Wiley & Sons.
R. Dale Walker, M.D. (a) & Douglas A. Bigelow, Ph.D. (b)
The authors wish to acknowledge the leadership of the Oregon Tribal Best Practices Initiative, Caroline Cruz, Founder, John Spence, Ph.D., and representatives of the Tribes of Oregon.
(a) Professor of Psychiatry, Oregon Health & Science University, Portland, OR; Director, One Sky Center. (b) professor Emeritus, Oregon Health & Science University, Portland, OR.
Please address correspondence to Douglas A. Bigelow, Ph.D., Professor Emeritus, One Sky Center, Oregon Health & Science University, Gaines Hall 155, 3181 SW Sam Jackson Park Road, Portland, OR 97239, email: firstname.lastname@example.org.
|Gale Copyright:||Copyright 2011 Gale, Cengage Learning. All rights reserved.|