Improving follow-up of abnormal cancer screens using electronic health records: trust but verify test result communication
© Singh et al; licensee BioMed Central Ltd. 2009
Received: 11 August 2009
Accepted: 9 December 2009
Published: 9 December 2009
Early detection of colorectal cancer through timely follow-up of positive Fecal Occult Blood Tests (FOBTs) remains a challenge. In our previous work, we found 40% of positive FOBT results eligible for colonoscopy had no documented response by a treating clinician at two weeks despite procedures for electronic result notification. We determined if technical and/or workflow-related aspects of automated communication in the electronic health record could lead to the lack of response.
Using both qualitative and quantitative methods, we evaluated positive FOBT communication in the electronic health record of a large, urban facility between May 2008 and March 2009. We identified the source of test result communication breakdown, and developed an intervention to fix the problem. Explicit medical record reviews measured timely follow-up (defined as response within 30 days of positive FOBT) pre- and post-intervention.
Data from 11 interviews and tracking information from 490 FOBT alerts revealed that the software intended to alert primary care practitioners (PCPs) of positive FOBT results was not configured correctly and over a third of positive FOBTs were not transmitted to PCPs. Upon correction of the technical problem, lack of timely follow-up decreased immediately from 29.9% to 5.4% (p < 0.01) and was sustained at month 4 following the intervention.
Electronic communication of positive FOBT results should be monitored to avoid limiting colorectal cancer screening benefits. Robust quality assurance and oversight systems are needed to achieve this. Our methods may be useful for others seeking to improve follow-up of FOBTs in their systems.
Fewer than 75% of patients with abnormal cancer screening examinations receive follow-up diagnostic care subsequent to the initial screening [1–5]. This inadequate follow-up of abnormal cancer screens compromises the benefits of population-based screening programs [6–9]. For instance, the rate of follow-up for positive fecal occult blood test (FOBT) results in the Veterans Affairs health care system is low; more than 40% of veterans with positive FOBTs may not be receiving timely diagnostic colonoscopies [10, 11]. Lack of timely follow-up has also been documented outside the VA system [12, 13].
An important, largely preventable but relatively unexplored reason for lack of follow-up is a problem in communication of the positive test result from the laboratory to the clinician who ordered it [14, 15]. The use of electronic health records, especially those that utilize such features as automated communication of abnormal results from laboratories to clinicians, can potentially improve follow-up of abnormal cancer screens [16–19]. Electronically "alerting" the ordering provider about an abnormal test result such as positive FOBT can improve the availability of vital information at the point of care . As one of several multifaceted interventions to improve follow-up of positive FOBTs, our institution previously implemented standard operating procedures for the electronic health record's test result communication system , including the transmission of a mandatory alert to the patient's clinician for every positive FOBT result. This procedure was expected to reduce breakdowns in communication between the laboratory and clinicians.
A significant increase in timely responses to positive FOBT notifications (defined as a documented response within two weeks of the test) followed implementation of this and several related interventions. However, we found that 40% of automated notifications of FOBT results had no documented response by a treating clinician at two weeks even though all of the patients with these positive FOBT tests were eligible to receive a diagnostic colonoscopy. Our research question was to determine why a large number of FOBT alerts were not followed by clinician response at 2-weeks and to investigate if technical and workflow-related aspects of automated communication in the electronic health record were responsible. We also sought to implement and evaluate a potential solution to the issue(s) we identified.
The study was conducted at the Michael E. DeBakey Veterans Affairs Medical Center and its satellite clinics and was approved by the local institutional review board. We used a mixed methods approach analogous to root cause analysis  to uncover potential workflow or technical reasons for lack of clinician response to positive FOBT results. We conducted eleven semi-structured interviews with key informants from the laboratory, primary care, and information technology sections to gather details related to FOBT alert generation, transmission, and receipt. Concurrently, we obtained quantitative data to track the alert receipt and follow-up actions by providers.
Clinicians in the VA health care system receive notifications of high-priority information such as abnormal test results in a "View Alert" window of the electronic health record. To understand the technical issues surrounding electronic communication, we analyzed and mapped the associated system-level processes involved. We discovered that the FOBT alert communication system is driven by an underlying component of the electronic health record that continually monitors test order and result entry. Alerts are automatically generated and recipients selected based on a set of predefined rules and parameters. For instance, entry of a test result such as positive FOBT (which was pre-determined to be a high-priority test result) will generate an automated notification to one or more clinicians. The proper recipients for this notification are chosen based on the setting of certain system parameters. After delivery to recipients, alerts stay active in the clinician's inbox up to two weeks, or until acknowledged.
Using the alert tracking system of the electronic health record, we identified all positive FOBT alerts transmitted daily during our study period. Approximately three weeks after alert generation, a trained physician reviewed the electronic health record for evidence of timely FOBT follow-up using a standardized data collection form that had been pilot tested in previous work . Any documented response to the FOBT, such as colonoscopy referral, patient notification, or mention of exclusion criteria for colonoscopy, was considered timely follow-up. If no follow-up action was documented, an additional investigator confirmed the findings and called the ordering clinician (usually the primary care practitioner-PCP). If the clinician gave convincing information to support any undocumented actions, we considered this response as evidence of timely follow-up as well. We also recorded clinicians' comments and actions.
Following a trail of positive FOBTs that were found to have lack of timely follow-up, we used purposeful sampling and snowball techniques to identify our study subjects . We initially purposefully sampled three PCPs whose FOBT results were found on chart review to have not received follow-up. Information from these PCPs led to further interviews with 1 additional provider (a subspecialist) and representatives that were involved with FOBT performance (laboratory personnel) and FOBT reporting (laboratory and Information Technology personnel). Additionally, 3 institutional representatives from leadership and administration that oversee workflow related to FOBT results were also interviewed.
Data from PCP interviews and reported comments suggested that PCPs were not receiving positive FOBT alerts consistently, leading us to further investigate the processes associated with FOBT alert generation. Workflow analysis revealed that a large number of patients who are given FOBT cards never return them to the lab for processing, and therefore an order for the test (through a computerized order-entry system) is only placed upon receipt of the card by the lab. However, in the absence of a provider-generated computerized order, the ordering provider is not easily identifiable unless written on the card. Because lab technicians use a different order-entry system, it is difficult for them to identify the ordering provider (and hence the primary recipient of the alert).
Further analysis of alert generation revealed that, regardless of an identifiable ordering provider in the system, the alert management software is designed to communicate all high priority alerts to the PCP as long as a primary alert recipient is identified. We discovered that in positive FOBTs where the ordering provider was not identified, a laboratory staff member served as the designated "ordering" provider, i.e. the primary recipient for the alert. This workaround (nonstandard procedures typically used because of deficiencies in system or workflow design)  was intended to enable the completion of the order and subsequent transmission of any alert generated to the patient's PCP; a fail-safe or safety-net mechanism designed to prevent loss of FOBT follow-up.
However, additional technical analysis of the alert tracking data revealed that in all cases where the designated lab provider was alerted as the "ordering" provider, there was no concomitant alert transmission to the PCP. Thus, only the lab provider was receiving the positive FOBT results and had no knowledge of this technical problem. We categorized such alerts as designated lab provider alerts and found that lack of timely follow-up was much more prevalent in this subgroup of alerts (29.9% vs. 4.5% in non-designated lab provider alerts).
We surmised that a lack of PCP awareness (in over a third of cases with positive FOBTs) contributed substantially to the prevalence of FOBT results with no documented follow-up, and that a software configuration error was the root of the problem. Once the electronic health record determines the need to generate an alert, proper recipients are selected based on their relationship to the patient (i.e., ordering provider, PCP, etc.). We found an improper configuration of the parameter that defines these default recipients, such that the PCP was not selected as a recipient for designated lab provider alerts (i.e. when PCPs were not listed as ordering providers). However, we could not determine when and how this error occurred in the system. Nevertheless, we posited that a problem-specific fix of this incorrect software configuration would reduce the risk of loss of follow-up for these alerts. The solution to this problem, an addition of a code to link patients to their PCP for tests ordered by others, was implemented on November 28, 2008 (date of intervention).
We investigated reasons why follow-up actions on a large proportion of positive FOBT results that needed a diagnostic colonoscopy were not documented by clinicians despite the use of a system to electronically communicate positive results. In addition to order-entry workarounds in the electronic health record, we discovered that the communication system intended to alert PCPs of positive FOBT results was not configured correctly, leading to certain situations in which PCPs never received the test result. Upon correction of the software configuration error, the percentage of positive FOBT results lacking follow-up were dramatically reduced. Although the rate did not drop to zero, it was comparable to the rate of lack of timely follow-up we found for other types of non-life threatening, high-priority lab notifications in the same system . Our findings suggest that communication of cancer-related test results in the electronic health record must be monitored to avoid compromising the promise of cancer screening programs.
Of the over 800 patients each year who have positive FOBTs at our institution, about 10-15% of them are eventually diagnosed with some form of colon disease (including cancer). None of the patients in our study had any delay in cancer diagnosis or related harm. Although it is possible that follow-up may have occurred beyond our 30 day "timely response" window had we not intervened, previous work suggests that many of these findings would ultimately never be followed-up [10, 11, 19]. Thus, our seemingly small intervention could potentially have a large impact on decreasing time to referral for colonoscopy, thereby reducing the risk of a missed or delayed diagnosis of colorectal cancer, a common reason for ambulatory malpractice claims [24–26]. Previous literature has highlighted the need for system-based interventions to improve follow-up of positive cancer screens and our study is one of few that contributes to this body of knowledge .
Our findings also highlight how electronic health record use can have dramatic effects on follow-up care of patients. Electronic health records have potential to address the fragmented and discontinuous care that usually characterizes care in the outpatient setting. Critical information flow between different practitioners, settings and systems of care is essential to high quality care. Through good decision support systems, transmission of information to the right provider at the right time is within the reach of integrated electronic health records. However, as we find, electronic health record use must take into account the effect electronic communication will have on workflow and vice versa. Not doing this correctly would lead to circumstances that reduce the situational awareness of providers and perhaps other unintended adverse effects.
A limitation of our study was a lack of comparable data from other VA or non-VA facilities. However, our work illustrates how electronic test result communication systems are susceptible to errors that may limit their intended outcomes. Furthermore, it should be noted that other VA investigators [10, 11] have demonstrated high rates of lack of positive FOBT follow-up, so it is possible that this problem exists at other VA sites. We are currently investigating whether this problem exists in other VA facilities or if this was an isolated event. Additionally, in this study we did not address many other systems issues that should be considered to address follow-up of abnormal test results in addition to provider, technology and work-flow. In our work, we are now using a socio-technical model that accounts for many other systems issues beyond the responsible provider, including the role of organizations and policies and procedures to address monitoring of abnormal test results . For instance, an institutional policy that all FOBTs are ordered through computerized order entry would be another intervention to address this area. In our future work, we will propose multifaceted solutions to address the many complex issues related to abnormal test result follow-up.
Although electronic health records likely offer many benefits over paper-based systems for improving communication of abnormal cancer screening results , our findings highlight the need to account for inherent complexities of clinical practice. This complexity may introduce circumstances requiring special attention to EHR workflow to prevent loss of follow-up of important clinical information. In our setting, several workarounds of the FOBT ordering and reporting process resulted in disruption of the normal electronic health record workflow, creating a reliance on a secondary PCP notification system, which was not functioning as intended. The challenge of recognizing these complexities and their effects underscores the need for continuous monitoring of key electronic health record features that may impact safety. The work described here was a direct result of quality assurance work that is highly regarded in the VA health care system. Other institutions could use our methods to track the effectiveness of electronic communication. However, quality monitoring procedures such as used by the VA to ensure system safety must also be used to identify red flags that would lead to similar future investigations. Without the safeguards used by the VA, the problems related to test result communication may go undetected.
Health care systems should aim to achieve a high reliability for tracking delivery of abnormal cancer screening results. An example viewed as an ideal model for tracking systems is that of FedEx, which is considered to have 99.6% tracking reliability for its packages . To achieve such high tracking reliability would not only require implementation of comprehensive technology-based systems for communication, but also formal policies and procedures regarding their use . The test result communication system evaluated in our study addressed several criteria  for effective critical results reporting systems, such as computerized tracking and back-up procedures. However, to achieve tracking comparable to other industries, cancer screening programs should continuously monitor and oversee the timely delivery of positive cancer screening results to the right clinicians. For example, we recommend that cancer screening programs using electronic health record systems should develop and monitor multiple metrics of performance of automated communication processes. Failure to implement such monitoring systems could lead to sub-optimal screening success, which may otherwise be difficult if not impossible to trace.
In conclusion, we believe that electronic health records are beneficial in communicating abnormal cancer screening results to clinicians and will improve their follow-up care; however, we cannot assume that electronic communication is always working exactly as expected especially when workarounds are used. To achieve the most benefits of cancer screening programs, robust monitoring systems are necessary in electronic health record systems to ensure that abnormal cancer screening results are being delivered to the correct providers in a timely manner.
The study was supported by an NIH K23 career development award (K23CA125585) to Dr. Singh, the VA National Center of Patient Safety, and in part by the Houston VA HSR&D Center of Excellence (HFP90-020). These sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and the preparation, review, or approval of the manuscript. The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.
We would like to thank Ms. Roxy Pierce, Section of Information Technology for support of this work.
All authors had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
- Yabroff K, Washington KS, Leader A, Neilson E, Mandelblatt J: Is the Promise of Cancer-Screening Programs Being Compromised? Quality of Follow-Up Care after Abnormal Screening Results. Med Care Res Rev. 2003, 60: 294-331. 10.1177/1077558703254698.View ArticlePubMed
- Baig N, Myers RE, Turner BJ: Physician-reported reasons for limited follow-up of patients with a positive fecal occult blood test screening result. The American Journal of Gastroenterology. 2003, 98: 2078-2081. 10.1111/j.1572-0241.2003.07575.x.View ArticlePubMed
- Levin B, Hess K, Johnson C: Screening for colorectal cancer. A comparison of 3 fecal occult blood tests. Arch Intern Med. 1997, 157: 970-976. 10.1001/archinte.157.9.970.View ArticlePubMed
- Morris JB, Stellato TA, Guy BB, Gordon NH, Berger NA: A critical analysis of the largest reported mass fecal occult blood screening program in the United States. Am J Surg. 1991, 161: 101-105. 10.1016/0002-9610(91)90368-N.View ArticlePubMed
- Burack RC, Simon MS, Stano M, George J, Coombs J: Follow-up among women with an abnormal mammogram in an HMO: is it complete, timely, and efficient?. Am J Manag Care. 2000, 6: 1102-1113.PubMed
- Bastani R, Yabroff KR, Myers RE, Glenn B: Interventions to improve follow-up of abnormal findings in cancer screening. Cancer. 2004, 101: 1188-1200. 10.1002/cncr.20506.PubMed CentralView ArticlePubMed
- Mandel JS, Church TR, Bond JH: The effect of fecal occult-blood screening on the incidence of colorectal cancer. N Engl J Med. 2000, 343: 1603-1607. 10.1056/NEJM200011303432203.View ArticlePubMed
- Mandel JS, Bond JH, Church TR: Reducing mortality from colorectal cancer by screening for fecal occult blood. Minnesota Colon Cancer Control Study. N Engl J Med. 1993, 328: 1365-1371. 10.1056/NEJM199305133281901.View ArticlePubMed
- Kronborg O, Jorgensen OD, Fenger C, Rasmussen M: Randomized study of biennial screening with a faecal occult blood test: results after nine screening rounds. Scand J Gastroenterol. 2004, 39: 846-851. 10.1080/00365520410003182.View ArticlePubMed
- Etzioni D, Yano E, Rubenstein L: Measuring the Quality of Colorectal Cancer Screening: The Importance of Follow-Up. Diseases of the Colon & Rectum. 2006, 49: 1002-1010. 10.1007/s10350-006-0533-2.View Article
- Fisher DA, Jeffreys A, Coffman CJ, Fasanella K: Barriers to full colon evaluation for a positive fecal occult blood test. Cancer Epidemiol Biomarkers Prev. 2006, 15: 1232-1235. 10.1158/1055-9965.EPI-05-0916.View ArticlePubMed
- Myers RE, Hyslop T, Gerrity M: Physician Intention to Recommend Complete Diagnostic Evaluation in Colorectal Cancer Screening. Cancer Epidemiol Biomarkers Prev. 1999, 8: 587-593.PubMed
- Myers RE, Turner B, Weinberg D: Impact of a physician-oriented intervention on follow-up in colorectal cancer screening. Prev Med. 2004, 38: 375-381. 10.1016/j.ypmed.2003.11.010.View ArticlePubMed
- Jimbo M, Myers RE, Meyer B: Reasons Patients With a Positive Fecal Occult Blood Test Result Do Not Undergo Complete Diagnostic Evaluation. Ann Fam Med. 2009, 7: 11-16. 10.1370/afm.906.PubMed CentralView ArticlePubMed
- Wahls T: Diagnostic errors and abnormal diagnostic tests lost to follow-up: a source of needless waste and delay to treatment. J Ambul Care Manage. 2007, 30: 338-343.View ArticlePubMed
- Poon EG, Wang SJ, Gandhi TK, Bates DW, Kuperman GJ: Design and implementation of a comprehensive outpatient Results Manager. J Biomed Inform. 2003, 36: 80-91. 10.1016/S1532-0464(03)00061-3.View ArticlePubMed
- Singh H, Arora HS, Vij MS, Rao R, Khan M, Petersen LA: Communication outcomes of critical imaging results in a computerized notification system. J Am Med Inform Assoc. 2007, 14: 459-466. 10.1197/jamia.M2280.PubMed CentralView ArticlePubMed
- Singh H, Naik A, Rao R, Petersen L: Reducing Diagnostic Errors Through Effective Communication: Harnessing the Power of Information Technology. Journal of General Internal Medicine. 2008, 23: 489-494. 10.1007/s11606-007-0393-z.PubMed CentralView ArticlePubMed
- Singh H, Kadiyala H, Bhagwath G: Using a multifaceted approach to improve the follow-up of positive fecal occult blood test results. Am J Gastroenterol. 2009, 104: 942-952. 10.1038/ajg.2009.55.PubMed CentralView ArticlePubMed
- Bagian JP, Gosbee J, Lee CZ, Williams L, McKnight SD, Mannos DM: The Veterans Affairs root cause analysis system in action. Jt Comm J Qual Improv. 2002, 28: 531-545.PubMed
- Ash JS, Smith AC, Stavri PZ: Performing subjectivist studies in the qualitative traditions responsive to users. Evaluation Methods in Biomedical Informatics. Edited by: Friedman CP, Wyatt JC. 2006, Springer New York, 267-300. full_text. 2View Article
- Koppel R, Wetterneck T, Telles JL, Karsh BT: Workarounds to Barcode Medication Administration Systems: Their Occurrences, Causes, and Threats to Patient Safety. J Am Med Inform Assoc. 2008, 15: 408-423. 10.1197/jamia.M2616.PubMed CentralView ArticlePubMed
- Singh H, Thomas E, Petersen LA: Automated Notification of Laboratory Test Results in an Electronic Health Record: Do Any Safety Concerns Remain?. American Journal of Medicine.
- Gandhi TK, Kachalia A, Thomas EJ: Missed and delayed diagnoses in the ambulatory setting: A study of closed malpractice claims. Ann Intern Med. 2006, 145: 488-496.View ArticlePubMed
- Phillips RL, Bartholomew LA, Dovey SM, Fryer GE, Miyoshi TJ, Green LA: Learning from malpractice claims about negligent, adverse events in primary care in the United States. Qual Saf Health Care. 2004, 13: 121-126. 10.1136/qshc.2003.008029.PubMed CentralView ArticlePubMed
- Singh H, Sethi S, Raber M, Petersen LA: Errors in cancer diagnosis: current understanding and future directions. J Clin Oncol. 2007, 25: 5009-5018. 10.1200/JCO.2007.13.2142.View ArticlePubMed
- Sittig DF, Singh H: Eight Rights of Safe Electronic Health Record Use. JAMA. 2009, 302: 1111-1113. 10.1001/jama.2009.1311.View ArticlePubMed
- Bates DW, Leape LL: Doing better with critical test results. Jt Comm J Qual Patient Saf. 2005, 31: 66-67.PubMed
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6947/9/49/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.