What clinical information do doctors need?BMJ 1996; 313 doi: http://dx.doi.org/10.1136/bmj.313.7064.1062 (Published 26 October 1996) Cite this as: BMJ 1996;313:1062
- Richard Smith (), editora
Doctors use some two million pieces of information to manage patients, but little research has been done on the information needs that arise while treating patients
Textbooks, journals, and other existing information tools are not adequate for answering the questions that arise: textbooks are out of date, and “the signal to noise” ratio of journals is too low for them to be useful in daily practice
Computer systems that have been developed to help doctors are not widely used—perhaps because they have not been developed to meet doctors' information needs
When doctors see patients they usually generate at least one question; more questions arise than the doctors seem to recognise
Most of the questions concern treatment
Many of the questions are highly complex, simultaneously asking about individual patients and particular areas of medical knowledge
Often doctors are asking not simply for information but for support, guidance, affirmation, and feedback
Many of the questions go unanswered, but most could be answered; it is, however, time consuming and expensive to answer them
Doctors are most likely to seek answers to these questions from other doctors
The best information sources provide relevant, valid material that can be accessed quickly and with minimal effort
New information tools are needed: they are likely to be electronic, portable, fast, easy to use, connected to both a large valid database of medical knowledge and the patient record, and a servant of patients as well as doctors
This is the first article in a new section on managing clinical infor mation that will appear regularly in the journal. This week, Richard Smith considers the information needs of doctors. He reviews existing research and draws conclusions about what information doctors need, what they think they need, how they try to find it, and how they determine its value
The aims of this section are:
To help doctors understand that better management of clinical information will improve their treatment of patients and the management of their practices
To encourage rigorous evaluation of information management systems, particularly with respect to patient care
To empower doctors to shape the development of information management projects so that clinical needs are put before financial and administrative needs
To generate enthusiasm among doctors by demystifying clinical infor mation management
To help doctors understand the information demands that will be made of them
To consider how information management can enhance doctors' relationships with patients and the public
The central job of doctors is to meet the needs of patients by drawing on the knowledge accumulated by medicine over 5000 years. Medicine, in modern jargon, is a knowledge based business, and experienced doctors use about two million pieces of information to manage their patients.1 2 Clinical information can be defined as “the commodity used to help make patient care decisions.”3 About a third of doctors' time is spent recording and synthesising information,4 and a third of the costs of a hospital are spent on personal and professional communication.4 Yet most of the information doctors use when seeing patients they keep in their heads in what has been called “a constantly expanding and reinterpreted database.”5
Unfortunately, some of the information in doctors' heads is out of date and wrong, new information may not have penetrated, and the information may not be there to deal with patients with uncommon problems. These deficiencies have become more serious as the rate of change in medical knowledge has accelerated: the doubling time of the biomedical knowledge base is currently about 19 years,2 meaning that medical knowledge will increase fourfold during a professional lifetime. Inevitably, doctors cannot practise high quality medicine without constantly updating the information within their heads and finding information to help them with particular patients. This realisation has led to the development of medical informatics, which can be defined “as the development, use, and evaluation of information technology in health care.”4
Those interested in medical informatics have tried to develop systems that will help doctors in their daily clinical practice by providing them with information. Very few of these systems have, however, been adopted, and most doctors continue to practise without them. One reason why these systems have failed to penetrate routine practice may be that they have been designed without any close study of the information needs of doctors. Medical informatics has been dominated by concern with the technology and has developed solutions that have to search for problems. Now the time has come to understand the needs that the technology must meet, but there have been few studies of the information needs of doctors.
This paper examines studies of the needs and wants of doctors for medical knowledge; I pay less attention to the other information needs and wants of doctors. Wants and needs for information are of course different, but a good information system should provide for both. Most studies are of wants rather than needs because they are easier to identify, but some have touched on needs.
The primary focus of this review is on how to connect the rapidly expanding knowledge base of medicine with the doctor treating an individual patient. The review is aimed at ordinary doctors, not at informatics experts, but attempts to review comprehensively and critically what we know about doctors' information needs. A secondary objective is to identify where further research is required.
Categorising information needs
One of the problems with the few studies that have looked at the information needs of doctors is that they have not defined or categorised the needs.6 The studies look at different needs, giving one explanation of why they reach such different results. Gorman has proposed a classification of doctors' information needs, using work that went before and his own research among primary care doctors.6 I have extended the classification a little (see table 1). Those who want to meet the information needs of doctors should consider all these categories, but the emphasis of this review is on how doctors connect their daily practice to the rapidly changing knowledge base of medicine.
This is not a systematic review, but I have gathered papers on the information needs of doctors by searching Medline, using a database of 406 references on the information of doctors compiled by Margaret Thompson from Allegheny University of the Health Sciences Library, asking for references from people I know to be interested in the subject, and following up the references from papers I have found. A particularly useful source of information is the December 1995 issue of Journal of the American Society for Information Science, which is devoted to medical informatics.7 It includes a review by Paul Gorman, an assistant professor of medicine and medical informatics from Oregon, on the information needs of physicians.6
I identified 13 studies that collected original data on the information needs and wants of doctors. table 2 summarises the studies in chronological order.8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 Many of the studies are of only moderate quality, and all but one is American. The studies use different methods, and most are on small populations of doctors. It would not be sensible to try to combine the studies' results, but I have used all of them to try to identify what we do and do not know. I have summarised in the text studies that illustrate how the study of doctors' information needs has developed.
Survey of the information problems of American primary care physicians
Concerns about the failures of doctors to keep up with medical knowledge go back a long way. In 1983 the Massachusetts Medical Society, the publishers of the New England Journal of Medicine, became concerned with how doctors identified their needs for scientific information, how they found the information, and how they decided its value. The society commissioned research to try to answer those questions. Williamson and others surveyed a stratified random sample of 625 office based primary care physicians and 100 physician opinion leaders in the United States to identify self perceived problems in managing scientific information.13 The researchers used a telephone survey and reached 79% of the practising doctors and 90% of the opinion leaders.
Nearly two thirds of the respondents from both samples said that the current volume of scientific information was unmanageable, and more than a third said that “most physicians find the effort to get information from the literature to be a major problem.” The researchers asked about “markers” of knowledge selected by a consensus of advisory panels and found severe deficiencies: half the respondents did not know that digoxin should be withdrawn in elderly patients with uncomplicated congestive heart failure (interestingly, later papers have disputed this medical “fact”23); more than a third of all doctors (and two thirds of those in general practice) did not know that glycated haemoglobin was a good measure of diabetic control; and more than a third of obstetricians did not know that experts advised a trial of labour in patients who had undergone previous caesarean sections.
The biggest problem with this study is that it reports self perceived estimates and opinions, but doctors are perhaps unlikely to be biased to reporting the picture as even worse than it is—and it seems bad. Doctors face a serious problem in keeping up to date. They do not know about important advances, feel overwhelmed by new scientific information, are not good at finding new information, and do not know how to evaluate it when it is found. “Science information management,” conclude the authors, “is a critical professional skill that is not adequately taught in undergraduate medical education.” Too often practitioners “don't know what they don't know.”13
Information needs arising in the consultation
A questionnaire survey is a blunt instrument, and a better method for identifying doctors' information needs is to ask about them as doctors are seeing patients. This is what David Covell and others did in the most widely quoted study of information needs of doctors.11 They looked at the information needs of 47 physicians practising internal medicine in “office practice” in Los Angeles county during a half day: 12 of the physicians were generalists, and 35 were subspecialists. The physicians saw one to 16 patients during the half day. The authors identified the physicians' self perceived needs through a closed questionnaire completed before the office interviews, an interview after each patient was seen, and an interview at the end of the office visit.
The physicians answered the questionnaire by saying that they needed information about once a week, but 269 questions were raised during the interviews after 409 patient visits—about two questions for every three patients seen. Of these questions, 107 (40%) were questions of fact (“What are the side effects of bromocriptine?”), 120 (45%) were questions of medical opinion (“How do you manage a patient with labile hypertension?”), and 42 (16%) were questions of non-medical information (“How do you arrange home care for a patient?”). About a third of the questions were about treatment of specific conditions, a quarter about diagnosis, and 14% about drugs. For the subspecialists, about a third of their questions related to their own specialties and about two thirds to other subspecialties.
One important finding was that the questions were asked in a “non-generalised but practice related fashion” that would make it hard to find the answers. Thus doctors would ask “Should I test the serum procainamide level in this patient?” rather than “What are the indications for measuring serum procainamide?”
table 3 shows the sources of information that the doctors claimed they used on the questionnaire and those that they reported using when interviewed immediately after the patients' visits. The doctors believed that they used print sources a lot, but in fact they were most likely to consult other doctors. Most of the doctors thought that they could find answers to their questions, more often from books than from journals. Although the doctors thought that they could answer their questions, only a third were answered while the interviewer was present. Each doctor had an average—from one half day—of four questions that were not answered immediately. Lack of time, cost, poor organisation and non-availability of sources, and “a glut of sources of differing reliability” were seen as the barriers to finding information.11
This study generally produces a bleak picture: all but one of the 47 doctors had a question relating to a patient during the half day, and on average doctors had four; most of the questions were not answered, but the doctors thought that they were able to answer their questions. “In a typical half day of office practice,” conclude Covell et al, “four management decisions might have been altered if needed information had been available at the time of the patient visit.”11 We do not, of course, know how important the questions were or whether the patients would have done any better if these questions had been answered. The interviewers may also have prompted doctors to ask questions that they would not normally have asked.
Observing doctors' information needs
The next methodological step was to observe doctors' information needs directly. Jerry Osheroff and colleagues studied 24 doctors and medical students in a university based general medical service in Pittsburgh.16 An anthropologist, Diana Forsythe, observed the needs of the doctors and students, and internal medicine physicians then identified information requests by reviewing texts prepared from field notes. Generalisability is a big problem with this study, as many of the questions may have been generated because teaching was as important as patient care. Forsythe identified 519 information requests during 17 hours of observing inpatient and outpatient activity. During this time the 24 doctors and students cared for about 90 patients. A total of 454 information requests were “strictly clinical,” an average of five for each patient. Three quarters of the questions were related to patient care, and three fifths were about specific patients. A quarter of the questions were about treatment, and 16% were specifically about drug treatment.
The doctors conducting the study anticipated that 42% of the questions would be answered by patients' records, 39% by a hospital information system, and a quarter by the library. About a quarter of the 337 questions asked about patient care that the doctors anticipated could be answered via a library, a textbook, a journal, or Medline. A quarter of the questions required a synthesis of patient data and medical knowledge—for example, “Why do you think this patient has had three deep vein thromboses?” The researchers did not investigate how many of the questions could actually be answered.
A second paper from the same study—this time on 35 hours of observation—produced a more complicated picture of doctors' information needs.17 Firstly, Forsythe and others observed that many information needs were not expressed as grammatical questions: indeed, they might not even be verbalised. Secondly, information needs might be hard to identify, and “the information seeking messages” might be interpretable only within the particular context. Thirdly, the needs might be for much more than specific clinical information. Doctors and students might be asking for support, guidance, and approval of what they are doing. Thus, one doctor asked “What do you do for the treatment of breast cancer?” about a patient who had exhausted all known treatments. The doctor was asking how she as a doctor could cope with being able to offer no further treatment to a woman dying of breast cancer. Such “information needs” are never likely to be met by a computer—or by books or journals—and may be one explanation why doctors tend to turn first to colleagues for information.
The Oregon studies: when are questions pursued and can they be answered?
Gorman and others have extended the work on doctors' information needs by identifying which questions doctors pursue and why and by measuring the amount of effort needed to answer the questions.6 19 20 They used methods modified from those of Covell et al to study the information needs of 50 non-academic “physicians providing ambulatory care in a primary care specialty” in Oregon (25 rural and 25 non-rural doctors). Participation and response rates were disappointingly low.
The doctors were interviewed immediately after each patient for half a day of practice and asked “Do you have any questions about the diagnosis or management of this patient's problems?” The researchers asked doctors to state “all the questions which occur to you during patient care, no matter whether you would pursue them or not, nor what source you might consult for an answer.” At the end of the half day doctors were asked about 12 factors that might motivate them to seek answers to the questions (see table 4), and two to five days after the interview they were phoned to ask if they had pursued the questions. In addition, seven medical librarians conducted online searches to try to answer a random sample of 60 of the questions. The librarians then selected the “best” evidence they had found (such as meta-analyses and randomised controlled trials) and sent it back to the doctors, who were asked about the usefulness and relevance of the material. Sadly, there was a 6–12 month delay.
Examples of unanswered questions of primary care doctors (from Gorman6)
“How best to sort out diabetic neuropathy, vascular pain, discogenic pain, and musculoskeletal causes of back and leg pain in a 70 year old woman (without testing for all of them)?”
“For diagnosis of deep vein thrombosis, how good is ultrasound? Does it obviate the need for a venogram? Can it exclude the diagnosis?”
“Is there a connection between asthma and migraines?”
“Can isosorbide dinitrate and/or Lopresor be responsible for itching and rash?”
“In an 88 year old woman with dysphagia due to past laryngeal cancer, now in respiratory failure due to aspiration, what is the physician's role in aggressiveness of care decisions when the patient's family has unrealistic expectations?”
“In an 60 year old woman with an elevated sed rate, guiaic-positive stool attributed to haemorrhoids, and 35 pound weight loss, what work up is appropriate, taking cost and potential benefit into account?”
Complete data were gathered for 49 doctors, who had cared for 514 patients and asked 295 questions about diagnosis or management. The doctors had pursued less than a third of the questions and found answers for less than a quarter. A multiple logistic regression model showed that, of the 12 factors that might prompt doctors to pursue questions, only two were statistically significant predictors—the doctor's belief that a definitive answer existed and the urgency of the patient's problem. Doctors most commonly used colleagues to answer the questions. The librarians succeeded in answering most questions, but the mean cost for each question was $27 and the mean time taken was 43 minutes. The doctors thought that the information retrieved was would have had an impact on the patient in 40% of cases and on them in about half.
The limitations of this study are that it was small, had low participation and response rates, used volunteers, and depended on self reporting. Nevertheless, it shows yet again that many questions are generated during consultations, many remain unanswered that could be answered, the answers that are generated seem relevant and important for patient care, and the ones most likely to be answered are those which the doctors think they will find an answer to and when the patient's needs are urgent. The unanswered questions are an important missed opportunity to educate doctors and improve medical practice. But, conclude Gorman and Helfand, “reducing the cost of using medical information systems is, by itself, insufficient to increase their use by practising physicians. New medical information systems, no matter how fast, inexpensive, and easy to use, will not be used more widely until it has been demonstrated to practitioners that these systems provide answers that help solve the problems of patient care.”20
AIDS: information needs when knowledge is growing fast
The last study that I have summarised in full concerns the information needs of doctors (and other health professionals) treating patients infected with HIV. I included this study because it is one of the most recently published; it uses yet another method; it finds which questions can be answered; and it concerns a part of medicine where information is growing faster than in almost any other, where electronic databases are growing fast, and where some of the patients are as smart as the doctors—and possibly more adept at using modern information technology. The body of information on HIV doubles every 22 months,24 and, although half of that information is concentrated in 30 journals, the other half is spread through 593.25
Guise and others asked seven health professionals to look through the medical records of 10 patients who had died of AIDS to “Identify questions (for example, points you would like more information about) that you think could be answered through information sources in printed or electronic form, such as journal articles, medical textbooks, printed abstracts, treatment protocols, etc.”22 The health professionals collected their questions, and then two of the paper's authors trained in library and information science tried to answer them. In the first phase of their search they searched standard textbooks and then four electronic databases. In the second phase they tried to answer questions that remained unanswered by reading printed material specifically related to AIDS and conducting additional online searches. The authors stopped their search as soon as they found an answer.
The 10 medical records covered 120 patient encounters. The health professionals identified 266 questions, an average of 38 each or 3.8 per patient. A quarter of the questions were about treatment. The authors were able to answer three quarters of the questions in phase one in an average time of 10 minutes and 18% in phase two in an average time of 22 minutes; 8% of questions could not be answered. Of the questions answered in phase one, 87% were answered from online sources.
Again this was a small study, and the questions were not generated by actual consultations of health professionals with patients. The health professionals were not invited to judge the correctness or the usefulness of the “answers,” and the results cannot be generalised to other health care settings. However, the results show—as in most of the other studies—that large numbers of questions are generated by caring for this set of patients. Most of the questions can be answered, but it is time consuming and expensive to do so. Something better than present information systems is needed.
What do we know about doctors' information needs?
Despite the many deficiencies of these studies of the information needs, I feel able to draw some tentative conclusions.
Firstly, all the studies show that information needs do arise regularly when doctors see patients. The doctors are generally aware of the needs themselves, but questioning the doctors immediately after they have seen patients shows that these needs are greater than doctors admit to in general surveys. Observing the doctors suggests that the needs are still greater. Only one of the studies considered information needs that the doctors themselves did not identify,16 17 and it found a much higher rate of questions for each patient encounter than any other study. There is an urgent need to repeat a study along these lines in other settings. I think that it is conservative to conclude that every interaction between a patient and a doctor is likely on average to generate at least one question. The real need is likely to be higher.
Studies of information needs that still need to be done
Studies that use an expanded view of the type of information need—to include the psychological component
Studies that use common definitions so as to allow better comparisons
Studies of what motivates doctors to seek to answer questions
Intervention studies that try to meet the information needs of doctors
Studies that use patient outcome as the endpoint
Studies in different contexts and health systems, including the British NHS
Secondly, these questions are most likely to be about treatment, and many of these are likely to be about drugs.
Thirdly, the questions are often complex and multidimensional. They are often questions about both particular patients and different areas of medical knowledge—for example, “In an octogenarian with anaemia, angina, and a history of transient ischaemic attacks, with a normal creatinine, iron, and mean corpuscular volume, who refuses a bone marrow exam, what diagnostic and therapeutic options are there?”
Fourthly, the need for information is often much more than a question about medical knowledge. Doctors are looking for guidance, psychological support, affirmation, commiseration, sympathy, judgment, and feedback. This “information need” is particularly poorly explored, and yet it may well be the most important need and the biggest stumbling block to a technical solution.
Fifthly, most of the questions generated in consultations go unanswered. We do not know from any of these studies whether answering the questions would lead to better patient outcomes or better doctors, but surely it would. We do know that many surveys of how much doctors know about important developments show severe deficiencies.2 13
Sixthly, doctors are most likely to seek answers to their questions from other doctors. Weinberg et al found that 81 American doctors in a discrete geographical area consulted 23 experts, logging 11 calls to experts each month. Six experts received over 90% of the calls, about 90 each a month.26 Doctors consult experts presumably because it is a quick, cheap, and easy method and because other doctors can also provide the psychological benefits that are not available from books, journals, and computers. One problem is that the doctors “answering” the questions may sometimes not be much more knowledgeable than the doctors seeking answers.
Seventhly, most of the questions generated by doctors can be answered, usually from electronic sources, but it is time consuming and expensive to do so—and demands information skills that many doctors do not have.
Eighthly, doctors in developed countries seem to be overwhelmed by the information provided for them. The amount of information is enormous and disorganised, and it is hard to find the answers to questions that arise in consultations.
Providing for doctors' information needs
I have not attempted in this paper to provide an answer to doctors' information needs, but I want to make a few observations about existing and future methods of supplying information. The “answer” to the problem can be considered by using a formula described by Shaughnessy et al.27
Usefulness of medical information = relevance × validity/work to access
The relevance of any information is based on the frequency of your exposure to the problem being addressed and the type of evidence being presented. Validity is the likelihood of the information being true, and the work to access the information is the time and effort that must be spent extracting the information. The ideal information source will be directly relevant, contain valid information, and be accessed with a minimal amount of work. table 4 shows my judgment of how existing information sources score against these criteria. Seven sources score highly: colleagues and drug reference books, both of which are widely used; journals like ACP Journal Club or Evidence-Based Medicine, which scan medical journals and select abstracts of those that are clinically relevant and scientifically sound; evidence based, regularly updated textbooks, which are not yet available; systematic journal reviews, which are only just becoming available; a portable and comprehensive summary of systematic reviews, which is planned but does not yet exist; and the Internet in 10 years' time, which is pure speculation.
The deficiencies in current information sources make it clear that something more is required, and new sources are beginning to emerge. Nobody knows the form of the information tool that may transform medicine, but we can begin to discern some of its characteristics. The tool must be able to answer highly complex questions and so will have to be connected to a large valid database. Inevitably, it will be electronic, but it must be portable, fast, and easy to use. Almost certainly the tool will have to prompt doctors rather than simply answer questions, but it will have to do so in ways that doctors find helpful rather than demeaning. It will probably be connected to the patient record, and perhaps it will be as much a servant of patients as doctors, possibly changing fundamentally the doctorpatient relationship. A successful information tool will also meet the information needs of doctors that go beyond the need for answers to questions and begin to be a need for psychological support and affirmation.
“The big challenge for the next decade,” writes Nicholas Negroponte, guru of the information age, “is to make computers that know you, learn about your needs, and understand verbal and non-verbal languages…[We need computers that] exhibit intelligence to such a degree that the physical interface almost goes away. Therein lies the secret to interface design: make it go away.”28 The doctor's information tool of the future might be some sort of combination between the patient record and the Internet, with the doctor and the patient positioned together at the intersection but not having to pay attention to the technology. Probably there will be no single tool but a family of tools, and I suspect that, whatever sophisticated tools may be developed, the major source of information will remain colleagues.
I thank Brian Haynes, Jerry Osheroff, John Roberts, Jane Smith, Alison Tonks, and Jeremy Wyatt for reading this paper and making many suggestions for improvement. I also thank Margaret Thompson for supplying references.