Analysis of User Messages to MedlinePlus.gov*
Miller, Naomi, Journal of the Medical Library Association
MedlinePlus, the National Library of Medicine's (NLM) consumer health web site, debuted in October, 1998. Providing information to consumers via a web site was a new direction for NLM, and MedlinePlus staff have continuously monitored email feedback from users to measure its effectiveness and user satisfaction . When users select the "Contact Us" link from any page on MedlinePlus, they access a feedback form. Forms feed into the Siebel customer relations management software program . Each message receives a unique identifying "ticket" number and flows into a queue for triage.
Feedback can be anonymous; if the user desires a reply, he or she must supply an email address. All feedback with a return email address receives an automatic response saying that NLM has received the communication and that a response will come within four working days, if one is required. NLM customer service staff respond to messages as needed, using a knowledge base that includes hundreds of standardized answers to frequently asked questions. They assign tickets about MedlinePlus to a group of MedlinePlus staff, who reviews the responses and may provide additional responses to users if required.
This paper examines the feedback messages that NLM receives about MedlinePlus to explore what users tell NLM about the site and to compare their feedback with observations from published studies of health web site users.
All messages (N = 3,061) originating from the "Contact Us" link on MedlinePlus pages during March and July of 2005 were examined. These months were chosen because historically they have had high (March) and low (July) usage as determined by Internet log files. The messages generally consisted of two subsets:
1. General reference questions about diseases, conditions, or treatment, or how to use MedlinePlus to find information about them.
2. Compliments, complaints, questions, or suggestions relating directly to the content of specific MedlinePlus pages.
Table 1 shows the number of reference questions submitted through MedlinePlus. Given the large number of reference questions, a random sample for each month was analyzed, with sample numbers generated by an online randomization program . Each reference question received one of the following investigator-assigned categorization labels: directory assistance, general medical reference, purchase drugs or financial assistance, question about current condition requiring professional advice, reference question or drug/laboratory test, request for collaboration, and request for materials or homework assistance. Table 2 (online only) lists examples of each. Each comment received one of the following investigator-assigned categorization labels: collaboration, complaint about content, compliment, request for materials or license, link to me, correction or suggestion for improvement, technical or access, and other. The categories were defined prior to coding. Table 3 (online only) lists examples of each.
As noted, messages broadly fell into two types: general reference questions and comments about MedlinePlus itself. There were 3,061 reference questions submitted to MedlinePlus during the study period. After randomization and excluding duplicate messages, mis-categorizations by the customer service staff and unintelligible communications, there were 261 messages appropriate for analysis (Table 1). With regards to the comments about MedlinePlus, after excluding duplicate messages, mis-categorizations by customer service staff, and uninterpretable communications, there were 284 queries in March and 207 in July for a total of 491 messages appropriate for analysis (Table 4; online only).
Figure 1 (online only) summarizes the content of the sample of the reference questions originating from MedlinePlus for both months. Over one-third (103/261, 39%) of reference questions originating from MedlinePlus concerned the current condition of the writer, family member or friend. …