Eric van Sonderen, Robbert Sanderman and I recently published a paper in PLOS One, “Ineffectiveness of Reverse Wording of Questionnaire Items: Let’s Learn from Cows in the Rain” that is available open access here.
Basically we pursued Eric’s hunch that the almost universal practice of reverse wording of items in constructing psychological and marketing questionnaires was self-defeating. The goal of reverse wording (or reverse oriented items) is that investigators want to disrupt the tendency of respondents to mindlessly pick the same response categories without having to think about the content of the item. Put differently, they want to decrease the influence of response sets.
‘I am healthy’
Is reversed as
‘I am unhealthy’
The intention of reverse wording is that it should force respondents to think about each item more carefully before answering. However Eric was convinced that it had another effect. Rather than disrupting response sets, reverse wording elicited contradictory responses from respondents who did notice the direction of the wording of the item had changed. Respondents who were on a roll quickly checking off items that were worded in a consistent fashion, positive or negative failed to notice when they encountered items reversed in their wording and kept going as if there had been no change.
We turned Eric’s hunch into a hypothesis and tested it with data set of patient responses to the Multidimensional Fatigue Inventory (MFI-20) a fatigue questionnaire widely used in the Netherlands. Ten of the 20 items are reverse worded.
Physically I feel only able to do a little
Is reverse worded as
Physically I can take on a lot
We found that response patterns were consistent with our hypothesis, not the intention of investigators. We believe our results have implications for other psychological measures and may even explain some of the inexplicable variations in factor structure obtained for the Hospital Anxiety and Depression Scale (HADS), but that remains to be demonstrated.
The implication is that researchers using questionnaires with reverse worded items might get more valid responses with the reverse worded items removed.
We finished our manuscript just after having another paper had just been accepted,
The Hospital Anxiety and Depression Scale (HADS) is Dead, But like Elvis, There Will Still Be Citings
so Eric wanted to come up with another provocative subtitle.
Wording of titles can be very important because titles are the key means of potential readers deciding whether to click to view an abstract when searching on Google Scholar or electronic bibliographic sites like ISI Web of Science. A smaller number of those who see a title will click to see the abstract, and a even smaller number will go on to reading the article or bookmarking or downloading it. So there is a funneling down from the title to actually seeing the paper. The pathway to the article eventually being cited can be interrupted by a title that does not encourage moving on to view the abstract.
I often have titles with a colon (:) in them, putting critical keywords on one side and having a bit of fun on the other side of the colon that is intended to arouse additional interest. The idea is to provoke readers to click and view the abstract.
So, there was the Elvis paper and the Ain’t Necessarily So paper. The forthcoming PLOS One article, Problems in Cross-Cultural Use of the Hospital Anxiety and Depression Scale: “No Butterflies in the Desert” refers to problems translating an odd item making use of a British colloquial expression into other languages and cultural contexts.
I get a sort of frightened feeling like ‘butterflies’ in the stomach.
Is meant to assess anxiety. It does not even work well in the United States, where butterflies in the stomach connotes being in love. In pilot work, we had asked a South Asian author how he had translated the item. He replied that in his country there were few butterflies and people who endorsed having butterflies in the stomach would be considered psychotic, not anxious. He simply substituted another item. That was not noted in his article. So the idea for a title was born.
I really do not have empirical data for whether this strategy of constructing titles is effective, but I at least have some fun and hope that prospective readers will click and view the abstract.
Limited evidence of the effectiveness of the “colon strategy” can be seen in the wild success of
An in-depth analysis of a piece of shit: distribution of Schistosoma mansoni and hookwork eggs in human stool
Although in a less accessed PLOS journal, PLOS: Neglected Tropical Diseases, as of August 3, 2013,the article has been accessed over 145,000 times and shared over 10,000 times. Of course, it is an obscure topic and at this date the article had only been cited once. An author can lead a reader to an abstract with a catchy title. But if the article has an obscure topic, they will simply satisfy their curiosity and stop there. Apparently only one of the 145,000 hits was by a reader who could possibly find a way to cite the paper because of an interest in Schistosoma mansoni, hook worms or other aspects of tropical diseases.
So, we did we get “cows in the rain”? I have to admit that having grown up in inner-city American housing projects, I had no idea why Eric chose this title. It certainly aroused my curiosity. Reviewers of the first submission of our manuscript were also puzzled. So Eric, a Dutch guy who grew up on a farm, barefoot when not in wooden shoes, gave the following explanation:
This subtitle captures a familiar image from the Dutch countryside that perhaps needs explanation for some urban readers: When it starts raining, all cows stand in the same direction, their tails pointing from where the rain comes. Enclosed are two typical pictures.
In submitting our revised manuscript in response to a revise and resubmit decision, he enclosed these photos
Sure enough, when a skeptical Robbert Sanderman had tried out the title at a gathering of Dutch health psychologists, at least one of them knew the source of ‘cows in the rain.’ But she had grown up in a small Dutch village surrounded by farms.
For those who try the colon strategy, I suggest avoiding clichés like “Lost in Translation” or “Size Matters” or “Back to the Future.” These phrases dull rather than increase readers’ interest. It is a good idea to avoid clichéd titles and you can always check and see what a Googling of your prospective title yields. Just try that with any of what I labeled as clichés.
We briefly considered “Lost in Translation” as a subtitle for our paper about problems in translating the HADS. But we checked and the many past papers with that in the title had variously that about linguistics, cross-cultural psychology, translational science, and engineering. The title was thus clichéd and might dissuade potential readers from clicking on our abstract because in a quick perusal of the title, they thought it might be about something else.
“Size Matters” was overused as a subtitle to the point of cliché back when many fields were dominated by men.
An author has a word or character limit for a title because of the restrictions imposed by journals because of the limits of electronic bibliographic searches. Choosing “Size Matters” would waste 12 characters or two words without either provoking interest or conveying content. And now that there are more women in most fields, the authors risk that they would be put off or simply disagree that size matters Regardless, potential readers would potentially be distracted rather than having their motivation decreased to click on the title to get to the abstract.
The title of this blog post was constructed using the colon strategy.
I would be interested in readers’ response to this strategy with titles. Certainly titles, more than size, matter.