The Chump Effect
Reporters are credulous, studies show.
Dec 5, 2011, Vol. 17, No. 12 • By ANDREW FERGUSON
Lots of cultural writing these days, in books and magazines and newspapers, relies on the so-called Chump Effect. The Effect is defined by its discoverer, me, as the eagerness of laymen and journalists to swallow whole the claims made by social scientists. Entire journalistic enterprises, whole books from cover to cover, would simply collapse into dust if even a smidgen of skepticism were summoned whenever we read that “scientists say” or “a new study finds” or “research shows” or “data suggest.” Most such claims of social science, we would soon find, fall into one of three categories: the trivial, the dubious, or the flatly untrue.
A rather extreme example of this third option emerged last month when an internationally renowned social psychologist, Diederik Stapel of Tilburg University in the Netherlands, was proved to be a fraud. No jokes, please: This social psychologist is a fraud in the literal, perhaps criminal, and not merely figurative, sense. An investigative committee concluded that Stapel had falsified data in at least “several dozen” of the nearly 150 papers he had published in his extremely prolific career.
Perhaps “falsified” is too mild a word. Stapel didn’t just tweak and twist numbers, he made stuff up. With his colleagues, Science Insider reported, “he would discuss in detail experimental designs, including drafting questionnaires, and would then claim to conduct the experiments at high schools and universities with which he had special arrangements. The experiments, however, never took place.” Questionnaires are the mother’s milk of social science, given (most often) to collections of students who are easily accessible to the scientist. After being rewarded with course credits or money, the students go on to serve as proxies for humanity in general, as the scientist draws from their questionnaires large conclusions about the way human nature compels us, all of us, to think and act.
The conclusions that Stapel drew were large indeed. One thing he liked to demonstrate in his studies was the exploitive nature of democratic capitalism. Last year, the New York Times reported on a typical Stapel study, called “The Self-Activation Effect of Advertisements.” It proved that advertising for cosmetics and fancy shoes “makes women feel worse about themselves,” as the Times put it. Another study, released at the end of the scandal-ridden year 2009, was called “Power Increases Hypocrisy.” Quite a timely little study it was. Stapel and his colleagues’ research revealed that powerful people were more likely to be “moral hypocrites.” And which powerful people did the researchers have in mind? “Politicians [who] use public funds for private benefits while calling for smaller government” and CEOs “accepting executive bonuses while simultaneously asking for government bailouts.”
Both of these studies purported to employ the usual social-psychology method: Students in psychology or marketing classes were asked to “role-play” or perform some artificial task under the observation of graduate students. Then they’d fill out those questionnaires to report their thoughts or feelings.
Sometimes, though, social psychologists move beyond the lab. A good example is a more recent study from Stapel’s corpus, released last spring to wide publicity. It touched on another of Stapel’s favorite themes: white racism.
“Disorder can encourage stereotyping, study says,” read the headline in the Los Angeles Times. Stapel discovered—scientifically, of course—that white heterosexuals used racism and homophobia as defense mechanisms. Confronted with disorder in their “social environment,” Stapel showed, they quickly reverted to their natural inclination to stereotype “the other” and draw comfort from their prejudice.
At this writing, investigators are not yet clear to what extent the results of these particular studies are discredited by Stapel’s fakery. And nobody knows how extreme an anomaly Stapel’s behavior will prove to be. Leslie John of Harvard Business School recently surveyed more than 2,000 social psychologists about their research methods. She found a rash of research practices she deemed “questionable.” Indeed, she wrote, in social psychology, “some questionable practices may constitute the prevailing research norm.”
But it hardly seems to matter, does it? The silliness of social psychology doesn’t lie in its questionable research practices but in the research practices that no one thinks to question. The most common working premise of social-psychology research is far-fetched all by itself: The behavior of a statistically insignificant, self-selected number of college students or high schoolers filling out questionnaires and role-playing in a psych lab can reveal scientifically valid truths about human behavior.
And when the research reaches beyond the classroom, it becomes sillier still.
Consider this recent study by Stapel, demonstrating the relationship between “disorder” and white racism and homophobia. Several news reports outlined the methodology as Stapel explained it.
The experiment began after janitors at the Utrecht railroad station went on strike. Stapel and colleagues leapt into action. As the garbage in the station piled up, they cornered 40 white passengers. One by one the travelers were asked to take a seat in a row of folding chairs. They were given a questionnaire. If they filled it out, they were told, they would get a piece of chocolate or an apple as a reward.
The questionnaire asked to what degree the travelers agreed with stereotypes about certain types of people. (Are gays “creative and sweet” or “strange and feminine” or “impatient and intelligent”?) And then came the twist! Stapel had planted a person at the end of the row of chairs—sometimes a black person, sometimes a white. Researchers measured how far away from the person each respondent chose to sit. Meanwhile, thanks to the questionnaire, they could measure the degree of racism or homophobia each was feeling. On average, the travelers sat 25 percent closer to the white man than to the black man.
In time the janitors came back to work. The station was cleaned spick-and-span. Stapel and his gang returned and performed the experiment again, on another 40 white travelers. There in the tidy environment, their questionnaires showed they were less racist and homophobic than their counterparts from the earlier experiment. And on average, they sat the same distance from the white person as the black person. Hence, as the headline read: “Messy surroundings make people stereotype others.”
But Stapel, as an internationally respected social psychologist, wasn’t satisfied. So he designed another experiment to confirm his finding. The Stapel gang went to a wealthy neighborhood. They threw a bicycle on the ground, tore up paving stones, and, as the L.A. Times noted, parked Stapel’s Subaru on the sidewalk. Chaos! Disorder! Forty-seven passersby were collared, given a new questionnaire to test their racism, and asked to donate money to (I’m not making this up) a charity called “Money for Minorities.”
Then the bike was removed. The stones were replaced. Stapel moved his Subaru. Now it was just a nice, rich, tidy neighborhood. More passersby were collared, more questionnaires were filled out, and—here’s the scientific finding—less racism and homophobia were revealed. And the passersby in the tidy neighborhood gave more money to minorities on average: to be precise, 0.65 euro more.
Social psychologists around the world gazed on these findings when they were published this spring. They gave their chins a good, firm tug. “This need for order matters a lot more than we might have thought,” a Duke psychologist told the Times.
Did Stapel fake his research? Did he and his students really make all those people fill out forms for an apple? Did Stapel really cross-tabulate the data? Did he really park his car on the sidewalk?
Who cares? The experiments are preposterous. You’d have to be a highly trained social psychologist, or a journalist, to think otherwise. Just for starters, the experiments can never be repeated or their results tested under controlled conditions. The influence of a hundred different variables is impossible to record. The first group of passengers may have little in common with the second group. The groups were too small to yield statistically significant results. The questionnaire is hopelessly imprecise, and so are the measures of racism and homophobia. The notions of “disorder” and “stereotype” are arbitrary—and so on and so on.
Yet the allure of “science” is too strong for our journalists to resist: all those numbers, those equations, those fancy names (say it twice: the Self-Activation Effect), all those experts with Ph.D.’s!
To their credit, the Stapel scandal has moved a few social psychologists to self-reflection. They note the unhealthy relationship between social psychologists and the journalists who bring them attention—each using the other to fill a professional need. “Psychology,” one methodologist told the Chronicle of Higher Education, “has become addicted to surprising, counter-intuitive findings that catch the news media’s eye.”
That’s a scandal, all right. Stapel’s professional treachery is a scandal, too. But the biggest scandal is that the chumps took him seriously in the first place.
Andrew Ferguson is a senior editor at The Weekly Standard and the author, most recently, of Crazy U.
Recent Blog Posts