A recent article in the prestigious Proceedings of the National Academy of Sciences(PNAS) claims to explore genomic aspects of the pursuit of meaning versus happiness. The considerable press coverage the article generated accepted it provided a scientific basis for resolving a classical philosophical question: should we pursue meaning ( termed “eudaimonism”) in our lives or happiness (“hedonism”)?
As the Atlantic Monthly notes
The terms hedonism and eudaimonism bring to mind the great philosophical debate, which has shaped Western civilization for over 2,000 years, about the nature of the good life. Does happiness lie in feeling good, as hedonists think, or in doing and being good, as Aristotle and his intellectual descendants, the virtue ethicists, think? From the evidence of this study, it seems that feeling good is not enough. People need meaning to thrive. In the words of Carl Jung, “The least of things with a meaning is worth more in life than the greatest of things without it.” Jung’s wisdom certainly seems to apply to our bodies, if not also to our hearts and our minds.
The title of the Atlantic Monthly article announces the resolution this research offers
Meaning Is Healthier Than Happiness
And the header further declares
People who are happy but have little-to-no sense of meaning in their lives have the same gene expression patterns as people who are enduring chronic adversity.
If, as an Academic Editor for PLOS One I had received this article as a manuscript, I would probably have recommended Rejection without sending it out for further review. But if I had sent the manuscript out for review, I would have chosen at least some reviewers with relevant psychometric backgrounds. I would be disappointed if they did not immediately notice some fatal flaws hiding in plain sight.
The article is highly technical, with basic details presented in unnecessarily ponderous sentences:
Primary analyses examined the relationships of hedonic and eudaimonic well-being to expression of a 53-gene contrast score summarizing three a priori-defined components of the CTRA profile (12, 33–35): up-regulated expression of proinflammatory genes, down-regulated expression of genes mediating type I IFN responses, and down-regulated expression of genes involved in antibody synthesis.
There are many such sentences, some even more complex.
I doubt that many people who have an opinion about the article have actually read it in its entirety. But with such a formidably complex article making such strong claims, it is all the more important to start with checking basic statistics and measurement issues. Problems often start in these details that preclude important questions being addressed. Think of the aliens invading Earth in the War of the Worlds being brought down by common microbes.
In the case of this article, the validity of sophisticated methodologies, analyses, and interpretations depend entirely on some distinctions being captured in self-report assessments of hedonic versus eudaimonic well-being with the Short Flourishing Scale. What we can say about all the complicated biomedical assessments is limited by what we can say about some very simple questions delivered by Internet So, I started by examining the correlation between these two subscales and it was r = 0.79, ( p < 0.0001).
This correlation is as high as the reliability of these two subscales allows, meaning they are essentially interchangeable and measure the same thing. Except for by random variance, that finding is highly unlikely that something will be related to one variable and not the other.
The high correlation between these variables is reflected in shared associations with other variables:
Analyses found both forms of well-being to show similarly strong inverse relationships to symptoms of depression [Center for Epidemiological Studies–Depression (CES-D) correlation with hedonic well-being, r = −0.67, p < 0.0001; correlation with eudaimonic well-being, r = −0.66, p< 0.0001; difference in dependent correlations, p = 0.8550]. Similarly strong inverse relationships were also observed for CES-D subscales assessing affective symptoms of depression (hedonic, r = −0.75, P < 0.001; eudaimonic, r = −0.71, P < 0.001; difference, P = 0.3228) and vegetative symptoms of depression (hedonic, r = −0.45, P < 0.001; eudaimonic, r = −0.48, P < 0.001; difference, P = 0.6297).
The investigators attempted to escape their problems by introducing statistical controls:
each well-being dimension treated as a continuous measure and adjusted for correlation with the other dimension of well-being and for age, sex, race/ethnicity, body mass index (BMI), smoking, alcohol consumption, recent minor illness symptoms, and leukocyte subset prevalence.
But given the close association between the two variables, what is portrayed in the multivariate analyses, eudaimonic-well-being-controlling-for-hedonic-well-being-and-many-other-things is very different than eudaimonic well-being without such controls. If we were talking about people, we probably couldn’t even recognize a family resemblance between the two. Shared similarities were removed, so one would not be recognizable from a photo of the other.
This represents statistical malpractice and we are well on our way to nonsense. Any divergent associations with genomic transcriptome profiles are likely to be artificial and not replicated in future studies. The investigators’ particular choice of variables to control was arbitrary. There was no particular reason given for this combination. Other variables were simply left out. Other choices of what to control would lead to other results. But this is not a meaningful exercise for another reason: there are too many variables being controlled with too few research participants.
And more generally, we cannot just dump possible confounding variables into an equation expect to find anything meaningful
Statistical adjustment by an excessive number of variables or parameters, uninformed by substantive knowledge (e.g. lacking coherence with biologic, clinical, epidemiological, or social knowledge)…can obscure a true effect or create an apparent effect when none exists.
Faced with such highly correlated measures of the crucial variables of hedonic versus eudaimonic well-being, what could the investigators possibly have done to remedy matters and proceed? I don’t think that there is anything. It should have been mission aborted.
When I encounter such high correlations, I look to the items that went into the construction of scales, deliberately ignoring the labeling of the scales. In the case of these subscales:
Participants completed online assessments of hedonic and eudaimonic well-being [Short Flourishing Scale, e.g., in the past week, how often did you feel. . . happy? (hedonic), satisfied? (hedonic), that your life has a sense of direction or meaning to it? (eudaimonic), that you have experiences that challenge you to grow and become a better person? (eudaimonic), that you had something to contribute to society? (eudaimonic); answered on a six-point frequency metric whereby 0 indicates never, 1 indicates once or twice, 2 indicates approximately once per week, 3 indicates two or three times per week, 4 indicates almost every day, and 5 indicates every day]
These questions are odd, vague, and unlikely to be encountered in everyday life unless someone happens to be Barack Obama or Bill Gates. I don’t know about you, but my wife has not recently asked me at dinner nor have I been asked another friend at a happy hour get together, “hey Jim, what did you do today to contribute to society?” “Did you happen to run into any problems that challenge you to grow and become a better person?”
It’s not surprising that research participants requested to answer these questions came up with something vague and affectively toned, i.e., related to their mood at the moment. I’m sure that if investigators had done a cognitive interview, it would’ve revealed that respondents struggle with trying to find answers and the basis of the answers vary widely from responded to respond. This is a particularly poorly constructed assessment instrument. And what ever solid biomedical science was done, it is sustained or falls on an empirically indefensible distinction derived from poor assessment of what people have to say about themselves on the Internet.
Press coverage for this study is pure hokum. Over centuries, lots of people have offered advice about whether we should pursue meaning or happiness. They will undoubtedly continue to do so, but let’s have no illusions that Barbara Fredrickson’s study can do anything to settle the issue.
Distorted press coverage can often be traced to distorted abstracts, and so I always recommend that skeptical readers compare press coverage to the abstracts of scientific articles. I know well that the abstracts too are often distorted, but it is sort of a rule-out assessment. If the press coverage does not fit with what is said in the abstract, it could be that the journalist is making something up. On the other hand if the two seem to fit, we might have to proceed to the laborious process of checking the abstract against what is said in the rest of the paper. And maybe we can establish that exaggerated coverage in the press is churnaled from a quick read of the abstract.
The abstract states
Hedonic and eudaimonic well-being showed similar affective correlates but highly divergent transcriptome profiles.
Translation? The two scales had similar correlates with self-report measures, but widely different profiles associated with them.
Perhaps, but there is a sleight of hand going on here, or if you like, a bait and switch. The correlates of Hedonic and eudaimonic well-being were examined at the simple bivariate level and their association with transcriptome profiles was examined in dubious multivariate analyses.
I am extremely doubtful that with transcriptome profiles of hedonic and eudaimonic well-being would be that different if statistical controls had not been applied. I challenge the authors to present the relevant data. It is quite statistically improbable that to self-report variable so highly correlated could have distinctive different profiles.
Note that this is the same Barbara Fredrickson who brought us the 2.900 positivity/negativity ratio. Check out the NeuroSkeptic’s straight-shooting coverage of the debunking of that, as well as his call for a retraction.
In a recent posting at my other blog, I cited John Ioannidis to suggest that hot areas like genomics tend to bring lowered standards, false discoveries, and exaggerations and very little real advancement of science. This application of genomics to philosophical questions about how we should lead our life certainly seems to fit the bill. But I think something else is going on here. A huge market has been generated by people desperately searching in self-help books for answers that religion once provided. I wouldn’t be surprised if the contract for another self-help book has been signed already.