During the first part of the 21st century, the legal rights and social roles of lesbian and gay people have been in flux. Positive and complex depictions of lesbian and gay television characters have become far more common in the U.S. . Further, U.S. news media have extensively covered the suicides and peer rejection of LGBT adolescents as well as responses like Savage’s It Gets Better campaign . In 2010, the U.S. Congress and President Obama repealed the U.S. military’s Don’t Ask, Don’t Tell policy, which had prohibited LGBT people from serving openly in the armed forces. Perhaps the most discussed issue was change in the legal recognition of same-sex couple relationships. In 2003, marriages between two people of the same gender were not legally recognized anywhere in the United States. In contrast, as of the end of 2014, 35 states, the District of Columbia, and the federal government legally recognize such marriages.
Accompanying the changes in legal rights, self-reported attitudes towards lesbian and gay people have also been shifting . National Gallup polls show that moral approval of lesbian and gay people in the U.S. has increased from 44% in 2006 to 59% in 2013 [3, 4]. In a worldwide poll, the Pew Global Attitudes Project found that in 2003 a slim majority of 51% of respondents felt that homosexuality should be accepted by society. By 2013, this number had increased to 60%. In both cases, people who expressed positive attitudes towards lesbian and gay people were in the minority in the 2000s but constituted the majority of respondents by 2013. Horn (2012) presented a multidimensional framework to describe predictors of self-reported attitudes. These include participant demographic (e.g., gender) and personality (e.g., gender role attitude) characteristics, as well as perceptions of the target and the type of self-report attitude measured. Despite change in self-reported attitudes, these predictors have remained relatively consistent across this time period .
This shift in legal rights and cultural attitudes is dramatic. It prompts a question of significant theoretical interest in attitude research: what has changed? Self-reported attitudes can shift because people change their minds or because expressing one’s mind becomes fraught with social consequences [2, 10]. It is possible that the shifts towards greater positivity toward lesbian women and gay men in past research are merely indicators of decreased willingness to express negative attitudes. In other words, are these changes in mental representation or have people just become more reluctant to admit to bias?
Implicit measures of evaluation offer an opportunity to test explanations for the observed change in attitudes. Should we attribute shifts in reported attitudes to shifts in evaluation? Or, are we observing shifts in willingness to express negative evaluations of lesbian and gay people – due to people’s unwillingness to admit to or endorse those evaluations? Implicit measures assess evaluations without requiring an act of introspection and are relatively uncontrollable ([16, 17]; and for discussion of interpretation of implicit and explicit attitude constructs see [1, 5, 15, 19]). If preferences for straight people compared to lesbian women and gay men show evidence of change on implicit measures as well as explicit measures, this would suggest that evaluations, not just reporting tendencies, are changing.
The present research used the Implicit Association Test (IAT) to investigate whether implicit attitudes towards lesbian and gay people have changed over time. The IAT is a computerized reaction time measure that assesses the comparative strength of associations between two pairs of concepts. Participants use two keys on a keyboard to categorize target stimuli appearing one at a time on the screen as belonging to one of two categories (“Gay people” or “Straight people”) or one of two attributes (“Good” or “Bad”). If people are faster at accurately categorizing stimuli when “Gay people” and “Bad” are paired on the same side of the screen (compared to when “Straight people” and “Bad” are paired on the same side of the screen), it suggests that people have stronger negative associations with gay people than with straight people. The IAT thus can provide a measure of implicit evaluations of lesbian and gay people.
Moreover, there is considerable interest and evidence for the malleability of implicit evaluation in the short term (e.g., [11, 20]), but much less evidence for long-term or cultural shifts in evaluation (see  for a review). In order to determine the magnitude of change at the cultural level, researchers must examine shifts in mean bias across time. Knowing how and when implicit attitudes change on a cultural level is vitally important to understanding how long-term enduring changes in implicit attitudes occur. When large numbers of individuals experience small changes, these changes accumulate into large differences at the societal level. Thus, it is important to examine both change on the individual level (i.e., the association between date and an individual’s bias score) as well as change in bias over time aggregated across many individuals (i.e., how much the sample mean changes over time).
To date, no researchers have identified any instances of implicit evaluation change on a cultural level. Such knowledge may help identify interventions and situations most likely to lead to lasting changes in implicit attitude evaluations. In the most similar study, Schmidt and Nosek (; N = 479,405) found little evidence for change in implicit racial preferences for Whites over Blacks between September 2006 and May 2009 that could have been attributed to the rise and election of Barack Obama as the first Black U.S. president. That lack of an effect was particularly interesting in light of evidence from laboratory research, which demonstrated that Obama could operate as a means of shifting implicit racial preferences short-term in the lab . Clearly, short-term malleability in the lab does not always translate to long-term social change. Thus, the present research may provide the first opportunity to demonstrate evidence of shifting implicit preferences across time on a cultural scale.
Materials and Method
Participants self-selected the “Sexuality IAT” from a list of available IATs after arriving on the Project Implicit demo site at https://implicit.harvard.edu and completing informed consent. The presentation order of the IAT, explicit measures, and demographics measures was randomized. After completing all measures, participants were debriefed with feedback on their IAT performance. All data were collected with human subjects approval from the University of Virginia.
Participants were volunteers who visited Project Implicit’s demonstration site and selected the “Sexuality” Implicit Association Test (IAT) between February 6, 2006 and August 10, 2013, for a total of 7.48 years (2742 days) of data collection. February 6, 2006 marked the introduction of demographic items in the Sexuality IAT used in this analysis, and August 10, 2013 marked the initiation of data analysis for this project. We included only participants who had (a) complete data for all key measures and covariates and (b) an overall error rate of less than 30% or error rates below 40% on any individual block and a latency of 400ms or less on fewer than 10% of IAT trials [7, 18]. We excluded 3.6% of eligible sessions on the latter criterion. The final dataset consisted of 683,976 sessions, with a mean of 250 (SD = 215.23; median = 214) sessions per day. For the final complete dataset, the smallest daily sample consisted of 1 session (during website downtime), and the largest consisted of 4664 sessions. The middle 80% of daily sample sizes ranged from 94 to 424 sessions. There was no evidence of systematic variation across time in the number of IAT sessions per day, b = –.001, t(2729) = –.183, p = .86, R2 < .0001.
Participants’ average age was 24.9 years (SD = 10.6), and 62.3% of participants identified as female. Among participants, 70.8 percent reported their ethnicity as White (not of Hispanic origin), 6.4% as Asian or Pacific Islander, 6.3% as Black (not of Hispanic origin), 5.2% multiracial (Other), 4.7% as “other” or did not know, 4.6% Hispanic, 1.5% multi-racial (Black and White), and 0.6% as American Indian or Alaskan Native. Eighty percent of participants identified as heterosexual, 10.4% as lesbian, gay, or homosexual, and 9.6% as bisexual. Participants were relatively highly educated; 62.5% reported some college or an undergraduate college degree, 12.2% were completing or had completed an advanced degree, and 25.4% reported a high school education or less. Politically, 52% of participants identified as liberal, 20.2% conservative, and 27.8% moderate. Most participants were U.S. residents (83.2%).
Below, we report all measures that we analyzed for this study, and manipulations of order and procedural conditions for the IAT. Additional measures are available in the dataset. Complete codebooks and datafiles are available at: https://osf.io/p7va5/ and https://osf.io/ctqxo/.
Explicit preferences for straight people compared to lesbian and gay people. Explicit attitudes for the first half of 2006 were assessed on a five-point Likert scale, ranging from –2 “I strongly prefer straight people to gay people” to +2 “I strongly prefer gay people to straight people.” Explicit attitudes from the end of 2006 through 2013 were assessed on a seven-point Likert scale ranging from –3 “I strongly prefer straight people to gay people” to +3 “I strongly prefer gay people to straight people.” In both cases, the midpoint represented equal liking of straight and gay people. Scores from both scales were Z-transformed and combined into one standardized scale retaining the rational zero midpoint of no preference.
Implicit preferences for straight people compared to lesbian women or gay men. The IAT is a computerized reaction time measure that assesses the comparative strength of associations between two pairs of concepts [6, 14]. Participants use two keys on a keyboard to categorize target stimuli appearing one at a time on the screen as belonging to one of four categories (“Gay people”, “Straight people”, “Good”, “Bad”). Average response latencies in categorizing stimuli correctly are compared between two response conditions: (1) Gay people and Good items are categorized with one response key (‘e’) and Straight people and Bad items are categorized with the other (‘i’); or, (2) Gay people and Bad items are categorized with one response key and Straight people and Good items are categorized with the other. Participants who categorize items faster in the first condition compared to the second are said to have an implicit preference for gay people compared to straight people.
The IAT followed the standard 7-block format described in  and followed the same procedure as the sexuality IAT in  that reviewed its psychometric properties, variation across demographic groups, and relations with explicit measures. Order of the two category pairing conditions was randomized across participants, as was whether good items were categorized with either the ‘e’ or ‘i’ key (and bad items with the other key). Target stimuli for Straight people, Good, and Bad categories were the same for all participants. Target stimuli for Gay people consisted of pictures and words representing either lesbian women or gay men. The images consisted of 1) same-sex vs different-sex wedding toppers and 2) abstract restroom-style images of either two women, two men, or a woman and a man standing side-by-side. Participants were randomly assigned to see stimuli representing either lesbian women or gay men. This variation is known to shift the main effect such that participants show somewhat weaker implicit preferences for straight people over gay people when the Gay people category is represented with lesbian women compared to gay men (, Supplement). We computed scores using the D algorithm recommended by Greenwald et al. (2003) with response latencies recorded for the time used to make the correct response, even if an error occurred during the trial.
We tested whether implicit evaluations of lesbian and gay people have changed over time, with 683,976 Project Implicit volunteers who participated between February 6, 2006 and August 10, 2013, for a total of 7.48 years (2742 days) of data collection. Overall, participants showed a preference for straight people over gay people both implicitly (IAT D M = .33, SD = .47, Cohen’s d = .70) and explicitly1 (M = .60, SD = 1.26, d = .48). The implicit and explicit sexuality attitude measures were positively correlated, r(654,328) = .437, 95% CI [.435, .439]. We standardized the IAT and explicit scores, retaining the rational zero midpoint of no preference for the remainder of the analyses.2
Examining shifts in sample demographics over time
Inferring change with cross-sectional data is difficult. A particular inferential limitation would occur if the composition of the sample changed over time and that composition could account for observed shifts in evaluation. We examined shifts in demographics by regressing age, ethnicity, education, sexual orientation, political ideology, gender, and US residency on date of participation. There were slight shifts in demographic representation over the course of data collection (Total R2 = .015). The largest shifts were for age and education: the sample became slightly younger and more educated over time3. To a lesser extent, the sample became slightly more female, White, conservative, less straight, and more composed of U.S. residents.
To account for potential influences, the hierarchical regression models4 evaluating the effects of date on implicit and explicit preferences first examined the effect of date in isolation, then tested whether the effect survived after adding these demographics to the model as covariates. Finally, we tested whether the effects of date on preferences were moderated by any of the demographic variables. In sum, when date effects on preferences were evaluated within demographic groups, the shift observed in the aggregate analysis persisted, suggesting that the small shifts in demographics over time could not account for the effects of date on implicit or explicit preferences reported next.
Hierarchical regression evaluating shift in implicit preferences across time
Though our primary research question is about societal change over time (i.e., sample mean change over time), we first examined variation in implicit preferences at the individual level. Many factors predict variation in attitudes across individuals – sex, race, education, political orientation, religiosity, and one’s own sexual orientation – so variance accounted for by date of participation should be minor in comparison to these other sources of variation. Nevertheless, any indication of change at the individual level could accumulate to sizable evidence of change at the societal level.
We regressed IAT score on participation date to determine whether the magnitude of implicit sexuality preferences changed over time from February 2006 to August 2013 (see Table 1). Date was negatively related with implicit preference for straight people compared to gay people suggesting that the effect has weakened over time. Date accounted for roughly 0.1% of the variance in implicit preferences across individuals (R2 = .0009). In the second step, we added procedural variables for the implicit measure to the model: order of stimuli, gay men vs. lesbian women stimuli, and whether good items were categorized to the right or left. These variables accounted for about 4.2% of the variation in implicit evaluations but did not affect the relationship between date and implicit evaluation.
|Sexual orientation||2||0.13337||52629.40||< .0001|
|Political orientation||1||0.04846||34833.17||< .0001|
|US Resident||1||0.00037||251.44||< .0001|
|Sexual orientation||2||0.13325||52533.89||< .0001|
|Political orientation||1||0.04849||34880.86||< .0001|
|US Resident||1||0.00038||257.54||< .0001|
|Date × Side||1||0.00004||25.65||< .0001|
|Date × Order||1||0||0.00||0.9534|
|Date × Stimuli||1||0.00002||10.81||0.0010|
|Date × Age||1||0.00007||45.21||< .0001|
|Date × Gender||1||0.00001||4.67||0.0307|
|Date × Race||7||0.00006||5.79||< .0001|
|Date × Sexual orientation||2||0.00002||5.79||0.0031|
|Date × Political orientation||1||0.00004||24.47||< .0001|
|Date × Education||4||0.00001||1.63||0.1624|
|Date × US Resident||1||0||0.42||0.5146|
In the third step, we added demographic variables to the model: sexual orientation, political ideology, gender, race/ethnicity, age, education, and U.S. residency. As expected, these variables substantially increased the model’s prediction of variation in implicit preferences to 26.8%. In other words, individual differences in identity account for much more variation in implicit preferences than the date they completed the measure. Participants’ sexual orientation and political ideology were particularly predictive. However, including demographic characteristics did not substantially reduce the effect of date predicting implicit preferences.
Finally, in the fourth step, we added tests of moderation: date (mean centered) interacting with each of the procedural and demographics variables entered in steps 2 and 3. These interactions added very little additional prediction of implicit preferences to the model (change in model R2 = .0002), and did not qualify date predicting implicit preferences.
In sum, we observed a very small effect of date on implicit preferences that survived the inclusion of many covariates and moderating terms. It is not surprising that at the individual level, demographic variables are far superior predictors of variation in implicit preferences than which date the participant completed the implicit measure. Substantial individual differences in implicit preferences persist over the timespan across sexual orientation, political ideology, and other variables. On an individual basis, time is a trivial predictor of implicit preferences compared to those identities. While the gaps between gay and straight people and liberals and conservatives may persist across time, the small date effect suggests that the entire culture may be shifting toward more positive evaluations of gay men and lesbians. In other words, if everyone develops more positive attitudes toward gay men and lesbians, then the variation across individuals by demographic factors will be preserved, making date a small contributor in accounting for variation across individuals, while still demonstrating a substantial societal shift.
Estimating the magnitude of change at the cultural level requires examining how the mean is shifting across time. Many small effects at the level of the individual may add up to large cumulative effects at the cultural level. To examine cultural change, we computed estimated effect sizes for the sample as a whole on the first and final days of data collection (see Table 3). For these estimates, we predicted standardized IAT D-scores for Day 1 and Day 2742 of data collection, calculated from the hierarchical regression analyses presented here. The effect size for date corresponds to an overall decline in the estimated effect size of d = .77 on Day 1 of the data collection to a d = .67 on Day 2742 (point estimate delta = –.103). Descriptive data for the mean standardized IAT D-scores for each month from 2006 to 2013 can be seen in Figure 1. In a separate analysis (see SI Appendix), we also tested for non-linear effects of time and found support for a quadratic effect in addition to the linear effect (R2 = .0013) suggesting that the rate of change accelerated slightly over the time period. In sum, across the nearly 7.5 years of data collection, the mean implicit preference for straight people over gay people declined by 13.4%, a notable decline.
|Sexual orientation||2||0.15620||103850.00||< .0001|
|Political orientation||1||0.10497||76738.04||< .0001|
|Sexual orientation||2||0.15650||56799.35||< .0001|
|Political orientation||1||0.10519||72035.93||< .0001|
|Date × Age||1||0||0.02||0.8842|
|Date × Gender||1||0.00003||19.47||< .0001|
|Date × Race||7||0.00002||2.26||0.0267|
|Date × Sexual orientation||2||0.00067||219.77||< .0001|
|Date × Political orientation||1||0.00016||105.02||< .0001|
|Date × Education||4||0.00011||17.37||< .0001|
|Date × US Resident||1||0.00008||50.47||< .0001|
Hierarchical regression evaluating shift in explicit preferences across time
We used the same analysis plan to evaluate shifts in explicit preferences across time, excluding the step adding IAT procedural variables to the model. In the first step, we regressed participation date on explicit preferences (see Table 2). Date was negatively related to explicit preferences for straight people compared to gay people suggesting that the effect has weakened over time. Date accounted for about 0.17% of the variance in explicit preferences across individuals (R2 = .0017).
In the second step, adding demographics to the model substantially increased the model’s prediction of variation in explicit attitudes to 33%. As with implicit attitudes, participants’ sexual orientation and political ideology accounted for the most variation. Even so, the effect of date predicting explicit preferences persisted. In the third step, adding interactions between date and the demographics added little additional prediction of explicit preferences to the model. Furthermore, these variables did not qualify the effect of date predicting explicit preferences.
As noted above, change at the cultural level is indicated by the mean change over time. We therefore computed regression estimated effect sizes for the sample as a whole on the first and final days of data collection. The effect size for date corresponds to a decline in estimated effect size from d = .54 on day 1 of the data collection to a d = .40 on day 2742 (Point estimate delta = –.139). In a separate analysis (see SI Appendix), we also tested for non-linear effects of time and found support for a quadratic effect in addition to a linear effect of time (R2 = .0030), suggesting that the rate of change accelerated slightly over the time period.
In sum, paralleling the results for implicit preferences, we found an effect of date on explicit attitudes that survived the inclusion of many covariates and moderating terms. All told, across the nearly 7.5 years of data collection, the mean explicit preference for straight people compared to gay people declined by 26% at the cultural level. This effect was approximately twice the size of the cultural shift observed using implicit measures.
Demographic moderators of shifts in implicit preferences across time
We conducted exploratory analyses to learn whether some demographic groups showed more evidence of shifting implicit preferences over time than others (i.e., the moderation effects in Step 4 of the hierarchical regression). Table 3 displays predicted standardized IAT D-scores for Day 1 and Day 2742 of data collection, calculated from the hierarchical regression analyses presented above. Predicted scores for the first and last day were used in place of raw scores to represent the estimated trend in preferences across the time period. Positive point estimates represent pro-straight implicit preference, and negative point estimates represent pro-gay implicit preference, with a rational zero midpoint.
Overall, race/ethnicity, age, political ideology were the strongest moderators. White and Hispanic people showed greater shifts than Asian and Black people, younger adults showed greater shifts than older adults, and liberals showed greater shifts than conservatives. Women showed slightly greater shifts in implicit preferences than men. Shifting implicit preferences were not confined to straight people; straight, bisexual, and lesbian or gay people all showed moderate shifts in implicit preferences during this time period, with bisexual people showing evidence of the largest shifts (see Figure 1). Overall, attitudes towards lesbian women shifted more than attitudes towards gay men. There was little evidence for differential change as a function of education or U.S. residency.
Demographic moderators of shifts in explicit preferences across time
We conducted similar exploratory analyses for explicit preferences (i.e., the moderation effects in Step 3 of the hierarchical regression). Overall, sexual orientation, political ideology, education, and U.S. residency were the largest moderators. Lesbian and gay participants reported weaker explicit preferences for gay people over straight people over time, whereas straight participants reported weaker explicit preferences for straight people over gay people over time. In other words, both groups reported weaker absolute preferences on the basis of sexual orientation over time. Conservatives shifted more than liberals, U.S. residents shifted more than non-U.S. residents, and less-educated people shifted more than highly educated people (college degree or higher). We observed a smaller effect for gender and race, with men shifting more than women and Black people shifting slightly more than White people. There was little evidence for differential change as a function of age.
The cultural shift in legal rights and attitudes toward gay men and lesbians is not limited to what people are willing and able to report on explicit measures. In a very large sample collected between 2006 and 2013, we replicated evidence from public opinion polls for change over time in explicit attitudes towards lesbian and gay people. For the first time, we contributed evidence that this attitude shift has occurred implicitly, as well. This is the first documented evidence for a cultural shift in implicit evaluations about any topic. Implicit preference for straight people over gay people declined overall by 13.4% from 2007 to 2013, and explicit preferences declined by 26% during that same period. Notably, the shift in explicit preferences was about twice the size of the shift in implicit preferences in this sample. This suggests that change is more rapid in evaluations that people report willingly than those that may operate outside of conscious awareness or conscious control.
Nearly all demographic groups showed weakening of implicit preferences for straight people over gay people over time, with the greatest changes occurring among Hispanic, White, female, liberal, and young adult participants. The smallest changes in implicit evaluations occurred among Black, Asian, male, conservative, and older adult participants. Interestingly, however, the relative magnitude of change was largely opposite for explicit preferences. Although nearly all groups reported weaker explicit preferences for straight people over gay people, shifts in explicit preferences were greatest among straight, male, conservative, Black, and less-educated participants. These are also the groups that held the strongest overall preferences for straight people. The two findings together offer a speculative interpretation: substantial explicit change may occur first and perhaps enable the slower implicit change that occurs later. Here, increases in explicit positivity toward gay men and lesbians may have already occurred among women and liberals prior to the assessed time period, setting the stage for later change in implicit preferences. Men and conservatives, however, may be currently undergoing explicit change, which could accelerate implicit change in the future. With the present data, however, this possibility is only speculation.
Evidence for the changing preferences of gay men and lesbian women furthers this speculation. Implicitly, these groups showed more substantial increases than straight participants in their positive evaluations of gay people compared to straight people. Explicitly, however, gay participants’ reported preferences for gay compared to straight people weakened across the time period. That is, implicit and explicit changes were moving in opposite directions. It is as if explicit pro-gay preferences had become relatively extreme and were now becoming more moderate in the face of an increasingly tolerant culture, while implicit pro-gay preferences were continuing to strengthen, indicating the slow cumulative change over time. In the coming epoch, implicit preferences among lesbian women and gay men may likewise become more moderate, following the explicit trends. In sum, might explicit change be a leading indicator of eventual implicit change? Systematic investigation of this possibility is left to future research.
The current study has limitations. In particular, it is a cross-sectional comparison of a non-representative sample of the U.S. and world populations who may be younger and less likely to identify themselves as male, conservative, or straight than the general population. Given that we do not have experimental control over the sampling, we cannot eliminate the possibility that an undetected selection effect is responsible for the apparent shift. However, this seems unlikely, as the effects are robust to controlling for the demographic variables available in the dataset, and our explicit data mirrors attitudinal shifts observed in Gallup and other representative polling. Also, the IAT stimulus set featured symbolic representations of gay and lesbian couples rather than individuals, and it may be difficult to compare relative change between the implicit and explicit measures due to variance in the measures’ estimates. Finally, we were not able to examine religion as a potential moderator due to lack of data spanning the time period of interest. However, one strength is that we conducted daily assessments of implicit and explicit preferences towards gay and lesbian people, across more than seven years, with a massive sample, controlling for demographic characteristics. It is reasonable, but not definitive, to interpret the observed decline over time in preferences for straight people over gay people as indicating actual cultural change. Notably, despite the overall decline, participants still exhibited pronounced pro-straight preferences at the end of the time period. It would be useful to accumulate longitudinal evidence on representative samples in order to confirm this effect and provide parameter estimates for the magnitude of change across the culture.
A notable feature of this shift is that while we observed large shifts at the cultural level, the change in attitude for any single individual was trivial in effect size. The passage of time, and the cultural change occurring with it, accounts for an extremely small amount of a single person’s implicit and explicit preferences, especially when compared to features of their identity such as sexual orientation and politics. What makes this observed change important is the fact that it is occurring across the entire sample, and across demographic categories, indicating a cultural shift in evaluation of lesbian and gay people. The tiny change for one individual accumulates into a substantial cultural effect expressed in all individuals (13.4% decline for implicit preferences and 26% decline for explicit preferences). For comparison, the effect size of time observed in this study was one hundred times greater than that found by Schmidt and colleagues (2010) when they examined change in implicit racial attitudes before and after President Obama’s election. It is unclear whether a large change in a few people or a small change in everyone ultimately has a more substantial impact on society. We anticipate that most social change starts with large attitude changes within a subset of society and eventually ripples out to produce small changes in all members of society . In the early 21st century, we appear to be bearing witness to this effect for evaluations of lesbian women and gay men.
Conflict of Interest Declaration
Nosek is an officer of Project Implicit Inc., a non-profit organization that includes in its mission “To develop and deliver methods for investigating and applying phenomena of implicit social cognition, including especially phenomena of implicit bias based on age, race, gender or other factors.”