A new study published in the Journal of Experimental Psychology: General suggests that widely used research methods for studying racial differences in perceptions of racism may be misleading. While people in different racial categories often show average differences in how they perceive racism, the new findings indicate just as much disagreement exists within those categories as between them. The findings challenge the common assumption that racial groups have cohesive, shared perspectives on what constitutes racism and suggest that current methods may unintentionally reinforce the illusion that race categories represent real, psychologically distinct groups.
Social scientists have long been interested in how people perceive racist events. One important question is what kinds of actions or words are considered racist in the first place. How people use the term “racist” can reveal how they define racism, how they recognize racist experiences, and how they position themselves in relation to events considered racist.
Previous research has found that racial classifications are often associated with different perspectives on what counts as racist, with studies showing that, on average, people from racialized groups that experience discrimination are more likely to perceive racism compared to those who are not targeted. However, most of this research relies on statistical averages to compare racial categories, a practice that assumes within-group agreement and treats race categories as if they represent stable, coherent groups.
Joel E. Martinez, a former postdoctoral researcher at Harvard University’s Data Science Initiative, questioned whether this common approach might be reinforcing, rather than merely measuring, race as a real social entity. Drawing on the concept of “racecraft”—which describes how the belief in race as a real category is sustained through social practices and research methods—Martinez hypothesized that average-based analyses might be concealing a more complex reality. Instead of assuming that race categories predict uniform perspectives on racism, the study sought to map the actual variation in perceptions within and across racial categories.
“This paper had a long 8 year history before it was published, so inspirations came and went as I read more and grew as a scholar,” Martinez told PsyPost. “But what kept my interest and motivation going was this gut feeling (which I eventually developed into a research agenda) that the methods we (psychologists) are trained with and the ways of interpreting data and norms around communicating our findings were really just sophisticated ways of creating caricatures of the people we studied. Caricatures draped in the illusion of objectivity given by the authority status of scientific procedures.”
To investigate these questions, Martinez designed a study in which participants evaluated the perceived racism of anti-immigrant tweets. He recruited 368 participants living in the United States, with roughly 100 participants from each of three racial categories: Black, Latino, and White. After excluding participants whose responses were not reliable across repeated measures, the final sample included 306 participants. A second study replicated the findings with a separate sample of 301 participants.
Participants were shown a series of tweets related to two Trump-era policies—the construction of a border wall and the travel ban targeting Muslim-majority countries. These tweets were selected to range from blatantly racist statements (e.g., calls for violence against immigrants) to more coded political rhetoric about immigration and national security. Each participant rated how racist they perceived each tweet to be on a scale from 1 (not at all) to 9 (extremely). Importantly, each participant rated the same tweets multiple times across three different blocks, allowing Martinez to assess the consistency of their perceptions.
In addition to traditional average-based analyses, Martinez used a novel approach called variance component analysis (VCA). This method allowed him to measure the degree of agreement and disagreement among participants, both within and across racial categories.
As expected, Martinez found that there were average differences in how participants from different racial groups rated the tweets. Specifically, participants who identified as White tended to rate the anti-immigrant tweets as less racist compared to those who identified as Black or Latino. This aligns with previous research showing that different racial groups often perceive discrimination at different rates.
However, the core finding of the study was that there was a substantial amount of disagreement about what constituted racism within each racial group. When the researcher used variance component analysis to examine the variation in ratings, he found that the level of disagreement within each racial group was just as high as the disagreement across all participants combined. This means that even though there were average differences between groups, these averages did not reflect a shared or unified viewpoint within each group. In other words, knowing a person’s racial identity did not reliably predict how they would rate the tweets.
“I was trained as a social psychologist, so I actually came into this paper thinking I would find the kind of perceptual patterns theorized or discussed in this field: big racial gaps in perceptions of racism accompanied by relatively more agreement within than between race categories,” Martinez said. “That disagreement (of various kinds) dominated perceptual patterns across and within race categories was a surprising wake up call.”
“The surprise was mostly about how theories can sometimes replace your vision/logic. If I think about what I experienced all throughout my personal life, people who would be categorized as all sorts of different race categories have had all sorts of contradictory beliefs on all sorts of issues. The data I collected were just a continuation of that experience, which left me with a different kind of gap: how do I reconcile this with the theories about racial cognition I was reading in social psychology papers? This is what led me to read widely beyond this field, in search of alternative ways of thinking about the same problem.”
Martinez also explored what factors contributed to agreement and disagreement in the ratings. Agreement across the entire sample was largely driven by the aggressiveness of the language used in the tweets. This suggests that people, regardless of their racial background, tended to agree that more aggressive tweets were more racist.
On the other hand, disagreements were partly explained by individual characteristics such as a person’s level of symbolic racism (a measure of beliefs that racial disparities are due to lack of effort rather than discrimination), social dominance orientation (a preference for hierarchical social structures), and general political ideology.
Another important finding was that a significant portion of the disagreement came from the way individuals ranked specific tweets relative to each other. Two people might have similar average ratings overall, but one might consistently rate certain tweets as more racist than others, while the other person might do the opposite. This highlights the fact that even when people appear to agree on average, there can be substantial differences in how they perceive and interpret specific instances of potential discrimination.
In short, the findings suggest that while race does correlate with perceptions of racism on average, it is not a strong predictor of individual views. There is a great deal of variation within racial groups, highlighting the problem of treating them as distinct entities.
“When people hear ‘race is not real,’ they immediately think you’re advocating for colorblindness – disregarding the reality of racism by rejecting the reality of race,” Martinez told PsyPost. “This is because mainstream race talk seems to be governed by two poles where race and racism are both either real or not real.”
“What my paper is attempting to showcase is the potentials of an alternative stance: that we can take racism seriously while resisting its alluring illusion of race, which functions to make racism’s violence appear as if natural. And if we start from that place, which the paper calls the race anti-realist perspective, then our science starts to look different.
“The commonsense analytic and theorizing practices for studying ‘race’ (especially in social psychology) start to feel wrong because they are deeply connected to a race-realist perspective (aka racecraft, where races are real, even if socially constructed),” Martinez explained. “And that’s what led me to explore and experiment with alternative practices that start by asking ‘what could a race-anti realist social science look like?’ The switch from examining averages (which contain a lot of assumptions about the groupiness and thus realness of race categories) to mapping out variation felt like a good entry point.”
One limitation of the study is that it focused on perceptions of anti-immigrant discourse, which some people may not consider a “racial” topic. Future research could examine whether similar patterns of disagreement are found in contexts that are more explicitly related to race. Additionally, the variance component analysis used in this study requires a specific study design that may not always be practical.
“The variance component analysis is just one tool for mapping variation – and to use it requires a specific study design that isn’t always feasible (multiple people rating multiple stimuli multiple times),” Martinez noted. “So figuring out how to widen the practicality of mapping variation to other kinds of data and study designs will be an incredibly useful step in resisting the analytic essentialization of social categories.”
“I have left academia, so I’m just hoping other researchers can continue this line of thinking, where we don’t take race for granted but instead trace the dynamic processes of racialization and broader forces (such as anti-blackness, capitalism, and sociogeny) that enable us to see and treat each other as racial beings in the first place.”
“One barrier I see is that mainstream social psychology as a discipline is very insular in its reading and citation practices, it barely engages with the humanities,” Martinez added. “In the 8 years of working on this paper, I became radically interdisciplinary, and I purposefully left a citation trail within this paper and my other related papers (this and this) so that others could follow my line(s) of thinking. My hope is that it inspires researchers to read widely and look inward at their own practices and imagine new ones that take seriously their political and societal embeddedness.”
“The point of this paper is not that ‘in-group’ variation exists and is important, but to think about what that variation means for the coherence of the analytic categories (like race) we use to produce knowledge (or caricatures) about the world and the people forced to inhabit/navigate those categories.”
The study, “Analytic Racecraft: Race-Based Averages Create Illusory Group Differences in Perceptions of Racism,” was published October 7, 2024.
