In the ever-evolving world of online dating, a new study has brought to light the intricacies of matchmaking algorithms used by these platforms. Researchers from Carnegie Mellon University and the University of Washington have uncovered a ‘popularity bias’ in these algorithms, a tendency to recommend more popular and attractive users over their less popular counterparts. The findings of this study were published in Manufacturing & Service Operations Management.
Previous research in the realm of online dating platforms has often focused on user behavior and preferences. However, there’s been a growing interest in understanding how the platforms themselves, through their algorithms, influence matchmaking. This study was motivated by a need to explore the so-called “popularity bias” – a tendency for dating apps to favor users who are deemed more attractive, successful, or engaging. The researchers sought to understand the implications of this bias not only on individual users but also on the overall efficacy of these platforms in creating successful matches.
“Online dating has become the prevalent way for people to find their potential significant others, and most of the research in this area focused on understanding people’s own preferences rather than the platforms/websites that provide this domain in the first place. At the end of the day, it is these platforms’ algorithms that make or break users’ experience in online dating,” said study author Musa Eren Celdir, a senior data scientist at United Airlines who led the study while he was a Ph.D. student at Carnegie Mellon’s Tepper School of Business.
The study was a blend of theoretical modeling and empirical data analysis. Researchers modeled the decision-making process of online dating platforms and user interactions as a three-stage game. This model included two types of users: the ‘popular’, who generally have more options both within the app and in real life, and the ‘unpopular’, who do not attract as much attention. The team aimed to understand how a platform’s pursuit of maximizing revenue or the number of successful matches influenced its recommendations.
To ground their theoretical work in reality, the researchers utilized data from a major online dating platform, involving approximately 243,000 users and over 30 million interactions over a three-month period. This rich dataset included detailed demographics, user preferences, and a record of user decisions such as seeking more information about others, sending messages, and responding to received messages. This comprehensive analysis allowed the researchers to not only validate their theoretical assumptions but also to predict future user behavior and test various recommendation strategies.
Celdir and his colleagues found that the recommendations aimed at maximizing the platform’s revenue and those aimed at maximizing successful matches were not necessarily conflicting goals. However, revenue-maximizing strategies tended to discriminate more against unpopular users. This is because popular users, by boosting engagement through likes and messages, help in revenue generation. Additionally, they contribute to more successful matches as long as they don’t become overly selective and thus unapproachable to less popular users.
“Our work contributes to the research on online matching platforms by studying fairness and bias in recommendation systems and by building a new predictive model to estimate users’ decisions,” said Elina H. Hwang, an associate professor at the University of Washington’s Foster School of Business, who also co-authored the study. “Although we focused on a specific dating platform, our model and analysis can be applied to other matching platforms, where the platform makes recommendations to its users and users have different characteristics.”
A significant gender difference was also observed. The data showed that popular female users were more selective than their unpopular counterparts, leading to less bias against unpopular female users in match-maximizing recommendations. However, both revenue-focused and match-focused recommendations showed a similar level of bias against unpopular male users.
Another intriguing finding is the ‘congestion effect’ – when a user receives numerous messages and faces significant effort to screen them. The study found that in scenarios with lower congestion, unbiased recommendations led to fewer messages and matches compared to biased ones. However, as the congestion effect increased, both revenue-maximizing and match-maximizing recommendations began to include both popular and unpopular users more equally.
“I think there are two key takeaways: 1) Even though some online dating platforms claim they employ highly-sophisticated algorithms for their users to find the best matches, their algorithms are susceptible to simple biases. 2) For online dating platforms, users’ interactions with others (sending likes/messages etc.) are very important in recommending new users. Therefore, users who are mindful in showing interest to others are more likely to find good matches in the future,” Celdir told PsyPost.
Despite its in-depth analysis, the study isn’t without limitations. One significant constraint is its reliance on data from just one online dating platform, which may not capture the full spectrum of user behavior across different platforms. Furthermore, the model used, while robust, still simplifies the complex nature of human interactions and preferences. Future research could expand on these findings by exploring a variety of platforms, incorporating longitudinal data to understand changes over time, and delving deeper into the psychological aspects of user interaction in online dating contexts.
The study, “Popularity Bias in Online Dating Platforms: Theory and Empirical Evidence“, was authored by Musa Eren Celdir, Soo-Haeng Cho, and Elina H. Hwang.
