Traditional tests of intelligence and literacy may be fundamentally flawed because they force test-takers to choose a single answer rather than allowing them to express their level of confidence in different options. When people are given financial incentives and allowed to distribute their answers based on how sure they are, women actually score higher than men. The research was published in the Journal of Political Economy.
For decades, psychologists and economists have measured cognitive ability using multiple-choice tests. These assessments score responses as strictly right or wrong. Glenn W. Harrison of Georgia State University, Don Ross of University College Cork, and J. Todd Swarthout of Georgia State University suspected this format misses a vital component of human cognition. Knowing how strongly to believe in an answer is a skill in itself.
The researchers note that the standard format forces people to mask their thought processes. If someone is somewhat confident in an answer but still perceives some risk of being wrong, the rigid format does not capture that nuance. The test format demands absolute certainty even when a person possesses healthy skepticism.
To address this, the team examined the Raven Advanced Progressive Matrices test. This assessment presents a grid of shapes with one missing piece and asks the test-taker to identify the pattern. It is widely used to measure fluid intelligence, which is the ability to solve new logic problems without relying on prior knowledge.
The researchers wrote that, “The measurement of intelligence should identify and measure an individual’s subjective confidence that a response to a test question is correct.” They noted that existing tests completely fail to achieve this goal.
The standard version of this puzzle allows test-takers unlimited time and offers no financial motivation. The researchers created a computerized version that offered monetary rewards for correct answers. They divided participants into different groups to test how the structure of the task changed their performance.
In the baseline group, participants took a traditional version for a flat fee of five dollars. In another group, participants were paid based on their accuracy but were still forced to pick just one answer. A third group experienced a radically different test structure.
These participants were given eighty digital tokens to allocate across eight possible answers. If they were completely sure, they could place all eighty tokens on a single choice for a maximum reward of two dollars per puzzle. If they were unsure, they could spread their tokens out over multiple likely answers to guarantee a smaller payout.
This token system measures what the researchers refer to as confidence. In this context, confidence does not mean optimism. It refers to the precision of a person’s belief. A person who places ten tokens on every single answer is safely guarding against risk because they have no idea which shape is correct.
When financial incentives were combined with the ability to express varying degrees of confidence, the results shifted dramatically. In the traditional format, female participants scored lower than male participants. When participants could assign tokens based on their confidence, women outperformed men.
The data showed that female participants were better at calculating the risk of their answers and distributing their tokens efficiently. Knowing when you are unsure is a core part of cognition. The researchers consider this risk assessment to be a fundamental element of fluid intelligence.
The researchers also altered the order of the puzzles. The standard test starts with easy puzzles and gradually progresses to difficult ones. The researchers call this sequence a structured progression, meaning it is an environmental clue that helps a person think.
When the researchers scrambled the order of the puzzles so that difficulty varied randomly, overall performance dropped. The gap in performance between the group forced to pick one answer and the group allowed to use tokens widened even further. This confirmed that the ability to express uncertainty is a distinct cognitive advantage when facing unpredictable problems.
This discovery regarding gender prompted the researchers to revisit other areas where men possess a supposed advantage. They looked at studies regarding competitiveness. Past behavioral studies suggest that women back away from competitive environments, such as workplace tournaments, in favor of flat payment schedules.
The researchers recreated these experiments using the token system and discovered that women were making the mathematically correct risk management choices. Participants had to solve logic problems under a time limit, choosing either a guaranteed payment per correct answer or a tournament style where only the top performer received a large payout.
Men tended to choose the competitive tournament even when it resulted in a monetary loss for them. Men proved to be overly optimistic about their chances of winning. Women evaluated the risk accurately and chose the safer compensation structure, which resulted in better financial outcomes.
The team also looked at financial literacy tests. Standard surveys report that women choose the “do not know” option much more often than men when asked financial questions. This has led to the assumption that women possess lower financial literacy.
The researchers presented participants with a standard question about calculating purchasing power based on interest and inflation rates. When the researchers allowed subjects to use tokens to answer the question, they found that women were just more open about their lack of complete certainty. The bias in their actual knowledge was tiny and not statistically significant.
Many women distributed their tokens broadly, meaning they were aware that they lacked the exact knowledge and guarded their bets accordingly. This behavior signals an intellectual awareness of uncertainty. Someone who knows they are guessing is more likely to seek out a financial advisor or a textbook to learn the correct answer.
Individuals who place all their tokens on a highly incorrect answer represent a much larger danger. The researchers noted that these individuals are completely confident in their incorrect knowledge. These are the people most likely to make catastrophic financial decisions without consulting outside help.
The authors specify that their findings on motivation might involve variables that are difficult to isolate. Participants might bring personal motivations into the laboratory that interact with the monetary incentives offered by the experimenters.
Future studies could attempt to separate these personal drives from the financial rewards to see how they impact token distribution. The research team also plans to further investigate data suggesting that Black participants similarly perform drastically better when allowed to express their confidence through the token system.
The study, “Gender, Confidence, and the Mismeasure of Intelligence, Competitiveness, and Literacy,” was authored by Glenn W. Harrison, Don Ross, and J. Todd Swarthout.
