The Invisible Gorilla: And Other Ways Our Intuitions Deceive Us (14 page)

In July 1998, the average U.S. Chess Federation rating of the 27,562 people who’d played at least twenty tournament games was 1337. Masters are players rated 2200 or higher. Chris achieved this milestone when he was in college. Dan had a rating just under 1800 in high school but hasn’t played competitively since then. Comparing two players’ ratings gives the odds that one will defeat the other. Ratings are set and adjusted so that over a long series of games, a player rated two hundred points higher than his opponent should score about 75 percent of the points (wins count as one point, draws count as half a point). A player rated four hundred points higher than his or her opponent is expected to win almost every game.

Despite playing hundreds of tournament games in high school and being well above average for a tournament player, Dan never beat a master-level player, and he would stand effectively no chance of beating Chris in a tournament game. Similarly, Chris has defeated only one grandmaster in tournament play, despite once being ranked among the top 2 percent of players nationally. The differences in skill between these levels are just too large. If you consistently beat a player with the same rating as yours, then your rating will go up and theirs will go down
so that the forecasts predict that you will beat them in the future. Unlike the rankings published for most sports, the chess rating system is extremely accurate; for practical purposes, your rating is a nearly perfect indicator of your ability. Armed with knowledge of their own ratings, and the workings of the rating system, players ought to be exquisitely aware of how competent they are. But what do they actually think about their own abilities?

Together with our friend Dan Benjamin, who was then an undergraduate student at Harvard and is now an economics professor at Cornell University, we ran an experiment at the World Open in Philadelphia and at another tournament, the U.S. Amateur Team Championship in Parsippany, New Jersey. As players walked by on their way to or from their games we asked them to fill out a short questionnaire. We posed two simple questions: “What is your most recent official chess rating?” and “What do you think your rating should be to reflect your true current strength?”
²

As expected, the players knew their ratings: Half reported their ratings exactly, and most of the rest were off by only a few points. Since the players know what they are rated, they ought to be able to correctly answer the second question about what they
should
be rated. The correct answer
is
their current rating, because the rating system’s design ensures that ratings are an accurate measure of skill. But only 21 percent of the players in our experiment actually said their current rating reflected their true strength. About 4 percent thought they were overrated, and the other 75 percent believed they were underrated. The magnitude of their overconfidence in their own playing ability was stunning: On average, these competitive chess players thought they were ninety-nine points underrated, which means they believed that they would win a match against another player
with the exact same rating as their own
by a two-to-one margin—a crushing victory. Of course, in reality the most likely outcome of a match against a player with the same rating as their own would be a tie.

What explains this extreme overconfidence in the face of concrete evidence for their actual skills? Not a lack of familiarity with chess:
These players had played the game for an average of twenty years. Not a lack of feedback about their competitive skill levels: They had been playing in rated tournaments for thirteen years, and their average rating was 1751, well above the average player’s. Not being out of touch with their own skill level (from being out of practice): Over half had played at least one other tournament within the two months before we surveyed them.

Perhaps the players interpreted our question slightly differently than we had intended. Maybe they were predicting what their ratings
would be
once the system caught up to their true strength. Because ratings are adjusted only after tournaments, and the updated ratings sometimes take a month or two to be published, it is possible for rapidly improving players to be systematically underrated in the official lists, because they are improving at a rate too fast for their ratings to keep up. We checked our subjects’ ratings a year later, and the players were rated almost exactly as they had been when we first did the experiment: one hundred points lower than their own estimates of their skill. In fact, even after five years, they still hadn’t reached the levels they had estimated as their actual strength. The overconfidence that players displayed cannot be explained by a reasonable expectation of future improvement.
³
Our tournament chess players, despite their long and intimate experience with competitive chess ratings, overestimated their abilities. They suffered from our third everyday illusion: the
illusion of confidence
.

The illusion of confidence has two distinct but related aspects. First, as with the chess players, it causes us to overestimate our own qualities, especially our abilities relative to other people. Second, as Chris experienced in the doctor’s office, it causes us to interpret the confidence—or lack thereof—that other people express as a valid signal of their own abilities, of the extent of their knowledge, and of the accuracy of their memories. This wouldn’t be a problem if confidence in fact had a close relationship with these things, but the reality is that confidence and ability can diverge so far that relying on the former becomes a gigantic mental trap, with potentially disastrous consequences. Thinking you’re better at chess than you really are is only the beginning.

Charles Darwin observed that “ignorance more frequently begets confidence than does knowledge.”
⁴
In fact, those who are the least skilled are the most likely to think better of themselves than they should—they disproportionately experience the illusion of confidence. Some of the most striking examples of this principle come from criminals, an idea captured in Woody Allen’s first feature film,
Take the Money and Run.
⁵
Allen stars as Virgil Starkwell, a boy raised in difficult circumstances who turns to a life of crime as a teenager. But Virgil never manages to achieve success in his profession. As a child he tries to steal gumballs but gets his hand stuck and has to run down the street carrying the entire machine. As an adult he tries to rob a bank, but the tellers can’t read his holdup note and the police arrive before he can explain it to them. He tries to break out of jail by carving a gun out of soap and coating it with black shoe polish, but as he leaves, it pours rain and the guards notice suds frothing from his weapon.

Stupid criminals are a staple of film and television comedy in part because they violate the stereotype of the criminal mastermind—the genius-turned-psychopath James Bond villain. But this stereotype is not representative of actual criminals, at least not those who get caught. Smut Brown, the murder suspect whom Kenneth Conley chased down in Boston, was a high school dropout who was arrested eight times in a single year.
⁶
People convicted of crimes are, on average, less intelligent than noncriminals.
⁷
And they can be spectacularly foolish. A high school classmate of Dan’s decided to vandalize the school—by spray-painting his own initials on the back wall. A Briton named Peter Addison went one step further and vandalized the side of a building by writing “Peter Addison was here.” Sixty-six-year-old Samuel Porter tried to pass a one-million-dollar bill at a supermarket in the United States and became irate when the cashier wouldn’t make change for him.

In a brilliant article entitled “Unskilled and Unaware of It,” social psychologists Justin Kruger and David Dunning of Cornell University tell the story of McArthur Wheeler, who robbed two banks in
Pittsburgh in 1995 without using a disguise.
⁸
Security camera footage of him was broadcast on the evening news the same day as the robberies, and he was arrested an hour later. According to Kruger and Dunning, “When police later showed him the surveillance tapes, Mr. Wheeler stared in incredulity. ‘But I wore the juice,’ he mumbled. Apparently, Mr. Wheeler was under the impression that rubbing one’s face with lemon juice”—a substance used by generations of children to write hidden messages—“rendered it invisible to videotape cameras.”
⁹

Kruger and Dunning wondered whether Wheeler’s combination of incompetence and obliviousness was unusual (perhaps a profile peculiar to failed criminals) or whether it might be a more general phenomenon. In their first experiment, they zeroed in not on criminal ability, which is (we hope) uncommon, but on a quality that most people believe they possess: a sense of humor. They asked whether people who are bad at understanding which jokes are funny and which are not mistakenly believe they have a perfectly good sense of humor. But how to measure sense of humor?

Unlike chess, there is no rating system for sense of humor, but one clear lesson of the past century of psychology research is that almost any quality can be measured well enough to be studied scientifically. We don’t mean to say that it’s easy to capture the ineffable qualities that make something funny. If it were, then someone with no sense of humor could write a computer program to generate good jokes. What we mean is that people are remarkably consistent in judging what’s funny and what’s groan-worthy. The same is true for many other seemingly immeasurable qualities. You might think that beauty is in the eye of the beholder, but it isn’t—when asked to judge the attractiveness of a set of faces, people give remarkably consistent ratings despite individual differences in tastes and preferences. This is the reason why most people will never be models.
¹⁰

To create their sense-of-humor test, Kruger and Dunning selected thirty jokes written by Woody Allen, Al Franken, Jack Handey, and Jeff Rovin, and e-mailed them to professional comedians, eight of whom agreed to rate how funny the jokes were. Kruger and Dunning
had them use a funniness scale that ranged from 1 to 11, with 1 meaning “not at all funny” and 11 meaning “very funny.” You can test your own sense of humor right now. Decide which of these two jokes is funnier:

Question: What is as big as a man, but weighs nothing? Answer: His shadow.


If a kid asks where rain comes from, I think a cute thing to tell him is “God is crying.” And if he asks why God is crying, another cute thing to tell him is “Probably because of something you did.”

The experts generally agreed about which jokes were funny and which were not. That’s not surprising, considering that expert comedians succeed as comedians because they know what most people will find humorous. The first joke listed above received the lowest rating (1.3) of the thirty that were tested, and the second one, from Jack Handey’s “Deep Thoughts” on
Saturday Night Live
, received the highest rating (9.6). Kruger and Dunning then asked undergraduate students at Cornell to rate the same jokes. The idea was that people with a good sense of humor would rate the jokes similarly to the professional funny people, but people with a bad sense of humor would rate them differently. The top scorers agreed with the comedians 78 percent of the time on whether or not a joke was funny. The bottom scorers—those in the bottom quarter of the subjects on the sense-of-humor test—actually
disagreed
with the comedians about whether a joke was funny more often than they agreed with them. They thought only 44 percent of the funny jokes were funny, but that 56 percent of the unfunny jokes were.
¹¹

Next, Kruger and Dunning asked their subjects to assess their own “ability to recognize what’s funny” by writing down the percentage of other Cornell students they thought were worse than themselves in this skill. The average student is, by definition, better than 50 percent of other students. But 66 percent of the subjects thought they had a better sense of humor than most of their peers.
¹²
Where did that sixteen-percentage-point
overconfidence effect come from? Almost exclusively from those participants with the worst sense of humor! People who scored in the lowest 25 percent on the sense-of-humor test thought they had an above-average sense of humor.

The same pattern held in our study of chess players who thought they should have been more highly rated than they actually were. The players who considered themselves most underrated were disproportionately found in the
bottom half
of the ability range. On average, these weaker players thought they were underrated by 150 points, whereas the players in the top half in ability claimed to be only 50 points underrated.
¹³
Stronger players thus were somewhat overconfident, but weaker players were extremely overconfident.

These findings help to explain why competitive reality shows like
America’s Got Talent
and
American Idol
attract so many people who audition but have no hope of qualifying, let alone winning. Many are just trying to get a few seconds of TV time, but some, like William Hung with his famously awful rendition of Ricky Martin’s “She Bangs,” seem to believe that they’re much more skilled than they actually are.

In other experiments, Kruger and Dunning showed that this unskilled-and-unaware effect can be measured in many areas besides humor, including logical reasoning and English grammar skills. It probably applies to any area of human experience. Whether in real life or on the television comedy
The Office
, we have all encountered obliviously incompetent managers. People who graduate last in their medical school classes are still doctors—and probably think they are pretty good ones.

Aside from showing that the depth of a stupid criminal’s plight can be quantified, can psychology offer any help to the McArthur Wheelers of the world? The answer to this depends on the source of their problem. The incompetent face two significant hurdles. First, they are below average in ability. Second, since they don’t realize that they are below average, they are unlikely to take steps to improve their ability. McArthur Wheeler didn’t know that he needed to become a better criminal before taking on the challenge of robbing banks. But what kept him from that
realization? Why couldn’t he imagine executing his plan for robbing a bank and realize that he didn’t fully grasp everything involved? Why didn’t he question his own competence?