The Invisible Gorilla: And Other Ways Our Intuitions Deceive Us (30 page)

Subsequent research reports from the Mozart effect team also were covered extensively in the press. Just like the original, these new experiments
found dramatic improvements in mental task performance immediately after the Mozart sonata, but not after silence or relaxation.
¹²
Meanwhile, psychologists interested in music and cognition began to examine this discovery, which was intriguing because no previous research had shown that merely listening to music could have such a large effect on mental ability.

The first independent research group to publish its findings was headed by Con Stough of the University of Auckland in New Zealand.
¹³
They used the same Mozart sonata and silence conditions as in the original study, and added a new one: dance music, specifically ten minutes of “Fake 88 (House Mix)” and “What Can I Say to Make You Love Me? (Hateful Club Mix)” by Alexander O’Neal. Thirty subjects participated in each listening condition and worked on part of the Raven’s Advanced Progressive Matrices test after each one. This test is considered an excellent measure of general intelligence. Stough’s team found that the Mozart group outperformed the control groups by only about one IQ point, not even close to the eight to nine points reported by Rauscher. A one-point difference is small enough that it could easily have arisen just from the random variations in the measures of cognitive abilities, or from accidental differences among the subjects assigned to the Mozart and control groups. Other researchers reported similar experiences.
¹⁴

Along with two of his students, Kenneth Steele, a psychology professor at Appalachian State University in North Carolina, tried a Mozart experiment in 1997. They used a “digit span” test, which measures the longest list of digits that you can hold in short-term memory accurately enough to repeat it back, either forward or backward. This test is strongly associated with general intelligence: the smarter you are, the longer your backward digit span. But listening to Mozart had no effect on digit span. Steele tried again the next year, this time copying the design of Rauscher and Shaw’s 1995 follow-up study, which had also produced a large Mozart effect. Steele used the paper-folding task rather than digit span, but again he found no benefits of Mozart.
¹⁵
The next year the American Psychological Society’s flagship journal,
Psychological Science
,
published these new results under the title “Mystery of the Mozart Effect: Failure to Replicate,” and the society issued a press release headlined “‘Mozart Effect’ De-Bunked.” Almost immediately, the headline was changed to “‘Mozart Effect’ Challenged” after Gordon Shaw threatened the APS with a lawsuit.
¹⁶

Steele wrote later that when he started his experiments, he expected to replicate the Mozart effect.
¹⁷
Indeed, researchers rarely conduct experiments that they think will fail! Experiments can fail for many reasons even when the theory that motivated them is correct. In this case, the theory that listening to Mozart increases cognitive performance could be true, but any particular experiment intended to test the theory could fail to support it because of a variety of errors in design or execution, none of which have anything to do with the correctness of the theory. But after repeated failures to find any cognitive improvement after listening to Mozart, Steele came to believe that there was no Mozart effect to be found.

The studies by Stough, Steele, and others received little notice, but the publications of the original discoverers continued to influence public perceptions and even public policy—Rauscher even testified about her findings before a committee of the U.S. Congress. The media gives tremendous weight and coverage to the
first
study published on a research question, and essentially ignores all of those that come later. This bias is unsurprising—fame goes to the discoverer, not to the person who got there a few months later, or who just followed up on the original work. But even in science, the judgment of greatness is a retrospective one that only history can render, and journalism is well known to be only the first draft of history. When a new finding is announced, journalists and other observers might be hard-pressed to say, “I won’t report this story until I see it replicated by at least two other laboratories.” And restraint is all the less likely when the impact might be as great as nine IQ points in ten minutes. The first report of a new scientific
finding is analogous to the front-page coverage granted to a high-profile criminal indictment; the news that the results didn’t hold up winds up in the back pages (if it is covered at all), next to the story about the suspect’s eventual exoneration.

As the Mozart effect story evolved, it became even more fantastical. Even though all of the relevant studies had been conducted with college students or adults, the legend spread that Mozart was great for children, babies, and even fetuses. A Chinese newspaper columnist wrote, “According to studies conducted in the West, babies who hear Cosi Fan Tutte or the Mass in C Minor during gestation are likely to come out of the womb smarter than their peers.”
¹⁸

Social psychologists Adrian Bangerter and Chip Heath measured the news coverage devoted to the initial Rauscher-Shaw study and found that in 1993, the year of its publication, it received plenty of media attention, but no more than the other widely covered research studies published in
Nature
around the same time. (These concerned topics like schizophrenia, the orbit of Pluto, skin cancer, and even how many sexual partners men and women claim to have.) In the ensuing eight years, though, the Mozart effect paper received more than ten times as much coverage as those studies. The media’s interest in the others diminished sharply after the initial reports, but coverage of the Mozart effect only grew.
¹⁹

Chris’s interest in the Mozart effect was piqued in early 1998 when he was writing an article about the concept of intelligence. The enthusiastic public reaction to the Mozart effect stems partly from the way that the concept of intelligence is presented in the media. Intelligence tests are thought by many to be a simplistic, arbitrary, inaccurate, and even racist way of understanding human cognition.
²⁰
What better way to debunk IQ tests than to show that just listening to a few minutes of music can dramatically change your score? The reception of the Mozart effect among experts on cognition was different. Chris noticed that the failures to replicate the original Rauscher, Shaw, and Ky finding were piling up, and that almost all of the successful replications came from the original team, not from independent researchers. In science, whenever just one or a few labs can produce an effect, and others cannot (as
in the celebrated case of cold fusion), scientists and skeptics begin to doubt the effect itself. Was the Mozart effect real, or just a myth?

Chris decided to conduct a meta-analysis, a statistical procedure that combines all of the available data from all of the studies on a research question to determine the best answer. The value of meta-analysis can perhaps be best understood by analogy to the classic carnival game of guessing the number of jelly beans in a jar that we discussed in
Chapter 3
. If you have a large group of people who want to come up with their best collective estimate of an unknown quantity, the way to do it is to have everyone make his or her guess privately, and then average together all the guesses. Each person’s guess is unlikely to be right, but it is equally likely to be too high or too low. As a result, if you average all of the independent guesses, the estimates that are too large will cancel out the ones that are too small, and you will end up with a more accurate estimate of the actual total.
²¹

The same principle applies to scientific research. Any individual study might be affected by inadvertent biases or errors that distort its results, leading to an imprecise estimate of the true effect (here, how much your IQ increases after listening to Mozart). By averaging across a number of studies, though, any random errors that led to over-or underestimates of the size of an effect will tend to average out, leaving a better estimate of the truth. Because they are based on
all
of the relevant studies, the results of a meta-analysis are not unduly influenced by a single memorable or well-publicized finding, such as the original Rauscher-Shaw article.

After scouring scientific journals for experiments like the original one, Chris noticed that—aside from Steele’s article in
Psychological Science
—all of the followup studies were published in journals that most researchers never read, and many have never even heard of. He wrote to the authors of many of the articles to request additional data or information he needed to evaluate their results. In total, he found sixteen experiments (including the original) that tested the Mozart effect and were published in peer-reviewed scientific journals. All of them used the same sonata and compared it with silence, relaxation, or both. For
each experiment, Chris calculated the size of the difference in performance between those subjects who had listened to Mozart and those who had not. When compared with silence, Mozart improved performance by the equivalent of 1.4 IQ points, only one-sixth as much as the Rauscher-Shaw team had found. For experiments comparing the sonata with relaxation, the advantage for Mozart turned out to be three IQ points, about a third as much as the original article reported, but still twice as large as in the comparison between Mozart and silence. There may be good reason for this small benefit: Relaxation reduces anxiety and arousal, but being in a “laid-back” state is not ideal for solving difficult problems on IQ tests. Nor is being excessively anxious, of course—a happy medium is best. Compared with relaxation, sitting in silence likely has a similar, but weaker effect—without external stimulation, your mind may wander, making you less prepared for hard work.

Chris concluded that the entire “Mozart effect” might have nothing to do with a positive effect of listening to music. Rather than Mozart making you smarter, sitting in silence or getting relaxed might make you dumber! Viewed this way, Mozart’s music is a control condition that resembles the general level of mental stimulation we encounter during everyday life, and silence and relaxation are “treatments” that reduce cognitive performance. In either case, though, there is little or no Mozart effect to explain.

Several additional studies could not be included in Chris’s meta-analysis because they did not include the relaxation or silence control conditions. However, they did reveal another possible explanation for the apparent benefit of Mozart. In one, British researcher Susan Hallam arranged for the BBC to conduct a massive experiment on
eight thousand
children in two hundred schools around the United Kingdom. The children listened to either a Mozart string quintet, a discussion about scientific experiments, or three popular songs (“Country House” by Blur, “Return of the Mack” by Mark Morrison, and “Stepping Stone” by PJ and Duncan), and then performed cognitive tests like those originally used by Rauscher. The children who listened to popular music did the best, and there was no difference in performance between those who
listened to Mozart and those who heard the science discussion. An article on this finding cheekily dubbed it the “Blur Effect.”
²²

A second study by Kristin Nantais and Glenn Schellenberg of the University of Toronto found no overall difference in cognitive task performance after listening to the Mozart sonata or the short story “The Last Rung on the Ladder” by Stephen King. But subjects did do better after listening to what they
liked
best.
²³
The most sensible explanation for this finding, as well as for the “Blur Effect,” is that your mood improves when you hear what you like, and you do modestly better on IQ tests when you are in a better mood. The effect has nothing to do with increasing your intelligence per se.

Chris submitted his meta-analysis to
Nature
, the journal that published the initial 1993 article. He did not expect the editors to accept it, because its conclusion—that any small benefits that do exist result from arousal and positive mood rather than any special property of Mozart’s music—could be interpreted as questioning the journal’s decision to publish the first paper. To his surprise and delight, they accepted the paper and published it in August 1999 alongside another report of a failure to replicate by Kenneth Steele and his colleagues. Rauscher was given space to reply, and
Nature
highlighted the exchange in its weekly press bulletin. The media, loving a good fight, even among staid academics, sprang into action: Chris was interviewed for CNN, CBS, and NBC news programs. Rauscher and Steele debated on the
Today
show, with Matt Lauer as referee. Chris’s article even earned him a short appearance on an episode of
Penn and Teller: Bullshit!
entitled, charmingly, “Baby Bullshit.”

Recall the media analysis done by Adrian Bangerter and Chip Heath. They found a spike in coverage of the Mozart effect in 1999, coincident with these articles in
Nature
, and then things died down again. Did Chris’s meta-analysis, and the studies by Steele and Schellenberg, finally debunk the Mozart effect? Yes and no. Bangerter and Heath found that news articles mentioning the positive effect of listening to Mozart for adults became less and less frequent, but that articles falsely claiming that Mozart made babies smarter became more common! Indeed, this
trend started just one year after the original Rauscher-Shaw report. To be clear, we repeat that no published studies had ever examined the effect with babies!
²⁴
Our national survey of fifteen hundred adults was conducted in 2009, ten years after Chris’s meta-analysis was published. It found that 40 percent of people agreed that “listening to music by Mozart will increase your intelligence.” A majority disagreed, but keep in mind that the scientific evidence does not support this claim at all. It would be better if almost everybody disagreed, as they would with a statement like “on average, women are taller than men.”