How We Decide (42 page)

* The TD-learning model used by Tesauro was based on the pioneering work of computer scientists Rich Sutton and Andrew Barto. In the early 1980s, when they were grad students at the University of Massachusetts, Amherst, Sutton and Barto tried to develop a model of artificial intelligence that could learn simple rules and behaviors and apply them to achieve a goal. It was an audacious idea; their academic advisers tried to dissuade them from even trying, but the young scientists were stubborn. "It had always been this kind of untouchable goal in computer science," Sutton says. "Marvin Minsky had done his thesis on reinforcement learning and basically given up. He said it was impossible and left the field. Luckily for us, it wasn't impossible. We knew even simple animals could learn like this—nobody teaches a bird how to find a worm—we just didn't know how."

[back]

***

* Slot machines account for about 70 percent of the $48 billion a year Americans spend at casinos, which means that the average citizen spends five times more on slot machines than he or she does on movie tickets. There are now twice as many slot machines as ATMs in America.

[back]

***

* This misconception is known as the gambler's fallacy. It occurs when people assume that an event is more or less likely to occur based on whether or not that event has recently occurred. As a result, people are surprised when a shuffled song repeats or when a flipped coin exhibits extended streaks of heads or tails. The most famous example of such a phenomenon occurred in a Monte Carlo casino in the summer of 1913 when a roulette wheel landed on black
twenty-six
times in a row. During that staggeringly improbable run, most gamblers bet against black, since they felt that the red must be "due." In other words, they assumed that the randomness of the roulette wheel would somehow correct the imbalance and cause the wheel to land on red. The casino ended up making millions of francs.