The Most Human Human (27 page)

Add to all the above the fact that the Turing test is, at the end of the day, a race against the clock. A five-second Turing test would be an obvious win for the machines: the judges, barely able even to say “hello,” simply wouldn’t be able to get enough data from their respondents to make any kind of judgment. A five-hour one would be an obvious win for the humans. The time limit at the Loebner Prize contest has fluctuated since its inception, but in recent years has settled on Turing’s original prescription of five minutes: around the point where conversation starts to get interesting.

Part of what I needed to do was simply to make as much engagement happen in those minutes as I physically and mentally could. Against the terseness of the deponent I offered the prolixity and logorrhea of the author. In other words, I talked a
lot
. I only stopped typing when to keep going would have seemed blatantly impolite or blatantly suspicious. The rest of the time, my fingers were moving.

If you look at Dave’s transcripts, he warms up later on, but starts off like he’s on the receiving end of a deposition, answering in a kind of minimal staccato:

Like a
good
deponent, he lets the questioner do all the work
⁷
—whereas I went out of my way to violate that maxim of “A bore is a man who, being asked ‘How are you?’ starts telling you how he is.” (And I might add: “And doesn’t stop until you cut him off.”)

When I saw how stiff Dave was being, I confess I felt a certain confidence—I, in my role as the world’s worst deponent, was perhaps in fairly good shape as far as the Most Human Human award was concerned.

This confidence lasted approximately sixty seconds, or enough time to glance to my other side and see what Doug and
his
judge had been saying.

In 2008, London
Times
reporter Will Pavia misjudged a human as a computer (and thus voted the computer in the other window a
human) when a confederate responded “Sorry don’t know her” to a question about Sarah Palin—to which he incredulously replied, “How can you possibly not know her? What have you been doing for the last two months?” Another judge that year opened his conversations with a question about the “Turner Prize shortlist,” the annual award to a contemporary British visual artist, with similarly hit-or-miss results: Most Human Computer winner Elbot didn’t seem to engage the question—

—but neither, really, did the confederate in that round:

Runner-up for 2008’s Most Human Computer was the chatbot “Eugene Goostman,” which pretended to be an
immigrant
, a non-native speaker of English with an occasionally shaky command of the language:

Is this cheating, or merely clever? Certainly it’s true that if language is the judge’s sole means of determining which of his correspondents is which, then any limitations in language use become limitations in the judge’s overall ability to conduct the test. There’s a joke that goes around in AI circles about a program that models catatonic patients, and—by saying nothing—perfectly imitates them in the Turing test. What the joke illustrates, though, is that seemingly the less fluency between the parties, the less successful the Turing test will be.

What, exactly, does “fluency” mean, though? Certainly, to put a human who only speaks Russian in a Turing test with all English speakers would be against the spirit of the test. What about dialects, though? What exactly counts as a “language”? Is a Turing test peopled by English speakers from around the globe easier on the computers than one peopled by English speakers raised in the same country? Ought we to consider, beyond national differences, demographic ones? And where—as I imagine faltering against a British judge’s cricket slang—do we draw the line between
language
and
culture
?

It all gets a bit murky, and because in the Turing test all routes to and from intelligence pass through language, these become critical questions.

All of a sudden I recalled a comment that Dave Ackley had made to me on the phone, seemingly offhand. “I really have no idea how I would do as a confederate,” he said. “It’s a little bit of a crapshoot whether the judges are your kind of people.” He’s right: if language is the medium with which we confederates must prove ourselves to the judges, then there are any number of things that can aid or impair it, from shared interests or reference points, to generational gaps, to nuances of allusion and slang.

Among the four confederates, Dave and I are Americans, Doug is Canadian, and Olga is a Russian-born South African. Among the four judges, two are English, one is an American expatriate to England, and one is a Canadian. I had read logs of Loebner Prizes past and
had seen the problems that arise when cultural mismatch or cultural disfluency rears its head.

I wondered: Would any such cultural issues come to bear in 2009? All my preparations, my investigations, all the good advice I’d gotten from lawyers, linguists, researchers, and interviewers, wilted compared to actually having something in common and hitting it off with someone. To “speaking the same language,” however literally or figuratively. Would that play in this year?

I didn’t have to wait long for my answer; any uncertainty I’d had on that score, not to mention the optimism I’d begun to feel about my own chances, faded fast when I glanced at Doug’s terminal:

Doug and his judge had just discovered that they were both from Canada. And they started to let rip with abbreviations and nicknames and slang and local references.
And they started to talk about hockey
.

I was in trouble.

As I read through the Loebner Prize transcripts, reams (literally) of small talk and chitchat and “how’s it going?” and basic world-knowledge questions, I was struck by the following dialogue from the 1997 contest:

I have to say, this is up there as one of the most trenchant, coherent, focused, and substantive conversations in the entire annals of the Turing test.

Which is why it’s so surprising that Catherine is a computer.

Specifically, she’s a program called Converse, written by science author, entrepreneur, and International Computer Games Association president David Levy, along with University of Sheffield AI professor (now emeritus) Yorick Wilks. This conversation really shook me—how did they do that? It shook me, that is, until I read the
other
judges’ conversations with “Catherine.” Observe: