Children are willing to work hard, really hard, to win the robots’ affection. They dance for the robots and sing favorite childhood songs: “The Farmer in the Dell,” “Happy Birthday,” “Three Blind Mice.” They try to make the robots happy with stuffed animals and improvised games. One ten-year-old boy makes clay treats for Kismet to eat and tells us that he is going “to take care of it and protect it against all evil.” But because Cog and Kismet cannot like or dislike, children’s complicity is required to give the impression that there is an emerging fondness. Things can get tense. These more sophisticated robots seem to promise more intimacy than their simpler “cousins.” So when they do not gratify, they seem more “withholding.”
During our study Cog has a broken arm, and Kismet is being modified for research purposes. On many days both robots are “buggy.” Children work gamely around these limitations. So, on a day when there are problems with Kismet’s microphone, some children try out the idea that Kismet is having trouble talking because it speaks a foreign language. A five-year-old decides that this language is Korean, his own language. A twelve-year-old argues for French, then changes her mind and decides on Spanish. When Kismet finally does speak to her, she is pleased. She says that she was right about the Spanish. “He trusts me,” she says happily, and bids the robot good-bye with a wave and an
adios
. Of course, children are sometimes exhausted by a robot’s quirky malfunctions or made anxious when attempts to charm a broken machine fail. There are disappointments and even tears. And yet, the children persevere. The robots are alive enough to keep them wanting more.
As we saw with simpler robots, the children’s attachments speak not simply to what the robots offer but to what children are missing. Many children in this study seem to lack what they need most: parents who attend to them and a sense of being important. Children imagine sociable machines as substitutes for the people missing in their lives. When the machines fail, it is sometimes a moment to revisit past losses. What we ask of robots shows us what we need.
BUILDING A “THOU” THROUGH THE BODY
When children realize that Cog will not speak, they do not easily give up on a feeling that it should. Some theorize that it is deaf. Several of the children have learned a bit of American Sign Language at school and seize on it as a way to communicate. They do not question the idea that Cog has things it wants to say and that they would be interested to hear.
When Allegra, nine, meets Cog, she reaches out to shake its hand. Cog returns her gesture, and they have a moment when their eyes and hands lock. Allegra then wants to know if it is possible to make a mouth for Cog. The robot has a mouth, but Allegra means a mouth that can speak. Like the five-year-old who thought that a Furby should have arms “because it might want to hug me,” Allegra explains that Cog “probably wants to talk to other people . . . and it might want to smile.” Allegra also thinks that an “improved” Cog should know how to dance. Scassellati asks, “Should it just dance for you or should it be able to dance with you?” Allegra’s answer is immediate: “Dance with me!” Inspired, she begins to dance, first hip-hop, then the slow and graceful turns of ballet. In response, Cog moves its head and its one functional arm. Robot and child are bound together. After a few minutes, Allegra says, “If his [Cog’s] other arm could move, I think that I would teach him to hug me.” Cog has become alive enough to love her. Later, Allegra makes her dance steps more complex and rapid. Now she dances not with but for Cog. She wants to please it, and she says, “a little bit I want to show off for him.”
Brooke, seven, comes to her session with Cog hoping that it has “a heart . . . and tonsils” so that it will be able to talk and sing with her. When this doesn’t work out, she moves on to teaching Cog to balance its toys—stuffed animals, a slinky, blocks—on its arms, shoulders, and neck. When things go awry, as they often do (Cog can rarely balance the toys), she gently chides the robot: “Are you paying attention to me, mister?” She says that Cog’s failures are perhaps due to her not having identified its favorite toy, and she remains Cog’s dedicated tutor. Cog finally succeeds in balancing its slinky and this reanimates the robot in her eyes. When Cog fails in successive attempts, Brooke assumes it has lost interest in her game. She asks it, “What’s the matter?” She never questions her pupil’s competency, only its desire.
But Brooke yearns to talk to the robot. She tells Cog that at home she feels ignored, in the shadow of her eleven-year-old sister Andrea, who is scheduled to meet Cog later that day: “Nobody talks to me. . . . Nobody listens to me.” When Cog responds with silence, she is distressed. “Is he trying to tell me to go away?” she asks. “Cog, Cog, Cog . . . why aren’t you listening to me?” Suddenly, she has an idea and declares, “I didn’t think of this before.... This is what you have to do.” She begins to use sign language. “I know how to say ‘house’. . . . I can teach him to say ‘house’ [she taps her head with her right palm, making the sign for house].” Then she signs “eat” and “I love you” as Cog focuses on her hands. She is happy that Cog pays attention: “He loves me, definitely.”
Now, feeling both successful and competitive, Brooke boasts that she has a better relationship with Cog than her sister will have: “She’s probably just going to talk to Cog. I’m not just talking. I’m teaching.” As Brooke leaves, she announces to the research team, “I wanted him to speak to me. I know the robot down the hall [Kismet] is the talking one. But I really wanted
him
to talk.”
Scassellati is used to hearing such sentiments. He has worked on Cog for seven years and seen a lot of people behave as though smitten with his robot and frustrated that it will not talk with them. He uses the first-encounters study for an experiment in what he considers “responsible pedagogy.” Thirty of the children in our study participate in a special session during which Scassellati demystifies Cog. One by one, Scassellati disables each element of Cog’s intelligence and autonomy. A robot that began the session able to make eye contact and imitate human motion ends up a simple puppet—the boy Pinocchio reduced to wood, pins, and string.
So later that day, Scassellati “debriefs” Brooke and Andrea. He shows the sisters what Cog sees on its vision monitors and then covers its “eyes”—two cameras for close vision, two for distance vision—and the girls watch the four monitors go blank, one after another. They are given a computer mouse that controls Cog’s movement and they get to “drive” it.
Together, the sisters direct Cog’s eyes toward them. When Cog “sees” them, as evidenced by their appearance on its vision monitors, the quiet, didactic tone of the debriefing breaks down. Brooke screams out, “He’s looking at us” and the carefully built-up sense of Cog as mechanism is gone in a flash. Even as the girls control the robot as though it were a puppet, they think back to the more independent Cog and are certain that it “likes” looking at them.
As Scassellati proceeds with this debriefing, he tries to demonstrate that Cog’s “likes and dislikes” are determined by its programming. He shows the girls that what has Cog’s attention appears in a red square on a computer screen. They can control what gets into the square by changing what its program interprets as being of the highest value. So, for example, Cog can be told to look for red things and skin-colored things, a combination that would have Cog looking for a person with a red shirt.
Despite this lesson, the sisters refer to the red square as “the square that says what Cog likes,” and Brooke is joyful when Cog turns toward her hand: “Yep, he likes it.” They try to get Cog’s interest with a multicolored stuffed caterpillar, which, to their delight, makes it into Cog’s red square as well. Cog also likes Brooke’s leg. But she is troubled that Cog does not like a Mickey Mouse toy. On one hand, she understands that Cog’s lack of interest is due to Mickey’s coloration, half black and half red. The black is keeping Mickey from being registered as a favorite. “I see,” says Brooke, “Mickey is only half red.” But she continues to talk as though it is within Cog’s power to make Mickey a favorite. “I really want Cog to like Mickey. I like Mickey. Maybe he’s
trying
to like Mickey.”
The children imbue Cog with life even when being shown, as in the famous scene from the
Wizard of Oz
, the man (or, in this case, the machines) behind the magic. Despite Scassellati’s elegant explanations, the children want Cog to be alive enough to have autonomy and personality. They are not going to let anyone take this away. Scassellati’s efforts to make the robot “transparent” seem akin to telling someone that his or her best friend’s mind is made up of electrical impulses and chemical reactions. Such an explanation is treated as perhaps accurate but certainly irrelevant to an ongoing relationship.
Scassellati is concerned that Cog’s lifelike interface is deceptive; most of his colleagues take a different view. They want to build machines that people will relate to as peers. They don’t see lifelike behaviors as deceptions but as enablers of relationship. In
The Republic
, Plato says, “Everything that deceives may be said to enchant.”
10
The sentiment also works when put the other way around. Once Cog enchants, it is taken as kin. That which enchants, deceives.
Children have met this idea before; it is a fairy tale staple. More recently, in the second volume of the Harry Potter series, a tale of young wizards in training, Harry’s friend Ginny Weasley falls under the spell of an interactive diary. She writes in it; it writes back. It is the wizarding version of the ELIZA program. Even in a world animated by living objects (here, people in photographs get to move around and chat), a caution is served. Ginny’s father, himself a wizard, asks, “Haven’t I taught you anything? What have I always told you? Never trust anything that can think for itself
if you can’t see where it keeps its brain
.”
11
But, of course, it is too late. When something seems to thinks for itself, we put it in the category of “things we form relationships with.” And then we resist having information about mechanisms—or a detail such as where it keeps its brain—derail our connection. Children put Cog in that charmed circle.
When Scassellati turns Cog into a limp puppet, showing where Cog “keeps its brain,” children keep the autonomous and responsive Cog in mind. They see Cog’s malfunctions as infirmities, reasons to offer support. Part of complicity is “covering” for a robot when it is broken. When Cog breaks its arm, children talk about its “wounds.” They are solicitous: “Do you think it needs some sort of, well, bandage?”
BUILDING A THOU THROUGH A FACE AND A VOICE
As with Cog, children will describe a “buggy” Kismet as sick or needing rest. So, on days when Kismet does not speak, children talk to the “deaf ” Kismet and discuss how they will chat with it when it “gets better.” Robyn, nine, is chatting with an expressive and talkative Kismet that suddenly goes mute and immobile. Robyn’s reaction: “He is sleeping.”
Sometimes children weave complex narratives around Kismet’s limitations. Lauren, ten, gets into a happy rhythm of having Kismet repeat her words. When Kismet begins to fail, Lauren likens the robot’s situation to her own. It is not always possible to know what Kismet is learning just from watching “what is happening on the outside” just as we cannot observe what is happening inside of her as she grows up. Despite its silence, Lauren believes that Kismet is growing up “inside.” Lauren says that Kismet is “alive enough” to have parents and brothers and sisters, “and I don’t see them around here.” Lauren wonders if their absence has caused Kismet to fall silent.
Fred, eight, greets Kismet with a smile and says, “You’re cool!” He tells us that he is terrorized by two older brothers whose “favorite pastime is to beat me up.” A robot might help. He says, “I wish I could build a robot to save me from my brothers.... I want a robot to be my friend.... I want to tell my secrets.” Fred stares intently into Kismet’s large blue eyes and seems to have found his someone. In response to Fred’s warm greeting, Kismet vocalizes random sounds, but Fred hears something personal. He interprets Kismet as saying, “What are you doing, Rudy [one of Fred’s brothers]?” Fred is not happy that Kismet has confused him with one of his roughhousing brothers and corrects Kismet’s error. “I’m Fred, not Rudy. I’m here to play with you.” Fred is now satisfied that Kismet has his identity squared away as the robot continues its soft babble. Fred is enchanted by their interchange. When Fred presents a dinosaur toy to Kismet, it says something that sounds like “derksherk,” which Fred inteprets as Kismet’s pronunciation of dinosaur. During one back-and-forth with Kismet about his favorite foods, Fred declares victory: “See! It said cheese! It said potato!”
When Kismet sits in long silence, Fred offers, “Maybe after a while he gets bored.” When Kismet shows no interest in its toys, Fred suggests, “These toys probably distract Kismet.” At this point, the research team explains Kismet’s workings to Fred—the Kismet version of Scassellati’s “Cog demystification” protocol. We show Fred the computer monitor that displays what Kismet is “hearing.” Fred, fascinated, repeats what he sees on the monitor, hoping this will make it easier for Kismet to understand him. When this strategy doesn’t prompt a response, Fred blames Kismet’s bad hearing. But in the end, Fred concludes that Kismet has stopped talking to him because it likes his brothers better. Fred would rather feel rejected than see Kismet as a less than adequate relational partner.
Amber, six, also fights to keep Kismet alive enough to be a friend. On the day Amber visits MIT, Kismet’s face is expressive but its voice is having technical difficulty. The young girl, unfazed, attends to this problem by taking Kismet’s part in conversation. So, Amber engages Kismet with a toy and asks Kismet if she is happy. When Kismet doesn’t answer, Amber answers for it with a hearty “Yep!”