When after many minutes, Kismet haltingly begins to speak, Amber’s response is immediate: “He likes me!” Now, Kismet babbles, and Amber interprets. The young girl says aloud what Kismet
meant
to say and then engages in a conversation with Kismet based on her interpretation. Before leaving Kismet, Amber tries hard to have the robot say, “I love you.” After a half dozen prompts, Kismet says something close enough. Amber thanks Kismet, says, “I love you too,” and kisses the robot good-bye.
In some ways, Amber’s time with Kismet resembles play with a traditional doll, during which a child must “fill in” both sides of the interaction. But even at its worst, Kismet gives the appearance of
trying
to relate. At its best, Kismet appears to be in continuous, expressive conversation. As with Cog, Kismet’s failures can be interpreted as disappointments or rejections—very human behaviors. Your Raggedy Ann doll cannot actively reject you. When children see a sociable robot that does not pay attention to them, they see something alive enough to mean it.
BUILDING A THOU BY CARING
Children try to get close to Cog and Kismet by tending them. Children ask the robots how they are feeling, if they are happy, if they like their toys. Robyn, the nine-year-old who imagines Kismet asleep when it mysteriously stops speaking, thinks that the robot is alive because “it talks and moves like a person.” When Kismet develops problems, Robyn wants to take it home to “feed it and give it water to drink so that it wouldn’t die; I would give it a Tylenol if it felt sick and I would make Kismet his own room.” The room, Robyn explains, would have a television on which Kismet could “see other robots so it wouldn’t miss its family and friends.”
As children see it, they teach the robots, and the robots appreciate it, even if they are imperfect pupils. Over half the children in the first-encounters study say, unprompted, that they love the robots and the robots love them back. From those who don’t speak of love, there is still talk about Cog and Kismet having made a “good effort” during their lessons. When children congratulate the robots one hears something akin to parental pride. When the robots succeed, in even the smallest thing, children take credit and present each success as evidence that their own patience has borne fruit. During our study the robots’ performance is subpar. But the children’s investment—their desire, connection, and pride—makes the sessions sparkle.
This is clear in the relationship that Neela, eleven, forms with Cog. When Neela first sees Cog, she exclaims, “Oh, it’s so cute!” and then explains, “He has such innocent eyes, and a soft-looking face.” After teaching the robot to balance a stuffed caterpillar on its arm, she says, “I could never get tired of Cog. . . . It’s not like a toy because you can’t teach a toy; it’s like something that’s part of you, you know, something you love, kind of like another person, like a baby.” When Cog raises its arm, Neela says, “I wonder what he’s thinking?” She asks, “What do you want?” “What do you like?” When Cog hesitates in his play—for example, when he is slow to raise his arm in response to her actions—Neela never uses a mechanical explanation for Cog’s trouble. Her reasoning is always psychological. She says that Cog reminds her of the “slow kids” in her class and she is sympathetic. “He’s slow—it takes him a while to run through his brain.” And she wants to help. “I want to be its friend, and the best part of being his friend would be to help it learn.... In some ways Cog would be better than a person-friend because a robot would never try to hurt your feelings.” (This is an eleven-year-old’s version of the comment made by the graduate student who wanted a robot boyfriend.) For Neela, a silent Cog is simply disabled: “Being with Cog was like being with a deaf or blind person because it was confused, it didn’t understand what you were saying.” In fact, Neela says that Cog does “see”—just not very well during her visit. To compensate, Neela treats the robot as a person having a bout of temporary blindness. “I was just like, ‘Hello!’ because a blind person would have to listen.” Neela hopes that Cog will get over its problems or that “he might grow out of it.... He’s very young you know.”
Neela has recently arrived from India and is having trouble fitting in at school. She explains that a group of girls seemed to accept her but then made fun of her accent: “Girls are two-faced. They say they like you and then they don’t. They can’t make up their mind.” Cog poses fewer risks. At school the girls who taunted her finally begged her forgiveness, but Neela hasn’t been able to accept their apology. In this last regard, “Cog could be a better friend than a person because it is easier to forgive.... It’s easier to forgive because it doesn’t really understand.” Recall that Neela speaks of Cog as “part of you . . . something you love.” This is love safe from rejection. Like any object of love, the robot becomes “part of you.” But for Neela, Cog, unlike a person, does not have enough independence to hurt you. In Neela’s feelings for Cog we see how easily a robot can become a part object: it will meet our emotional needs because we can make it give us what we want. Is this an object for our times? If so, it is not an object that teaches us how to be with people.
Some children, particularly with Kismet, explicitly put themselves in the role of sibling or parent. In either of these roles, the relationship with Kismet may become a place to reenact the tensions in a family, something we have already seen with AIBO and My Real Baby. In the pursuit of Kismet, brothers come to blows and sisters bitterly compete. And efforts to parent Kismet can be a critique of what goes on at home. Rain, ten, lives with her mother and is preoccupied by her father’s absence. She explains that she would never abandon Kismet: “My father doesn’t live at home; he moved away. If Kismet came to live with me, I would never move away, ever. I would leave him juice every morning. I would make him a comfortable bed. And I would teach it to really talk, not just the little bit it knows now.” There is much similarity between this kind of talk and what happens in a therapist’s office when children act out their conflicts with their dolls. A doll can let you vent feelings, enjoy imaginary companionship, and teach you what is on your mind. But unlike dolls, these robots “push back.” Children move beyond using the robot to relive past relationships. They hope for a relationship with the robot in the real.
Madison, nine, works with Kismet on a day when the robot is at its best. Its emotive face is responsive and appropriate. It remembers words and repeats them back in humanlike cadences. The result looks like Madison is speaking earnestly to someone whose inflection and tone make her feel perfectly understood.
Madison asks Kismet questions in a gentle and soft-spoken manner, “What is your name? Do you have parents?” Kismet responds warmly. Encouraged, Madison continues. “Do you have brothers and sisters?” Kismet moves its head in a way that suggests to Madison that the answer is yes. Madison tells us that Kismet is a little girl (she was “born from a stomach”), but a new kind of little girl. And like any baby, “she” doesn’t know when “her” birthday is. Madison wants to be “her” good parent. “Do you like ice cream?” Madison asks, and when Kismet quietly responds to this question, the two go on to discuss ice cream flavors, favorite colors, and best toys.
Madison begins to dangle one toy after another in front of Kismet’s face, laughing at its changing expressions. Madison tells Kismet that some of the girls in her school are mean; she says that Kismet is nicer than they are. Kismet looks at Madison with interest and sounds encouraging. In this warm atmosphere, Madison tells Kismet that she looks forward to introducing the robot to her baby sister. Playing with her sister, Madison says, is her favorite thing to do, and she expects Kismet will feel the same way. Kismet nods and purrs happily. Again, projection onto an object becomes engagement with a subject; Rorschach gives way to relationship.
Madison believes that Kismet learns from every child who comes to play. But you can’t be impatient. “Babies learn slowly,” she offers. Like a baby, Kismet, too, will learn over time. “I taught Kismet to smile,” Madison says. “[Kismet] is still little, but it grows up.” To justify this claim, Madison, like Lauren, distinguishes between what you can see of a child’s learning and what is hidden from view: “You can’t always tell what babies are learning by looking at them on any day.” The same is true for Kismet. Kismet is learning “inside” even if we can’t see it. A mother knows her child has secrets.
In the hour she plays with Kismet, Madison becomes increasingly happy and relaxed. Watching girl and robot together, it is easy to see Kismet as increasingly happy and relaxed as well. Child and robot are a happy couple. It is almost impossible not to see Madison as a gratified mother and Kismet as a content child. Certainly, Kismet seems to prefer Madison to the children who have visited with it earlier that day. For me, their conversation is one of the most uncanny moments in the first-encounters study, stunning in its credibility because Kismet does not know about ice cream flavors, baby sisters, or mean girls. Kismet does not like Madison; it is not capable of liking anything or anybody.
BUILDING A THOU IN DISAPPOINTMENT AND ANGER
The children in the study care about having the robots’ attention and affection far more than I anticipated. So their interpretation of robot malfunctions as illness is ingenious; they can walk away without feeling dismissed. But the most vulnerable children take disappointments with a robot very personally. The children most upset by a robot’s indifference are those who feel least tended to. They seem almost desperate for Kismet and Cog to recognize and respond to them. Since the children in our study come from a wide range of backgrounds, some tell us that the snack they get during their session at MIT is the best meal of their day. Some find ways to make it clear that their time at MIT is the most attention they have received that week. Children from affluent as well as economically disadvantaged homes talk about parents they rarely see. When these children interpret robotic technical limitations as rejection, they become withdrawn, depressed, or angry. Some take foolish chances.
My field notes taken after one session with Kismet describe a conversation with the junior members of my research team, two college seniors and two graduate students: “Emergency meeting with team after session with Estelle. Disappointment with Kismet provokes her binge eating, withdrawal. Team feels responsible. How to handle such children? What did child want? A friend? A future?” My team meets at a local coffee shop to discuss the ethics of exposing a child to a sociable robot whose technical limitations make it seem uninterested in the child.
We have spent the afternoon with twelve-year-old Estelle, who had seen the flyer describing our work on the bulletin board of her after-school center: “Children wanted for study. Meet MIT Robots!” She brought it to her counselor and asked to participate. Estelle tells us that she “stood over my counselor while she called MIT.” Estelle has taken special care with her appearance in preparation for her day with us. She is dressed in her best clothes, her hair brushed to a fine polish. As soon as we picked her up, Estelle talks nonstop about “this wonderful day.” She has never been to MIT, but she knows it is a “very important place.” No one in her family has been to college. “I am the first one to go into a college . . . today.”
On the day of Estelle’s visit, Kismet engages people with its changing facial expressions but is not at its vocal best. We explain Kismet’s technical problems to Estelle, but nonetheless, she makes every effort to get Kismet to speak. When her efforts bear no fruit, Estelle withdraws, sullen. She goes to the room where we interview children before and after they meet the robots. There we have set out some simple snacks. Estelle begins to eat, not stopping until we finally ask her to leave some of the crackers, cookies, and juice boxes for other children. She briefly stops eating but begins again as we wait for the car service that will bring her back to the after-school program. She tells us that the robot does not like her. We explain this is not the case. She is unappeased. From her point of view, she has failed on her most important day. As Estelle leaves, she takes four boxes of cookies from our supply box and puts them into her backpack. We do not stop her. Exhausted, we reconvene to ask ourselves a hard question: Can a broken robot break a child? We would not consider the ethics of having children play with a damaged copy of Microsoft Word or a torn Raggedy Ann doll. But sociable robots provoke enough emotion to make this ethical question feel very real.
The question comes up again with Leon, twelve. Timid and small for his age, Leon usually feels like the odd man out. In Cog, Leon sees another figure who “probably doesn’t have a lot of friends,” and Leon says they have a good chance to connect. But, like Estelle, Leon has not come to the laboratory on a good day. Cog is buggy and behaves as though bored. The insecure child is quick to believe that the robot is not interested in him. Leon had been shown Cog’s inner workings, and Scassellati gently reminds Leon that Cog’s “interests” are set by people adjusting its program. Leon sees the monitor that reflects these preset values, but he insists that “Cog doesn’t really care about me.” He explodes in jealousy when he sees Cog looking at a tall, blond researcher, even as Scassellati points to the researcher’s red T-shirt, the true lure that mobilizes Cog’s attention. Leon cannot focus. He insists that Cog “likes” the researcher and does not like him. His anxieties drive his animation of the robot.
Now Leon embarks on an experiment to determine whether Cog cares about him. Leon lifts and then lowers his arm and waits for Cog to repeat what he has done. Cog lifts its arm and then, as the robot’s arm moves down, Leon puts his head directly in its path. This is a love test: if Cog stops before hitting him, Leon will grant that Cog cares about him. If the falling arm hits Leon, Cog doesn’t like him. Leon moves swiftly into position for the test. We reach out to stop him, appalled as the child puts his head in harm’s way. Cog’s arm stops before touching Leon’s head. The researchers exhale. Leon is jubilant. Now he knows that Cog is not indifferent. With great pleasure, he calls out “Cog!” and the robot turns toward him. “He heard me! He heard me!”