By studying human interactions, computer-science expert Timothy Bickmore creates animated characters that help people lead happier, healthier lives
By Lewis I. Rice - Photography by Webb Chappell
Yeah, yeah, most people know they should exercise regularly. But this knowledge doesn’t always—or often—translate into action. Witness the roughly two-thirds of American adults classified as overweight or obese.
The problem of inactivity is most acute in a particular group: the elderly. Only 12 percent of adults over age seventy-five get the minimum level of physical activity recommended by the Centers for Disease Control and Prevention.
Timothy Bickmore wanted to get more of this population moving. So he connected a group of elderly volunteers with a personal exercise adviser named Laura, who worked with them every day. She’d monitor their progress, encourage them—sometimes just chitchat with them.
“She’s nice,” one participant confided. “She asks you the right questions. She tells you if you’re not doing up to par. . . . And if you’re doing good, she’ll tell you.”
The clients who worked with Laura walked more than the others who had simply received pamphlets about the benefits of exercise. In addition to going the extra distance, many clients reported they enjoyed interacting with her. A personal connection, it seemed, motivated them more than any piece of paper could.
Then again, Laura isn’t actually a person. She’s an animated character who speaks to users from a computer screen. But the people using her saw her as something more than pixels and a synthesized voice. They started, well, caring about the relationship they had with her.
One user even joked that, when she went away on a trip, “I began to feel bad about Laura, stuck in that box.”
Laura is what’s known as a relational agent, a computer animation designed to build and maintain long-term social-
emotional relationships. She’s the brainchild of Bickmore, an assistant professor in the College of Computer and Information Science. Not long after arriving at Northeastern in 2005, Bickmore launched the Relational Agents Group, a circle of researchers who study how such animations may improve the lives of their human users.
In the process, he has found evidence that people exercise more and engage in other healthy practices when communicating with relational agents. More than that, they form a bond with the animations. They trust them.
As Bickmore explains in a research paper he wrote on spoken interaction between man and machine, “[N]ot only will users readily engage in a wide range of social behavior appropriate to the task context, but . . . the social behaviors have the same effect on them as if they had been interacting with another person. This trend seems to indicate a human readiness, or even need, to engage computational artifacts in deeper and more substantive social interactions.”
This sounds like science fiction, like storylines from the movies A.I. and 2001: A Space Odyssey. But although no one would ever confuse a relational agent with a person (despite the fact that almost everyone calls Laura a “she”), Bickmore believes the computer representations can reveal something important about the human experience.
“I got into artificial intelligence because I was always interested in how the mind works and in looking into psychology in a deeper way,” he says. “How human-
computer interfaces can play with people’s psychology is really fascinating to me.”
“You were going to make me some hands”
An observer at a recent weekly Relational Agents Group meeting might have thought he had wandered into the wrong department by mistake.
Daniel Schulman, a PhD student, was giving a presentation on “Visual Attention and Eye Gaze During Multiparty Conversations with Distractions,” a paper written by University of Pennsylvania researchers. It’s a cognitive-psychology take on how eyes really are the windows to the human soul. “You can get a good idea of who’s paying attention and who isn’t just by looking at eye movements,” Schulman tells the group.
This is not a psychology department meeting. Ten or so people from the College of Computer and Information Science have gotten together in a West Village H conference room to talk about computer applications.
Midway through the meeting, Bickmore asks about a pointing gesture for a computer animation. “You were going to make me some hands,” he says to one of the team.
The tech talk, though, is threaded with information about human responses, such as that offered in Schulman’s presentation. As Bickmore explains, the most effective relational agents mimic human behavior. “The way I was trained,” he says, “was always to go to human-human interaction as the inspiration and the baseline for building things.”
In his research, Bickmore writes about the ways humans relate to humans—rules of etiquette, for instance. Emily Post, he notes in one paper, urged people to show their conversational partners that they are important and valued. He also outlines the interpersonal constants friends are supposed to provide one another, such as reliability and emotional stability.
In a paper called “Towards Caring Machines,” Bickmore explains one of his studies of “simulated caring by a computer.” For a month, thirty computer-savvy MIT students interacted with a relational agent who showed thoughtful qualities (nice Laura). Another thirty interacted with an agent who didn’t (no-nonsense Laura). The students who used the caring version were more apt to report they were willing to continue the relationship.
Some participants were ambivalent, of course. There were those who emphasized that Laura seemed like a robot or
a machine.
Although all the MIT students were certainly aware of the simulated aspect of the caring, many seemed to transcend this barrier and embrace the results. “I don’t know if she cared about me,” one participant reported. “She’s a character and has a role, but I don’t know if she has feelings. But it worked for me, and I’m happy.”
A new platform
Some people seem destined to work with computers. Maybe their parents are engineers or science buffs, who convey to their offspring the joys of technology.
Timothy Bickmore did not seem destined to work with computers. Not too many computer scientists grow up as a human juggling ball in the circus.
The child of traveling circus performers, Bickmore began working in the family business when he was three. His parents did an acrobatic foot-juggling act called Risley (named for the nineteenth-century acrobat Richard Risley Carlisle), in which performers lying on their back appear to juggle people.
Throughout his teenage years, he was literally supported by his parents, as part of the act.
Bickmore says the circus was an insular society, and he expected he would continue with the family tradition as an adult. “I had no concept of what people on the outside did or why anybody would want to live in one place for more than a few days,” he says.
His mother, however, urged him to go to college. Bickmore figured he’d study electrical engineering at Arizona State University, then go back to the only life he had known and design light shows for the circus. But his college studies, especially his work in an artificial intelligence lab, changed his mind about all that. He was inspired to think beyond the big top to other career possibilities.
After earning a bachelor’s in computer systems engineering and a master’s in computer science at Arizona State, Bickmore worked in research and development labs for such companies as Lockheed and Fuji Xerox.
With a dozen years in industry under his belt, eager for more training in research methods and a more advanced degree, he decided to pursue a PhD at MIT’s Media Lab, where he collaborated with Rosalind Picard, the founder and director of the Affective Computing Research Group. Picard says Bickmore advanced the notion that computer agents could help people accomplish tasks by addressing their emotional needs.
“He had a lot of bold ideas,” she says, “about how agents could not just appear to be human, but how they could actually forge a relationship with people, responding to their emotions and acting more like someone you’d get to know gradually over time and start to like, as opposed to something that presumes it’s there to help you and mostly irritates you.”
At the time, Bickmore was developing a computer real-estate agent for a fellow Media Lab faculty member. Like actual realtors, the computer agent asked questions probing the kind of property the user wanted to buy, and also engaged in conversation not directly related to home buying, to build a connection.
For his own research, though, Bickmore wanted to work on something more altruistic than a sales agent. “I like to be able to build systems that actually help people,” he says. He soon discovered a large amount of academic literature that indicates relational aspects can influence people’s health.
And so a direction was born. Laura, his first agent, focused on exercise because of its nearly universal appeal: Most people can use more of it. And her effect could be easily measured by having test subjects wear a pedometer.
Currently, Bickmore plans to add to his physical-fitness research by conducting a one-year study of a personal digital assistant loaded with both a relational agent and a motion sensor that detects people’s movements.
At the same time, he and the Relational Agents Group are exploring ways to use agents for health-care practices. For instance, he’s collaborating with the University of Pittsburgh School of Nursing on a study of patients with schizophrenia, who often fail to take the medications prescribed to treat the disease. Participants will discuss taking their medications and managing side effects with a relational agent on a home desktop computer.
And there’s a five-year study with the Boston Medical Center that aims to improve hospital patients’ ability to care for themselves after discharge and reduce the need for readmission. Bickmore has devised two characters, virtual nurses Louise and Elizabeth, who speak to patients at their bedside about care and medication. The study will compare patients counseled by the relational agents with those counseled by a human health-care professional.
In general, Bickmore says, patients don’t take their medications properly about half the time, thereby endangering their recoveries. He doesn’t blame health-care professionals for this. He does, however, point to the ways a relational agent can serve people with health concerns, perhaps more effectively than even the most committed human.
“You can interact with [an agent] much more frequently,” Bickmore says. “It’s at home. You talk to it every day. Or it’s on a PDA and with you all the time. It learns all the details of your life to a much greater degree than a health professional could.”
Would the relationship be as strong if you interacted with a computer agent month after month, year after year? Bickmore is trying to find out, with a five-year study that will test an agent’s long-term effect on physical activity among older adults.
It’s possible a relational agent could also promote long-term disease management, Bickmore says, coaching people who require daily treatment for diabetes, for instance. He’d like to know what would sustain a long agent-patient relationship, potentially over the course of a lifetime.
Should these studies indicate that relational agents can make people healthier, there will undoubtedly be commercial interest in making them widely available. If someone wanted to market Laura and her cohorts, that would be fine, Bickmore says.
Right now, however, he’s intently focused on the research, on melding science and psychology to create a new kind
of relationship.
Laura doesn’t yell
Sometimes a Relational Agents Group meeting sounds like a screenwriting class, with a lot of talk about writing scripts.
Group members aren’t making a movie. But they are, in a sense, fashioning a drama.
As yet, no relational agent can conduct a free-flowing conversation, like HAL 9000 debating the astronauts in 2001. Even in 2008, Bickmore says, the technology to equip a computer with human-level intelligence is still a long way away. So, for each step of a conversation between an agent and a user, group members have to write dialogue, which they base on observation of and academic research on human interaction.
Displayed on a computer screen, the branches of dialogue look like an organizational chart for a massive company. Laura has about a thousand different conversational stages, with three or four utterances that the user might elicit at each level, and rephrasings that occur randomly every day, to give the user a sense of variety over time and repeated discussions.
“I come up with an explicit schedule of how I think the relationship should evolve over the range of the interaction,” says Bickmore.
When he was designing Laura, his first relational agent, he hired an athletic trainer to interact with students daily for a few weeks, so he could observe the conversations that happen naturally in that relationship.
Likewise, a relational agent’s look is based on data—in this case, on the literature that studies people’s preferences for a certain kind of provider, such as an exercise adviser or a nurse. Bickmore will test several different looks to see what users are most comfortable with.
Laura, the exercise adviser, appears to be in her twenties, dark-complexioned, and fit. Once she’s powered up, she offers a typical greeting, then segues into small talk or social dialogue, which is supposed to build trust. (Laura was built a few years ago, so her synthesized voice sounds somewhat more artificial than a voice created by current technology would, Bickmore says.)
The user answers not by speaking back but by choosing from options on the screen. The character then responds on the basis of what the user has chosen, each step a branch of the scripted dialogue. If you tell her you’re not feeling well, for instance, Laura may say, “I am sorry to hear that. Tell me more about how you are feeling.”
Laura’s not all talk. She moves while speaking—smiling, frowning, blinking, nodding her head—showing about thirty different behaviors, all based on research on and observation of people’s nonverbal reactions.
She also “remembers” past interactions and demonstrates that she knows what you talked about last time. Such familiarity encourages speakers to engage more with her, Bickmore says.
As in human relationships, the small talk soon transitions to more substantive matters—in this case, finding out how much the user has walked. Laura offers positive reinforcement for meeting goals, which are measured with a pedometer.
If users don’t walk enough, Laura doesn’t yell at them. According to Bickmore, the literature says negative reinforcement doesn’t work. Rather, Laura may say, “I know it’s hard to find time to exercise, but you know how important it is to exercise regularly.”
Lisa Caruso certainly knows the value of exercise for the population she treats. An assistant professor of medicine at Boston University School of Medicine, she works with geriatric patients and assisted Bickmore in his study on inspiring older adults to walk.
Caruso says she wondered whether older adults would be receptive to the technology. In fact, she acknowledges, “we thought it was kind of weird when Tim presented the idea to us.”
But she came to see the computer agent as akin to having an extra employee who serves as a resource for follow-up care and counseling. The study’s results showed how a computer agent could change people’s behavior for the better, she says.
If using a computer agent is “done in the right way,” Caruso says, “and it could be done as a complement to what [a patient’s] plan of care is with a medical provider, I think it could really add a lot.”
Human concerns
Of course, not everyone appreciates computer agents like Laura. In an essay on the phenomenon, Jaron Lanier, a technology expert who coined the term “virtual reality,” calls them “agents of alienation” that devalue human creativity.
“An agent is a way of using a program in which you have ceded your autonomy,” he writes.
For his part, Bickmore believes that, rather than alienating people from one another, relational agents can help to bring them together.
“People who are significantly isolated, who have no social networks, have much greater mortality rates, much greater risk of dying,” he says. “If we could build an intervention that helps them make friends and maintain their social networks by providing tips and setting goals, it could maybe help enhance their human social lives.”
Currently, Bickmore and his team are working on agents for the National Cancer Institute; the National Heart, Lung, and Blood Institute; the National Library of Medicine; the National Institute on Aging; and the National Science Foundation, as well as several privately funded projects.
They’re also working on tools and methods that will let other researchers build health-counseling agents by combining reusable parts that Bickmore’s team has already developed.
Not all of Bickmore’s work is focused on health care. One new project is a guide agent for Boston’s Museum of Science. Situated in the ComputerPlace exhibition, the agent, named Tinker, looks like a human-sized robot and offers information about the museum and its exhibitions.
Though Tinker’s area of expertise is science education, not health, she still focuses on getting museum visitors to like and trust her over time. Thanks to a hand scanner, she can even recognize return visitors, causing more than one user to do an astonished double take.
Like any technology, relational agents could be used for unethical purposes, Bickmore acknowledges: “You could envision having a product that builds trust over time and then persuades [people] to do things they wouldn’t ordinarily do.”
But he carefully designs his relational-agent models to help people, not fool them. He always explains to users—some of whom, like in the exercise study for older adults, are unsophisticated about technology—that the system is a computer program, not a person. To reinforce this notion, a character like Laura will sometimes joke that she lives in a box, or was created in Boston.
Picard says that Bickmore brings a rigorous honesty to his creations’ design. For instance, he’d never have characters say they have trouble finding the time to exercise. Computer agents don’t exercise, and they never have to worry about the clock. Instead, a character may talk about interacting with people who have trouble exercising.
“There’s the fear we’ll deceive people,” Picard says. “I really appreciate Tim’s sensitivity to the subtleties. People may not notice, but it’s a real attempt to put integrity into the technology—what [a character] can know and what it can’t know.”
On the other side of the equation, users who speak to computer agents as they would speak to another person are not really deceiving themselves either, says Bickmore. People seem to have the cognitive machinery, he says, to relate to anything that exhibits human characteristics. Observe the way people behave with their pets or a doll that talks.
Again and again, Bickmore finds that people don’t have to be persuaded to interact with Laura. They are, in a uniquely human way, attracted to her.
Lewis I. Rice, MA’96, is a freelance writer living in Arlington, Massachusetts. He wrote about assistant communication- studies professor Walter Carl’s study of word-of-mouth marketing in the Winter 2006 issue.
|
A date with Louise Things aren’t going well. I’m taking a math test, and I’m failing miserably. A message on the computer screen where the questions appear tells me the average person gets 60 percent of these problems right. I thought I was pretty good at math, but somehow I keep running out of time, then a loud buzzer goes off. I’ve answered only 30 percent of the questions correctly. It appears I’m not as smart as I think I am. Luckily, I can turn to Louise for a shoulder to cry on. Okay, maybe not literally. Louise is one of Timothy Bickmore’s relational agents. She consoles me from a large video screen a few feet away. I’m taking part in an experiment in the HCI (Human-Computer Interaction) lab at the College of Computer and Information Science. As its name implies, the lab hosts many human test subjects, including those involved in research for the Relational Agents Group. While I’m struggling with my math problems, PhD candidate Daniel Schulman is watching me on a monitor outside the room. He’s wrapped a couple of sensors around my thumb and two fingers to measure my heart and perspiration rates. I can feel my hands moisten as I jab my finger at the keyboard, usually too late. Both before and after the test, I answer questions about how I feel. Then I get to talk to Louise, who asks innocuous questions, such as how the weather is today. She says she wants to know because she never gets out. Louise also wants to know how I’m feeling. After I choose “Not so great” from a menu of options, she says she’s sorry. She asks me what I think of Boston. This time, I can respond aloud any way I like, though the agent can’t understand what I say. Most people prefer the agent to converse with them, even with a limited choice of responses, Schulman later says. Following similarly dismal results on another math test, I’m asked how much I’d like to speak to Louise again. Maybe, I think, a little more than I did before I showed how stupid I am. Actually, as Schulman explains after the experiment is over, the math test was rigged to make me fail. The program timed my responses to practice questions and sped up the time I had to complete the real questions. Manipulating my self-esteem was in the name of science: The experiment measures, through physical responses and the questionnaire, the capacity of the relational agent to make a subject feel better. “The idea is that the negative feelings you don’t want to be feeling are going to go down after talking with the agent,” says Schulman. He and Bickmore have coauthored a paper titled “The Comforting Presence of Relational Agents,” which looks at preliminary data from similar kinds of experiments. When Schulman interviews volunteers afterward, a substantial number say that talking to the agent helped them, he says. I’m not sure how much better Louise made me feel. But I’d definitely rather talk with her than take that test again. — Lewis I. Rice |