Emotion science for homeland security?

Photo cour­tesy of Thinkstock.

Ear­lier this month the Affec­tive Sci­ence Insti­tute hosted another great event called Reading the Face: Trans­lating Sci­ence to Secu­rity. Three emo­tion sci­en­tists, including Northeastern’s Lisa Feldman Bar­rett, dis­cussed their unique research approaches to studying the human ability to detect a person’s emo­tion by looking at his or her face.

In the wake of 9/​11, large-​​scale secu­rity pro­grams were rolled out at air­ports across the country in which secu­rity per­sonnel were trained to stop indi­vid­uals that looked sus­pi­cious. In addi­tion to the researchers, the event brought Peter DiDomenica, former Director of Secu­rity Policy at Boston Logan Inter­na­tional air­port who devel­oped the TSA’s behavior-​​based screening pro­gram. Speaking from expe­ri­ence, DiDomenica said that secu­rity guards have used the tech­nique to arrest crim­i­nals trying to fly for var­ious rea­sons, from drug smug­gling to skip­ping the country to launching a ter­rorist attack on-​​board.

But the sci­ence, said Bar­rett and her col­leagues Jon Freeman of Dart­mouth Col­lege and Philippe Schyns of the Uni­ver­sity of Glasgow, is unable to sup­port the hypoth­esis that we can deter­mine who is a threat by simply looking at peo­ples’ faces.

Freeman uses mouse tracking to mon­itor how quickly research sub­jects are able to cat­e­go­rize faces. When a gender ambiguous face is pre­sented on a com­puter screen for instance, Freeman’s group looks at the actual tra­jec­tory of the mouse toward its ulti­mate des­ti­na­tion: a button with either male or female written in the right or left corner of the screen. He has also looked at how our cat­e­gor­ical deci­sions change based on con­text. In one exper­i­ment, he pre­sented study sub­jects with sim­ilar faces across a range of skin-​​colors. When the faces belonged to men wearing “high-​​status” clothing, such as a busi­ness suit, sub­jects cat­e­go­rized darker faces as Cau­casian whereas the oppo­site was true when those same men wore “low-​​status” clothing, such as a jan­i­to­rial uniform.

Schyns uses brain imaging to under­stand the neu­ro­log­ical under­pin­nings to these cat­e­go­riza­tion deci­sions, which happen in an instant but actu­ally require very com­plex machinery to take place. He’s shown that there are very subtle but impor­tant dif­fer­ences across cul­tures for what sig­nals dif­ferent emo­tions like hap­pi­ness or anger. For example, indi­vid­uals from eastern cul­tures transmit more emo­tional infor­ma­tion with their eyes whereas western indi­vid­uals use their mouth more.

Bar­rett brought home the mes­sage that there is no single, uni­versal facial expres­sion for any single emo­tion, a hypoth­esis that has long dom­i­nated psy­chology teaching without much data to sup­port it. She talked about research in which mem­bers of her lab looked at the dif­fer­ences between Amer­i­cans and people of a remote Namibian tribe who have had very little expo­sure to western cul­ture. They were able to dis­tin­guish smiling from neu­tral and wide-​​eyed fear faces, but not able to pick up on more subtle expres­sions, regard­less of the face’s ethnic back­ground. Amer­i­cans were only to do a little better at the task.

So, to recap: we’re not ter­ribly good at dis­tin­guishing indi­vid­uals from one another. Dif­ferent cul­tures demon­strate (and pick up on) emo­tion using dif­ferent parts of the face.  And, finally, we’re not all that sus­cep­tible to facially trans­mitted emo­tional infor­ma­tion to begin with. If all of this is true, then how can DiDomenico’s behavior-​​based screening pro­gram be even remotely successful?

Emo­tion detec­tion is an impor­tant part of the eval­u­a­tion, he said, but it’s not every­thing. The screening is designed to look at dozens of fac­tors, from base­line attrib­utes like the mood of the crowd, to how an indi­vidual answers benign ques­tions like “how’s it going today?” to where the person is trav­eling to and from. Ulti­mately, threat detec­tion experts use a range of infor­ma­tion to do their jobs and only some of it comes from the face.

Much of how we per­ceive each other is based on knowl­edge we have from our cul­ture,” says Bar­rett. “This presents cer­tain chal­lenges for threat detec­tion. The sci­ence of how we per­ceive each other gives us some clues, how­ever, to what experts are doing, and how to train people to become experts.”

Appar­ently only one in 173 mil­lion pas­sen­gers will be a ter­rorist. “We’re not nec­es­sarily going to catch him, but it puts one more obstacle in the way,” said DiDomenico. “It’s hard to mea­sure the impact when some­thing doesn’t happen. You don’t have to catch ter­ror­ists for it to be successful.”