Fundamental facial expressions: are there really 21?

All face photos this page courtesy of Ohio State University.

For centuries, psychologists have studied six basic facial expressions: happy, angry, sad, surprised, fearful and disgusted. But as cognitive scientist Aleix Martinez of the Ohio State University studied those six, a heretical question flitted through his mind: Could the human face be capable of registering a much wider variety of emotions?

“Something struck me as odd,” says Martinez, an associate professor of electrical and computer engineering. “Why would we have only one positive emotion in the six categories? We are not that negative of a people, are we?”

Guillaume Duchenne de Boulogne electrically stimulated facial muscles during the middle 19th century. Duchenne was a French physician and pioneering neurophysiologist who investigated which muscles activate different facial expressions. He published his results with a photo album in 1862.

Photo: Public Domain

And why such a short list? Could six emotions be the limit of what we can express with our marvelously muscular mugs? Martinez started to wonder about “compound emotions” like happily surprised or angrily surprised, and assembled a new list. To the original six, he added 12 compound emotions, three new emotions (appalled, hatred, awed) and one neutral face. That list of 22 faces became the basis for a study published this week in the Proceedings of the National Academy of Sciences.

To test the legitimacy of the new categories, Martinez asked 230 research subjects to make a face expressing, say, disgusted surprise or sadly angry. To clarify the request, the subjects read a sample situation (like watching a gross but humorous movie for happily disgusted). Before the researchers took a photo, the subjects could use a mirror to compose the face, but they were not coached on what type of face was wanted.

Data crunching time

Armed with 5,000 photos, Martinez built a computer model to measure similarity between two images. Following the common strategy in facial expression studies, the computer estimated which facial muscles were in use to make each face. The result was “a very small difference,” between pairs of images showing the same emotion, Martinez says.

MOUSE-OVER IMAGE TO SEE ANSWERS

When he compared pairs of images showing different emotions, he found a much larger difference. “This suggests that not only do we use the same muscles for the same emotion, but the expressions are different from one category to another.”

The computer analysis also showed that the compound categories are mergers of the simple ones. “Happily disgusted,” for instance, creates an expression that combines the scrunched-up eyes and nose of “disgusted” with the smile of “happy.”

Overall, Martinez says, “pretty much everyone used the same face to express all 21 categories. I was happily surprised. I was expecting several to match, but all of them matching? That was unexpected.”

Like recognizing faces, recognizing emotions is based on fine distinctions called “configurational features,” Martinez says. In a real smile, muscles at the outer edge of the eye contract, creating a little wrinkle at the corner of the eye and a slight squint. “We can tell a fake smile, because it does not include the squint, just the curved mouth,” Martinez says.

MOUSE-OVER IMAGE TO SEE ANSWERS

Martinez says the subjects’ use of similar faces to express the same emotion, and the computer analysis showing that the same muscles were employed to express a particular emotion, support the compound emotions as real, inherent emotional states. “The results suggest that all of us have all these emotional categories, or we’d not express them in the same way. It seems all of us have these since birth.”

Emotions: Not all are created equal

We asked about some other expressions: disbelief, reverence, love and loathing, but Martinez said their definitions were too squishy to be included on the master list, “especially for love.”

Mr. Yuk is widely used to label poisons in the United States. Notice how Mr. Y exaggerates the archetypal expression of “disgust”?

TM The Children’s Hospital of Pittsburgh

And while the 21 emotions are common, even perhaps universal, at least in the United States, Martinez acknowledges that personal receptivity to them depends on experience. “Babies express happily surprised very readily, because they are always surprised and always happy when people show them something new.”

Some studies, he adds, “found that abused children are very sensitive to anger, and detect its expression with high precision.” On the other hand, “There are some emotions that you are bad at recognizing or expressing, because you never use them.”

On a surprisingly happy note, Martinez says his research on the original six categories showed that Ohio State students are “terrible at recognizing fear. But when we think about it, they are happy kids, they have never experienced fearful events, never been to war, have not been abused as children.”