Social Engineering Blogs

An Aggregator for Blogs About Social Engineering and Related Fields

Emotional Consequences of Digital Communication

Video Call Facetime Chatting Communication Concept

Several weeks ago, we published a blog on how the recent shift from in-person to digital communication has changed the way we read people and their body language. We discussed how humans did not evolve to do 2-dimensional communication and that there are drastic differences in the nonverbal messages we receive in real-life versus digitally.

We’re now talking to people more than ever using digital platforms such as Microsoft Teams, Slack and Zoom. Last month, Teams reached 44 million daily users, up from 20 million in November. But what is gained and what’s lost through these types of digital communication? What are some of the long-term emotional implications of these types of interactions?

We sat down with Humintell Director Dr. David Matsumoto to discuss his thoughts on these issues.

Human emotions evolved to facilitate human social bonds

In real-life interactions, emotions arise as consequences of the interaction. In a live setting, as we are expressing ourselves verbally, we’re also emoting non-verbally. Simultaneously, we’re also perceiving each other’s verbal expressions (words) as well as their nonverbal behavior in total, some of which may be unconscious (including sights, sounds and smells). With all those receiving channels operating, it in turn elicits some kind of emotional reactions in the receiver.

Dr. Matsumoto believes that these reactions allow for the development of emotional bonds and connections among people. As humans, our whole communication package is not only about sending signals across all the modalities, it’s about receiving them in all the channels as well. Not only that, we’ve evolved for these emotional bonds to develop. This reciprocal exchange through words through language but also emotions and feelings and sentiments, non-verbally, is an important part of human interaction.

Mirror neurons are further evidence that receivers in an interaction are not passive clinical recipients. These neurons in the brain are specific to producing or mimicking other people’s expressions. And as result of them, interactants are actively processing information being received and then having thoughts, feelings and emotions triggered by what they’re perceiving.

These emotional, strong human bonds are critical to our evolutionary success.

They also play a part in our overall well-being. Strong social bonds have been proven to be beneficial to one’s life satisfaction and overall health. In addition, high levels of social support appear to buffer or protect against the full impact of mental and physical illness.

When using digital communication, there is a drastic reduction in our senses.

Including the loss of smells and sounds, we lose the complete nonverbal package we often get in real life. In addition, we lose the ability to read facial expressions of emotion as well as we would in a live setting; the observable contours, the wrinkle patterns and the shading of the face are greatly diminished. The amount of overall stimuli is greatly reduced.

Therefore, even though the empathetic response system of the body (including mirror neurons) may be turned on during digital communication, Dr. Matsumoto can’t see the thoughts, feelings, emotions in reaction to the other person are going to be triggered as to the extent they are in real-life.

As a result, with digital communication, we lose an emotional connection to the people with whom we’re interacting and there are several emotional consequences:

  • It’s difficult to develop strong relationships with others
  • It’s harder to maintain good relationships with others
  • It’s easier for existing relationships to degrade

Dr. Matsumoto states that there are potentially major social and individual consequences to this kind of exclusivity of interaction and recent research seems to back up his concerns. According to new research released by Well Being Trust (WBT) and the Robert Graham Center for Policy Studies in Family Medicine and Primary Care, the growing epidemic of “deaths of despair” is increasing due to the coronavirus pandemic—and they anticipate as many as 75,000 more people will die from drug or alcohol misuse and suicide.

So where do we go from here?

Having an awareness of these potential negative side-effects of digital communication equips us to insulate ourselves from such possibilities. Knowing these consequences can occur give us the gateway to then think about ways to mitigate and even counter such effects. Proactively planning around it and beyond it could be beneficial. For example, perhaps having longer discussions remotely or when safe, plan to have more in-person interactions with those you are close to in order to re-group what may have been lost.

Developing and maintaining relationships takes conscious effort and work. Devote your time and prioritize relationships that are important to you. Realize the importance of emotional and social bonds.

We are all going through this difficult time together. But if we remain cognizant of these potential negative consequences of digital communication and proactively plan for the future, we can overcome these challenges.

Addressing Arguments Against Facial Expressions of Emotion

Within the past few months, many people have reached out to Humintell and asked us to comment on recent research articles that argue against facial expressions of emotion. After a lot of deliberation, Director Dr. David Matsumoto addresses those issues in the video above. 

First and foremost, I’d like to express my deep and sincere respect for all the researchers on both sides of this issue. I encourage healthy debates and more importantly, data about those debates. I think those debates are very healthy for science as well as for scientists, practitioners but most importantly, for the general public.

Dr. Matsumoto has researched and read the vast majority of all the studies that have been cited as evidence for and against the various positions that exist. In the video above, he does not get into technical issues of claims or the nature of the studies or exact data. Though he is happy to get into those discussions, they would require some knowledge of methodology. More importantly, he thinks the message that he wants to impart gets lost really easily if that path is gone down.

Dr. Matsumoto agrees with all the data he has seen from all the researchers. What he doesn’t agree with are all the interpretations or claims made about that data.

I believe data and findings are generated within the limitations of the methodologies that are used to produce that data.

If you look at the papers that argue against facial expressions of emotion, they typically don’t encompass all of the evidence for facial expressions of emotion or their universality including:

  • 100s of Judgement studies
  • Production studies
  • Studies of blind individuals
  • Studies of children and infants around the world
  • Studies of kin vs non-kin
  • Studies of family vs non-family members
  • Non-human primate studies

These types of debates have been occurring for a century. Ever since Darwin started this work and published it in 1872, these ideas have been debated hotly both in the lay public and academic discourse. Within the academic discourse, the start of these debates came from early anthropologists like Margaret Mead and Ray Birdwhistell. Those debates carried on to the 50s, 60s, and 70s. The original universality studies were conducted in the 60s and 70s. And even from the 80s these same debates and arguments have been occurring. 

To tell you the truth, the nature of the arguments made are essentially the same today as they were 30 or 40 years ago when I started being involved with them directly myself. 

A lot of the thinking that’s dominated this field and much of academia is what Dr. Matsumoto calls “logical determinism”. Logical determinism is a way of thinking that things are mutually exclusive; they’re either or, it’s this way or that way. They are either or dichotomies. Dr. Matsumoto thinks this is true for a lot of academic debates as well as much every day thinking.

What are the limitations of logical determinism?

  • Leads easily to confirmation bias. This confirmation bias exists in the way academics think about their phenomenon. It also biases the way they create studies and the way they, Dr. Matsumoto included, interpret data. 
  • Leads to what might call straw person arguments. One straw person argument heard all the time is “facial expressions of emotion are the only things that faces do” or that “they’re always reflective of an emotional state”. 

The thought that facial expressions are always reliable indicators of emotion is a straw person argument because no one who studies facial expressions of emotion today seriously believes that. 

There’s actually a recent survey of all of the most contemporary emotion researchers in the field that was published in 2016. A survey went out to about 250 of those researchers around the world and 88% of them believed there was compelling evidence for universals in any aspect of emotion. The vast majority of researchers in this field believe the existence of facial expressions of emotion but they don’t believe these extreme straw person arguments and no one does.

Faces do many, many things.

One very special thing that faces do is create facial expressions of emotion. We know that our faces can create thousands of behaviors.

We also know that our facial behavior has many other different functions such as:

  • Signal cognition and cognitive processes
  • Signal specific verbal words or phrases
  • Speech articulation
  • Signals of physical exertion or physical effort
  • Idiosyncratic things

Because of these multiple functions of facial expressions, it makes perfect sense that some experiments will find (under some conditions), that facial behaviors are not necessarily a signal of an emotion. There’s no question about that.

But what is also true is that when a true and strong emotional reaction is spontaneously triggered, and the closer that reaction is to something that is really meaningful in our lives, that will produce the impulse to create a facial expression of emotion in people all around the world.

The link between a spontaneous, strong, intense, meaningful emotional reaction and a corresponding facial expression has never been refuted by any study.

There have been many other studies about other aspects of the face, especially studies where people are judging faces. But no study that has actually elicited a meaningful, intense, emotional reaction *spontaneously* has shown otherwise. In addition, there are a lot of studies that have shown that the face does many, many other things sometimes with the same muscles we use for emotion signaling. 

It is necessary to understand the entirety of the data in terms of the complexity of the face.

What is it about the question of universality or not that gets people so heated?

Perhaps the question about universality is somehow related to how we see ourselves and humankind; whether we see humans as fundamentally similar or somehow different. It is a deep, philosophical question with no clear answers.

Although he doesn’t agree with all the interpretations that are made of the data, Dr. Matsumoto believes that we can find ways to understand the totality of the data without negating one side or the other. 

To learn more about the seven basic emotions, visit this past blog

Covid-19, Remote Communication, and Body Language

We are all facing an unprecedented challenge regarding the coronavirus pandemic. Many countries around the world are on lockdown. Businesses have been forced to close and most people have been mandated to stay at home for weeks, if not months.

As a result, communication has also undoubtedly changed. Face to face meetings, interviews and negotiations are now taking place virtually or by phone. In fact, Zoom, a cloud-based video conferencing service, has seen daily users more than quadruple during the past several weeks.

How has this shift from in-person to digital communication changed the way we read people and their body language? What are some of the long-term implications of exclusive or predominant use of virtual meetings? And finally, given the loss of contextual and nonverbal cues in remote meetings, what are some tips on how to have more effective communication?

We sat down with Humintell Director and Psychologist Dr. David Matsumoto to discuss his thoughts on the questions above.

Humans Did Not Evolve To Do 2-dimensional Communication 

The recent shift to almost exclusive digital communication has undoubtedly changed the way we read people. Dr. Matsumoto suggests that this largely stems from the fact that humans did not evolve to do 2-dimensional communication, such as through a computer screen. On the contrary, we have evolved our perceptual senses to live in a 3-dimensional world and our sense of reality is grounded in that fact.

Interactions are based on being live and in person. Being live and in person with somebody and interacting with them is what we’ve evolved to do and what we’ve learned to do all of our lives.

It is true that when communicating digitally, the visual stimulation of a person’s behavior or in their facial expressions is almost the same as being face to face, especially given the level of technology that we have now. In addition, your auditory perception (hearing ability) is present, since you can clearly hear the other person’s tone of voice. But there are clearly some substantial differences between being live vs virtual, including a reduction in nonverbal communication and nonverbal behavior.

What are some of those differences?

Real-Life vs Digital Communication Differences

In a live context when we interact with others, we receive what Dr. Matsumoto calls “full nonverbal packages”. This full package not only includes the face and facial expressions, but also includes voice, gestures, body posture, as well as leg, hand and foot movements.  In-person we also have the additional ability to sense spacing between one another, hear sounds and experience various smells.

Even though we can communicate through these remote communication technologies, there is an overall reduction in the information that we can get nonverbally.

Virtually we have much less of these observable cues. You typically cannot see the whole body and therefore cannot see many of the body movements from the waist down that you can typically see live. There are more distractions: on screen, you can see yourself and focus on what you look like. People’s backgrounds are static. It’s easy to pay less attention to the person you’re speaking to.

Digitally there is also a big data reduction in the ability to detect subtle expressions of the face. 3-dimensional (3D) settings are especially important for reading faces. Expressions are appearance changes that occur because of wrinkle patterns and how light falls on those wrinkle contours. Thus, 2-dimensional (2D) perception cuts down on the number of stimuli we can get to make inferences to read facial expressions of emotion.

What other differences are there?

Context Matters

It’s clear that seeing people’s faces is different than just having audio. However, it’s also clear that even though you may be able to see people’s faces in a remote meeting, that’s different than being live.

Perhaps the biggest way digital communication has changed the way we interact with each other, has to do with context. 

In addition to the 2D and 3D differences mentioned above, Dr. Matsumoto suggests differences between virtual and in-person interactions also has to do with the priming in our minds of the setting or context we are in. In any interaction, there is contextualization that occurs in our head that primes us to be more sensitive to certain things.

From the time we are a few months old, social and cultural rules are all contextualized and transmitted through specific contexts. Everyone around the world learns rules about how to behave, how to think, how to feel, how to act from when we’re infants. Dr. Matsumoto calls this enculturation.

By the time we’re adults, our two major contexts are typically work and home. We learn to think different, act differently, and engage with people differently in those contexts. But now that work is in our home, it’s easy to be confused. We have a different mindset and all our neural networks that tie all our learning, thinking and feelings together are disjointed. There is a lack of depth of visual cues and lack of depth of cognitive processing.

Long Term Implications

In addition to the context and priming differences, being at home means a vast reduction in human contact and interaction we normally have in our daily lives, especially with a variety of people. We probably “see” our family members and friends less frequently, if at all.

The sobering truth is that the current shelters in place around the country and world although necessary, facilitate more social isolation, the effects of which are wide and pervasive. These implications are greater if social isolation is longer.

In addition, the tracking of psychological effects of the increased use of mobile technology and social media has been studied widely. Psychologist Jean Twenge wrote an excellent article for The Atlantic entitled “Have Smartphones Destroyed a Generation?”

Rates of teen depression and suicide have skyrocketed since 2011. It’s not an exaggeration to describe iGen as being on the brink of the worst mental-health crisis in decades. Much of this deterioration can be traced to their phones.

Because it is clear social isolation and withdrawal can have extreme negative impacts, Dr. Matsumoto suggests we need to ensure that we also create time to have actual in-person human interactions. This is good not only for the relationship but keeping our mental health.

Tips on How to Have More Effective Communication

Many of us are currently trying to strike a balance while working from home. What are some tips Dr. Matsumoto has for having more effective communication while communicating virtually? See below!

  • Create a context that is conducive to have work meetings

If possible, carve out a physical workspace at home and treat that space like you would a work environment. If you have a meeting, dress like you’re going to a meeting. Context matters!

  • Check your appearance

Now that we can now see ourselves on screen, it’s good to know how we appear. See what you look like before the meeting; check your camera angle. Check your background, sounds, lighting to ensure it’s appropriate.

  • Stay focused on the interaction

It’s easy to get distracted and to think that others not paying attention

  • Use nonverbal cues to confirm understanding

Use nonverbal cues – faces and voices – along with verbal content and confirm understanding. These remote meetings may take longer, and we may need to make more effort to make sure we’re all on the same page. Ask questions. Confirm understanding verbally. Not being on the same page can have dire consequences.

  • Record and re-watch

If your service allows recording of your interactions, check out how you come off. It’s a great learning experience to see yourself.

The differences between in-person and virtual communication are pervasive and we must be realistic about our expectations and understand that there will be a reduction in nonverbal communication and nonverbal behavior.

Yet, we must also understand that our current situation is temporary. With patience and understanding we will overcome these unprecedented challenges!