Using Wearable Device to Reconstruct Facial Expressions

A new wearable earphone device known as earable bounces sound off the cheeks and changes the echoes into an avatar of a person’s entire moving face, utilizing acoustic technology to offer better privacy. A team led by Cheng Zhang, assistant professor of information science, and François Guimbretière, professor of information science, designed the system, named EarIO. It transmits facial movements to a smartphone in real time and is compatible with commercially available headsets for hands-free, cordless video conferencing.

The team described their earable in “EarIO: A Low-power Acoustic Sensing Earable for Continuously Tracking Detailed Facial Movements”.

’Earable’ Uses Sonar for Facial Expressions Reconstruction

The EarIO works like a ship sending out pulses of sonar. A speaker on each side of the earphone sends acoustic signals to the sides of the face and a microphone picks up the echoes. As wearers talk, smile or raise their eyebrows, the skin moves and stretches, changing the echo profiles. A deep learning algorithm developed by the researchers uses artificial intelligence to continually process the data and translate the shifting echoes into complete facial expressions.

By collecting sound instead of data-heavy images, the earable can communicate with a smartphone through a wireless Bluetooth connection, keeping the user’s information private. With images, the device would need to connect to a Wi-Fi network and send data back and forth to the cloud, potentially making it vulnerable to hackers.

“People may not realize how smart wearables are – what that information says about you, and what companies can do with that information,” Guimbretière said. With images of the face, someone could also infer emotions and actions. “The goal of this project is to be sure that all the information, which is very valuable to your privacy, is always under your control and computed locally.”

Source-Eurekalert