Smile, You're in VR: HTC Device Can Translate Lip Movement Into Virtual Realty

A new device from HTC can translate your facial expressions, such as a smile or a laugh, into virtual reality. 

HTC's Vive Facial Tracker can capture your facial and mouth movements and then render them on a VR avatar in real-time. According to HTC, the product can track up to 38 facial “shapes” made via your lips, jaw, teeth, tongue, cheeks, and chin movement. All the facial expressions can be captured in under 10 milliseconds, and then rendered back at a 60Hz refresh rate. As a result, the device can take your lip movements as you talk and reflect them onto a VR avatar.  

A VR game developer called Frooxious has uploaded a video showing the technology in action. As you can see, the device can reproduce facial gestures in VR to varying degrees of success.

The facial tracking is done through two cameras in the device, which can capture the movements even in low light via infrared. 

HTC plans on selling the product for $129 as an accessory. However, the company has yet to say when it’ll officially go on sale. For now, HTC seems to be marketing the device to software developers who want to create more life-like interactions in their VR applications.  

The other limitation is how the product will only be compatible with the Vive Pro VR headsets, and not the more consumer-focused Cosmo series. In addition, HTC says the device works best with the enterprise-grade $1,399 Vive Pro Eye headset for full facial tracking. 

Recommended by Our Editors

Doom 3 Announced for PlayStation VR

Sony Teases Next-Gen PlayStation VR for PS5

Germany: Requiring Oculus VR Users to Have Facebook Account May Be 'Abuse of Dominance'

Whether any upcoming games will tap the technology was left unsaid. But the device does support the two biggest 3D game engines on the market, Unreal Engine and Unity.

HTC isn't alone in trying to bring facial movements into VR. Facebook's Oculus division has also been working on the technology in the hopes of creating realistic VR avatars.