Researchers at Rutgers University-New Brunswick have published "Face-Mic," the first work examining how voice command features on virtual reality headsets could lead to major privacy leakages, known as "eavesdropping attacks."
The research shows that hackers could use popular virtual reality (AR/VR) headsets with built in motion sensors to record subtle, speech-associated facial dynamics to steal sensitive information communicated via voice-command, including credit card data and passwords.
Common AR/VR systems on the market include the popular brands Oculus Quest 2, HTC Vive Pro, and PlayStation VR.
Led by Yingying "Jennifer" Chen, associate director of WINLAB and graduate director of Electrical and Computer Engineering at Rutgers University-New Brunswick, the study will be presented at the annual International Conference on Mobile Computing and Networking in March. Other research collaborators include Nitesh Saxena of Texas A&M University and Jian Liu at University of Tennessee at Knoxville.
To demonstrate the existence of security vulnerabilities, Chen and her fellow WINLAB researchers developed an eavesdropping attack targeting AR/VR headsets, known as "Face-Mic."
"Face-Mic is the first work that infers private and sensitive information by leveraging the facial dynamics associated with live human speech while using face-mounted AR/VR devices," said Chen. "Our research demonstrates that Face-Mic can derive the headset wearer's sensitive information with four mainstream AR/VR headsets, including the most popular ones: Oculus Quest and HTC Vive Pro."
The researchers studied three types of vibrations captured by AR/VR headsets' motion sensors, including speech-associated facial movements, bone-borne vibrations and airborne vibrations. Chen noted that bone-borne vibrations in particular are richly encoded with detailed gender, identity and speech information.
"By analyzing the facial dynamics captured with the motion sensors, we found that both cardboard headsets and high-end headsets suffer security vulnerabilities, revealing a user's sensitive speech and speaker information without permission," Chen said.
Although vendors usually have policies regarding utilizing the voice access function in headset microphones, Chen's research found that built-in motion sensors, such as an accelerometer and gyroscope within a VR headset, do not require any permission to access. This security vulnerability can be exploited by malicious actors intent on committing eavesdropping attacks.
