Determining What Video Conference Participants Are Typing from Watching Shoulder Movements

guest · Nov 4, 2020

Determining What Video Conference Participants Are Typing from Watching Shoulder Movements
November 4, 2020
https://www.schneier.com/blog/archi...-typing-from-watching-shoulder-movements.html

Accuracy isn’t great, but that it can be done at all is impressive.

Murtuza Jadiwala, a computer science professor heading the research project, said his team was able to identify the contents of texts by examining body movement of the participants. Specifically, they focused on the movement of their shoulders and arms to extrapolate the actions of their fingers as they typed.

Given the widespread use of high-resolution web cams during conference calls, Jadiwala was able to record and analyze slight pixel shifts around users’ shoulders to determine if they were moving left or right, forward or backward. He then created a software program that linked the movements to a list of commonly used words.

In a controlled setting, with specific chairs, keyboards and webcam, Jadiwala said he achieved an accuracy rate of 75 percent.
Click to expand...

longshots · Nov 4, 2020

In a controlled setting, with specific chairs, keyboards and webcam, Jadiwala said he achieved an accuracy rate of 75 percent.
Click to expand...

That's for the promo press release.

However, in uncontrolled environments, accuracy dropped to only one out of every five words being correctly identified.
Click to expand...

Even this is mind boggling.

What a shame we can't watch this experiment on YouTube and verify for ourselves that this is not just another chest beating exercise.

guest · Nov 5, 2020

Zoom Snooping: How Body Language Can Spill Your Password
November 5, 2020
https://threatpost.com/zoom-snooping-passwords/161000/

This week, a group of University of Texas researchers published a technical brief on what they claim is a reliable framework – using shoulder movements (PDF) – that can determine what someone on the other end of a Zoom, Microsoft Skype or Google Hangouts video call is typing.
Can Your Shoulders REALLY Reveal What You’re Typing?
He added most users are typically doing other stuff while participating in video calls and a lot of those tasks involve typing.

“One of the reasons our attack framework targets image frames (in the video call) containing upper body/shoulders of the user is because that is the only portion of the body that is typically visible in most video calls.”
Public-interest technologist Bruce Schneier recently highlighted the research on his blog, writing the, “Accuracy isn’t great, but that it can be done at all is impressive.”
Click to expand...

Users who are concerned about their keystrokes being mapped over video conferencing can take a few simple steps to protect their data, according to Jadliwala and his team.

First, use existing tools to blur the background during calls. The researchers experimented with blurring and found it cut their ability to decipher words from 65 percent down to as low as 13 percent.

“These results show that blurring is an effective mitigation technique, which imposes little efficiency and quality overheads,” the report said.
Pixelation and frame skipping, much like blurring, were also effective at mitigating the team’s ability to read keystrokes, according to the report.
Click to expand...

Jadliwala adds there is no reason for alarm and no threat attempts using video conferencing to interpret keystrokes have been observed.
Click to expand...

arXiv: Zoom on the Keystrokes: Exploiting Video Calls for Keystroke Inference Attacks
(PDF): https://arxiv.org/pdf/2010.12078.pdf

Log in or Sign up

Determining What Video Conference Participants Are Typing from Watching Shoulder Movements

guest Guest

longshots Registered Member

guest Guest

Log in or Sign up

Determining What Video Conference Participants Are Typing from Watching Shoulder Movements

guest Guest

longshots Registered Member

guest Guest

Useful Searches