Your Smartphone Spies On What You Type
Written by Mike James   
Sunday, 29 September 2013

All you have to do is place your phone next to your keyboard to provide a direct channel for anyone to read what you are typing - and it's all down to the vibration of the keys.

We all do it - place our phones down on the desk next to the keyboard. This might not be such a good idea if you want to keep your work to yourself. 

keyboardspy

A team of researchers from MIT and the Georgia Institute of Technology have provided proof of concept for logging keystrokes using nothing but the sensors inside a smartphone - an iPhone 4 to be precise.  

The first interesting finding is that the accelerometer inside an iPhone 3GS wasn't good enough to pick up the vibrations of the keys but the iPhone 4 produced a clear signal.  This demonstrates not only that sensors are getting better, but the improvement makes a real difference. 

The iPhone 4 was placed a few inches away from a keyboard and the keys were pressed while a sensor data was fed to two neural networks. The data was processed to produce a feature vector based on a range of temporal and frequency information. One network  was trained to recognize the horizontal position of the key and the other the vertical position. The scan rate was on the low side, 100Hz, due to the limitations fo the hardware. 

Just detecting raw keypresses didn't produce a very accurate result, but switching to picking up pairs of keywords and then using a word dictionary did produce useful data extraction. Accuracies of around 80% were achieved, but the accuracy reduced with the number of keypresses. Word recognition only achieved a 46% accuracy, but this increased to 73% if second choice words were included. Clearly semantic analysis could push the accuracy up. 

 

keyprocess

 

There have been experiments that recover text by listening to the noise a keyboard makes and these have achieved slightly higher accuracies that this vibration-based approach. The key difference is that this experiment used a smartphone rather than a desktop machine with a big microphone and high-speed sampling. The method may be less accurate, but it could be packaged into smartphone malware and be used without anyone knowing it was going on.  

The paper describes some problems with the method - if the phone is too far from the keyboard then the vibrations aren't strong enough, if  the user taps on the desk or plays a radio then there is too much interference and if the user types too fast the processor can't keep up.

So how to stop such eavesdropping?

Leave the phone outside the room or keep in in a bag next to the desk rather than on it. The researchers also suggest that in future APIs could be modified to restrict the sampling rate to below the Nyquist rate for the data being listened into - say 30Hz in this case. 

It is interesting to speculate on how much better an engineered design might do than the neural network learning approach. The accelerometer acts much like a seismometer and geologists have a lot of techniques for using such data to work out where an earthquake happened. Perhaps the same signal processing techniques could pinpoint the location of the key vibrations without the need to use supervised learning.

Whatever the final outcome, it is clear that with a smartphone in your possession secrecy and privacy are far from certain. Be careful the next time someone leaves their phone on your desk. 

keyboardspy

More Information

Decoding Vibrations From Nearby Keyboards Using Mobile Phone Accelerometers

Related Articles

Smartphone Apps Track Users Even When Off

PlaceRaider - Your Phone Can Steal A 3D Model Of Your Location

 UnLoc - An App That Learns Where You Are Indoors

Siri a Security Risk for IBM 

 

To be informed about new articles on I Programmer, install the I Programmer Toolbar, subscribe to the RSS feed, follow us on, Twitter, FacebookGoogle+ or Linkedin,  or sign up for our weekly newsletter.

 

raspberry pi books

 

Comments




or email your comment to: comments@i-programmer.info

 

Banner


AWS Adds Support For Llama2 And Mistral To SageMaker Canvas
12/03/2024

As part of its effort to enable its customers to use generative AI for tasks such as content generation and summarization, Amazon has added these state of the art LLMs to SageMaker Canvas.



GitHub Enterprise Server Adds Deployment Rollout Controls
11/03/2024

Version 3.12 of GitHub Enterprise Server, the self-hosted version of GitHub that organizations can install on their own servers, has been released with support for restricting deployment rollouts [ ... ]


More News

 

 

 

Last Updated ( Sunday, 29 September 2013 )