Your Phone or Mine? Fusing Body, Touch and Device
Written by Kay Ewbank   
Thursday, 31 May 2012

Another novel use for the Kinect. It can work out which of multiple users is interacting with a multi-user interactive
touch display.

How can you work out who’s touching a multi-user interactive touch display?

It’s all very well writing apps that users can interact with by touching, but at the moment there’s no way your app can automatically tell who’s doing the touching if you allow multiple people to touch the screen.

Researchers at Microsoft Research have published a paper suggesting a way that you could work out who it is.

 

 shake1

 

The method makes use of data from a smartphone’s on-board sensors together with camera-based body tracking. ShakeID can work out who’s touching the screen while holding their mobile phone.

So long as each user is holding a smartphone or other portable device that can sense its own movement, ShakeID matches the motion sensed by the device to motion observed by a Microsoft Kinect camera pointed at the users standing in front of the touch display. By comparing the movements of each phone in the scene with the movements of each user, the system can associate each phone to a specific user’s hand.

It then transforms the 2D space of the display to the 3D camera space, and associates the touches on the display to specific users. This means the app can work out which users are doing the touching, and which devices they’re holding.

The research paper gives the example of two users touching a display simultaneously in different locations to grab content, and ShakeID being able to associate each touch to a specific user and transfer the correct content to each user’s personal device. By using Kinect tracking data, the technique can also work out which hand is holding the device, and can detect when users move away from the screen and log the user out from the session automatically.

 

shake2

 

In the tests carried out of the technique, the users were told that phone movement was being used to associate devices with people and if they experienced any problems with the system not recognising their touch, they could shake their phone to improve the accuracy - what the researchers call ‘shake to associate’.

The current work matches the phone to the users’ hands, but the researchers suggest the technique may work when matching to body parts other than hands. They suggest it may be possible to successfully associate a device to a user if it is in the user’s pants pocket by matching to users’ hip joints data from the Kinect SDK. Or putting it another way, wiggle your hips to associate.

Not a suggestion often heard in software development, but it may just catch on.

More Information

Your Phone or Mine? Fusing Body, Touch and Device Sensing for Multi-User Device-Display Interaction (pdf)

Related Articles

Kintinuous - Kinect Creates Full 3D Models

Kinect SDK 1.5 - Now With Face Tracking

LightGuide: Kinect Teaches Movement

Kinect 3D Full Body "Hologram"

 

To be informed about new articles on I Programmer, install the I Programmer Toolbar, subscribe to the RSS feed, follow us on, Twitter, Facebook, Google+ or Linkedin,  or sign up for our weekly newsletter.

Banner


Ibis 8 Adds Streaming
05/03/2024

Ibis 8.0 has been released with stream processing backends. The new release includes Apache Flink as a streaming backend, and RisingWave, a streaming database backend. There's also a new batch backend [ ... ]



Quantum Computers Really Are A One Trick Pony
17/03/2024

Google is offering $5 million if you can think up a use for a quantum computer. Wait, I thought quantum computers were the next big thing, a revolution! Surely we know what they can do?


More News

raspberry pi books

 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Friday, 01 June 2012 )