Your Phone or Mine? Fusing Body, Touch and Device
Written by Kay Ewbank   
Thursday, 31 May 2012

Another novel use for the Kinect. It can work out which of multiple users is interacting with a multi-user interactive
touch display.

How can you work out who’s touching a multi-user interactive touch display?

It’s all very well writing apps that users can interact with by touching, but at the moment there’s no way your app can automatically tell who’s doing the touching if you allow multiple people to touch the screen.

Researchers at Microsoft Research have published a paper suggesting a way that you could work out who it is.

 

 shake1

 

The method makes use of data from a smartphone’s on-board sensors together with camera-based body tracking. ShakeID can work out who’s touching the screen while holding their mobile phone.

So long as each user is holding a smartphone or other portable device that can sense its own movement, ShakeID matches the motion sensed by the device to motion observed by a Microsoft Kinect camera pointed at the users standing in front of the touch display. By comparing the movements of each phone in the scene with the movements of each user, the system can associate each phone to a specific user’s hand.

It then transforms the 2D space of the display to the 3D camera space, and associates the touches on the display to specific users. This means the app can work out which users are doing the touching, and which devices they’re holding.

The research paper gives the example of two users touching a display simultaneously in different locations to grab content, and ShakeID being able to associate each touch to a specific user and transfer the correct content to each user’s personal device. By using Kinect tracking data, the technique can also work out which hand is holding the device, and can detect when users move away from the screen and log the user out from the session automatically.

 

shake2

 

In the tests carried out of the technique, the users were told that phone movement was being used to associate devices with people and if they experienced any problems with the system not recognising their touch, they could shake their phone to improve the accuracy - what the researchers call ‘shake to associate’.

The current work matches the phone to the users’ hands, but the researchers suggest the technique may work when matching to body parts other than hands. They suggest it may be possible to successfully associate a device to a user if it is in the user’s pants pocket by matching to users’ hip joints data from the Kinect SDK. Or putting it another way, wiggle your hips to associate.

Not a suggestion often heard in software development, but it may just catch on.

More Information

Your Phone or Mine? Fusing Body, Touch and Device Sensing for Multi-User Device-Display Interaction (pdf)

Related Articles

Kintinuous - Kinect Creates Full 3D Models

Kinect SDK 1.5 - Now With Face Tracking

LightGuide: Kinect Teaches Movement

Kinect 3D Full Body "Hologram"

 

To be informed about new articles on I Programmer, install the I Programmer Toolbar, subscribe to the RSS feed, follow us on, Twitter, Facebook, Google+ or Linkedin,  or sign up for our weekly newsletter.

Banner


$10K Contest To Solve A Problem Worth Millions
06/10/2014

VistaPrint is offering $10,000 for algorithms that help it solve its packing problems. Is this a fair challenge or just downright exploitative?



Doctor Who Teaches Programming
22/10/2014

Dr Who is a sci fi hero to a great many kids and not so kid like techies. What could be more logical than to get him, and everyone's favourite killing machine, the daleks to teach programming and all  [ ... ]


More News

blog comments powered by Disqus

Last Updated ( Friday, 01 June 2012 )
 
 

   
RSS feed of news items only
I Programmer News
Copyright © 2014 i-programmer.info. All Rights Reserved.
Joomla! is Free Software released under the GNU/GPL License.