This is another simple idea with results that go well beyond what you might expect. Take a tub of cloudy water, a projector, a Kinect to sense objects on its surface and you have the first "immersive" interface - literally. You can place your hands in the water and play with the virtual objects projected onto it. See it!
If you put some bath salts in a tub of water the result is a milky liquid, which makes a really good projection surface - so why not project something on to it. This is what the team at Tokyo's University of Electro-Communications Koike Laboratory and the results are more impressive than you might expect.
A projector and a Kinect depth camera work together with some software to allow the user to interact with the water in some amazing ways.
To start off with the Kinect can track the hand and finger positons so that it can interpret gestures. If the system projects photos on the surface, for example, you can move them around, resize them using the usual two-finger pinch, but you can also pick them up in cupped hands and transfer them somewhere else. The gesture I really liked was "sink to delete" - yes, that's often how I feel about a file.
But there is more.
Add some waterproof loudspeakers under the surface and allow the computer to run them at low frequency. The result is that you can now make the surface "boil" in response to the sound. You can make fountains of water appear and project the right colors onto it to make it look like an explosion. The same system can provide haptic feedback so that you feel the water as you push against in different ways. Put all of this together and you have the first water-based shoot-em-up. Throw bolts of fire at squid that explode when hit. It looks fun, if slightly wet.
Watch the video and be patient, it's a slow start but well worth the wait:
You have to admit its impressive. It looks fun and who wouldn't want to have a go at the games. But water? It is difficult to imagine a way that AquaTop could make it as a mainstream (pun intended) interface. So nice idea but...
What about using a non-Newtonian fluid-like cornstarch? Then the loudspeakers could instantly change the viscosity of the fluid You could be moving through it like water one minute and then have it set like concrete around your hand. The same Kinect-based feedback might just make it interesting.
What AquaTop does demonstrate is that we haven't yet seen the end of novel ideas using the Kinect.