Tom Cruise in Minority Report
When Tom Cruise used a gesture-driven video display to search for future criminals in the film Minority Report he was interacting with a CGI-enhanced setup designed in part by John Underkoffler, a former PhD student of MIT’s Tangible Media Group. In a fantasy-become-reality scenario, Underkoffler went on to form Oblong Industries, a design group that released a first version of G-Speak, a working version of the gestural interface from the movie, in November 2008.
A portable technology that takes a related role is SixthSense , developed by Pranav Mistry at MIT Media Lab’s Fluid Interfaces Group – a wearable computer that projects its display onto any surface and uses hand or finger gestures for interaction. For example, users can take a snapshot of a landscape scene simply by framing the scene with their (colour-coded) fingertips, similar to Tom’s gesture in the image above. This technology extends the “multi-touch” concept into a “multi-gesture” mode, allowing users to engage in more complex forms of visual interaction. Watch a video of the FIG’s Professor Patti Maes describing the system at a TED talk in February 2009 below.
Critics of gestural interfaces usually point out that existing interface devices such as touchpads, mice and keyboards require less physical effort than gesticulating in space with arms extended, as the G-Speak interface seems to require. Personally, I think this type of interface has a place in certain disciplines where creative manipulation of virtual or real 3D objects can enhance learning. Virtually conducting a synthesised orchestra could be one way of exploiting the potential of a gestural interface. Conductors could quickly experiment with the placement of virtual orchestral performers – maybe bring the French Horns in closer and push the Harpist off to the left, etc.
Adding force feedback to the interface opens up further potential for creative interaction or precise procedural activity. Intuitive Surgical’s da Vinci Surgical System provides surgeons with sufficient tactile feedback to allow precision surgery where the actual procedure is done entirely by a human-guided robot. Virtual musical instruments such as bowed devices can provide musicians with sufficient synthetic feedback to create a virtuoso performance or create entirely new forms of music. Get in touch with your inner interface soon.