I came across this interesting iPhone app today from the Innovation Engineblog, and thought I’d share. This is the future of mobile communications!
An astonishing app uses an iPhone’s accelerometer to sense the location of a tap on any surface and translate it into typed letters on a keyboard.
We have virtual keyboards, clip-on keyboards, magnetic keyboards and even laser keyboards. But what if your keyboard wasn’t a keyboard at all?
Florian Kräutli has developed an ingeniously simple alternative called the Vibrative Virtual Keyboard. By placing an iPhone on any surface, that surface becomes a keyboard. Technically, the phone’s accelerometer is measuring vibrations on that surface. Kräutli’s software maps those vibrations to a point of origin on the table. And when the phone can “
see” where you’re tapping, you can have a QWERTY keyboard on any tabletop.
“I wondered how a normal surface might become interactive, without using an elaborate combination of projectors and cameras, as Microsoft is currently doing in their research,” Kräutli explains. “There are already tons of sensors in current smartphones, so I thought there must be more that we can do with them.”
Kräutli was right. By using an Apple-approved iPhone app called SensorMonitor, he can access the raw sensor outputs of an iPhone via a network connection. The software he coded analyzes this sensor output on a networked Macbook. All the user needs to do is train a new surface–tap a few points and let the software know what letter those taps are supposed to be–and Kräutli’s software will number-crunch the positions for the rest of the keys. A user can then save this surface so the software won’t need a calibration for it again.
So how could this possibly be accurate? Machine learning is a powerful tool used successfully in many industries, but truth be told, the measurements still aren’t 100% clear. Instead of achieving flawless data, Kräutli has cleverly designed the software around this shortcoming.
“The important bit is that the software needs to deal with the fact that this recognition is not perfect,” Kräutli explains. “Therefore it also uses a kind of spell checker.”
Kräutli invisibly autocorrects typing, much like Apple’s own spell checker fixes mistakes, though at a deeper level of the application. Unfortunately, this design breakthrough won’t carry over very well to another killer application–gaming.
“When you play a game, you want every interaction to be recognized correctly,” he says. “Typically, you also want this when typing, but because the software ‘knows’ that you are writing with a certain vocabulary, it can correct errors more easily.”
For the same reasons, music creation is a no-go as well, since a computer could hardly predict your next move (unless you wanted every song to sound the same). Even still, Kräutli’s creation is a remarkable statement about the future of user interfaces, where conceivably, every surface becomes a conduit for digital input. But sadly, the OS X software isn’t available for download at this time.
