Microsoft Research Demos Motion Sensing Keyboard with Gestures

Microsoft Research has demoed a new augmented mechanical keyboard that allows users to use motion controls either on or slightly above the keyboard to interact with their computers. The prototype was curiously made using keycaps from an Apple keyboard.

gesture

A low-resolution matrix of infrared (IR) proximity sensors is interspersed with the keys of a regular mechanical keyboard. This results in coarse but high frame-rate motion data. We extend a machine learning algorithm, traditionally used for static classification only, to robustly support dynamic, temporal gestures.

The gestures that the keyboard can recognize include traditional gestures like "pinch-to-zoom" and swiping to static gestures like holding a single finger above the keyboard to advanced gestures like turning an imaginary steering wheel.


It's unclear whether the augmented keyboard, referred to as the Type-Hover-Swipe, could eventually turn into a real product, but it's another step in Microsoft exploring motion control after its Kinect device. Apple, too, has been exploring motion sensing with its purchase of PrimeSense, the 3D body sensing firm that developed the technology for Kinect. The Type-Hover-Swipe keyboard is also similar to the technology of Leap Motion, who former Apple iAd chief Andy Miller worked for for close to two years.

Popular Stories

iOS 26

15 New Things Your iPhone Can Do in iOS 26.2

Friday December 5, 2025 9:40 am PST by
Apple is about to release iOS 26.2, the second major point update for iPhones since iOS 26 was rolled out in September, and there are at least 15 notable changes and improvements worth checking out. We've rounded them up below. Apple is expected to roll out iOS 26.2 to compatible devices sometime between December 8 and December 16. When the update drops, you can check Apple's servers for the ...
Intel Inside iPhone Feature

Apple's Return to Intel Rumored to Extend to iPhone

Friday December 5, 2025 10:08 am PST by
Intel is expected to begin supplying some Mac and iPad chips in a few years, and the latest rumor claims the partnership might extend to the iPhone. In a research note with investment firm GF Securities this week, obtained by MacRumors, analyst Jeff Pu said he and his colleagues "now expect" Intel to reach a supply deal with Apple for at least some non-pro iPhone chips starting in 2028....
ive and altman

Jony Ive's OpenAI Device Barred From Using 'io' Name

Friday December 5, 2025 6:22 am PST by
A U.S. appeals court has upheld a temporary restraining order that prevents OpenAI and Jony Ive's new hardware venture from using the name "io" for products similar to those planned by AI audio startup iyO, Bloomberg Law reports. iyO sued OpenAI earlier this year after the latter announced its partnership with Ive's new firm, arguing that OpenAI's planned "io" branding was too close to its...
iPhone 17 Pro Cosmic Orange

10 Reasons to Wait for Next Year's iPhone 18 Pro

Monday December 1, 2025 2:40 am PST by
Apple's iPhone development roadmap runs several years into the future and the company is continually working with suppliers on several successive iPhone models at the same time, which is why we often get rumored features months ahead of launch. The iPhone 18 series is no different, and we already have a good idea of what to expect for the iPhone 18 Pro and iPhone 18 Pro Max. One thing worth...
Photos App Icon Liquid Glass

John Gruber Shares Scathing Commentary About Apple's Departing Software Design Chief

Thursday December 4, 2025 9:30 am PST by
In a statement shared with Bloomberg on Wednesday, Apple confirmed that its software design chief Alan Dye will be leaving. Apple said Dye will be succeeded by Stephen Lemay, who has been a software designer at the company since 1999. Meta CEO Mark Zuckerberg announced that Dye will lead a new creative studio within the company's AR/VR division Reality Labs. On his blog Daring Fireball,...
Apple John Ternus 2019

Will John Ternus Really Be Apple's Next CEO?

Friday December 5, 2025 9:01 am PST by
There is uncertainty about Apple's head of hardware engineering John Ternus succeeding Tim Cook as CEO, The Information reports. Some former Apple executives apparently hope that a new "dark-horse" candidate will emerge. Ternus is considered to be the most likely candidate to succeed Cook as CEO. The report notes that he is more likely to become CEO than software head chief Craig Federighi, ...
maxresdefault

iPhone Fold: Launch, Pricing, and What to Expect From Apple's Foldable

Monday December 1, 2025 3:00 am PST by
Apple is expected to launch a new foldable iPhone next year, based on multiple rumors and credible sources. The long-awaited device has been rumored for years now, but signs increasingly suggest that 2026 could indeed be the year that Apple releases its first foldable device. Subscribe to the MacRumors YouTube channel for more videos. Below, we've collated an updated set of key details that ...
ios 18 to ios 26 upgrade

Apple Pushes iPhone Users Still on iOS 18 to Upgrade to iOS 26

Tuesday December 2, 2025 11:09 am PST by
Apple is encouraging iPhone users who are still running iOS 18 to upgrade to iOS 26 by making the iOS 26 software upgrade option more prominent. Since iOS 26 launched in September, it has been displayed as an optional upgrade at the bottom of the Software Update interface in the Settings app. iOS 18 has been the default operating system option, and users running iOS 18 have seen iOS 18...
iOS 26

Apple Seeds iOS 26.2 and iPadOS 26.2 Release Candidates to Developers and Public Beta Testers

Wednesday December 3, 2025 10:33 am PST by
Apple today seeded the release candidate versions of upcoming iOS 26.2 and iPadOS 26.2 updates to developers and public beta testers, with the software coming two weeks after Apple seeded the third betas. The release candidates represent the final versions of iOS 26.2 and iPadOS 26.2 that will be provided to the public if no further bugs are found during this final week of testing....
Touchscreen MacBook Feature

Here Are the Four MacBooks Apple Is Expected to Launch Next Year

Monday December 1, 2025 5:00 am PST by
2026 could be a bumper year for Apple's Mac lineup, with the company expected to announce as many as four separate MacBook launches. Rumors suggest Apple will court both ends of the consumer spectrum, with more affordable options for students and feature-rich premium lines for users that seek the highest specifications from a laptop. Below is a breakdown of what we're expecting over the next ...

Top Rated Comments

Gasu E. Avatar
152 months ago
This is how you know Microsoft still doesn't have it. Regardless of whatever this technology is, You don't go out and make a video showing off how innovative you are with some demo technology. You use the thing in an actual product to prove it's real worth.

Microsoft Research does research. Hence the word "research" in its name.
Score: 21 Votes (Like | Disagree)
2IS Avatar
152 months ago
The point a lot of people here are trying to make is that at least with the Apple way, it's a shipped product that you can go out now and purchase with your money.

The Microsoft way just exists in this video. Good luck actually using it any time soon.

Yeah? And when was the last time Apple did something remotely exciting? There's nothing wrong with showing off a concept. If there is a viable application for it and can be successfully marketed and sold, someone will run with it.

Thinner iPhone 6? How exciting is that? I can tell you, when I purchased my iPhone 5, I wasn't thinking "I'll bet the iPhone 6 will be fatter"

Someone's thinking outside the box, and it isn't :apple:

I personally don't have any interest in this keyboard, but as an enthusiast of new technology, I'm glad to see someone isn't afraid to do something new.
Score: 15 Votes (Like | Disagree)
2IS Avatar
152 months ago
This is how you know Microsoft still doesn't have it. Regardless of whatever this technology is, You don't go out and make a video showing off how innovative you are with some demo technology. You use the thing in an actual product to prove it's real worth.

I will guarantee this thing is a PURE gimmick. They're comfortable showing you a video of something they themselves don't know the value of.

Contrary to what you may think, the Apple way isn't the only way, nor is it necessarily the correct way.
Score: 14 Votes (Like | Disagree)
imageWIS Avatar
152 months ago
Interactive porn in 3, 2, 1...
Score: 13 Votes (Like | Disagree)
Gasu E. Avatar
152 months ago
I don't want thing because it's a thing yet! :mad:

Ain't no thing.

----------

agreed, maybe they should put the time and money into a developing a secure OS before they make gimmicky gesture keyboard.

Ah, the traditional MacRumors "false trade-off" fallacy.
Score: 12 Votes (Like | Disagree)
therealtruth Avatar
152 months ago
This is how you know Microsoft still doesn't have it. Regardless of whatever this technology is, You don't go out and make a video showing off how innovative you are with some demo technology. You use the thing in an actual product to prove it's real worth.

I will guarantee this thing is a PURE gimmick. They're comfortable showing you a video of something they themselves don't know the value of.
Score: 10 Votes (Like | Disagree)