Microsoft Research Demos Motion Sensing Keyboard with Gestures

Microsoft Research has demoed a new augmented mechanical keyboard that allows users to use motion controls either on or slightly above the keyboard to interact with their computers. The prototype was curiously made using keycaps from an Apple keyboard.

gesture

A low-resolution matrix of infrared (IR) proximity sensors is interspersed with the keys of a regular mechanical keyboard. This results in coarse but high frame-rate motion data. We extend a machine learning algorithm, traditionally used for static classification only, to robustly support dynamic, temporal gestures.

The gestures that the keyboard can recognize include traditional gestures like "pinch-to-zoom" and swiping to static gestures like holding a single finger above the keyboard to advanced gestures like turning an imaginary steering wheel.


It's unclear whether the augmented keyboard, referred to as the Type-Hover-Swipe, could eventually turn into a real product, but it's another step in Microsoft exploring motion control after its Kinect device. Apple, too, has been exploring motion sensing with its purchase of PrimeSense, the 3D body sensing firm that developed the technology for Kinect. The Type-Hover-Swipe keyboard is also similar to the technology of Leap Motion, who former Apple iAd chief Andy Miller worked for for close to two years.

Popular Stories

AirPods Pro Firmware Feature

Apple Releases New Firmware for AirPods Pro 2, AirPods Pro 3, and AirPods 4

Thursday November 13, 2025 11:35 am PST by
Apple today released new firmware designed for the AirPods Pro 3, the AirPods 4, and the prior-generation AirPods Pro 2. The AirPods Pro 3 firmware is 8B25, while the AirPods Pro 2 and AirPods 4 firmware is 8B21, all up from the prior 8A358 firmware released in October. There's no word on what's include in the updated firmware, but the AirPods Pro 2, AirPods 4 with ANC, and AirPods Pro 3...
Tim Cook WWDC 2018

Report: Tim Cook to Step Down as Apple CEO 'as Soon as Next Year'

Saturday November 15, 2025 2:40 pm PST by
Apple is preparing for Tim Cook to step down as CEO of the company "as soon as next year," according to the Financial Times. The company's board of directors and senior executives "recently intensified preparations for Cook to hand over the reins," the report said. While the report said that Apple is unlikely to name a new CEO before its next earnings report in late January, it went on to ...
iPhone Pocket Short

iPhone Pocket Now Available to Order, But Already Selling Out

Friday November 14, 2025 6:20 am PST by
Apple recently teamed up with Japanese fashion brand ISSEY MIYAKE to create the iPhone Pocket, a limited-edition knitted accessory designed to carry an iPhone. iPhone Pocket is available to order on Apple's online store starting today, in the United States, France, China, Italy, Japan, Singapore, South Korea, and the United Kingdom. However, it is already completely sold out in the United...
apple silicon mac lineup 2024 feature purple m5

Apple's 2026 Mac Plans

Friday November 14, 2025 3:23 pm PST by
Most of Apple's Macs are slated to get M5 chips across 2026, and there's a possibility we'll even see the first M6 chips toward the end of the year. Updates are planned for everything from the MacBook Air to the Mac Studio. MacBook Air (Early 2026) The MacBook Air will be one of the first Macs to get a 2026 refresh, with an update planned for the first few months of the year. The MacBook...
best early black friday deals

Best Black Friday Apple Deals Live Now - Save on AirPods, iPads, and Apple Watches

Saturday November 15, 2025 1:45 pm PST by
We're officially in the month of Black Friday, which will take place on Friday, November 28 in 2025. As always, this will be the best time of the year to shop for great deals, including popular Apple products like AirPods, iPad, Apple Watch, and more. In this article, the majority of the discounts will be found on Amazon. Note: MacRumors is an affiliate partner with some of these vendors. When ...
tvOS 26 Profiles

tvOS 26.2 Adds a Useful New Feature to Your Apple TV

Friday November 14, 2025 10:02 am PST by
Starting with the upcoming tvOS 26.2 update, currently in beta, additional profiles created on the Apple TV no longer require their own Apple Account. In the Settings app on the Apple TV, under Profiles and Accounts, anyone can create a new profile by simply entering a name and indicating whether the profile is for a kid. The profile will be associated with the primary user's Apple Account,...
walmart new ornametns

Walmart Black Friday Deals Begin Today With Low Prices on Headphones, TVs, and More

Friday November 14, 2025 7:55 am PST by
Walmart's Black Friday sale has officially kicked off today, with an online shopping event that's also seeing some matching deals in retail locations. There are quite a few major discounts in this sale, including savings on headphones, TVs, and more. Note: MacRumors is an affiliate partner with Walmart. When you click a link and make a purchase, we may receive a small payment, which helps us...
CarPlay Pinned Messages

iOS 26.2 Adds New CarPlay Setting

Thursday November 13, 2025 6:48 am PST by
iOS 26 extended pinned conversations in the Messages app to CarPlay, for quick access to your most frequent chats. However, some drivers may prefer the classic view with a list of individual conversations only, and Apple now lets users choose. Apple released the second beta of iOS 26.2 this week, and it introduces a new CarPlay setting for turning off pinned conversations in the Messages...
CarPlay Liquid Glass Dark

Here's When Tesla is Expected to Add Support for Apple CarPlay

Sunday November 16, 2025 6:49 am PST by
In his Power On newsletter today, Bloomberg's Mark Gurman reiterated that Tesla plans to add Apple CarPlay support to its vehicles "in the coming months." This is easily the biggest news for the regular version of CarPlay in years, should Tesla actually follow through with offering the system in its vehicles. As noted by Gurman, this would be a stunning reversal for Tesla and its CEO Elon ...

Top Rated Comments

Gasu E. Avatar
151 months ago
This is how you know Microsoft still doesn't have it. Regardless of whatever this technology is, You don't go out and make a video showing off how innovative you are with some demo technology. You use the thing in an actual product to prove it's real worth.

Microsoft Research does research. Hence the word "research" in its name.
Score: 21 Votes (Like | Disagree)
2IS Avatar
151 months ago
The point a lot of people here are trying to make is that at least with the Apple way, it's a shipped product that you can go out now and purchase with your money.

The Microsoft way just exists in this video. Good luck actually using it any time soon.

Yeah? And when was the last time Apple did something remotely exciting? There's nothing wrong with showing off a concept. If there is a viable application for it and can be successfully marketed and sold, someone will run with it.

Thinner iPhone 6? How exciting is that? I can tell you, when I purchased my iPhone 5, I wasn't thinking "I'll bet the iPhone 6 will be fatter"

Someone's thinking outside the box, and it isn't :apple:

I personally don't have any interest in this keyboard, but as an enthusiast of new technology, I'm glad to see someone isn't afraid to do something new.
Score: 15 Votes (Like | Disagree)
2IS Avatar
151 months ago
This is how you know Microsoft still doesn't have it. Regardless of whatever this technology is, You don't go out and make a video showing off how innovative you are with some demo technology. You use the thing in an actual product to prove it's real worth.

I will guarantee this thing is a PURE gimmick. They're comfortable showing you a video of something they themselves don't know the value of.

Contrary to what you may think, the Apple way isn't the only way, nor is it necessarily the correct way.
Score: 14 Votes (Like | Disagree)
imageWIS Avatar
151 months ago
Interactive porn in 3, 2, 1...
Score: 13 Votes (Like | Disagree)
Gasu E. Avatar
151 months ago
I don't want thing because it's a thing yet! :mad:

Ain't no thing.

----------

agreed, maybe they should put the time and money into a developing a secure OS before they make gimmicky gesture keyboard.

Ah, the traditional MacRumors "false trade-off" fallacy.
Score: 12 Votes (Like | Disagree)
therealtruth Avatar
151 months ago
This is how you know Microsoft still doesn't have it. Regardless of whatever this technology is, You don't go out and make a video showing off how innovative you are with some demo technology. You use the thing in an actual product to prove it's real worth.

I will guarantee this thing is a PURE gimmick. They're comfortable showing you a video of something they themselves don't know the value of.
Score: 10 Votes (Like | Disagree)