Microsoft Research Demos Motion Sensing Keyboard with Gestures

Microsoft Research has demoed a new augmented mechanical keyboard that allows users to use motion controls either on or slightly above the keyboard to interact with their computers. The prototype was curiously made using keycaps from an Apple keyboard.

gesture

A low-resolution matrix of infrared (IR) proximity sensors is interspersed with the keys of a regular mechanical keyboard. This results in coarse but high frame-rate motion data. We extend a machine learning algorithm, traditionally used for static classification only, to robustly support dynamic, temporal gestures.

The gestures that the keyboard can recognize include traditional gestures like "pinch-to-zoom" and swiping to static gestures like holding a single finger above the keyboard to advanced gestures like turning an imaginary steering wheel.


It's unclear whether the augmented keyboard, referred to as the Type-Hover-Swipe, could eventually turn into a real product, but it's another step in Microsoft exploring motion control after its Kinect device. Apple, too, has been exploring motion sensing with its purchase of PrimeSense, the 3D body sensing firm that developed the technology for Kinect. The Type-Hover-Swipe keyboard is also similar to the technology of Leap Motion, who former Apple iAd chief Andy Miller worked for for close to two years.

Popular Stories

Apple Shopping Event 2025

Apple Announces 2025 Black Friday Event, Here's What You Can Get

Thursday November 20, 2025 6:28 am PST by
Apple's annual four-day Black Friday through Cyber Monday shopping event is returning on Friday, November 28 through Monday, December 1 in many countries, including the United States, Canada, Australia, New Zealand, France, Germany, Italy, Spain, the United Kingdom, Belgium, the Netherlands, Sweden, Thailand, and others. During the shopping event, customers can get an Apple gift card with...
iOS 26

iOS 26.2 Adds These New Features to Your iPhone

Thursday November 20, 2025 10:50 am PST by
iOS 26.2 is currently in beta testing. The upcoming update includes a handful of new features and changes on the iPhone, including a new Liquid Glass slider for the Lock Screen's clock, offline lyrics for Apple Music, and more. In a recent press release, Apple confirmed that iOS 26.2 will be released to all users in December, but it did not provide a specific release date. Keep reading...
hikawa phone grip stand apple%402x

Apple Launches Second Limited-Edition iPhone Accessory in a Month

Friday November 21, 2025 3:53 am PST by
Apple has begun selling the Hikawa Phone Grip and Stand, a new limited-edition iPhone accessory designed with accessibility in mind. Designed by LA-based Bailey Hikawa to celebrate the 40th anniversary of accessibility at Apple, the grip uses magnets to securely snap onto any iPhone with MagSafe. Apple says it can be removed with ease, and doubles as a stand with two different viewing...
iPhone 17 Pro Cosmic Orange

10 Reasons to Wait for Next Year's iPhone 18 Pro

Wednesday November 19, 2025 4:00 am PST by
Apple's iPhone development roadmap runs several years into the future and the company is continually working with suppliers on several successive iPhone models at the same time, which is why we often get rumored features months ahead of launch. The iPhone 18 series is no different, and we already have a good idea of what to expect for the iPhone 18 Pro and iPhone 18 Pro Max. One thing worth...
ipad black friday 2025

The Best Early Black Friday iPad Deals

Thursday November 20, 2025 10:20 am PST by
Black Friday is just over a week away, and iPad deals have finally started to flood in at retailers like Amazon and Best Buy. Below we're tracking discounts on every current generation iPad, including lowest-ever prices on M3 iPad Air and M5 iPad Pro, plus steep markdowns on iPad and iPad mini. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a ...
ipad mini 7 feature red and blue

iPad Mini 8: Four Major New Features to Expect

Wednesday November 19, 2025 7:50 am PST by
Apple's eighth-generation iPad mini is highly likely to arrive next year, offering a significant refresh of the device with at least four major new features. OLED Display The next-generation version of the iPad mini could feature an OLED display, as part of Apple's plan to expand the display technology across many more of its devices. Apple's first OLED device was the Apple Watch in 2015, ...
watchos 26 workout app

Apple Watch Users Claim Workout App Is Now Worse in Every Way

Thursday November 20, 2025 7:01 am PST by
Apple Watch owners have been voicing their frustration online over changes to the Workout app that Apple introduced in watchOS 26, with many finding the redesigned interface makes starting exercises difficult and exasperating. When Apple launched watchOS 26 in September, the Workout app went from large, easily tapped workout tiles to a scrolling, corner-button interface. Instead of tapping a ...
maxresdefault

The MacRumors Show: iPhone 18 Pro Looks Like a Huge Upgrade

Friday November 21, 2025 9:10 am PST by
On this week's episode of The MacRumors Show, we talk through all of the new features and improvements expected to come to next year's iPhone 18 Pro and iPhone 18 Pro Max models. Subscribe to The MacRumors Show YouTube channel for more videos Apple's next-generation iPhones are less than ten months away and we already have a good idea about what to expect based on corroborated leaks, rumors,...
Apple Foldable Thumb

Foldable iPhone to Debut These Two Breakthrough Features

Wednesday November 19, 2025 7:26 am PST by
Apple's first foldable iPhone is expected to launch alongside the iPhone 18 Pro models in fall 2026, and it's shaping up to include two standout features that could set it apart from the competition. The book-style foldable will reportedly feature an industry-first 24-megapixel under-display camera built into the inner display, according to a recent JP Morgan equity research report. That...

Top Rated Comments

Gasu E. Avatar
151 months ago
This is how you know Microsoft still doesn't have it. Regardless of whatever this technology is, You don't go out and make a video showing off how innovative you are with some demo technology. You use the thing in an actual product to prove it's real worth.

Microsoft Research does research. Hence the word "research" in its name.
Score: 21 Votes (Like | Disagree)
2IS Avatar
151 months ago
The point a lot of people here are trying to make is that at least with the Apple way, it's a shipped product that you can go out now and purchase with your money.

The Microsoft way just exists in this video. Good luck actually using it any time soon.

Yeah? And when was the last time Apple did something remotely exciting? There's nothing wrong with showing off a concept. If there is a viable application for it and can be successfully marketed and sold, someone will run with it.

Thinner iPhone 6? How exciting is that? I can tell you, when I purchased my iPhone 5, I wasn't thinking "I'll bet the iPhone 6 will be fatter"

Someone's thinking outside the box, and it isn't :apple:

I personally don't have any interest in this keyboard, but as an enthusiast of new technology, I'm glad to see someone isn't afraid to do something new.
Score: 15 Votes (Like | Disagree)
2IS Avatar
151 months ago
This is how you know Microsoft still doesn't have it. Regardless of whatever this technology is, You don't go out and make a video showing off how innovative you are with some demo technology. You use the thing in an actual product to prove it's real worth.

I will guarantee this thing is a PURE gimmick. They're comfortable showing you a video of something they themselves don't know the value of.

Contrary to what you may think, the Apple way isn't the only way, nor is it necessarily the correct way.
Score: 14 Votes (Like | Disagree)
imageWIS Avatar
151 months ago
Interactive porn in 3, 2, 1...
Score: 13 Votes (Like | Disagree)
Gasu E. Avatar
151 months ago
I don't want thing because it's a thing yet! :mad:

Ain't no thing.

----------

agreed, maybe they should put the time and money into a developing a secure OS before they make gimmicky gesture keyboard.

Ah, the traditional MacRumors "false trade-off" fallacy.
Score: 12 Votes (Like | Disagree)
therealtruth Avatar
151 months ago
This is how you know Microsoft still doesn't have it. Regardless of whatever this technology is, You don't go out and make a video showing off how innovative you are with some demo technology. You use the thing in an actual product to prove it's real worth.

I will guarantee this thing is a PURE gimmick. They're comfortable showing you a video of something they themselves don't know the value of.
Score: 10 Votes (Like | Disagree)