Microsoft Research Demos Motion Sensing Keyboard with Gestures

Microsoft Research has demoed a new augmented mechanical keyboard that allows users to use motion controls either on or slightly above the keyboard to interact with their computers. The prototype was curiously made using keycaps from an Apple keyboard.

gesture

A low-resolution matrix of infrared (IR) proximity sensors is interspersed with the keys of a regular mechanical keyboard. This results in coarse but high frame-rate motion data. We extend a machine learning algorithm, traditionally used for static classification only, to robustly support dynamic, temporal gestures.

The gestures that the keyboard can recognize include traditional gestures like "pinch-to-zoom" and swiping to static gestures like holding a single finger above the keyboard to advanced gestures like turning an imaginary steering wheel.


It's unclear whether the augmented keyboard, referred to as the Type-Hover-Swipe, could eventually turn into a real product, but it's another step in Microsoft exploring motion control after its Kinect device. Apple, too, has been exploring motion sensing with its purchase of PrimeSense, the 3D body sensing firm that developed the technology for Kinect. The Type-Hover-Swipe keyboard is also similar to the technology of Leap Motion, who former Apple iAd chief Andy Miller worked for for close to two years.

Popular Stories

Home Hub Command Center with Dome Base Feature

Apple Working on All-New Operating System

Saturday August 16, 2025 6:45 am PDT by
Apple is developing an all-new operating system codenamed "Charismatic," according to Bloomberg's Mark Gurman. Apple smart home hub concept This is likely Apple's long-rumored "homeOS" operating system. In a report this week, Gurman said both Apple's rumored smart home hub in 2026 and tabletop robot in 2027 will run the new operating system. He said the software platform will blend...
apple beta 26 lineup

Mark Gurman Responds to Last Week's Apple Device Leaks

Sunday August 17, 2025 7:03 am PDT by
Last week, Apple released and then pulled a software tool that accidentally contained identifiers for many unreleased devices and chips, according to MacRumors contributor Aaron Perris. His findings included new models of the Studio Display, Apple TV, Apple Watches, Apple Vision Pro, iPad mini, HomePod mini, and more. Here is what was uncovered in the file, according to MacRumors contributor ...
iOS 26 Feature

Here's Everything New in iOS 26 Beta 7

Monday August 18, 2025 11:59 am PDT by
The seventh developer beta of iOS 26 is now available. While we are now in the later stages of the iOS 26 beta cycle, there are still some changes. Below, we outline everything new that we have found in iOS 26 beta 7 so far. Redesigned Blood Oxygen Feature The seventh developer betas of iOS 26 and watchOS 26 include a redesigned Blood Oxygen feature on Apple Watch Series 9, Apple Watch ...
iPhone 17 Pro in Hand Feature Lowgo

iPhone 17 Pro Max's Internal Design With Metal Battery Allegedly Leaks

Friday August 15, 2025 9:13 am PDT by
Alleged images of the iPhone 17 Pro Max's internal design have surfaced, offering a potential look inside the device before it is announced by Apple next month. The images were shared by the account "yeux1122" this week, in a blog post on the Korean platform Naver. The account aggregates Apple rumors and leaks, so it is likely not the original source of the images, and it is unclear if they...
iPhone 16

No iPhone 18 in 2026, Another Report Claims

Monday August 18, 2025 7:34 am PDT by
Apple is expected to delay the launch of its base iPhone 18 model until spring 2027, marking a major shift in the company's long-established release cycle, according to South Korea's ETNews. The report claims that Apple has informed some of its suppliers that the iPhone 18 will not be part of the September 2026 iPhone lineup. Instead, the company will unveil only higher-end models in the...
Apple Watch Ultra 2 Complications

Apple Watch Reportedly Set to Receive 'Significant Redesign' Next Year

Friday August 15, 2025 1:31 pm PDT by
At least one new Apple Watch model launching next year will feature a "significant redesign," according to Taiwanese supply chain publication DigiTimes. In a paywalled report this week, citing supply chain insiders, DigiTimes claimed that a high-end 2026 Apple Watch model will feature "exterior design" changes, including but not limited to "eight sensors arranged in a ring pattern visible...
Golden Apple Logo

Every Apple Secret That Leaked Wednesday

Thursday August 14, 2025 4:13 am PDT by
Apple made a major slip Wednesday when it accidentally included hardware identifiers in software code linking to numerous unannounced products. The leaked information provided MacRumors with concrete evidence of Apple's hardware development across multiple product categories. Here's everything that was confirmed through the code discoveries: New HomePod mini with updated chip – New...
apple design award 2025

Apple Announces 2025 Design Award Winners Ahead of WWDC 2025

Tuesday June 3, 2025 10:14 am PDT by
As we wait for WWDC to kick off next Monday, Apple today announced the winners of its annual Apple Design Awards, recognizing apps and games for their innovation, ingenuity, and technical achievement. The 2025 Apple Design Award winners are listed below, with one app and one game selected per category: Delight and Fun - CapWords (App) and Balatro (Game) Innovation - Play (App) and PBJ -...

Top Rated Comments

Gasu E. Avatar
148 months ago
This is how you know Microsoft still doesn't have it. Regardless of whatever this technology is, You don't go out and make a video showing off how innovative you are with some demo technology. You use the thing in an actual product to prove it's real worth.

Microsoft Research does research. Hence the word "research" in its name.
Score: 21 Votes (Like | Disagree)
2IS Avatar
148 months ago
The point a lot of people here are trying to make is that at least with the Apple way, it's a shipped product that you can go out now and purchase with your money.

The Microsoft way just exists in this video. Good luck actually using it any time soon.

Yeah? And when was the last time Apple did something remotely exciting? There's nothing wrong with showing off a concept. If there is a viable application for it and can be successfully marketed and sold, someone will run with it.

Thinner iPhone 6? How exciting is that? I can tell you, when I purchased my iPhone 5, I wasn't thinking "I'll bet the iPhone 6 will be fatter"

Someone's thinking outside the box, and it isn't :apple:

I personally don't have any interest in this keyboard, but as an enthusiast of new technology, I'm glad to see someone isn't afraid to do something new.
Score: 15 Votes (Like | Disagree)
2IS Avatar
148 months ago
This is how you know Microsoft still doesn't have it. Regardless of whatever this technology is, You don't go out and make a video showing off how innovative you are with some demo technology. You use the thing in an actual product to prove it's real worth.

I will guarantee this thing is a PURE gimmick. They're comfortable showing you a video of something they themselves don't know the value of.

Contrary to what you may think, the Apple way isn't the only way, nor is it necessarily the correct way.
Score: 14 Votes (Like | Disagree)
imageWIS Avatar
148 months ago
Interactive porn in 3, 2, 1...
Score: 13 Votes (Like | Disagree)
Gasu E. Avatar
148 months ago
I don't want thing because it's a thing yet! :mad:

Ain't no thing.

----------

agreed, maybe they should put the time and money into a developing a secure OS before they make gimmicky gesture keyboard.

Ah, the traditional MacRumors "false trade-off" fallacy.
Score: 12 Votes (Like | Disagree)
therealtruth Avatar
148 months ago
This is how you know Microsoft still doesn't have it. Regardless of whatever this technology is, You don't go out and make a video showing off how innovative you are with some demo technology. You use the thing in an actual product to prove it's real worth.

I will guarantee this thing is a PURE gimmick. They're comfortable showing you a video of something they themselves don't know the value of.
Score: 10 Votes (Like | Disagree)