Microsoft Research Demos Motion Sensing Keyboard with Gestures

Microsoft Research has demoed a new augmented mechanical keyboard that allows users to use motion controls either on or slightly above the keyboard to interact with their computers. The prototype was curiously made using keycaps from an Apple keyboard.

gesture

A low-resolution matrix of infrared (IR) proximity sensors is interspersed with the keys of a regular mechanical keyboard. This results in coarse but high frame-rate motion data. We extend a machine learning algorithm, traditionally used for static classification only, to robustly support dynamic, temporal gestures.

The gestures that the keyboard can recognize include traditional gestures like "pinch-to-zoom" and swiping to static gestures like holding a single finger above the keyboard to advanced gestures like turning an imaginary steering wheel.


It's unclear whether the augmented keyboard, referred to as the Type-Hover-Swipe, could eventually turn into a real product, but it's another step in Microsoft exploring motion control after its Kinect device. Apple, too, has been exploring motion sensing with its purchase of PrimeSense, the 3D body sensing firm that developed the technology for Kinect. The Type-Hover-Swipe keyboard is also similar to the technology of Leap Motion, who former Apple iAd chief Andy Miller worked for for close to two years.

Popular Stories

iOS 18

Apple Releases iOS 18.5 With New Wallpaper, Screen Time Changes, Carrier Satellite Support for iPhone 13 and More

Monday May 12, 2025 10:06 am PDT by
Apple today released iOS 18.5 and iPadOS 18.5, the fifth updates to the iOS 18 and iPadOS 18 operating systems that came out last September. iOS 18.5 and iPadOS 18.5 come a little over a month after Apple released iOS 18.4 and iPadOS 18.4. The new software can be downloaded on eligible iPhones and iPads over-the-air by going to Settings > General > Software Update. The iOS 18.5 update has a...
tvOS 18 Feature

Apple Releases tvOS 18.5

Monday May 12, 2025 10:01 am PDT by
Apple today released tvOS 18.5, the latest version of the tvOS operating system. tvOS 18.5 comes a little over a month after the launch of tvOS 18.4, and it is available for the Apple TV 4K and Apple TV HD models. tvOS 18.5 can be downloaded using the Settings app on the ‌Apple TV‌. Open up Settings and go to System > Software Update to get the new software. ‌Apple TV‌ owners who have...
iPhone 17 Pro Blue Feature Tighter Crop

WSJ: Apple Weighing Price Hikes for iPhone 17 Lineup Without Blaming Tariffs

Monday May 12, 2025 3:36 am PDT by
Apple is considering raising prices for its upcoming iPhone 17 models set to release this fall, according to people familiar with the matter cited by The Wall Street Journal. The company reportedly aims to pair the potential price hikes with new features and design changes to justify the increased cost to consumers, rather than attributing them to U.S. tariffs on goods from China. The...
macOS Sequoia Feature

Apple Releases macOS Sequoia 15.5

Monday May 12, 2025 10:10 am PDT by
Apple today released macOS Sequoia 15.5, the fifth major update to the macOS Sequoia operating system that launched last September. macOS Sequoia 15.5 comes a little over a month after the launch of macOS Sequoia 15.4. Mac users can download the ‌‌‌macOS Sequoia 15.5‌‌‌ update through the Software Update section of System Settings. It is available for free on all Macs able to run ...
iOS 18

iOS 18.5 Expected This Week With These New Features

Monday May 12, 2025 7:20 am PDT by
Following more than a month of beta testing, Apple is expected to release iOS 18.5 to the general public this week. While the software update is relatively minor, it still includes a handful of new features and changes for iPhones. Below, we recap everything new in iOS 18.5. Pride Wallpaper Apple recently announced its 2025 Pride Collection, including a new Apple Watch band, watch face,...
Beyond iPhone 13 Better Triad

20th Anniversary iPhone Will Be Mostly Glass With All-Screen Design

Monday May 12, 2025 2:52 am PDT by
Apple will mark the 10th anniversary of the iPhone X in 2027 by launching a mostly glass, curved iPhone without any cutouts in the display, according to Bloomberg's Mark Gurman. Writing in his latest Power On newsletter, Gurman said the all-screen device will arrive later in 2027, suggesting a fall release. The model will be preceded by Apple's first foldable iPhone, claims the reporter....
AirPods Pro 3 Mock Feature

AirPods Pro 3 Just Months Away – Here's What We Know

Tuesday April 29, 2025 1:30 am PDT by
Despite being more than two years old, Apple's AirPods Pro 2 still dominate the premium wireless‑earbud space, thanks to a potent mix of top‑tier audio, class‑leading noise cancellation, and Apple's habit of delivering major new features through software updates. With AirPods Pro 3 widely expected to arrive in 2025, prospective buyers now face a familiar dilemma: snap up the proven...

Top Rated Comments

Gasu E. Avatar
144 months ago
This is how you know Microsoft still doesn't have it. Regardless of whatever this technology is, You don't go out and make a video showing off how innovative you are with some demo technology. You use the thing in an actual product to prove it's real worth.

Microsoft Research does research. Hence the word "research" in its name.
Score: 21 Votes (Like | Disagree)
2IS Avatar
144 months ago
The point a lot of people here are trying to make is that at least with the Apple way, it's a shipped product that you can go out now and purchase with your money.

The Microsoft way just exists in this video. Good luck actually using it any time soon.

Yeah? And when was the last time Apple did something remotely exciting? There's nothing wrong with showing off a concept. If there is a viable application for it and can be successfully marketed and sold, someone will run with it.

Thinner iPhone 6? How exciting is that? I can tell you, when I purchased my iPhone 5, I wasn't thinking "I'll bet the iPhone 6 will be fatter"

Someone's thinking outside the box, and it isn't :apple:

I personally don't have any interest in this keyboard, but as an enthusiast of new technology, I'm glad to see someone isn't afraid to do something new.
Score: 15 Votes (Like | Disagree)
2IS Avatar
144 months ago
This is how you know Microsoft still doesn't have it. Regardless of whatever this technology is, You don't go out and make a video showing off how innovative you are with some demo technology. You use the thing in an actual product to prove it's real worth.

I will guarantee this thing is a PURE gimmick. They're comfortable showing you a video of something they themselves don't know the value of.

Contrary to what you may think, the Apple way isn't the only way, nor is it necessarily the correct way.
Score: 14 Votes (Like | Disagree)
imageWIS Avatar
144 months ago
Interactive porn in 3, 2, 1...
Score: 13 Votes (Like | Disagree)
Gasu E. Avatar
144 months ago
I don't want thing because it's a thing yet! :mad:

Ain't no thing.

----------

agreed, maybe they should put the time and money into a developing a secure OS before they make gimmicky gesture keyboard.

Ah, the traditional MacRumors "false trade-off" fallacy.
Score: 12 Votes (Like | Disagree)
therealtruth Avatar
144 months ago
This is how you know Microsoft still doesn't have it. Regardless of whatever this technology is, You don't go out and make a video showing off how innovative you are with some demo technology. You use the thing in an actual product to prove it's real worth.

I will guarantee this thing is a PURE gimmick. They're comfortable showing you a video of something they themselves don't know the value of.
Score: 10 Votes (Like | Disagree)