Members of Apple's PrimeSense Team Patent Method of Interacting With Mac Using Hand Gestures

Apple today was granted a patent originally filed in August 2016, describing a method in which users would be able to control a Mac computer -- and potentially other devices -- using a "non-tactile three dimensional (3D) user interface" (via Patently Apple). The patent's inventor credits go in part to Amir Hoffnung and Jonathan Pokrass, two current Apple employees who joined the company from PrimeSense after Apple acquired it November 2013.

Some of PrimeSense's tech, which was originally used in Microsoft's Kinect devices on Xbox platforms, now resides in the front-facing TrueDepth camera of the iPhone X, and the new patent hints at a potential future where this technology expands in function to Macs as well. Instead of recognizing faces, Apple's patent describes a Mac that recognizes a "gesture by a hand," allowing users to interact with the computer without tactile inputs like a keyboard, mouse, or trackpad.

Images via United States Patent and Trademark Office

The patent includes a variety of gestures that users would use to control the 3D user interface, including what are called "push," (figure 2) "wave," (figure 3) and "up" (figure 5) interactions, which are all grouped into a category of "focus gestures." According to the patent, some of these could be used to perform basic app interactions, like scrolling through a menu, as well as change the state of the system from locked to unlocked.

Gestures described herein include focus gestures and unlock gestures. A focus gesture enables the user to engage (i.e., take control of) an inactive non-tactile 3D user interface. An unlock gesture enables the user to engage a locked non-tactile 3D user interface, as pressing a specific sequence of keys unlocks a locked cellular phone. In some embodiments, the non-tactile 3D user interface conveys visual feedback to the user performing the focus and the unlock gestures.

Examples of unlock gestures include an "up" gesture (e.g., raising hand 30 a specified distance), a sequence of two sequential wave gestures, and a sequence of two sequential push gestures, as described in detail hereinbelow.

Apple has previously been granted patents related to 3D sensing and gesture controls, but the company has yet to release a product that takes advantage of these features. Some of these previous patents included gesture controls on iPads and iPhones as well as on Magic Keyboards, which would allow users to gain access to virtual buttons and potentially streamline certain elements of the user interface.


Of course, with Face ID already launched on iPhone X, it's predicted that Apple will look into adding the facial recognition software into future iMacs and MacBooks before it focuses on a new unlocking system. It's also unclear whether the new 3D user interface patent would include both pieces of technology -- 3D gestures and Face ID -- to further augment a Mac's unlocking process.

As with any patent, the technology in question might not make it to an Apple product in the near future, if at all, but it is an interesting glimpse into what Apple might be planning to do with PrimeSense's technology down the line.

Tag: patent

Top Rated Comments

(View all)
Avatar
37 months ago
I want my unlock to be a middle finger. That'll be fun.

Wait, no... I want the middle finger to force-close everything.
Score: 13 Votes (Like | Disagree)
Avatar
37 months ago
How can this ever be novel or non-obvious? US patent system is soooo broken :-(
Score: 7 Votes (Like | Disagree)
Avatar
37 months ago
I got a hand gesture for this "technology"....
Score: 7 Votes (Like | Disagree)
Avatar
37 months ago
Remind me again, whether or not fictional representations can be used as prior art?

Score: 6 Votes (Like | Disagree)
Avatar
37 months ago
It should recognize American Sign Language, which would give you a gestural keyboard that is already well worked out, and some people already know it.
Score: 6 Votes (Like | Disagree)
Avatar
37 months ago


Because we want a bunch of lasers in our eyes and tracking our every move?

Think what Google analytics and Amazon can do with that...

We see you're reaching for a pen... how about a uni-ball Vision Rollerball Pens, Fine Point (0.7mm), Black, 12 Count?

Would you like some SweetLeaf Natural Stevia Sweetener to go in you tea?

Oops, looks like you spilled your coffee. How about some Bounty Select-a-Size Paper Towels, the quicker picker upper!

We're noticing a few unruly nose-hairs, we've ordered you a Fancii Professional Nose & Ear Hair Trimmer with LED Light, Water Resistant, Stainless Steel Blades, and Battery Power!
Score: 6 Votes (Like | Disagree)

Top Stories

When Will the iPhone 12 Launch? Here's What We Know

Wednesday September 16, 2020 6:12 am PDT by
Yesterday's "Time Flies" Apple event saw the release of the Apple Watch Series 6, Apple Watch SE, iPad 8, and iPad Air 4, but no new iPhone models. Rumors before the event strongly alleged that it would not see the unveiling of new iPhones, with many reports pointing to an October launch. The lack of new iPhone models yesterday seems to confirm that the iPhone 12 lineup will not appear...

iOS 14 Picture in Picture No Longer Working With YouTube's Mobile Website in Safari [Without Premium]

Friday September 18, 2020 12:21 pm PDT by
Apple in iOS 14 added Picture in Picture to the iPhone, a feature designed to let you watch a video in a small screen on your device while you continue to do other things on the phone. When Picture in Picture was working with YouTube The YouTube app doesn't support Picture in Picture, but up until yesterday there was a functional workaround that allowed videos from YouTube.com to be watched...

Hands-On With the New Apple Watch Series 6 and Apple Watch SE

Friday September 18, 2020 1:19 pm PDT by
Today's the official launch date for the Apple Watch Series 6 and the Apple Watch SE, both of which Apple announced on Tuesday. We picked up a couple of the new models and thought we'd give them a quick look for MacRumors readers thinking of ordering a new watch. Apple Watch Series 6 & Apple Watch SE Hands-On! When it comes to design, both the $399 Series 6 and the $279 SE look just like...

Here's How You Can Download iOS 14 and iPadOS 14 Around the World [It's Out]

Wednesday September 16, 2020 2:36 am PDT by
Apple's official public release of iOS 14 and iPadOS 14 dropped on Wednesday, September 16, just a day after the company released the Golden Master to third-party developers. Also set to be made available to the general public for the first time are watchOS 7 and tvOS 14. Getting Started With iOS 14 Video Click image to watch iOS 14 Getting Started While that's left a lot of developers...

Apple Releases iOS 14 and iPadOS 14 With Home Screen Redesign, App Library, Compact UI, Translate App, Scribble Support, App Clips, and More

Wednesday September 16, 2020 12:48 pm PDT by
Apple has released iOS 14 and iPadOS 14, the newest operating system updates designed for the iPhone and iPad. As with all of Apple's software updates, iOS 14 and iPadOS 14 can be downloaded for free. iOS 14 is available on the iPhone 6s and later, while iPadOS 14 is available on the iPad Air 2 and later. The updates are available on all eligible devices over-the-air in the Settings app. To ...

iOS 14.2 Beta Adds New Shazam Music Recognition Feature for Control Center

Thursday September 17, 2020 3:36 pm PDT by
Apple today released the first beta of iOS 14.2 to developers for testing purposes, and the new update introduces a Music Recognition control for the Control Center. The new feature lets you discover music playing around you and it recognizes the music playing with in apps, even when you're wearing AirPods. Songs pop up as notifications, and you can tap to listen in Apple Music....

Apple Updates AirPods 2 and AirPods Pro Firmware to Version 3A283

Monday September 14, 2020 11:24 am PDT by
Apple today released new 3A283 firmware updates for the second-generation AirPods and the AirPods Pro. The second-generation AirPods are being updated from the 2D15 firmware they were previously running, while the AirPods Pros are being updated from the 2D27 firmware they had installed previously. Apple does not provide details on what's included in refreshed firmware so we don't know what's ...

Rumor Report Card: Assessing the Accuracy of Leaks After Apple's Event

Friday September 18, 2020 12:57 pm PDT by
Apple hosted its virtual "Time Flies" event this week, where it introduced four new products, including the Apple Watch Series 6, lower-cost Apple Watch SE, a 10.9-inch iPad Air with an all-screen design, and an updated 10.2-inch iPad with a faster A12 Bionic chip. As expected, there were no new iPhones, which are believed to be coming in October instead. Apple also announced that it will be ...

Epic Games Announces 'Fortnite: Save the World' Will No Longer Be Playable on macOS

Friday September 18, 2020 4:50 am PDT by
Epic Games has announced that "Fortnite: Save the World" will no longer be playable on macOS, after Apple terminated Epic Games' developer account. Fortnite has been in violation of the ‌App Store‌ rules since August 13, when it introduced a direct payment option that skirted Apple's in-app purchase system by allowing payments directly to ‌Epic Games‌. Shortly after Epic blatantly...

Deals: Cellular Carriers Introduce First Offers on Apple Watch Series 6 and SE

Friday September 18, 2020 7:43 am PDT by
With the launch of the Apple Watch Series 6 and Apple Watch SE today, cellular carriers have now introduced special offers for these new wearable devices. Note: MacRumors is an affiliate partner with these vendors. When you click a link and make a purchase, we may receive a small payment, which helps us keep the site running. Starting with AT&T, if you buy one Apple Watch Series 3, Series...