Members of Apple's PrimeSense Team Patent Method of Interacting With Mac Using Hand Gestures

Apple today was granted a patent originally filed in August 2016, describing a method in which users would be able to control a Mac computer -- and potentially other devices -- using a "non-tactile three dimensional (3D) user interface" (via Patently Apple). The patent's inventor credits go in part to Amir Hoffnung and Jonathan Pokrass, two current Apple employees who joined the company from PrimeSense after Apple acquired it November 2013.

Some of PrimeSense's tech, which was originally used in Microsoft's Kinect devices on Xbox platforms, now resides in the front-facing TrueDepth camera of the iPhone X, and the new patent hints at a potential future where this technology expands in function to Macs as well. Instead of recognizing faces, Apple's patent describes a Mac that recognizes a "gesture by a hand," allowing users to interact with the computer without tactile inputs like a keyboard, mouse, or trackpad.

apple 3d UI patent


The patent includes a variety of gestures that users would use to control the 3D user interface, including what are called "push," (figure 2) "wave," (figure 3) and "up" (figure 5) interactions, which are all grouped into a category of "focus gestures." According to the patent, some of these could be used to perform basic app interactions, like scrolling through a menu, as well as change the state of the system from locked to unlocked.

Gestures described herein include focus gestures and unlock gestures. A focus gesture enables the user to engage (i.e., take control of) an inactive non-tactile 3D user interface. An unlock gesture enables the user to engage a locked non-tactile 3D user interface, as pressing a specific sequence of keys unlocks a locked cellular phone. In some embodiments, the non-tactile 3D user interface conveys visual feedback to the user performing the focus and the unlock gestures.

Examples of unlock gestures include an "up" gesture (e.g., raising hand 30 a specified distance), a sequence of two sequential wave gestures, and a sequence of two sequential push gestures, as described in detail hereinbelow.

Apple has previously been granted patents related to 3D sensing and gesture controls, but the company has yet to release a product that takes advantage of these features. Some of these previous patents included gesture controls on iPads and iPhones as well as on Magic Keyboards, which would allow users to gain access to virtual buttons and potentially streamline certain elements of the user interface.

apple 3d ui patent 2
Of course, with Face ID already launched on iPhone X, it's predicted that Apple will look into adding the facial recognition software into future iMacs and MacBooks before it focuses on a new unlocking system. It's also unclear whether the new 3D user interface patent would include both pieces of technology -- 3D gestures and Face ID -- to further augment a Mac's unlocking process.

As with any patent, the technology in question might not make it to an Apple product in the near future, if at all, but it is an interesting glimpse into what Apple might be planning to do with PrimeSense's technology down the line.

Tag: patent

Top Rated Comments

JM Avatar
39 months ago
I want my unlock to be a middle finger. That'll be fun.

Wait, no... I want the middle finger to force-close everything.
Score: 13 Votes (Like | Disagree)
PaddyTB Avatar
39 months ago
How can this ever be novel or non-obvious? US patent system is soooo broken :-(
Score: 7 Votes (Like | Disagree)
jcmeyer5 Avatar
39 months ago
I got a hand gesture for this "technology"....
Score: 7 Votes (Like | Disagree)
zarmanto Avatar
39 months ago
Remind me again, whether or not fictional representations can be used as prior art?

Score: 6 Votes (Like | Disagree)
Lepton Avatar
39 months ago
It should recognize American Sign Language, which would give you a gestural keyboard that is already well worked out, and some people already know it.
Score: 6 Votes (Like | Disagree)
asleep Avatar
39 months ago


Because we want a bunch of lasers in our eyes and tracking our every move?

Think what Google analytics and Amazon can do with that...

We see you're reaching for a pen... how about a uni-ball Vision Rollerball Pens, Fine Point (0.7mm), Black, 12 Count?

Would you like some SweetLeaf Natural Stevia Sweetener to go in you tea?

Oops, looks like you spilled your coffee. How about some Bounty Select-a-Size Paper Towels, the quicker picker upper!

We're noticing a few unruly nose-hairs, we've ordered you a Fancii Professional Nose & Ear Hair Trimmer with LED Light, Water Resistant, Stainless Steel Blades, and Battery Power!
Score: 6 Votes (Like | Disagree)

Top Stories

windows 10

Developer Successfully Virtualizes Windows for Arm on M1 Mac

Friday November 27, 2020 7:16 am PST by
Developer Alexander Graf has successfully virtualized the Arm version of Windows on an M1 Mac, proving that the M1 chip is capable of running Microsoft's operating system (via The 8-Bit). Currently, Macs with the M1 chip do not support Windows and there is no Boot Camp feature as there is on Intel Macs, but support for Windows is a feature that many users would like to see. Using the...
iPhone black friday 20 sale feature

Apple Black Friday 2020: Best iPhone Deals

Friday November 27, 2020 12:56 pm PST by
Black Friday is halfway done, but there are still a few deals to shop for on iPhones at carriers like AT&T, Verizon, and T-Mobile/Sprint. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a small payment, which helps us keep the site running. AT&T Starting with AT&T, you'll find up to $700 off any iPhone 12 when...
AirPods Pro black friday 20 sale feature 2

Black Friday 2020: AirPods Pro Reach Lowest Price Ever [Updated]

Wednesday November 25, 2020 3:22 pm PST by
Black Friday has kicked off this week, and one of the first major sales for the AirPods Pro is available right now on Walmart. You can find this deal below, along with a few other solid discounts on the regular AirPods. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a small payment, which helps us keep the site...
Apple Watc black friday 20 sale feature

Apple Black Friday 2020: Best Apple Watch Deals [Updated]

Wednesday November 25, 2020 4:01 pm PST by
Black Friday sales have begun on a variety of products, including the Apple Watch. There are quite a few deals across the Apple Watch lineup this year, including one of the lowest price we've ever seen the Apple Watch Series 3. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a small payment, which helps us keep the...
iphone trade in store

UK Environmental Committee Says Apple Contributing to 'Throwaway Culture' of 'Short-Lived Products'

Thursday November 26, 2020 7:07 am PST by
Technology companies like Apple are contributing to e-waste by making their products difficult to repair, and charging expensive repair fees, according to a lengthy report published today by the UK Parliament's Environmental Audit Committee. "We were told that Apple glues and solders parts together on their laptops, which makes repairing them very difficult," the Committee wrote in a summary ...
MacBooks black friday

Apple Black Friday 2020: Best Mac and MacBook Deals

Thursday November 26, 2020 8:29 pm PST by
For Black Friday 2020, many retailers have solid deals on a variety of Macs. Apple's new M1 Macs are ones everyone wants right now, but because they're so new, deals on them are fairly sparse, although we are seeing some modest discounts. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a small payment, which helps...
Top Stories 38 Feature

Top Stories: Black Friday Deals, Redesigned MacBooks, Hands-On With Apple's M1 Macs

Saturday November 28, 2020 6:00 am PST by
With Apple's holiday hardware lineup seemingly all set, attention this week turned to the shopping end of things with Apple and other retailers rolling out their Black Friday deals. That wasn't the only news this week, however, as we've continued to learn more about Apple's new M1-based Macs and we've even heard some fresh rumors about redesigned Mac notebooks coming next year, so read on...
General black friday 20 sale feature 2

Apple Black Friday 2020: Our Top Picks

Friday November 27, 2020 6:16 am PST by
Black Friday shopping has officially kicked off across the United States, and we've been collecting the best deals for Apple's iPads, Macs, AirPods, the HomePod, Apple Watch, and more in separate articles. In order to further streamline your Black Friday shopping this year, we've put together this quick list of our top picks of the overall best Apple-related bargains happening today. Note:...
mac mini macbook pro macbook air

Apple M1 Hands-On Comparison: MacBook Air vs. MacBook Pro vs. Mac Mini

Monday November 23, 2020 3:40 pm PST by
Apple's M1 Macs are out in the wild now, but ahead of the holidays, you might still be trying to figure out which one to pick up, either for yourself or as a gift for someone else. We've got all three of the new Macs available, so we thought we'd give MacRumors readers a hands-on overview of each machine in our latest YouTube video. Subscribe to the MacRumors YouTube channel for more videos. ...
General black friday 20 sale feature

Thanksgiving Day Deals Still Available on AirPods, M1 Macs, Apple Watch Series 6, iPads

Thursday November 26, 2020 10:21 am PST by
Black Friday deals seem to start earlier and earlier every year, so there were already a wide variety of discounts available for Apple products on Thanksgiving Day. Many of the deals remain available even after Thanksgiving, but act fast, as inventory quickly fluctuates. Thanksgiving Day Deals on Apple Products — Still Available:AirPods with a wireless charging case remain available for...