Members of Apple's PrimeSense Team Patent Method of Interacting With Mac Using Hand Gestures

Apple today was granted a patent originally filed in August 2016, describing a method in which users would be able to control a Mac computer -- and potentially other devices -- using a "non-tactile three dimensional (3D) user interface" (via Patently Apple). The patent's inventor credits go in part to Amir Hoffnung and Jonathan Pokrass, two current Apple employees who joined the company from PrimeSense after Apple acquired it November 2013.

Some of PrimeSense's tech, which was originally used in Microsoft's Kinect devices on Xbox platforms, now resides in the front-facing TrueDepth camera of the iPhone X, and the new patent hints at a potential future where this technology expands in function to Macs as well. Instead of recognizing faces, Apple's patent describes a Mac that recognizes a "gesture by a hand," allowing users to interact with the computer without tactile inputs like a keyboard, mouse, or trackpad.

apple 3d UI patent


The patent includes a variety of gestures that users would use to control the 3D user interface, including what are called "push," (figure 2) "wave," (figure 3) and "up" (figure 5) interactions, which are all grouped into a category of "focus gestures." According to the patent, some of these could be used to perform basic app interactions, like scrolling through a menu, as well as change the state of the system from locked to unlocked.

Gestures described herein include focus gestures and unlock gestures. A focus gesture enables the user to engage (i.e., take control of) an inactive non-tactile 3D user interface. An unlock gesture enables the user to engage a locked non-tactile 3D user interface, as pressing a specific sequence of keys unlocks a locked cellular phone. In some embodiments, the non-tactile 3D user interface conveys visual feedback to the user performing the focus and the unlock gestures.

Examples of unlock gestures include an "up" gesture (e.g., raising hand 30 a specified distance), a sequence of two sequential wave gestures, and a sequence of two sequential push gestures, as described in detail hereinbelow.

Apple has previously been granted patents related to 3D sensing and gesture controls, but the company has yet to release a product that takes advantage of these features. Some of these previous patents included gesture controls on iPads and iPhones as well as on Magic Keyboards, which would allow users to gain access to virtual buttons and potentially streamline certain elements of the user interface.

apple 3d ui patent 2
Of course, with Face ID already launched on iPhone X, it's predicted that Apple will look into adding the facial recognition software into future iMacs and MacBooks before it focuses on a new unlocking system. It's also unclear whether the new 3D user interface patent would include both pieces of technology -- 3D gestures and Face ID -- to further augment a Mac's unlocking process.

As with any patent, the technology in question might not make it to an Apple product in the near future, if at all, but it is an interesting glimpse into what Apple might be planning to do with PrimeSense's technology down the line.

Tag: patent

Top Rated Comments

JM Avatar
45 months ago
I want my unlock to be a middle finger. That'll be fun.

Wait, no... I want the middle finger to force-close everything.
Score: 13 Votes (Like | Disagree)
PaddyTB Avatar
45 months ago
How can this ever be novel or non-obvious? US patent system is soooo broken :-(
Score: 7 Votes (Like | Disagree)
jcmeyer5 Avatar
45 months ago
I got a hand gesture for this "technology"....
Score: 7 Votes (Like | Disagree)
zarmanto Avatar
45 months ago
Remind me again, whether or not fictional representations can be used as prior art?

Score: 6 Votes (Like | Disagree)
Lepton Avatar
45 months ago
It should recognize American Sign Language, which would give you a gestural keyboard that is already well worked out, and some people already know it.
Score: 6 Votes (Like | Disagree)
asleep Avatar
45 months ago


Because we want a bunch of lasers in our eyes and tracking our every move?

Think what Google analytics and Amazon can do with that...

We see you're reaching for a pen... how about a uni-ball Vision Rollerball Pens, Fine Point (0.7mm), Black, 12 Count?

Would you like some SweetLeaf Natural Stevia Sweetener to go in you tea?

Oops, looks like you spilled your coffee. How about some Bounty Select-a-Size Paper Towels, the quicker picker upper!

We're noticing a few unruly nose-hairs, we've ordered you a Fancii Professional Nose & Ear Hair Trimmer with LED Light, Water Resistant, Stainless Steel Blades, and Battery Power!
Score: 6 Votes (Like | Disagree)

Top Stories

General Music and AirPod 3 Feature

Rumor: Apple to Announce Third-Generation AirPods and HiFi Apple Music Tier on May 18

Thursday May 13, 2021 10:32 pm PDT by
A new rumor suggests that Apple will announce the third-generation AirPods and the recently rumored HiFi, or high-fidelity Apple Music tier, on Tuesday, May 18, via a press release on its website. The new rumor comes from Apple YouTuber Luke Miani who shared the alleged exclusive news with the AppleTrack website. According to the YouTuber, Apple plans to release the next-generation AirPods...
apple park drone june 2018 2

Apple Fires Newly Hired Ex-Facebook Product Manager Following Revelations of Past Misogynistic Comments

Thursday May 13, 2021 12:10 am PDT by
Apple has fired Antonio García Martínez, an ex-Facebook product manager and author of the controversial book "Chaos Monkeys," following public and internal calls for removal and investigation due to past misogynistic statements, The Verge reports. Apple hired Martínez earlier this week to join its ads team, however, comments that Martínez made in the past sparked condemnation from users...
imac m1 blue isolated 16x9 500k

M1 iMac is Up to 56% Faster Than Prior-Generation High-End 21.5-Inch iMac

Wednesday May 12, 2021 10:03 am PDT by
Apple's M1 iMacs are set to start delivering to customers next week, and ahead of the official launch day, benchmarks for the machines have been showing up on Geekbench, likely from reviewers who are testing them. It will come as no surprise that M1 iMac benchmarks are right on par with benchmarks for the M1 MacBook Pro, MacBook Air, and Mac mini, coming in with an average single-core score...
2021 mbp hdmi slot 3d

2021 MacBook Pro Leaks Confirm Returning MagSafe and Ports

Friday May 14, 2021 3:06 am PDT by
Apple's upcoming MacBook Pro models are expected to feature a number of major changes such as larger display options and powerful new Apple silicon chips. Among the more surprising updates to this year's MacBook Pro models is the return of three ports that have been missing from the machines for over five years. Expected to come in 14- and 16-inch sizes, the 2021 MacBook Pro models are...
fortnite apple logo 2

Judge in Epic vs. Apple Case Floats Potential Compromise

Wednesday May 12, 2021 3:54 pm PDT by
In the ongoing legal battle between Apple and Epic Games, the two companies are this week calling up their expert witnesses to argue their points before Judge Yvonne Gonzalez Rogers, who will make a decision in the case after a three week trial. Expert testimony is not as exciting as some of the leaked App Store documents that were highlighted last week, especially as much of what's being...
google photos

PSA: Google Photos Unlimited Storage Ends Next Month, Here's How to Export Your Pictures to iCloud

Thursday May 13, 2021 5:26 am PDT by
For as long as it's existed, Google Photos has offered free unlimited storage for uploading images at a reduced yet good enough quality for most users. From June 1, 2021, however, all photos and videos uploaded to Google accounts will count against users' cloud storage. If you've been relying on Google to back up your media library, it may be time to move that content elsewhere. This article...
AirTag in Envelope Feature 2

AirTag Used to Successfully Track a Mailed Package Across the UK

Wednesday May 12, 2021 8:44 am PDT by
An Apple customer in the United Kingdom has successfully used Apple's Find My network to track an AirTag as it was being sent by mail to a friend in a completely different city. Outlined in a blog post at Intego, Kirk McElhearn said he taped an AirTag to a piece of card, wrapped it inside a small bubble envelope, and then sent it on its way. Kirk lives in the small town of...
m1 ipad pro chip

M1 iPad Pro Over 50% Faster Than Previous Generation in Early Benchmarks

Tuesday May 11, 2021 11:56 am PDT by
Last month, Apple introduced a new iPad Pro with the same M1 chip found in the latest Macs, and early benchmark results indicate that the M1 iPad Pro is over 50% faster than the previous-generation iPad Pro. Based on five legitimate Geekbench 5 results (here's the fifth) for the fifth-generation 12.9-inch iPad Pro with the M1 chip, the device has average single-core and multi-core scores of...
prosser macbook air colors stacked

Images Reveal Colorful New MacBook Air Design

Tuesday May 11, 2021 5:06 am PDT by
Apple's next MacBook Air will feature a completely new design and come in a range of colors like the 24-inch iMac, according to leaker Jon Prosser, who has now released supposedly accurate renders of the new machines based on leaked images. In a new video uploaded to YouTube channel Front Page Tech, Prosser elaborated on his previous prediction that Apple's next-generation MacBook Air models ...
iPhone 13 Camera Backs

iPhone 13 Models Will Be Slightly Thicker and Will Have Larger Camera Bumps

Monday May 10, 2021 10:41 am PDT by
Apple's upcoming iPhone 13 models will be slightly thicker than the iPhone 12 models and will also feature larger, thicker camera bumps with lenses that protrude less, according to iPhone 13 schematics seen by MacRumors. The new iPhone 13 and 13 Pro models are expected to feature a thickness of 7.57mm, up from 7.4mm in the iPhone 12 models. That's an increase of 0.17mm, which won't be hugely ...