Members of Apple's PrimeSense Team Patent Method of Interacting With Mac Using Hand Gestures

Apple today was granted a patent originally filed in August 2016, describing a method in which users would be able to control a Mac computer -- and potentially other devices -- using a "non-tactile three dimensional (3D) user interface" (via Patently Apple). The patent's inventor credits go in part to Amir Hoffnung and Jonathan Pokrass, two current Apple employees who joined the company from PrimeSense after Apple acquired it November 2013.

Some of PrimeSense's tech, which was originally used in Microsoft's Kinect devices on Xbox platforms, now resides in the front-facing TrueDepth camera of the iPhone X, and the new patent hints at a potential future where this technology expands in function to Macs as well. Instead of recognizing faces, Apple's patent describes a Mac that recognizes a "gesture by a hand," allowing users to interact with the computer without tactile inputs like a keyboard, mouse, or trackpad.

apple 3d UI patent


The patent includes a variety of gestures that users would use to control the 3D user interface, including what are called "push," (figure 2) "wave," (figure 3) and "up" (figure 5) interactions, which are all grouped into a category of "focus gestures." According to the patent, some of these could be used to perform basic app interactions, like scrolling through a menu, as well as change the state of the system from locked to unlocked.

Gestures described herein include focus gestures and unlock gestures. A focus gesture enables the user to engage (i.e., take control of) an inactive non-tactile 3D user interface. An unlock gesture enables the user to engage a locked non-tactile 3D user interface, as pressing a specific sequence of keys unlocks a locked cellular phone. In some embodiments, the non-tactile 3D user interface conveys visual feedback to the user performing the focus and the unlock gestures.

Examples of unlock gestures include an "up" gesture (e.g., raising hand 30 a specified distance), a sequence of two sequential wave gestures, and a sequence of two sequential push gestures, as described in detail hereinbelow.

Apple has previously been granted patents related to 3D sensing and gesture controls, but the company has yet to release a product that takes advantage of these features. Some of these previous patents included gesture controls on iPads and iPhones as well as on Magic Keyboards, which would allow users to gain access to virtual buttons and potentially streamline certain elements of the user interface.

apple 3d ui patent 2
Of course, with Face ID already launched on iPhone X, it's predicted that Apple will look into adding the facial recognition software into future iMacs and MacBooks before it focuses on a new unlocking system. It's also unclear whether the new 3D user interface patent would include both pieces of technology -- 3D gestures and Face ID -- to further augment a Mac's unlocking process.

As with any patent, the technology in question might not make it to an Apple product in the near future, if at all, but it is an interesting glimpse into what Apple might be planning to do with PrimeSense's technology down the line.

Tag: patent

Top Rated Comments

JM Avatar
50 months ago
I want my unlock to be a middle finger. That'll be fun.

Wait, no... I want the middle finger to force-close everything.
Score: 13 Votes (Like | Disagree)
PaddyTB Avatar
50 months ago
How can this ever be novel or non-obvious? US patent system is soooo broken :-(
Score: 7 Votes (Like | Disagree)
jcmeyer5 Avatar
50 months ago
I got a hand gesture for this "technology"....
Score: 7 Votes (Like | Disagree)
zarmanto Avatar
50 months ago
Remind me again, whether or not fictional representations can be used as prior art?

Score: 6 Votes (Like | Disagree)
Lepton Avatar
50 months ago
It should recognize American Sign Language, which would give you a gestural keyboard that is already well worked out, and some people already know it.
Score: 6 Votes (Like | Disagree)
asleep Avatar
50 months ago


Because we want a bunch of lasers in our eyes and tracking our every move?

Think what Google analytics and Amazon can do with that...

We see you're reaching for a pen... how about a uni-ball Vision Rollerball Pens, Fine Point (0.7mm), Black, 12 Count?

Would you like some SweetLeaf Natural Stevia Sweetener to go in you tea?

Oops, looks like you spilled your coffee. How about some Bounty Select-a-Size Paper Towels, the quicker picker upper!

We're noticing a few unruly nose-hairs, we've ordered you a Fancii Professional Nose & Ear Hair Trimmer with LED Light, Water Resistant, Stainless Steel Blades, and Battery Power!
Score: 6 Votes (Like | Disagree)

Top Stories

apple california streaming event

Apple Event Announced: 'California Streaming' on September 14 With iPhone 13, Apple Watch Series 7 Expected

Tuesday September 7, 2021 9:03 am PDT by
Apple today announced that it will be holding a special event on Tuesday, September 14 at 10:00 a.m. The event will take place at the Steve Jobs Theater on the Apple Park campus in Cupertino, California. As with WWDC and last year's fall events, this new event will be held digitally with no members of the media invited to attend in person. Apple will likely provide pre-taped segments for...
Apple Prefer Lightning Over USB C Feature

iPhone Sticking With Lightning Port Over USB-C for 'Foreseeable Future'

Tuesday March 2, 2021 9:32 am PST by
Apple will retain the Lightning connector on the iPhone for the "foreseeable future," with no intention of switching to USB-C, according to reliable analyst Ming-Chi Kuo. In spite of much of the industry moving toward USB-C, Apple will not be using it to replace the Lightning connector on the iPhone 13, or indeed on any iPhone model for the time being. In a note seen by MacRumors yesterday,...
iphone 12 colors 2021

iPhone 12 Colors: Deciding on The Right Color

Thursday November 5, 2020 8:35 am PST by
The iPhone 12 and iPhone 12 Pro arrived in October 2020 in a range of color options, with entirely new hues available on both devices, as well as some popular classics. The 12 and 12 Pro have different color choices, so if you have your heart set on a particular shade, you might not be able to get your preferred model in that color. iPhone 12 mini and iPhone 12 The iPhone 12 mini and iPhone...
it home ecommerce app iphone 13

iPhone 13 to Launch on September 17, AirPods 3 on September 30, Claims Report

Wednesday August 25, 2021 2:42 am PDT by
Apple may be planning to launch the iPhone 13 on Friday, September 17 and third-generation AirPods on Thursday, September 30, according to an image of an e-commerce app discovered by Chinese language site IT Home. The screenshot, originally posted by Weibo account @PandaIsBald, suggests all four iPhone 13 models will go on sale on September 17, followed by the AirPods 3 on September 30....
youtube apple tv

YouTube Discontinuing 3rd-Generation Apple TV App, AirPlay Still Available

Wednesday February 3, 2021 3:09 pm PST by
YouTube is planning to stop supporting its YouTube app on the third-generation Apple TV models, where YouTube has long been available as a channel option. A 9to5Mac reader received a message about the upcoming app discontinuation, which is set to take place in March.Starting early March, the YouTube app will no longer be available on Apple TV (3rd generation). You can still watch YouTube on...
original iphone

Phil Schiller Says iPhone Was 'Earth-Shattering' Ten Years Ago and Remains 'Unmatched' Today

Monday January 9, 2017 7:15 am PST by
To commemorate the tenth anniversary of the iPhone, Apple marketing chief Phil Schiller sat down with tech journalist Steven Levy for a wide-ranging interview about the smartphone's past, present, and future. The report first reflects upon the iPhone's lack of support for third-party apps in its first year. The argument inside Apple was split between whether the iPhone should be a closed...
studio buds family

Beats Studio Buds Debuting Today With Active Noise Cancellation, Stemless Design, and More for $150

Monday June 14, 2021 8:00 am PDT by
We've seen a lot of teasers about the Beats Studio Buds over the past month since they first showed up in Apple's beta software updates, and today they're finally official. The Beats Studio Buds are available to order today in red, white, and black ahead of a June 24 ship date, and they're priced at $149.99. The Studio Buds are the first Beats-branded earbuds to truly compete with AirPods...
maroon5memories

Apple Collaborates With Maroon 5 to Add 'Memories' Song to Photos App

Wednesday September 25, 2019 12:02 pm PDT by
Apple has teamed up with Maroon 5 to add the group's new song "Memories" to the Memories feature in the Photos app, allowing it to be used for photo slide show creations, reports Billboard. "Memories" will be available as a soundtrack option for a limited time and it is available to iPhone and iPad users running the latest iOS 13 and iPadOS software. Memories in the Photos app are created ...
ulysses Blog Publishing

Ulysses 22 Brings New Blogging Options and Visual Customizations

Monday March 22, 2021 5:24 am PDT by
Popular writing app Ulysses today received its 22nd major release, introducing new publishing features and additional options for users to customize the visual appearance of their writing environment. As well as providing a focused writing environment, Ulysses offers ways to publish texts from within the app to various blogging platforms. Version 22 adds the ability to publish to Micro.blog, ...
affinity designer contour tool

Serif Updates Affinity Photo, Designer, and Publisher With New Tools and Functions

Thursday February 4, 2021 1:58 am PST by
Serif today announced across-the-board updates for its popular suite of Affinity creative apps, including Affinity Photo, Affinity Designer, and the Apple award-winning Affinity Publisher for Mac, all of which were among the first professional creative suites to be optimized for Apple's new M1 chip. "After another year which saw record numbers of people switching to Affinity, it's exciting to...