Members of Apple's PrimeSense Team Patent Method of Interacting With Mac Using Hand Gestures

Apple today was granted a patent originally filed in August 2016, describing a method in which users would be able to control a Mac computer -- and potentially other devices -- using a "non-tactile three dimensional (3D) user interface" (via Patently Apple). The patent's inventor credits go in part to Amir Hoffnung and Jonathan Pokrass, two current Apple employees who joined the company from PrimeSense after Apple acquired it November 2013.

Some of PrimeSense's tech, which was originally used in Microsoft's Kinect devices on Xbox platforms, now resides in the front-facing TrueDepth camera of the iPhone X, and the new patent hints at a potential future where this technology expands in function to Macs as well. Instead of recognizing faces, Apple's patent describes a Mac that recognizes a "gesture by a hand," allowing users to interact with the computer without tactile inputs like a keyboard, mouse, or trackpad.

apple 3d UI patent


The patent includes a variety of gestures that users would use to control the 3D user interface, including what are called "push," (figure 2) "wave," (figure 3) and "up" (figure 5) interactions, which are all grouped into a category of "focus gestures." According to the patent, some of these could be used to perform basic app interactions, like scrolling through a menu, as well as change the state of the system from locked to unlocked.

Gestures described herein include focus gestures and unlock gestures. A focus gesture enables the user to engage (i.e., take control of) an inactive non-tactile 3D user interface. An unlock gesture enables the user to engage a locked non-tactile 3D user interface, as pressing a specific sequence of keys unlocks a locked cellular phone. In some embodiments, the non-tactile 3D user interface conveys visual feedback to the user performing the focus and the unlock gestures.

Examples of unlock gestures include an "up" gesture (e.g., raising hand 30 a specified distance), a sequence of two sequential wave gestures, and a sequence of two sequential push gestures, as described in detail hereinbelow.

Apple has previously been granted patents related to 3D sensing and gesture controls, but the company has yet to release a product that takes advantage of these features. Some of these previous patents included gesture controls on iPads and iPhones as well as on Magic Keyboards, which would allow users to gain access to virtual buttons and potentially streamline certain elements of the user interface.

apple 3d ui patent 2
Of course, with Face ID already launched on iPhone X, it's predicted that Apple will look into adding the facial recognition software into future iMacs and MacBooks before it focuses on a new unlocking system. It's also unclear whether the new 3D user interface patent would include both pieces of technology -- 3D gestures and Face ID -- to further augment a Mac's unlocking process.

As with any patent, the technology in question might not make it to an Apple product in the near future, if at all, but it is an interesting glimpse into what Apple might be planning to do with PrimeSense's technology down the line.

Tag: Patent

Top Rated Comments

JM Avatar
83 months ago
I want my unlock to be a middle finger. That'll be fun.

Wait, no... I want the middle finger to force-close everything.
Score: 13 Votes (Like | Disagree)
PaddyTB Avatar
83 months ago
How can this ever be novel or non-obvious? US patent system is soooo broken :-(
Score: 7 Votes (Like | Disagree)
jcmeyer5 Avatar
83 months ago
I got a hand gesture for this "technology"....
Score: 7 Votes (Like | Disagree)
zarmanto Avatar
83 months ago
Remind me again, whether or not fictional representations can be used as prior art?

Score: 6 Votes (Like | Disagree)
Lepton Avatar
83 months ago
It should recognize American Sign Language, which would give you a gestural keyboard that is already well worked out, and some people already know it.
Score: 6 Votes (Like | Disagree)
asleep Avatar
83 months ago


Because we want a bunch of lasers in our eyes and tracking our every move?

Think what Google analytics and Amazon can do with that...

We see you're reaching for a pen... how about a uni-ball Vision Rollerball Pens, Fine Point (0.7mm), Black, 12 Count?

Would you like some SweetLeaf Natural Stevia Sweetener to go in you tea?

Oops, looks like you spilled your coffee. How about some Bounty Select-a-Size Paper Towels, the quicker picker upper!

We're noticing a few unruly nose-hairs, we've ordered you a Fancii Professional Nose & Ear Hair Trimmer with LED Light, Water Resistant, Stainless Steel Blades, and Battery Power!
Score: 6 Votes (Like | Disagree)

Popular Stories

reset password request iphone

Warning: Apple Users Targeted in Phishing Attack Involving Rapid Password Reset Requests

Tuesday March 26, 2024 4:34 pm PDT by
Phishing attacks taking advantage of Apple's password reset feature have become increasingly common, according to a report from KrebsOnSecurity. Multiple Apple users have been targeted in an attack that bombards them with an endless stream of notifications or multi-factor authentication (MFA) messages in an attempt to cause panic so they'll respond favorably to social engineering. An...
Generic iOS 18 Feature Purple

iOS 18: What to Expect From 'Biggest' Update in iPhone's History

Wednesday March 27, 2024 11:10 am PDT by
At least some Apple software engineers continue to believe that iOS 18 will be the "biggest" update in the iPhone's history, according to Bloomberg's Mark Gurman. Below, we recap rumored features and changes for the iPhone. "The iOS 18 update is expected to be the most ambitious overhaul of the iPhone's software in its history, according to people working on the upgrade," wrote Gurman, in a r...
maxresdefault

Apple to Launch New iPad Pro and iPad Air Models in May

Thursday March 28, 2024 11:07 am PDT by
Apple will introduce new iPad Pro and iPad Air models in early May, according to Bloomberg's Mark Gurman. Gurman previously suggested the new iPads would come out in March, and then April, but the timeline has been pushed back once again. Subscribe to the MacRumors YouTube channel for more videos. Apple is working on updates to both the iPad Pro and iPad Air models. The iPad Pro models will...
maxresdefault

Apple Announces WWDC 2024 Event for June 10 to 14

Tuesday March 26, 2024 10:02 am PDT by
Apple today announced that its 35th annual Worldwide Developers Conference is set to take place from Monday, June 10 to Friday, June 14. As with WWDC events since 2020, WWDC 2024 will be an online event that is open to all developers at no cost. Subscribe to the MacRumors YouTube channel for more videos. WWDC 2024 will include online sessions and labs so that developers can learn about new...
apple maps 3d feature

Apple Maps May Gain Custom Routes With iOS 18

Tuesday March 26, 2024 3:10 pm PDT by
Apple may be planning to add support for "custom routes" in Apple Maps in iOS 18, according to code reviewed by MacRumors. Apple Maps does not currently offer a way to input self-selected routes, with Maps users limited to Apple's pre-selected options, but that may change in iOS 18. Apple has pushed an iOS 18 file to its maps backend labeled "CustomRouteCreation." While not much is revealed...
General iOS 17 Feature Orange Purple

Apple Releases Revised Versions of iOS 17.4.1 and iPadOS 17.4.1 With Updated Build Number

Wednesday March 27, 2024 5:59 am PDT by
Apple on late Tuesday released revised versions of iOS 17.4.1 and iPadOS 17.4.1 with an updated build number of 21E237, according to MacRumors contributor Aaron Perris. The updates previously had a build number of 21E236. The revised updates are available for all iPhone and iPad models that are compatible with iOS 17 and iPadOS 17, but they can only be installed via the Finder app on macOS...
applephilschiller

Apple's Phil Schiller Works 80 Hours a Week Overseeing App Store

Wednesday March 27, 2024 2:03 pm PDT by
With the App Store and app ecosystem undergoing major changes in the European Union, The Wall Street Journal today shared a profile on App Store chief Phil Schiller, who is responsible for the App Store. Though Schiller transitioned from marketing chief to "Apple Fellow" in 2020 to take a step back from Apple and spend more time on personal projects and friends, he is reportedly working...