Members of Apple's PrimeSense Team Patent Method of Interacting With Mac Using Hand Gestures

Apple today was granted a patent originally filed in August 2016, describing a method in which users would be able to control a Mac computer -- and potentially other devices -- using a "non-tactile three dimensional (3D) user interface" (via Patently Apple). The patent's inventor credits go in part to Amir Hoffnung and Jonathan Pokrass, two current Apple employees who joined the company from PrimeSense after Apple acquired it November 2013.

Some of PrimeSense's tech, which was originally used in Microsoft's Kinect devices on Xbox platforms, now resides in the front-facing TrueDepth camera of the iPhone X, and the new patent hints at a potential future where this technology expands in function to Macs as well. Instead of recognizing faces, Apple's patent describes a Mac that recognizes a "gesture by a hand," allowing users to interact with the computer without tactile inputs like a keyboard, mouse, or trackpad.

apple 3d UI patent


The patent includes a variety of gestures that users would use to control the 3D user interface, including what are called "push," (figure 2) "wave," (figure 3) and "up" (figure 5) interactions, which are all grouped into a category of "focus gestures." According to the patent, some of these could be used to perform basic app interactions, like scrolling through a menu, as well as change the state of the system from locked to unlocked.

Gestures described herein include focus gestures and unlock gestures. A focus gesture enables the user to engage (i.e., take control of) an inactive non-tactile 3D user interface. An unlock gesture enables the user to engage a locked non-tactile 3D user interface, as pressing a specific sequence of keys unlocks a locked cellular phone. In some embodiments, the non-tactile 3D user interface conveys visual feedback to the user performing the focus and the unlock gestures.

Examples of unlock gestures include an "up" gesture (e.g., raising hand 30 a specified distance), a sequence of two sequential wave gestures, and a sequence of two sequential push gestures, as described in detail hereinbelow.

Apple has previously been granted patents related to 3D sensing and gesture controls, but the company has yet to release a product that takes advantage of these features. Some of these previous patents included gesture controls on iPads and iPhones as well as on Magic Keyboards, which would allow users to gain access to virtual buttons and potentially streamline certain elements of the user interface.

apple 3d ui patent 2
Of course, with Face ID already launched on iPhone X, it's predicted that Apple will look into adding the facial recognition software into future iMacs and MacBooks before it focuses on a new unlocking system. It's also unclear whether the new 3D user interface patent would include both pieces of technology -- 3D gestures and Face ID -- to further augment a Mac's unlocking process.

As with any patent, the technology in question might not make it to an Apple product in the near future, if at all, but it is an interesting glimpse into what Apple might be planning to do with PrimeSense's technology down the line.

Tag: patent

Top Rated Comments

JM Avatar
54 months ago
I want my unlock to be a middle finger. That'll be fun.

Wait, no... I want the middle finger to force-close everything.
Score: 13 Votes (Like | Disagree)
PaddyTB Avatar
54 months ago
How can this ever be novel or non-obvious? US patent system is soooo broken :-(
Score: 7 Votes (Like | Disagree)
jcmeyer5 Avatar
54 months ago
I got a hand gesture for this "technology"....
Score: 7 Votes (Like | Disagree)
zarmanto Avatar
54 months ago
Remind me again, whether or not fictional representations can be used as prior art?

Score: 6 Votes (Like | Disagree)
Lepton Avatar
54 months ago
It should recognize American Sign Language, which would give you a gestural keyboard that is already well worked out, and some people already know it.
Score: 6 Votes (Like | Disagree)
asleep Avatar
54 months ago


Because we want a bunch of lasers in our eyes and tracking our every move?

Think what Google analytics and Amazon can do with that...

We see you're reaching for a pen... how about a uni-ball Vision Rollerball Pens, Fine Point (0.7mm), Black, 12 Count?

Would you like some SweetLeaf Natural Stevia Sweetener to go in you tea?

Oops, looks like you spilled your coffee. How about some Bounty Select-a-Size Paper Towels, the quicker picker upper!

We're noticing a few unruly nose-hairs, we've ordered you a Fancii Professional Nose & Ear Hair Trimmer with LED Light, Water Resistant, Stainless Steel Blades, and Battery Power!
Score: 6 Votes (Like | Disagree)

Popular Stories

Upcoming Products 2022 Feature

Gurman: Apple Preparing 'Widest Array of New Hardware Products in Its History' for Fall

Sunday January 23, 2022 10:32 am PST by
Apple is working on a number of new products that are set to launch this fall, and Bloomberg's Mark Gurman says that it will be "the widest array" of new devices that Apple has introduced in its history. In his latest "Power On" newsletter, Gurman explains that Apple is working on four new flagship iPhones (iPhone 14, iPhone 14 Max, iPhone 14 Pro, and iPhone 14 Pro Max), an updated low-end Ma...
macbook pro 14 16 2021

Three Months After Launch, Apple Still Struggling to Meet Demand for Redesigned 14-Inch and 16-Inch MacBook Pro

Monday January 24, 2022 7:12 am PST by
Three months after their launch, the 14-inch and 16-inch MacBook Pros continue to experience high demand and seemingly short supply, with shipping dates for both models stretching into multiple weeks in several of Apple's key markets. In the United States, the baseline 14-inch MacBook Pro with the M1 Pro chip is estimated to ship in three to four weeks, promising an arrival by at least...
General Dropbox Feature

macOS 12.3 Will Include Cloud Storage Changes Affecting Dropbox and OneDrive

Tuesday January 25, 2022 3:31 pm PST by
Dropbox today announced that users who update to macOS 12.3 once that software version becomes available may temporarily encounter issues with opening online-only files in some third-party apps on their Mac. In a support document and an email to customers, Dropbox said it is actively working on full support for online-only files on macOS 12.3 and will begin rolling out an updated version of...
Questionable Design Decisions

Apple's Most Questionable Design Decisions in Recent Memory

Sunday January 23, 2022 2:59 am PST by
Apple has always emphasized the depth of thought that goes into the design of its products. In the foreword to Designed by Apple in California, a photo book released by the company in 2016, Jony Ive explains how Apple strives "to define objects that appear effortless" and "so simple, coherent and inevitable that there could be no rational alternative." But every once in a while even Apple...
Apple Watch Red Yellow Green Feature 1

Apple Launches Black Unity Braided Solo Loop With 'Unity Lights' Watch Face

Wednesday January 26, 2022 6:05 am PST by
Apple today announced the Black Unity Braided Solo Loop for the Apple Watch, as well as a new downloadable watch face, to celebrate Black History Month. Following the launch of the limited edition Black Unity Apple Watch Series 6 and Sport Band in 2021, Apple today launched the Black Unity Braided Solo Loop as part of its celebrations for Black History Month this year.Apple is launching a...
att gigabit internet

AT&T Bringing $180/Month 5-Gigabit Internet to 70 Cities

Monday January 24, 2022 9:20 am PST by
AT&T today announced the launch of upgraded AT&T Fiber plans, which support speeds of up to 5 Gigabits for some customers. There are two separate plans, one "2 GIG" plan and one "5 GIG" plan, available to new and existing AT&T Fiber subscribers. According to AT&T, the new plans are available to nearly 5.2 million customers across 70 metro areas including Los Angeles, Atlanta, Chicago, San...
intel vs m1 max chip purple

Benchmarks Confirm Intel's Latest Core i9 Chip Outperforms Apple's M1 Max With Several Caveats

Wednesday January 26, 2022 8:56 am PST by
Benchmark results have started to surface for MSI's new GE76 Raider, one of the first laptops to be powered by Intel's new 12th-generation Core i9 processor. Intel previously said that its new high-end Core i9 processor is faster than Apple's M1 Max chip in the 16-inch MacBook Pro and, as noted by Macworld, early Geekbench 5 results do appear to confirm this claim, but there are several...
ios 15

Apple Releases iOS 15.3 and iPadOS 15.3 With Fix for Safari Bug That Leaks Browsing Activity

Wednesday January 26, 2022 10:00 am PST by
Apple today released iOS 15.3 and iPadOS 15.3, the third major updates to the iOS and iPadOS 15 operating systems that were released in September 2021. iOS and iPadOS 15.3 come almost two weeks after the release of iOS and iPadOS 15.2.1, minor bug fix updates. The iOS 15.3 and iPadOS 15.3 updates can be downloaded for free and the software is available on all eligible devices over-the-air in ...