Members of Apple's PrimeSense Team Patent Method of Interacting With Mac Using Hand Gestures

Apple today was granted a patent originally filed in August 2016, describing a method in which users would be able to control a Mac computer -- and potentially other devices -- using a "non-tactile three dimensional (3D) user interface" (via Patently Apple). The patent's inventor credits go in part to Amir Hoffnung and Jonathan Pokrass, two current Apple employees who joined the company from PrimeSense after Apple acquired it November 2013.

Some of PrimeSense's tech, which was originally used in Microsoft's Kinect devices on Xbox platforms, now resides in the front-facing TrueDepth camera of the iPhone X, and the new patent hints at a potential future where this technology expands in function to Macs as well. Instead of recognizing faces, Apple's patent describes a Mac that recognizes a "gesture by a hand," allowing users to interact with the computer without tactile inputs like a keyboard, mouse, or trackpad.

apple 3d UI patent


The patent includes a variety of gestures that users would use to control the 3D user interface, including what are called "push," (figure 2) "wave," (figure 3) and "up" (figure 5) interactions, which are all grouped into a category of "focus gestures." According to the patent, some of these could be used to perform basic app interactions, like scrolling through a menu, as well as change the state of the system from locked to unlocked.

Gestures described herein include focus gestures and unlock gestures. A focus gesture enables the user to engage (i.e., take control of) an inactive non-tactile 3D user interface. An unlock gesture enables the user to engage a locked non-tactile 3D user interface, as pressing a specific sequence of keys unlocks a locked cellular phone. In some embodiments, the non-tactile 3D user interface conveys visual feedback to the user performing the focus and the unlock gestures.

Examples of unlock gestures include an "up" gesture (e.g., raising hand 30 a specified distance), a sequence of two sequential wave gestures, and a sequence of two sequential push gestures, as described in detail hereinbelow.

Apple has previously been granted patents related to 3D sensing and gesture controls, but the company has yet to release a product that takes advantage of these features. Some of these previous patents included gesture controls on iPads and iPhones as well as on Magic Keyboards, which would allow users to gain access to virtual buttons and potentially streamline certain elements of the user interface.

apple 3d ui patent 2
Of course, with Face ID already launched on iPhone X, it's predicted that Apple will look into adding the facial recognition software into future iMacs and MacBooks before it focuses on a new unlocking system. It's also unclear whether the new 3D user interface patent would include both pieces of technology -- 3D gestures and Face ID -- to further augment a Mac's unlocking process.

As with any patent, the technology in question might not make it to an Apple product in the near future, if at all, but it is an interesting glimpse into what Apple might be planning to do with PrimeSense's technology down the line.

Tag: patent

Top Rated Comments

JM Avatar
43 months ago
I want my unlock to be a middle finger. That'll be fun.

Wait, no... I want the middle finger to force-close everything.
Score: 13 Votes (Like | Disagree)
PaddyTB Avatar
43 months ago
How can this ever be novel or non-obvious? US patent system is soooo broken :-(
Score: 7 Votes (Like | Disagree)
jcmeyer5 Avatar
43 months ago
I got a hand gesture for this "technology"....
Score: 7 Votes (Like | Disagree)
zarmanto Avatar
43 months ago
Remind me again, whether or not fictional representations can be used as prior art?

Score: 6 Votes (Like | Disagree)
Lepton Avatar
43 months ago
It should recognize American Sign Language, which would give you a gestural keyboard that is already well worked out, and some people already know it.
Score: 6 Votes (Like | Disagree)
asleep Avatar
43 months ago


Because we want a bunch of lasers in our eyes and tracking our every move?

Think what Google analytics and Amazon can do with that...

We see you're reaching for a pen... how about a uni-ball Vision Rollerball Pens, Fine Point (0.7mm), Black, 12 Count?

Would you like some SweetLeaf Natural Stevia Sweetener to go in you tea?

Oops, looks like you spilled your coffee. How about some Bounty Select-a-Size Paper Towels, the quicker picker upper!

We're noticing a few unruly nose-hairs, we've ordered you a Fancii Professional Nose & Ear Hair Trimmer with LED Light, Water Resistant, Stainless Steel Blades, and Battery Power!
Score: 6 Votes (Like | Disagree)

Top Stories

jon prosser imac 2021colors

Prosser: 2021 iMac to Come in Five Colors, Apple Silicon Mac Pro to Resemble 'Stacked' Mac Minis

Wednesday February 24, 2021 7:26 am PST by
Hit-and-miss leaker Jon Prosser has today alleged that the upcoming 2021 iMac models will offer five color options, mirroring the colors of the fourth-generation iPad Air, and revealed a number of additional details about the Mac Pro with Apple Silicon. In a new video on YouTube channel FrontPageTech, Prosser explained that the redesigned iMacs will come featuring options for Silver, Space ...
First Look Big Sur Feature2

Apple Releases macOS Big Sur 11.2.2 to Prevent MacBooks From Being Damaged by Third-Party Non-Compliant Docks

Thursday February 25, 2021 10:07 am PST by
Apple today released macOS Big Sur 11.2.2, the fourth update to the macOS Big Sur operating system that launched in November. macOS Big Sur 11.2.2 comes two weeks after the release of macOS Big Sur 11.2.1, a bug fix update. The new ‌‌‌‌macOS Big Sur‌‌‌ 11.2.2‌ update can be downloaded for free on all eligible Macs using the Software Update section of System Preferences....
microsoft edge ios android

Bill Gates Says His Preference for Android Over iPhone is Due to Pre-Installed Software

Friday February 26, 2021 3:35 am PST by
Microsoft co-founder Bill Gates this week participated in his first meeting on Clubhouse, the increasingly popular invite-only conversation app, where he fielded a range of questions as part of an ongoing book tour. Gates was interviewed by journalist Andrew Ross Sorkin, and given that the Clubhouse app is currently only available on iOS, naturally one of the questions that came up was...
flat mbp 14 inch feature yellow

Redesigned 14-Inch MacBook Pro Expected to Feature Brighter Mini-LED Display With Slimmer Bezels and More

Thursday February 25, 2021 7:48 am PST by
Apple plans to unveil new 14-inch and 16-inch MacBook Pro models with Mini-LED-backlit displays in the second half of this year, according to industry sources cited by Taiwanese supply chain publication DigiTimes. The report claims that Radiant Opto-Electronics will be the exclusive supplier of the Mini-LED backlight units, while Quanta Computer is said to be tasked with final assembly of the...
m1 mac mini

M1 Mac Users Report Excessive SSD Wear

Tuesday February 23, 2021 7:07 am PST by
Over the past week, some M1 Mac users have been reporting alarming SSD health readings, suggesting that these devices are writing extraordinary amounts of data to their drives (via iMore). Across Twitter and the MacRumors forums, users are reporting that M1 Macs are experiencing extremely high drive writes over a short space of time. In what appear to be the most severe cases, M1 Macs are sai...
steam apple logo

Valve Ordered to Give Apple Information on 436 Steam Games As Part of Epic Games Legal Case

Thursday February 25, 2021 1:50 am PST by
Valve, the makers behind popular game distribution platform Steam, will be forced to hand over aggregate historical sales, price, and other information on 436 games hosted on the store to Apple, as part of the Apple vs. Epic Games antitrust case. As reported in a paywalled report by Law360, during a virtual discovery hearing on Wednesday, U.S. Magistrate Judge Thomas S. Hixson ordered that...
2021 mbp sd slot feature2

Kuo: New MacBook Pro Models With HDMI Port and SD Card Reader to Launch Later This Year

Monday February 22, 2021 8:52 pm PST by
Apple plans to release two new MacBook Pro models equipped with an HDMI port and SD card reader in the second half of 2021, according to analyst Ming-Chi Kuo, who outlined his expectations in a research note obtained by MacRumors. The return of an SD card reader was first reported by Bloomberg's Mark Gurman last month. "We predict that Apple's two new MacBook Pro models in 2H21 will have...
apple store macarthur center

Apple Store at MacArthur Center in Virginia Permanently Closing Following Years of Safety Issues at Shopping Mall

Thursday February 25, 2021 4:45 pm PST by
Apple today indicated that its retail store at the MacArthur Center shopping mall in Norfolk, Virginia will be permanently closing after over 14 years of business, although an exact closure date has yet to be announced by the company. Apple has assured that it will be offering all employees at the store other positions within Apple, and said that it looks forward to continuing to serve...
qualcomm snapdragon x60 5g

iPhone 13 Lineup Expected to Use Qualcomm's Snapdragon X60 Modem With Several 5G Improvements

Wednesday February 24, 2021 8:10 am PST by
Apple's next-generation iPhone 13 lineup will use Qualcomm's Snapdragon X60 5G modem, with Samsung to handle manufacturing of the chip, according to DigiTimes. Built on a 5nm process, the X60 packs higher power efficiency into a smaller footprint compared to the 7nm-based Snapdragon X55 modem used in iPhone 12 models, which could contribute to longer battery life. With the X60 modem, iPhone...
anker magsafe powercore battery pack

Anker Releases MagSafe-Compatible Battery Pack for iPhone 12 Lineup

Tuesday February 23, 2021 7:49 am PST by
Following rumors that Apple is working on a MagSafe battery pack for iPhone 12 models, popular accessory maker Anker has beaten Apple to the punch with the release of its PowerCore Magnetic 5K Wireless Power Bank. First previewed at CES 2021, the PowerCore battery pack magnetically attaches to the back of any iPhone 12 model and provides 5W of wireless charging. With a 5,000 mAh capacity,...