Members of Apple's PrimeSense Team Patent Method of Interacting With Mac Using Hand Gestures

Apple today was granted a patent originally filed in August 2016, describing a method in which users would be able to control a Mac computer -- and potentially other devices -- using a "non-tactile three dimensional (3D) user interface" (via Patently Apple). The patent's inventor credits go in part to Amir Hoffnung and Jonathan Pokrass, two current Apple employees who joined the company from PrimeSense after Apple acquired it November 2013.

Some of PrimeSense's tech, which was originally used in Microsoft's Kinect devices on Xbox platforms, now resides in the front-facing TrueDepth camera of the iPhone X, and the new patent hints at a potential future where this technology expands in function to Macs as well. Instead of recognizing faces, Apple's patent describes a Mac that recognizes a "gesture by a hand," allowing users to interact with the computer without tactile inputs like a keyboard, mouse, or trackpad.

apple 3d UI patent


The patent includes a variety of gestures that users would use to control the 3D user interface, including what are called "push," (figure 2) "wave," (figure 3) and "up" (figure 5) interactions, which are all grouped into a category of "focus gestures." According to the patent, some of these could be used to perform basic app interactions, like scrolling through a menu, as well as change the state of the system from locked to unlocked.

Gestures described herein include focus gestures and unlock gestures. A focus gesture enables the user to engage (i.e., take control of) an inactive non-tactile 3D user interface. An unlock gesture enables the user to engage a locked non-tactile 3D user interface, as pressing a specific sequence of keys unlocks a locked cellular phone. In some embodiments, the non-tactile 3D user interface conveys visual feedback to the user performing the focus and the unlock gestures.

Examples of unlock gestures include an "up" gesture (e.g., raising hand 30 a specified distance), a sequence of two sequential wave gestures, and a sequence of two sequential push gestures, as described in detail hereinbelow.

Apple has previously been granted patents related to 3D sensing and gesture controls, but the company has yet to release a product that takes advantage of these features. Some of these previous patents included gesture controls on iPads and iPhones as well as on Magic Keyboards, which would allow users to gain access to virtual buttons and potentially streamline certain elements of the user interface.

apple 3d ui patent 2
Of course, with Face ID already launched on iPhone X, it's predicted that Apple will look into adding the facial recognition software into future iMacs and MacBooks before it focuses on a new unlocking system. It's also unclear whether the new 3D user interface patent would include both pieces of technology -- 3D gestures and Face ID -- to further augment a Mac's unlocking process.

As with any patent, the technology in question might not make it to an Apple product in the near future, if at all, but it is an interesting glimpse into what Apple might be planning to do with PrimeSense's technology down the line.

Tag: Patent

Top Rated Comments

JM Avatar
70 months ago
I want my unlock to be a middle finger. That'll be fun.

Wait, no... I want the middle finger to force-close everything.
Score: 13 Votes (Like | Disagree)
PaddyTB Avatar
70 months ago
How can this ever be novel or non-obvious? US patent system is soooo broken :-(
Score: 7 Votes (Like | Disagree)
jcmeyer5 Avatar
70 months ago
I got a hand gesture for this "technology"....
Score: 7 Votes (Like | Disagree)
zarmanto Avatar
70 months ago
Remind me again, whether or not fictional representations can be used as prior art?

Score: 6 Votes (Like | Disagree)
Lepton Avatar
70 months ago
It should recognize American Sign Language, which would give you a gestural keyboard that is already well worked out, and some people already know it.
Score: 6 Votes (Like | Disagree)
asleep Avatar
70 months ago


Because we want a bunch of lasers in our eyes and tracking our every move?

Think what Google analytics and Amazon can do with that...

We see you're reaching for a pen... how about a uni-ball Vision Rollerball Pens, Fine Point (0.7mm), Black, 12 Count?

Would you like some SweetLeaf Natural Stevia Sweetener to go in you tea?

Oops, looks like you spilled your coffee. How about some Bounty Select-a-Size Paper Towels, the quicker picker upper!

We're noticing a few unruly nose-hairs, we've ordered you a Fancii Professional Nose & Ear Hair Trimmer with LED Light, Water Resistant, Stainless Steel Blades, and Battery Power!
Score: 6 Votes (Like | Disagree)

Popular Stories

dynamic island

iPhone 15 Dynamic Island to Include New Integrated Proximity Sensor

Friday March 24, 2023 12:27 am PDT by
This year, all iPhone 15 models will include Apple's Dynamic Island that unifies the pill and hole cutouts at the top of the display, but there will also be a material change to the feature that wasn't included in the iPhone 14 Pro models. According to a new tweet by Apple industry analyst Ming-Chi Kuo, the proximity sensor on the iPhone 15 series will be integrated inside the Dynamic Island ...
apple park at night 1

Apple 'Tracking Employee Attendance' in Crackdown on Remote Working

Thursday March 23, 2023 3:41 am PDT by
Apple is tracking the attendance of its employees at offices using badge records in order to ensure they are coming in at least three times a week, according to Platformer's Zoë Schiffer. Since April 2022, Apple employees have been operating on a hybrid home/office work policy as part of a gradual return strategy following the pandemic, with staff required to work from the office at least...
iphone 14 pro max deep purple feature purple

iPhone 15 Pro Rumor Recap: 10 New Features and Changes to Expect

Thursday March 23, 2023 6:42 am PDT by
While the iPhone 15 series is still around six months away from launching, there have already been plenty of rumors about the devices. Many new features and changes have been rumored for the iPhone 15 Pro and iPhone 15 Pro Max in particular. Below, we have recapped 10 changes rumored for iPhone 15 Pro models that are not expected to be available on the standard iPhone 15 and iPhone 15 Plus:A1...
maxresdefault

Nothing Launches $149 Ear (2) Wireless Earbuds to Compete With AirPods Pro 2

Wednesday March 22, 2023 9:48 am PDT by
Nothing today announced the launch of its second-generation wireless earbuds, the Nothing Ear (2), which offer many of the same features as Apple's AirPods Pro 2 at a lower price point. We went hands-on with the Ear (2) earbuds to see whether they're a viable alternative to the AirPods Pro 2 for those who want to save some cash. The Ear (2) earbuds are the successor to the Nothing Ear (1),...
TMobile Sprint

Apple Stops Allowing Sprint iPhone Activations, Removes Sprint References From Online Store

Thursday March 23, 2023 12:06 pm PDT by
Apple is no longer allowing customers who purchase an iPhone, cellular iPad, or Apple Watch to activate a device with now-defunct mobile carrier Sprint. Apple has also removed remaining references to Sprint from its online store. When checking out with a new purchase, Sprint is no longer an option for connectivity, a change that Apple appears to have implemented today. Prior to now, Sprint...
iOS 16

iOS 16.4 for iPhone Nearing Launch With These 5 New Features

Monday March 20, 2023 11:50 am PDT by
Apple says iOS 16.4 is coming in the spring, which began this week. In his Sunday newsletter, Bloomberg's Mark Gurman said the update should be released "in the next three weeks or so," meaning a public release is likely in late March or early April. iOS 16.4 remains in beta testing and introduces a handful of new features and changes for the iPhone. Below, we have recapped five new features ...
Hero0009

Best Apple Deals of the Week: Samsung's Smart Monitor M8 Gets Massive $250 Discount, Along With Year's Best AirPods Prices

Friday March 24, 2023 10:23 am PDT by
We saw a lot of great deals on Apple products and related accessories this week, including Samsung's iMac-like Smart Monitor M8 for $250 off, a 30 percent off spring sale at Anker, and the year's best prices on numerous AirPods models. All of these deals are still available to purchase right now, so we're recapping them and more below. Note: MacRumors is an affiliate partner with some of these ...
top stories 25mar2023

Top Stories: iPhone 15 Pro Design Leak, iOS 16.4 Coming Soon, and More

Saturday March 25, 2023 6:00 am PDT by
We're still almost six months away from the official unveiling of the iPhone 15 lineup, but it seems like every day we're learning more about what to expect from the next-generation models. Notably, this week gave us our clearest look yet at what appear to be some changes for the volume and mute control hardware. iOS 16.4 and associated releases are also right around the corner with some new ...