Members of Apple's PrimeSense Team Patent Method of Interacting With Mac Using Hand Gestures

Apple today was granted a patent originally filed in August 2016, describing a method in which users would be able to control a Mac computer -- and potentially other devices -- using a "non-tactile three dimensional (3D) user interface" (via Patently Apple). The patent's inventor credits go in part to Amir Hoffnung and Jonathan Pokrass, two current Apple employees who joined the company from PrimeSense after Apple acquired it November 2013.

Some of PrimeSense's tech, which was originally used in Microsoft's Kinect devices on Xbox platforms, now resides in the front-facing TrueDepth camera of the iPhone X, and the new patent hints at a potential future where this technology expands in function to Macs as well. Instead of recognizing faces, Apple's patent describes a Mac that recognizes a "gesture by a hand," allowing users to interact with the computer without tactile inputs like a keyboard, mouse, or trackpad.

Images via United States Patent and Trademark Office

The patent includes a variety of gestures that users would use to control the 3D user interface, including what are called "push," (figure 2) "wave," (figure 3) and "up" (figure 5) interactions, which are all grouped into a category of "focus gestures." According to the patent, some of these could be used to perform basic app interactions, like scrolling through a menu, as well as change the state of the system from locked to unlocked.

Gestures described herein include focus gestures and unlock gestures. A focus gesture enables the user to engage (i.e., take control of) an inactive non-tactile 3D user interface. An unlock gesture enables the user to engage a locked non-tactile 3D user interface, as pressing a specific sequence of keys unlocks a locked cellular phone. In some embodiments, the non-tactile 3D user interface conveys visual feedback to the user performing the focus and the unlock gestures.

Examples of unlock gestures include an "up" gesture (e.g., raising hand 30 a specified distance), a sequence of two sequential wave gestures, and a sequence of two sequential push gestures, as described in detail hereinbelow.

Apple has previously been granted patents related to 3D sensing and gesture controls, but the company has yet to release a product that takes advantage of these features. Some of these previous patents included gesture controls on iPads and iPhones as well as on Magic Keyboards, which would allow users to gain access to virtual buttons and potentially streamline certain elements of the user interface.


Of course, with Face ID already launched on iPhone X, it's predicted that Apple will look into adding the facial recognition software into future iMacs and MacBooks before it focuses on a new unlocking system. It's also unclear whether the new 3D user interface patent would include both pieces of technology -- 3D gestures and Face ID -- to further augment a Mac's unlocking process.

As with any patent, the technology in question might not make it to an Apple product in the near future, if at all, but it is an interesting glimpse into what Apple might be planning to do with PrimeSense's technology down the line.

Tag: patent

Top Rated Comments

(View all)
Avatar
33 months ago
I want my unlock to be a middle finger. That'll be fun.

Wait, no... I want the middle finger to force-close everything.
Score: 13 Votes (Like | Disagree)
Avatar
33 months ago
How can this ever be novel or non-obvious? US patent system is soooo broken :-(
Score: 7 Votes (Like | Disagree)
Avatar
33 months ago
I got a hand gesture for this "technology"....
Score: 7 Votes (Like | Disagree)
Avatar
33 months ago
Remind me again, whether or not fictional representations can be used as prior art?

Score: 6 Votes (Like | Disagree)
Avatar
33 months ago
It should recognize American Sign Language, which would give you a gestural keyboard that is already well worked out, and some people already know it.
Score: 6 Votes (Like | Disagree)
Avatar
33 months ago


Because we want a bunch of lasers in our eyes and tracking our every move?

Think what Google analytics and Amazon can do with that...

We see you're reaching for a pen... how about a uni-ball Vision Rollerball Pens, Fine Point (0.7mm), Black, 12 Count?

Would you like some SweetLeaf Natural Stevia Sweetener to go in you tea?

Oops, looks like you spilled your coffee. How about some Bounty Select-a-Size Paper Towels, the quicker picker upper!

We're noticing a few unruly nose-hairs, we've ordered you a Fancii Professional Nose & Ear Hair Trimmer with LED Light, Water Resistant, Stainless Steel Blades, and Battery Power!
Score: 6 Votes (Like | Disagree)

Top Stories

Apple's First MacBook Pro With a Retina Display Will Become 'Obsolete' in 30 Days

Monday June 1, 2020 7:50 am PDT by
If you are still hanging on to a Mid 2012 model of the 15-inch MacBook Pro with a Retina display, and require a new battery or other repairs, be sure to book an appointment with a service provider as soon as possible. In an internal memo today, obtained by MacRumors, Apple has indicated that this particular MacBook Pro model will be marked as "obsolete" worldwide on June 30, 2020, just over...

Tim Cook Addresses George Floyd's Death and Ensuing Protests and Riots as Apple Temporarily Closes Some U.S. Stores

Sunday May 31, 2020 8:04 pm PDT by
Amid unrest in numerous U.S. cities following last week's killing of George Floyd by police in Minneapolis, Apple CEO Tim Cook has shared an internal memo with employees (via Bloomberg) addressing the pain that many are feeling and urging others to commit "to creating a better, more just world for everyone." Cook also announced that Apple is making donations to several groups challenging...

Apple Releases macOS Catalina 10.15.5 Supplemental Update With Security Fix

Monday June 1, 2020 10:56 am PDT by
Apple today released a supplemental update for macOS Catalina 10.15.5, the fifth update to the macOS Catalina operating system that was released in October 2019. The supplemental update comes a week after the release of the macOS Catalina 10.15.5 update. ‌macOS Catalina‌ 10.15.5 is a free update that can be downloaded from the Mac App Store using the Update feature in the System...

iOS 14 Again Said to Be Compatible With All iPhones Able to Run iOS 13

Monday June 1, 2020 2:08 pm PDT by
iOS 14 will be compatible with all iPhones and iPod touch models able to run iOS 13, according to information shared today by Israeli site The Verifier. The compatibility data was allegedly found in a leaked version of iOS 14 and confirmed by what The Verifier says is a "trusted source from the system development process." iOS 13 is compatible with the iPhone 6s and later, with a full...

Apple Releases iOS and iPadOS 13.5.1 With Fixes for Recent 'unc0ver' Jailbreak Vulnerability

Monday June 1, 2020 9:58 am PDT by
Apple today released iOS and iPadOS 13.5.1, minor updates that come a little over a week after the release of iOS and iPadOS 13.5, major updates that brought the Exposure Notification API, FaceTime changes, mask-related unlocking updates and more. The iOS and iPadOS 13.5.1 updates are available on all eligible devices over-the-air in the Settings app. To access the updates, go to Settings >...

Mockups Depict iPhone 12 Lineup Sizes, Relocated SIM Tray

Monday June 1, 2020 1:22 pm PDT by
A series of 3D printed iPhone 12 mockups designed based on leaked schematics and info from Apple's supply chain was today shared by Japanese site Mac Otakara, giving us a first look at what we can expect from the 2020 iPhone lineup. Rumors have suggested Apple will release four iPhone 12 models in 2020: A 5.4-inch iPhone 12, a 6.1-inch iPhone 12, a 6.1-inch iPhone 12 Pro, and a 6.7-inch...

Five Mac Apps Worth Checking Out - June 2020

Tuesday June 2, 2020 2:25 pm PDT by
Apps developed for the Mac often don't receive as much coverage as apps designed for iPhones and iPads, so we have a series at MacRumors that highlights interesting Mac apps that are worth taking a look at. This month's apps are designed to make working from home a little bit easier. Subscribe to the MacRumors YouTube channel for more videos. Meeter (Free) - Working from home often...

Apple Music Joins Music Industry's Blackout Tuesday Awareness Campaign

Tuesday June 2, 2020 1:31 am PDT by
Apple Music has cancelled its Beats 1 radio schedule for Blackout Tuesday and is suggesting that listeners tune in to a radio stream celebrating the best in black music. Blackout Tuesday is a campaign organized by the music industry to support Black Lives Matter after Minneapolis citizen George Floyd was killed by police in the course of his arrest. On launching Apple Music, many users...

6.1-inch 'iPhone 12' Production to Begin in July Ahead of Other 2020 Models

Monday June 1, 2020 2:36 am PDT by
Volume production of Apple's forthcoming 6.1-inch "iPhone 12" models will start in July-August ahead of the rest of the company's flagship iPhone lineup this year, according to a new report by DigiTimes. Apple is widely rumored to be launching four new ‌iPhone‌ models in the usual September or October timeframe, although supply constraints and delays in production ramp-up could cause a...

Apple Releases tvOS 13.4.6 for Fourth and Fifth-Generation Apple TV Models

Monday June 1, 2020 9:56 am PDT by
Apple today released tvOS 13.4.6, a minor update to the tvOS operating system that runs on the fourth and fifth-generation Apple TV models. tvOS 13.4.6 comes a little over a week after the release of tvOS 13.4.5. tvOS 13.4.6, a free update, can be downloaded over the air through the Settings app on the Apple TV by going to System > Software Update. Apple TV owners who have automatic software ...