These Gestures Are How You Control Apple Vision Pro

Apple Vision Pro, Apple's new "spatial computing" device, does not have a hardware-based control mechanism. It relies on eye tracking and hand gestures to allow users to manipulate objects in the virtual space in front of them. In a recent developer session, Apple designers outlined the specific gestures that can be used with Vision Pro, and how some of the interactions will work.

apple vision pro gestures

  • Tap - Tapping the thumb and the index finger together signals to the headset that you want to tap on a virtual element on the display that you're looking at. Users have also described this as a pinch, and it is the equivalent to tapping on the screen of an iPhone.
  • Double Tap - Tapping twice initiates a double tap gesture.
  • Pinch and Hold - A pinch and a hold is similar to a tap and hold gesture, and it does things like highlighting text.
  • Pinch and Drag - Pinching and dragging can be used to scroll and to move windows around. You can scroll horizontally or vertically, and if you move your hand faster, you'll be able to scroll faster.
  • Zoom - Zoom is one of two main two-handed gestures. You can pinch your fingers together and pull your hands apart to zoom in, and presumably zooming out will have a pushing sort of motion. Window sizes can also be adjusted by dragging at the corners.
  • Rotate - Rotate is the other two-handed gesture and based on Apple's chart, it will involve pinching the fingers together and rotating the hands to manipulate virtual objects.

Gestures will work in tandem with eye movements, and the many cameras in the Vision Pro will track where you are looking with great accuracy. Eye position will be a key factor in targeting what you want to interact with using hand gestures. As an example, looking at an app icon or on-screen element targets it and highlights it, and then you can follow up with a gesture.

Hand gestures do not need to be grand, and you can keep your hands in your lap. Apple is encouraging that, in fact, because it will keep your hands and arms from getting tired from being held in the air. You only need a tiny pinch gesture for the equivalent of a tap, because the cameras can track precise movements.

@macrumors Eye tracking and pinching is the interface on the new #AppleVisionPro and we could to try jt out! #Apple #VR #AR ♬ original sound - MacRumors

What you're looking at will let you select and manipulate objects that are both close to you and far from you, and Apple does anticipate scenarios where you might want to use larger gestures to control objects that are right in front of you. You can reach out and use your fingertips to interact with an object. For example, if you have a Safari window right in front of you, you can reach your hand out and scroll from there rather than using your fingers in your lap.

In addition to gestures, the headset will support hand movements such as air typing, though it doesn't seem like those who have received a demo have been able to try this feature as of yet. Gestures will work together, of course, and to do something like create a drawing, you'll look at a spot on the canvas, select a brush with your hand, and use a gesture in the air to draw. If you look elsewhere, you'll be able to move the cursor immediately to where you're looking.

While these are the six main system gestures that Apple has described, developers can create custom gestures for their apps that will perform other actions. Developers will need to make sure custom gestures are distinct from the system gestures or common hand movements that people might use, and that the gestures can be repeated frequently without hand strain.

To supplement hand and eye gestures, Bluetooth keyboards, trackpads, mice, and game controllers can be connected to the headset, and there are also voice-based search and dictation tools.

Multiple people who have been able to try the Vision Pro have had the same word to describe the control system - intuitive. Apple's designers seem to have created it to work similarly to multitouch gestures on the ‌iPhone‌ and the iPad, and so far, reactions have been positive.

MacRumors videographer Dan Barbera was able to try out the headset and he was impressed with the controls. You can see his full overview of his experience on our YouTube channel.

Related Roundup: Apple Vision Pro
Buyer's Guide: Vision Pro (Buy Now)
Related Forum: Apple Vision Pro

Popular Stories

AirPods Pro Firmware Feature

Apple Releases Firmware Updates for AirPods Pro 2 and AirPods 4

Monday November 11, 2024 11:28 am PST by
Apple today released firmware updates for both AirPods 4 models (version number 7B20) and the AirPods Pro 2 with both Lightning and USB-C charging cases (version number 7B21). All of these AirPods models were previously on firmware version 7B19. It is not immediately clear what new features or changes are included in firmware versions 7B20 and 7B21, but we will update this story if we find...
New Things Your iPhone Can Do in iOS 18

18 New Things Your iPhone Can Do in iOS 18.2

Wednesday November 13, 2024 2:09 am PST by
Apple is set to release iOS 18.2 next month, bringing the second round of Apple Intelligence features to iPhone 15 Pro and iPhone 16 models. This update brings several major advancements to Apple's AI integration, including completely new image generation tools and a range of Visual Intelligence-based enhancements. There are a handful of new non-AI related feature controls incoming as well....
iPhone SE 4 Thumb 1

iPhone SE 4 Camera Modules to Enter Mass Production Next Month

Tuesday November 12, 2024 2:56 am PST by
Apple's camera module supplier for the upcoming iPhone SE 4 is set to begin mass production of the components in December, according to a new report coming out of Korea. Economic newspaper Ajunews reports that LG Innotek will supply the front camera module for the budget-friendly fourth-generation device. Final tests are now said to be underway, with mass production of the module following...
iphone 6 thickness

iPhone 17 'Air' May Not Be Much Thinner Than iPhone 6

Monday November 11, 2024 5:18 am PST by
Next year's iPhone 17 "Air" model may not be as thin as Apple planned, according to a rumor originating in Korea. According to the news aggregator account "yeux1122" on Naver, citing industry sources, Apple has run into problems making the new iPhone 17 model sufficiently thin. The device's reduced thickness is apparently dependent on manufacturing a battery with a thinner substrate, but...
new mac holiday

The Best Early Black Friday Mac Deals

Monday November 11, 2024 7:45 am PST by
Black Friday is getting closer, and prices on MacBook Pro, MacBook Air, iMac, and Mac mini computers have started to drop as the shopping holiday nears. These deals include the latest models of the M4 MacBook Pro and iMac. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a small payment, which helps us keep the site...
iOS 18

Here's Everything New in iOS 18.2 Beta 3

Monday November 11, 2024 4:53 pm PST by
Apple seeded the third betas of iOS 18.2 and iPadOS 18.2 to developers for testing today. While the third betas of each update are minor relative to the first two betas, there are still a handful of changes across the Photos app, TV app, and more. A corresponding iOS 18.2 public beta with these changes will likely be released later this week, and Apple previously confirmed that the software...
General Final Cut Pro Feature

Apple Likely to Announce Final Cut Pro Update This Week With These New Features

Sunday November 10, 2024 12:13 pm PST by
In its announcement video for the new Mac mini last month, Apple teased an "upcoming" version of Final Cut Pro for the Mac. Apple will likely announce the update during the annual Final Cut Pro Creative Summit, which begins this Wednesday. The conference is held in association with Apple, and attendees will be visiting Apple Park on the first day. Apple already teased four new features...
iPad iOS 16 WP Display Feature eric edit

Apple to Launch AI-Powered Home 'Command Center' as Soon as March 2025

Tuesday November 12, 2024 1:09 pm PST by
Apple is planning to launch an AI-powered smart home display as soon as March 2025, according to Bloomberg's Mark Gurman. The display will measure in at approximately six inches, and while it is similar to an iPad, it is square rather than rectangular and it has thick bezels around the edges. There is a camera at the top front so that it can be used for FaceTime, plus there are internal speakers...

Top Rated Comments

MacWiz_007 Avatar
19 months ago
They forgot the "flipping the bird" gesture when your battery dies in less than two hours. ?
Score: 21 Votes (Like | Disagree)
Piggie Avatar
19 months ago
This will be the hand gesture after you ask Siri to do something simple for the 8th time and you are hoping THIS TIME is actually gets it right ;)

Score: 15 Votes (Like | Disagree)
vegetassj4 Avatar
19 months ago
They forgot Force Quit and responding to Siri when she asks for clarification:



Attachment Image

Attachment Image
Score: 11 Votes (Like | Disagree)
TheYayAreaLiving ?️ Avatar
19 months ago
We need a hand clap ?? gesture.
Score: 9 Votes (Like | Disagree)
Love-hate ? relationship Avatar
19 months ago
that's seriously amazing. Im wayyyy to broke to afford it, and in all fairness it's not something meant for everyone i feel , but in spite of what reports claimed (that it wasnt ready) , the software appears to work really well and super intuintively (gosh i cannot write )
Score: 7 Votes (Like | Disagree)
MacWiz_007 Avatar
19 months ago

What about pinch to zoom?
Did you not see the one where it looks like you are pinching someone's nipples to zoom?
Score: 7 Votes (Like | Disagree)