These Gestures Are How You Control Apple Vision Pro

Apple Vision Pro, Apple's new "spatial computing" device, does not have a hardware-based control mechanism. It relies on eye tracking and hand gestures to allow users to manipulate objects in the virtual space in front of them. In a recent developer session, Apple designers outlined the specific gestures that can be used with Vision Pro, and how some of the interactions will work.

apple vision pro gestures

  • Tap - Tapping the thumb and the index finger together signals to the headset that you want to tap on a virtual element on the display that you're looking at. Users have also described this as a pinch, and it is the equivalent to tapping on the screen of an iPhone.
  • Double Tap - Tapping twice initiates a double tap gesture.
  • Pinch and Hold - A pinch and a hold is similar to a tap and hold gesture, and it does things like highlighting text.
  • Pinch and Drag - Pinching and dragging can be used to scroll and to move windows around. You can scroll horizontally or vertically, and if you move your hand faster, you'll be able to scroll faster.
  • Zoom - Zoom is one of two main two-handed gestures. You can pinch your fingers together and pull your hands apart to zoom in, and presumably zooming out will have a pushing sort of motion. Window sizes can also be adjusted by dragging at the corners.
  • Rotate - Rotate is the other two-handed gesture and based on Apple's chart, it will involve pinching the fingers together and rotating the hands to manipulate virtual objects.

Gestures will work in tandem with eye movements, and the many cameras in the Vision Pro will track where you are looking with great accuracy. Eye position will be a key factor in targeting what you want to interact with using hand gestures. As an example, looking at an app icon or on-screen element targets it and highlights it, and then you can follow up with a gesture.

Hand gestures do not need to be grand, and you can keep your hands in your lap. Apple is encouraging that, in fact, because it will keep your hands and arms from getting tired from being held in the air. You only need a tiny pinch gesture for the equivalent of a tap, because the cameras can track precise movements.

@macrumors Eye tracking and pinching is the interface on the new #AppleVisionPro and we could to try jt out! #Apple #VR #AR ♬ original sound - MacRumors

What you're looking at will let you select and manipulate objects that are both close to you and far from you, and Apple does anticipate scenarios where you might want to use larger gestures to control objects that are right in front of you. You can reach out and use your fingertips to interact with an object. For example, if you have a Safari window right in front of you, you can reach your hand out and scroll from there rather than using your fingers in your lap.

In addition to gestures, the headset will support hand movements such as air typing, though it doesn't seem like those who have received a demo have been able to try this feature as of yet. Gestures will work together, of course, and to do something like create a drawing, you'll look at a spot on the canvas, select a brush with your hand, and use a gesture in the air to draw. If you look elsewhere, you'll be able to move the cursor immediately to where you're looking.

While these are the six main system gestures that Apple has described, developers can create custom gestures for their apps that will perform other actions. Developers will need to make sure custom gestures are distinct from the system gestures or common hand movements that people might use, and that the gestures can be repeated frequently without hand strain.

To supplement hand and eye gestures, Bluetooth keyboards, trackpads, mice, and game controllers can be connected to the headset, and there are also voice-based search and dictation tools.

Multiple people who have been able to try the Vision Pro have had the same word to describe the control system - intuitive. Apple's designers seem to have created it to work similarly to multitouch gestures on the ‌iPhone‌ and the iPad, and so far, reactions have been positive.

MacRumors videographer Dan Barbera was able to try out the headset and he was impressed with the controls. You can see his full overview of his experience on our YouTube channel.

Related Roundup: Apple Vision Pro
Related Forum: Apple Vision Pro

Top Rated Comments

MacWiz_007 Avatar
16 weeks ago
They forgot the "flipping the bird" gesture when your battery dies in less than two hours. ?
Score: 21 Votes (Like | Disagree)
Piggie Avatar
16 weeks ago
This will be the hand gesture after you ask Siri to do something simple for the 8th time and you are hoping THIS TIME is actually gets it right ;)

Score: 15 Votes (Like | Disagree)
vegetassj4 Avatar
16 weeks ago
They forgot Force Quit and responding to Siri when she asks for clarification:



Attachment Image

Attachment Image
Score: 11 Votes (Like | Disagree)
TheYayAreaLiving ?️ Avatar
16 weeks ago
We need a hand clap ?? gesture.
Score: 9 Votes (Like | Disagree)
Love-hate ? relationship Avatar
16 weeks ago
that's seriously amazing. Im wayyyy to broke to afford it, and in all fairness it's not something meant for everyone i feel , but in spite of what reports claimed (that it wasnt ready) , the software appears to work really well and super intuintively (gosh i cannot write )
Score: 7 Votes (Like | Disagree)
MacWiz_007 Avatar
16 weeks ago

What about pinch to zoom?
Did you not see the one where it looks like you are pinching someone's nipples to zoom?
Score: 7 Votes (Like | Disagree)

Popular Stories

iphone se 4 modified flag edges

iPhone SE 4 Details: Action Button, USB-C Port, Face ID, and More

Wednesday September 27, 2023 1:34 pm PDT by
Significant changes are expected to arrive with Apple's fourth-generation iPhone SE, in terms of both design and hardware, MacRumors has learned. The iPhone SE 4, known internally under the codename Ghost, is expected to receive a new design derived almost entirely from the base model iPhone 14. According to our sources, the iPhone SE 4 will use a modified version of the iPhone 14 chassis...
iOS 17

Apple Releases iOS 17.0.2 and iPadOS 17.0.2 for All iPhones and iPads

Tuesday September 26, 2023 12:47 pm PDT by
Apple today released iOS 17.0.2 and iPadOS 17.0.2 updates, with the software coming five days after the releases of iOS 17.0.1 and iPadOS 17.0.1. Today's iOS 17.0.2 and iPadOS 17.0.2 updates arrive as build 21A351 and can be downloaded on eligible iPhones and iPads over-the-air by going to Settings > General > Software Update. Note that iOS 17.0.2 was previously made available for iPhone...
iPhone 15 Pro Lineup Feature

Kuo: iPhone 15 Pro Overheating Issues Likely Due to Thermal Compromises, Not 3nm Node

Tuesday September 26, 2023 9:12 am PDT by
Complaints about heat issues with the iPhone 15 Pro models are not related to TSMC's 3-nanometer node that was used for the A17 Pro chip, according to well-respected Apple analyst Ming-Chi Kuo. Kuo says that overheating could be caused by "compromises made in the thermal system design" that allowed Apple to cut down on the weight of the iPhone 15 Pro models. Kuo says that the reduced heat...
iOS 17

Everything New in iOS 17.1 Beta 1

Wednesday September 27, 2023 1:57 pm PDT by
Just a week after releasing iOS 17, Apple has seeded the first beta of iOS 17.1 to developers. iOS 17.1 adds some features that Apple promised were coming to iOS 17 in the future, plus it refines and improves some existing features. This guide covers everything new in the first iOS 17.1 beta. Apple Music Favorites You can favorite songs, albums, playlists, and artists in the iOS 17.1...
macos sonoma 4

Apple Releases macOS Sonoma With New Widget Features, Safari Updates, Screen Sharing Improvements and More

Tuesday September 26, 2023 10:01 am PDT by
Apple today released macOS 14 Sonoma, the newest version of the operating system that runs on the Mac. macOS Sonoma has been in beta testing for several months, and it is compatible with the 2019 and later iMac, the iMac Pro, the 2018 and later Mac mini, the 2018 and later MacBook Pro, the 2019 and later Mac Pro, and the Mac Studio. The ‌macOS Sonoma update can be downloaded for free on...