xrOS: What to Expect From the Software Designed for Apple's AR/VR Headset

The Worldwide Developers Conference (WWDC), Apple's annual developer and software-oriented conference, is now just a few days away. As with previous years, Apple is expected to introduce major updates to all of its operating systems, but unique to WWDC 2023, Apple is also very likely to unveil xrOS, the software that will run its much-rumored AR/VR mixed-reality headset. Here's everything we're expecting from it.

xrOS: What's in a Name?

Apple has not publicly acknowledged the existence of its headset, but internal Apple sources have confirmed that the operating system will be called xrOS, and Apple has used hidden shell companies to trademark the name in various countries.

‌xrOS‌ is meant to stand for "extended reality," hinting at the augmented and virtual reality functions at the heart of the head-mounted device. The backend capitalization also conforms to the naming conventions Apple has adopted for all of the operating systems in its ecosystem.

According to reporting by Bloomberg's Mark Gurman, Apple sees the headset as a device for gaming, watching streaming video, video conferencing, health and fitness, and interfacing with other people. Therefore ‌xrOS‌ will be focused on providing the best possible environment for delivering that content.

xrOS User Interface

‌xrOS‌ is believed to have an iOS-like interface that will be familiar to iPhone and iPad users, but presented to them and interacted with in an entirely new way.

We are expecting a dedicated Home Screen containing apps and customizable widgets, all of which can be rearranged, and all viewed on dual high-resolution 4K micro OLED displays with up to 3,000 pixels per inch for a rich, realistic, and immersive viewing experience.

Both of the user's hands and eyes will be tracked by more than a dozen optical cameras, and the wearer is expected to be able to look at an on-screen item to select it, and use a hand gesture like a pinch to activate the item on the screen.

The ‌xrOS‌ software could even potentially project AR app interface elements onto actual objects in the real world for a mixed-reality overlay kind of effect.

Apple is also said to be working on an in-air typing feature that is expected to be a bit rudimentary at launch, with Apple to make notable improvements over time.

Apps

At the core of ‌xrOS‌ will be re-imagined versions of stock Apple apps like Safari, Photos, Messages, Maps, Apple TV+, Apple Music, Podcasts, Reminders, and Calendar, as well as a FaceTime app customized for the headset. The Maps app will also feature, and could incorporate some of Apple's Look Around and landmark functionality, allowing users to explore cities around the world.

Users will be able to work on several apps at the same time, while an ‌xrOS‌ App Store will host made-for-headset apps, as well as existing ‌App Store‌ content. Apple is also said to be adapting ‌iPad‌ apps for the headset's 3D interface.

In addition, Apple is set on transforming several existing services and apps into immersive viewing experiences. For example, users will be able to watch videos in virtual reality environments as if they were looking at a giant screen, or participate in guided meditations accompanied by immersive visuals, audio, and voiceovers.

Some of the apps and services said to offer radically different experiences in ‌xrOS‌ include Apple Fitness+, ‌Apple TV‌+ (with a focus on viewing sports content like MLB and MLS), and a 3D version of Apple's collaborative Freeform tool. As for ‌FaceTime‌, with all of the cameras that the headset is expected to have, one-on-one chats are said to feature realistic avatars which mimic the user's actual facial expressions and body movements.

Apple is also said to have created a Books app for reading in virtual reality, a Health app focusing on psychological wellness, and a Camera app that can take images from the headset's cameras.

Gaming and App Creation

To get people to purchase the mixed-reality headset, Apple will need to convince users that the software experiences are worth it. That will require not only compelling first-party experiences, but also innovative third-party apps from developers.

With gaming reportedly a focus, Apple is said to have been working with a select number of game developers to help them update their existing content for mixed reality, but Apple reportedly also has a robust set of tools available that will allow non-developers to create their own AR/VR experiences. According to one report, customers will be able to create and release AR apps for the headset using Siri as an assistant, even if they do not have the ability to code.

The report likens Apple's tool to Minecraft and Roblox, which allow anyone to easily create 3D tools and worlds. Apple is said to be utilizing technology that it acquired from Montreal-based startup Fabric Software in 2017, and customers would be able to distribute the AR apps they create on the ‌App Store‌ alongside developers.

It's worth noting that the report cites sources who last saw demonstrations of the tool in 2021, so the report is based on older information and it's possible that Apple's plans have changed at least slightly in the time that has passed since. Whether this latter report is still accurate is something we should learn soon.

When is xrOS Coming?

Apple's software tools for the headset are likely to be revealed at WWDC 2023, giving developers time to create AR and VR apps and experiences for the headset before it launches. In that sense, we won't see the full version of ‌xrOS‌ until the headset is officially released to the public, and that isn't expected to happen until later in the year.

For everything else we know about the headset, including all the hardware details, be sure to check our dedicated AR/VR headset roundup.