Apple to Integrate Augmented Reality Functionality Into iPhone's Camera App

Apple has a team of people working on integrating augmented reality functionality into the iPhone's camera app, reports Business Insider. Employees from several different augmented reality companies Apple has acquired are said to be involved in the effort, including those from Metaio and Flyby Media.

Citing sources familiar with Apple's plans, Business Insider says Apple's goal is for consumers to be able point the phone at a real-world object and have that object be recognized. Apple is already working on similar technology using machine learning techniques and has built basic object recognition capabilities into the Photos app.


Object recognition in the camera app would potentially allow for the real-time recognition of objects, but it is more complex than what is available in the Photos app because it involves 3D objects. Such a feature could be used for spacial recognition and mapping, among other things.

Facial recognition and manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people. Apple is said to be planning to release an augmented reality SDK for developers following the debut of the rumored camera feature, which would allow third-party apps access the AR camera technology too.

Another early feature for Apple's AR integration into the camera app could be to recognize and manipulate people's faces. Apple integrated facial recognition technology into the photos app in the most recent version of its iOS software, and purchased FaceShift, a company with similar technology in 2015.

Recent rumors have suggested Apple's ultimate augmented reality ambition may be a set of smart glasses, which would connect wirelessly to the iPhone and display "images and other information" to the wearer.

A recent hire, John Border, seems to confirm Apple's rumored interest in smart glasses. Border, who describes himself as a "subject matter expert" in head-worn displays, plastic optics manufacturing, and camera systems and image sensors, recently joined Apple after serving as the chief engineer at Osterhout Design Group, a company that developed a set of smart glasses.

Apple's work on smart glasses is said to be in the early stages, and no product is expected to debut before 2018 should the project continue. It is not clear when Apple might introduce AR functionality in the camera app.

Related Roundup: Apple Vision Pro
Buyer's Guide: Vision Pro (Buy Now)
Related Forum: Apple Vision Pro

Top Rated Comments

Zirel Avatar
95 months ago


VR. It's the future! Guys!
Score: 17 Votes (Like | Disagree)
AngerDanger Avatar
95 months ago
I propose Apple create a feature whereby you aim your camera at a new MacBook and nearby device, and the app overlays all the dongles and adaptors you'd need to connect the two devices. You could even buy them right from the camera app!
Score: 13 Votes (Like | Disagree)
ErikGrim Avatar
95 months ago
While object recognition would be of limited use to most users, imagine what it could do for the vision impaired? A set of smart glasses that translated what they were seeing into sound with Voice Over would be a game changer. Kind of like how VO can identify faces in the camera app already, but for a whole library of every day objects. The blind would literally be able to see and navigate the world by sound.
Score: 10 Votes (Like | Disagree)
FatherJack1980 Avatar
95 months ago
"[facial] manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people."

Oh goody, yet another social media thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event.
Score: 7 Votes (Like | Disagree)
62tele Avatar
95 months ago
The skeptic in me says that this is just the next piece of feces that Apple is throwing against the wall to see if it sticks. The Watch was not the next iPod. The car went down a dead end.
I hope this doesn't distract them from the next revolutionary breakthrough in emoji development.
Score: 5 Votes (Like | Disagree)
jclo Avatar
95 months ago
"[facial] manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people."

Oh goody, yet another Snapchat thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event :oops:
Seems like it would be a fun addition to FaceTime.
Score: 3 Votes (Like | Disagree)

Popular Stories

iOS 17

iOS 17.4 Will Add These New Features to Your iPhone

Monday February 19, 2024 6:52 am PST by
Apple last month confirmed that iOS 17.4 will be released in March, and the update includes several new features and changes for the iPhone. Key new features in iOS 17.4 include major App Store changes in the EU and Apple Podcasts transcripts. The update also adds new emoji and includes preparations for the launch of next-generation CarPlay later this year. More details about the new...
apple wallet drivers license feature

Apple Said iPhone Driver's Licenses Will Expand to These 8 U.S. States

Monday February 12, 2024 7:51 am PST by
In 2021, Apple introduced a feature that lets residents of participating U.S. states add their driver's license or ID to the Wallet app on the iPhone and Apple Watch, providing a convenient and contactless way to display proof of identity or age. Unfortunately, states have been slow to adopt the feature since it was first announced in September 2021, with IDs in the Wallet app only available ...
samsung galaxy ring

'Apple Ring' Allegedly in Development to Rival Samsung Galaxy Ring

Tuesday February 20, 2024 2:27 am PST by
Apple is speeding up development of a smart ring that can be worn on the finger to track users' health biometrics, claims a new report coming out of Korea. Teaser image of Samsung Galaxy Ring shown at Galaxy Unpacked in January Apple has toyed with the idea of a ring wearable for several years, as indicated by several patents, but with Samsung preparing to bring its own product to market, the ...
New Macs iPads iOS 17 4

New Macs and iPads Coming Soon: What to Expect

Saturday February 17, 2024 3:00 am PST by
Following the launch of Apple's Vision Pro headset, attention has shifted towards what is next on the company's agenda this year. As usual, March should be eventful for Apple, as it is expected to release new MacBook Air, iPad Pro, and iPad Air models during the month. In a recent press release, Apple also confirmed that iOS 17.4 will be released in March. Below, we recap what to expect...
m3 macbook pro 14 16

Apple Now Selling Refurbished M3 Pro and M3 Max MacBook Pro Models

Monday February 19, 2024 5:04 pm PST by
Apple today began offering refurbished versions of the 14-inch and 16-inch MacBook Pro models with M3 Pro and M3 Max chip options, offering the machines at a discount for the first time since their October 2023 release. The release of M3 Pro and M3 Max models on Apple's refurbished store comes almost two weeks after the entry-level 14-inch M3 MacBook Pro first appeared on the store....
Apple Watch Series 9

5 Major New Features Rumored for This Year's Apple Watch

Sunday February 18, 2024 3:00 am PST by
Thinking about upgrading your Apple Watch or buying one for the first time? If your current smartwatch is doing its job just fine and it's only the idea of a good deal that's piqued your interest, it could be worth holding out until later this year when Apple unveils its next-generation version. Here's why. There are already several rumors from reputable sources that suggest 2024 could be a...