Apple to Integrate Augmented Reality Functionality Into iPhone's Camera App

Apple has a team of people working on integrating augmented reality functionality into the iPhone's camera app, reports Business Insider. Employees from several different augmented reality companies Apple has acquired are said to be involved in the effort, including those from Metaio and Flyby Media.

Citing sources familiar with Apple's plans, Business Insider says Apple's goal is for consumers to be able point the phone at a real-world object and have that object be recognized. Apple is already working on similar technology using machine learning techniques and has built basic object recognition capabilities into the Photos app.


Object recognition in the camera app would potentially allow for the real-time recognition of objects, but it is more complex than what is available in the Photos app because it involves 3D objects. Such a feature could be used for spacial recognition and mapping, among other things.

Facial recognition and manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people. Apple is said to be planning to release an augmented reality SDK for developers following the debut of the rumored camera feature, which would allow third-party apps access the AR camera technology too.

Another early feature for Apple's AR integration into the camera app could be to recognize and manipulate people's faces. Apple integrated facial recognition technology into the photos app in the most recent version of its iOS software, and purchased FaceShift, a company with similar technology in 2015.

Recent rumors have suggested Apple's ultimate augmented reality ambition may be a set of smart glasses, which would connect wirelessly to the iPhone and display "images and other information" to the wearer.

A recent hire, John Border, seems to confirm Apple's rumored interest in smart glasses. Border, who describes himself as a "subject matter expert" in head-worn displays, plastic optics manufacturing, and camera systems and image sensors, recently joined Apple after serving as the chief engineer at Osterhout Design Group, a company that developed a set of smart glasses.

Apple's work on smart glasses is said to be in the early stages, and no product is expected to debut before 2018 should the project continue. It is not clear when Apple might introduce AR functionality in the camera app.

Related Roundup: Apple Vision Pro
Buyer's Guide: Vision Pro (Buy Now)
Related Forum: Apple Vision Pro

Top Rated Comments

Zirel Avatar
96 months ago


VR. It's the future! Guys!
Score: 17 Votes (Like | Disagree)
AngerDanger Avatar
96 months ago
I propose Apple create a feature whereby you aim your camera at a new MacBook and nearby device, and the app overlays all the dongles and adaptors you'd need to connect the two devices. You could even buy them right from the camera app!
Score: 13 Votes (Like | Disagree)
ErikGrim Avatar
96 months ago
While object recognition would be of limited use to most users, imagine what it could do for the vision impaired? A set of smart glasses that translated what they were seeing into sound with Voice Over would be a game changer. Kind of like how VO can identify faces in the camera app already, but for a whole library of every day objects. The blind would literally be able to see and navigate the world by sound.
Score: 10 Votes (Like | Disagree)
FatherJack1980 Avatar
96 months ago
"[facial] manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people."

Oh goody, yet another social media thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event.
Score: 7 Votes (Like | Disagree)
62tele Avatar
96 months ago
The skeptic in me says that this is just the next piece of feces that Apple is throwing against the wall to see if it sticks. The Watch was not the next iPod. The car went down a dead end.
I hope this doesn't distract them from the next revolutionary breakthrough in emoji development.
Score: 5 Votes (Like | Disagree)
jclo Avatar
96 months ago
"[facial] manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people."

Oh goody, yet another Snapchat thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event :oops:
Seems like it would be a fun addition to FaceTime.
Score: 3 Votes (Like | Disagree)

Popular Stories

iPhone 16 Camera Lozenge 2 Perspective Gray

Five Key Upgrades Coming to iPhone 16

Friday March 15, 2024 1:45 pm PDT by
The iPhone is Apple's top-selling product, and it gets an update every year. In 2024, we're expecting the iPhone 16 and iPhone 16 Pro lineup, with an arguably more interesting feature set than we got with the iPhone 15 and iPhone 15 Pro. Subscribe to the MacRumors YouTube channel for more videos. Capture Button All four iPhone 16 models are set to get a whole new button, which will be...
When To Expect New iPads Feature 1

Apple to Announce New iPads on March 26, Rumors Claim

Monday March 18, 2024 4:02 am PDT by
Apple is widely expected to release new iPad Air and OLED iPad Pro models in the next few weeks. According to new rumors coming out of Asia, the company will announce its new iPads on Tuesday, March 26. Chinese leaker Instant Digital on Weibo this morning 日发布%23">claimed that the date will see some sort of announcement from Apple related to new iPads, but stopped short of calling it an...
airpods 3 orange

Two New AirPods 4 Models Expected to Launch in September or October

Sunday March 17, 2024 7:56 am PDT by
Apple suppliers will begin production of two new fourth-generation AirPods models in May, according to Bloomberg's Mark Gurman. Based on this production timeframe, he expects the headphones to be released in September or October. Gurman expects both fourth-generation AirPods models to feature a new design with better fit, improved sound quality, and an updated charging case with a USB-C...
iphone se 4 modified flag edges

iPhone SE 4 Expected to Depreciate Heavily

Tuesday March 12, 2024 9:04 am PDT by
Resale value trends suggest the iPhone SE 4 may not hold its value as well as Apple's flagship models, according to SellCell. According to the report, Apple's iPhone SE models have historically depreciated much more rapidly than the company's more premium offerings. The third-generation iPhone SE, which launched in March 2022, experienced a significant drop in resale value, losing 42.6%...
M3 iPad Feature 3

New iPads Likely to Begin Shipping in April

Monday March 18, 2024 9:52 am PDT by
Apple's new iPad Pro models with OLED displays will likely begin shipping to customers in April, according to information shared today by Ross Young, CEO of display industry research firm Display Supply Chain Consultants. Bloomberg's Mark Gurman also said the new iPad Pro models might not ship until "deeper" into April in his Power On newsletter on Sunday:I've repeatedly said that new...
General iOS 17 Feature Orange Purple

iOS 17.4.1 Update for iPhone is Imminent

Monday March 18, 2024 5:27 am PDT by
iOS 17.4.1 and iPadOS 17.4.1 should be released within the next few days, with a build number of 21E235, according to a source with a proven track record. MacRumors previously reported that Apple was internally testing iOS 17.4.1. As a minor update for the iPhone, it will likely address software bugs and/or security vulnerabilities. It is unclear if the update will include any other changes. ...
Reasons to Not Upgrade to macOS Sonoma 14

macOS Sonoma 14.4: Reasons Not to Update

Monday March 18, 2024 9:36 am PDT by
Since Apple unveiled macOS Sonoma 14.4 on March 7, the transition to the latest software update has not been entirely smooth for everyone, and a number of issues have been reported by users that significantly impact their daily workflow. This article lists the most prominent challenges users have faced since updating to macOS Sonoma 14.4, and offers potential solutions where available. USB...