Apple to Integrate Augmented Reality Functionality Into iPhone's Camera App

Apple has a team of people working on integrating augmented reality functionality into the iPhone's camera app, reports Business Insider. Employees from several different augmented reality companies Apple has acquired are said to be involved in the effort, including those from Metaio and Flyby Media.

Citing sources familiar with Apple's plans, Business Insider says Apple's goal is for consumers to be able point the phone at a real-world object and have that object be recognized. Apple is already working on similar technology using machine learning techniques and has built basic object recognition capabilities into the Photos app.


Object recognition in the camera app would potentially allow for the real-time recognition of objects, but it is more complex than what is available in the Photos app because it involves 3D objects. Such a feature could be used for spacial recognition and mapping, among other things.

Facial recognition and manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people. Apple is said to be planning to release an augmented reality SDK for developers following the debut of the rumored camera feature, which would allow third-party apps access the AR camera technology too.

Another early feature for Apple's AR integration into the camera app could be to recognize and manipulate people's faces. Apple integrated facial recognition technology into the photos app in the most recent version of its iOS software, and purchased FaceShift, a company with similar technology in 2015.

Recent rumors have suggested Apple's ultimate augmented reality ambition may be a set of smart glasses, which would connect wirelessly to the iPhone and display "images and other information" to the wearer.

A recent hire, John Border, seems to confirm Apple's rumored interest in smart glasses. Border, who describes himself as a "subject matter expert" in head-worn displays, plastic optics manufacturing, and camera systems and image sensors, recently joined Apple after serving as the chief engineer at Osterhout Design Group, a company that developed a set of smart glasses.

Apple's work on smart glasses is said to be in the early stages, and no product is expected to debut before 2018 should the project continue. It is not clear when Apple might introduce AR functionality in the camera app.

Related Roundup: Apple Glasses

Top Rated Comments

Zirel Avatar
66 months ago


VR. It's the future! Guys!
Score: 17 Votes (Like | Disagree)
AngerDanger Avatar
66 months ago
I propose Apple create a feature whereby you aim your camera at a new MacBook and nearby device, and the app overlays all the dongles and adaptors you'd need to connect the two devices. You could even buy them right from the camera app!
Score: 13 Votes (Like | Disagree)
ErikGrim Avatar
66 months ago
While object recognition would be of limited use to most users, imagine what it could do for the vision impaired? A set of smart glasses that translated what they were seeing into sound with Voice Over would be a game changer. Kind of like how VO can identify faces in the camera app already, but for a whole library of every day objects. The blind would literally be able to see and navigate the world by sound.
Score: 10 Votes (Like | Disagree)
FatherJack1980 Avatar
66 months ago
"[facial] manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people."

Oh goody, yet another social media thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event.
Score: 7 Votes (Like | Disagree)
62tele Avatar
66 months ago
The skeptic in me says that this is just the next piece of feces that Apple is throwing against the wall to see if it sticks. The Watch was not the next iPod. The car went down a dead end.
I hope this doesn't distract them from the next revolutionary breakthrough in emoji development.
Score: 5 Votes (Like | Disagree)
jclo Avatar
66 months ago
"[facial] manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people."

Oh goody, yet another Snapchat thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event :oops:
Seems like it would be a fun addition to FaceTime.
Score: 3 Votes (Like | Disagree)

Related Stories

General black friday 20 sale feature 2

Best Black Friday Deals on Apple Watch, AirPods Pro, MacBook Pro, More

Wednesday November 24, 2021 8:06 am PST by
Black Friday 2021 has kicked off, and you can now get some of the year's best deals on numerous Apple products. In this article we're providing a quick summary of all the best sales we've seen so far this season. For more on the best sales happening this week, visit our Black Friday Roundup. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a...
iphone holiday

Best Black Friday iPhone Deals Available Today

Friday November 26, 2021 4:58 am PST by
Cellular carriers have always offered big savings on the newest iPhone models in holidays past, and Black Friday 2021 is no different. Right now we're tracking notable offers on the iPhone 13 and iPhone 13 Pro devices from AT&T, Verizon, and T-Mobile. For even more savings, keep an eye on older models like iPhone SE. Note: MacRumors is an affiliate partner with some of these vendors. When you...
airpods pro blue holiday 3

Black Friday: AirPods Pro With MagSafe Drop to $159 on Amazon

Tuesday November 23, 2021 9:12 am PST by
Amazon is now matching Walmart's price on the AirPods Pro with MagSafe, available for $159.00, down from $249.00. These are shipped and sold directly from Amazon, and although stock is low you can still get them in time for the holidays (mid December as of writing). Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a...
apple wallet drivers license feature

Apple Delays iOS 15 Feature for Adding Your Driver's License to Your iPhone Until Early 2022

Tuesday November 23, 2021 9:35 am PST by
Apple recently updated its website to indicate that an upcoming iOS 15 and watchOS 8 feature that will let you add your driver's license or state ID to your iPhone and Apple Watch in participating U.S. states has been delayed until early 2022. Apple previously said the feature would launch in late 2021. In September, Apple said Arizona and Georgia would be among the first states to introduce ...
airpods 3 blue holiday 2

Black Friday: AirPods 3 Hit Record Low Price of $149.99 ($29 Off)

Thursday November 25, 2021 6:10 am PST by
Amazon has introduced a new record low price on the AirPods 3, available for $149.99, down from $179.00. At $29 off, this sale price beats the one that we were tracking earlier this week by about $5 and it's only available on Amazon as of writing. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a small payment, which ...
airpods pro pink holiday

Black Friday: AirPods Pro With MagSafe Hit Lowest Ever Price

Tuesday November 23, 2021 7:47 am PST by
Apple's AirPods Pro with the new MagSafe Charging Case is now available for its lowest ever price thanks to Black Friday sales. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a small payment, which helps us keep the site running. The limited-time deal is available at both Amazon and Walmart, which are both...
apple watch cellular holiday

Best Black Friday Apple Watch Deals Available Today

Friday November 26, 2021 4:55 am PST by
The Apple Watch always makes a great gift around the holiday season, and for Black Friday 2021 we're tracking a few solid offers on numerous models of the Apple Watch. In this article, you'll find the best Black Friday sales on the new Apple Watch 7, but the best money-saving discounts will be found on older models like the Apple Watch Series 3 and SE. Note: MacRumors is an affiliate partner...
iPads black friday 20 sale feature

Best Black Friday iPad Deals Available Today

Friday November 26, 2021 4:48 am PST by
Although Black Friday sales began as early as October in 2021, the shopping holiday is now officially underway and we're highlighting the best sales for each of Apple's product lines. In this article, you'll find the best Black Friday sales on iPad Pro and iPad mini. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a...