Apple to Integrate Augmented Reality Functionality Into iPhone's Camera App

Apple has a team of people working on integrating augmented reality functionality into the iPhone's camera app, reports Business Insider. Employees from several different augmented reality companies Apple has acquired are said to be involved in the effort, including those from Metaio and Flyby Media.

Citing sources familiar with Apple's plans, Business Insider says Apple's goal is for consumers to be able point the phone at a real-world object and have that object be recognized. Apple is already working on similar technology using machine learning techniques and has built basic object recognition capabilities into the Photos app.


Object recognition in the camera app would potentially allow for the real-time recognition of objects, but it is more complex than what is available in the Photos app because it involves 3D objects. Such a feature could be used for spacial recognition and mapping, among other things.

Facial recognition and manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people. Apple is said to be planning to release an augmented reality SDK for developers following the debut of the rumored camera feature, which would allow third-party apps access the AR camera technology too.

Another early feature for Apple's AR integration into the camera app could be to recognize and manipulate people's faces. Apple integrated facial recognition technology into the photos app in the most recent version of its iOS software, and purchased FaceShift, a company with similar technology in 2015.

Recent rumors have suggested Apple's ultimate augmented reality ambition may be a set of smart glasses, which would connect wirelessly to the iPhone and display "images and other information" to the wearer.

A recent hire, John Border, seems to confirm Apple's rumored interest in smart glasses. Border, who describes himself as a "subject matter expert" in head-worn displays, plastic optics manufacturing, and camera systems and image sensors, recently joined Apple after serving as the chief engineer at Osterhout Design Group, a company that developed a set of smart glasses.

Apple's work on smart glasses is said to be in the early stages, and no product is expected to debut before 2018 should the project continue. It is not clear when Apple might introduce AR functionality in the camera app.

Related Roundup: Apple Glasses

Top Rated Comments

Zirel Avatar
79 months ago


VR. It's the future! Guys!
Score: 17 Votes (Like | Disagree)
AngerDanger Avatar
79 months ago
I propose Apple create a feature whereby you aim your camera at a new MacBook and nearby device, and the app overlays all the dongles and adaptors you'd need to connect the two devices. You could even buy them right from the camera app!
Score: 13 Votes (Like | Disagree)
ErikGrim Avatar
79 months ago
While object recognition would be of limited use to most users, imagine what it could do for the vision impaired? A set of smart glasses that translated what they were seeing into sound with Voice Over would be a game changer. Kind of like how VO can identify faces in the camera app already, but for a whole library of every day objects. The blind would literally be able to see and navigate the world by sound.
Score: 10 Votes (Like | Disagree)
FatherJack1980 Avatar
79 months ago
"[facial] manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people."

Oh goody, yet another social media thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event.
Score: 7 Votes (Like | Disagree)
62tele Avatar
79 months ago
The skeptic in me says that this is just the next piece of feces that Apple is throwing against the wall to see if it sticks. The Watch was not the next iPod. The car went down a dead end.
I hope this doesn't distract them from the next revolutionary breakthrough in emoji development.
Score: 5 Votes (Like | Disagree)
jclo Avatar
79 months ago
"[facial] manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people."

Oh goody, yet another Snapchat thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event :oops:
Seems like it would be a fun addition to FaceTime.
Score: 3 Votes (Like | Disagree)

Popular Stories

Cyber Monday Deals Feature 2022

Best Cyber Monday Apple Deals Still Available for AirPods, Apple TV, iPad, and More

Monday November 28, 2022 5:24 am PST by
The Black Friday and Cyber Monday holiday shopping rush is drawing to a close, but there are still some good deals to be had out there. For Apple products, many of the deals you've seen since last week are still available, though some have expired. So for anyone who missed out on Black Friday deals, there's still an opportunity to get some of the year's best prices on many Apple devices. Note: ...
iPhone 14 Pro Rear Camera

iPhone 15 to Use 'State-of-the-Art' Image Sensor From Sony for Better Low-Light Performance

Monday November 28, 2022 11:00 am PST by
Apple's upcoming iPhone 15 models will be equipped with Sony's newest "state of the art" image sensors, according to a report from Nikkei. Compared to standard sensors, Sony's image sensor doubles the saturation signal in each pixel, allowing it to capture more light to cut down on underexposure and overexposure. Nikkei says that it is able to better photograph a person's face even with...
Apple Watch Ultra Oceanic Plus App

Apple Announces Oceanic+ App Now Available for Apple Watch Ultra

Monday November 28, 2022 6:11 am PST by
Apple today announced that the Oceanic+ app is available for the Apple Watch Ultra starting today. Designed by Huish Outdoors in collaboration with Apple, the app serves as a dive computer for recreational scuba diving at depths up to 40 meters/130 feet. Apple already offers a basic Depth app on the Apple Watch Ultra for viewing your current depth, maximum depth reached, water temperature,...
app store awards 2021

Apple Announces 2022 App Store Award Winners, Highlighting Best Apps of the Year

Tuesday November 29, 2022 3:10 am PST by
Apple today announced its 2022 App Store Award winners, highlighting the 16 best apps and games selected by Apple's global App Store editorial team. The top apps were chosen by Apple for their quality, innovative technology, creative design, positive cultural impact, and ability to deliver "exceptional experiences." Apple CEO Tim Cook said: This year's App Store Award winners reimagined...
rapid security response

Apple Releases Another Rapid Security Response Update for iOS 16.2 Beta Users

Monday November 28, 2022 10:16 am PST by
Apple today released a Rapid Security Response update that is available for those running the iOS 16.2 beta, marking the launch of the second RSR update since the feature was released in iOS 16. The Rapid Security Response Update is designed to provide iOS 16.2 beta users with bug fixes without the need to install a full update. The initial RSR release for iOS 16.2 beta users was a test with ...
twitter elon musk

Elon Musk Claims Apple Has 'Mostly Stopped' Offering Ads on Twitter and Is Making Moderation Demands

Monday November 28, 2022 10:42 am PST by
Apple has cut back on its Twitter advertising, according to Twitter CEO Elon Musk. In a tweet, Musk said that Apple has "mostly stopped" its Twitter ads, asking if Apple hates "free speech." Musk went on to publish a poll asking if Apple should "publish all censorship actions" taken that impact customers and he began retweeting content from companies that Apple has had moderation discussions ...
iphone 11 tesla cybertruck close up

Elon Musk Pledges to Build iPhone Rival If Apple Ousts Twitter

Tuesday November 29, 2022 2:48 am PST by
Elon Musk has pledged to offer an "alternative phone" if Apple and Google remove Twitter from their app stores, adding to long-standing rumors about an iPhone rival from Tesla. Modified iPhone 11 Pro in the style of the Tesla Cybertruck, by Caviar. Musk's remark came after being asked about the potential scenario of Twitter being removed from app stores, which could conceivably happen if the...
General Black Friday Deals 2022 Green

All the Apple Black Friday Deals You Can Still Get

Friday November 25, 2022 4:40 am PST by
Although Black Friday is now technically over, many Apple products are still seeing major discounts through the weekend as we head into Cyber Monday. In this article, you'll find every Apple device with a notable Black Friday sale that's still available. We'll be updating as prices change and new deals arrive, so be sure to keep an eye out if you don't see the sale you're looking for yet. Note:...