Apple to Integrate Augmented Reality Functionality Into iPhone's Camera App

Apple has a team of people working on integrating augmented reality functionality into the iPhone's camera app, reports Business Insider. Employees from several different augmented reality companies Apple has acquired are said to be involved in the effort, including those from Metaio and Flyby Media.

Citing sources familiar with Apple's plans, Business Insider says Apple's goal is for consumers to be able point the phone at a real-world object and have that object be recognized. Apple is already working on similar technology using machine learning techniques and has built basic object recognition capabilities into the Photos app.

Subscribe to the MacRumors YouTube channel for more videos.

Object recognition in the camera app would potentially allow for the real-time recognition of objects, but it is more complex than what is available in the Photos app because it involves 3D objects. Such a feature could be used for spacial recognition and mapping, among other things.

Facial recognition and manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people. Apple is said to be planning to release an augmented reality SDK for developers following the debut of the rumored camera feature, which would allow third-party apps access the AR camera technology too.

Another early feature for Apple's AR integration into the camera app could be to recognize and manipulate people's faces. Apple integrated facial recognition technology into the photos app in the most recent version of its iOS software, and purchased FaceShift, a company with similar technology in 2015.

Recent rumors have suggested Apple's ultimate augmented reality ambition may be a set of smart glasses, which would connect wirelessly to the iPhone and display "images and other information" to the wearer.

A recent hire, John Border, seems to confirm Apple's rumored interest in smart glasses. Border, who describes himself as a "subject matter expert" in head-worn displays, plastic optics manufacturing, and camera systems and image sensors, recently joined Apple after serving as the chief engineer at Osterhout Design Group, a company that developed a set of smart glasses.

Apple's work on smart glasses is said to be in the early stages, and no product is expected to debut before 2018 should the project continue. It is not clear when Apple might introduce AR functionality in the camera app.

Related Roundup: Apple Glasses

Top Rated Comments

(View all)
Avatar
52 months ago


VR. It's the future! Guys!
Score: 17 Votes (Like | Disagree)
Avatar
52 months ago
I propose Apple create a feature whereby you aim your camera at a new MacBook and nearby device, and the app overlays all the dongles and adaptors you'd need to connect the two devices. You could even buy them right from the camera app!
Score: 13 Votes (Like | Disagree)
Avatar
52 months ago
While object recognition would be of limited use to most users, imagine what it could do for the vision impaired? A set of smart glasses that translated what they were seeing into sound with Voice Over would be a game changer. Kind of like how VO can identify faces in the camera app already, but for a whole library of every day objects. The blind would literally be able to see and navigate the world by sound.
Score: 10 Votes (Like | Disagree)
Avatar
52 months ago
"[facial] manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people."

Oh goody, yet another social media thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event.
Score: 7 Votes (Like | Disagree)
Avatar
52 months ago
The skeptic in me says that this is just the next piece of feces that Apple is throwing against the wall to see if it sticks. The Watch was not the next iPod. The car went down a dead end.
I hope this doesn't distract them from the next revolutionary breakthrough in emoji development.
Score: 5 Votes (Like | Disagree)
Avatar
52 months ago

"[facial] manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people."

Oh goody, yet another Snapchat thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event :oops:

Seems like it would be a fun addition to FaceTime.
Score: 3 Votes (Like | Disagree)

Top Stories

Apple References Unreleased 2020 16-Inch MacBook Pro in Boot Camp Update

Monday October 26, 2020 8:42 am PDT by
Last week, Apple released an update for Boot Camp, its utility for running Windows on a Mac. While this update would typically be unremarkable, several of our readers noticed that the release notes reference an unreleased 2020 model of the 16-inch MacBook Pro. While this could easily be a mistake, the 16-inch MacBook Pro is nearly a year old, so it is certainly a worthy candidate for a...

Google Reportedly Pays Apple $8-12 Billion Per Year to be Default iOS Search Engine

Sunday October 25, 2020 2:59 pm PDT by
The United States Justice Department is targeting a lucrative deal between Apple and Google as part of one of the U.S. government's largest antitrust cases, reports The New York Times. On Tuesday, the Justice Department filed an antitrust lawsuit against Google, claiming the Mountain View-based company used anticompetitive and exclusionary practices in the search and advertising markets to ...

iPhone 12 Pro Allows You to Measure Someone's Height Instantly Using LiDAR Scanner

Saturday October 24, 2020 11:12 am PDT by
iPhone 12 Pro models feature a new LiDAR Scanner for enhanced augmented reality experiences, but the sensor also enables another unique feature: the ability to measure a person's height instantly using the Measure app. You can even measure the seated height of a person in a chair, according to Apple. When the Measure app detects a person in the viewfinder, it automatically measures their...

MagSafe Charger Only Charges at Full 15W Speeds With Apple's 20W Power Adapter [Updated]

Monday October 26, 2020 3:38 pm PDT by
Alongside the iPhone 12 and 12 Pro models, Apple introduced a new MagSafe charger that attaches to the magnetic ring in the back of the devices, providing up to 15W of charging power, which is double the speed of the 7.5W Qi-based wireless charging maximum. Apple does not provide a power adapter with the $39 MagSafe charger, requiring users to supply their own USB-C compatible option. Apple...

Early iPhone 12 Tests Show Ceramic Shield is Stronger and More Scratch Resistant Than iPhone 11 Glass

Friday October 23, 2020 1:21 pm PDT by
Apple's new iPhone 12 models are protected by a Ceramic Shield cover glass that has nano-ceramic crystals infused right into the glass to improve durability. According to Apple, Ceramic Shield offers four times better drop protection than the glass used for the iPhone 11 models. YouTube channel MobileReviewsEh conducted some tests on the iPhone 12 using a force meter to compare its performance ...

iPhone 12 Six-Foot Drop Test Results: Ceramic Shield More Durable But Not Damage Proof

Monday October 26, 2020 5:00 am PDT by
Apple's new iPhone 12 and iPhone 12 Pro feature a new Ceramic Shield screen that Apple says offers 4x better drop performance. To test that claim, Allstate Protection Plans put the two models through a range of breakability tests and recorded the results. In a face down sidewalk drop test at six feet, the iPhone 12 suffered small cracks and scuffed corners and edges, leaving sharp grooves in ...

Bloomberg: New AirPods and AirPods Pro Coming in 2021, AirPods Studio Delayed, Third HomePod Model Also Possible

Monday October 26, 2020 3:34 am PDT by
Apple plans to update its AirPods line next year with two new models including third-generation AirPods and second-generation AirPods Pro, according to a new report from Bloomberg. The Cupertino, California-based technology giant is working on two new models: third-generation entry-level AirPods and the second version of the AirPods Pro earbuds, according to people familiar with the plans. ...

Report: Apple Silicon iMac Featuring Desktop Class 'A14T' Chip Coming First Half of 2021

Tuesday October 27, 2020 4:14 am PDT by
The first iMac powered by Apple Silicon is set to arrive in the first half of next year and will feature a desktop class "A14T" chip, according to Chinese-language newspaper The China Times. Codenamed "Mt. Jade," Apple's first custom-made desktop processor will be twinned with its first self-developed GPU, codenamed "Lifuka," both of which are being produced using TSMC's 5-nanometer process, ...

iPhone 11 Pro Outlasts iPhone 12 and 12 Pro in Extensive Battery Life Test

Friday October 23, 2020 8:36 am PDT by
Arun Maini today shared a new side-by-side iPhone battery life video test on his YouTube channel Mrwhosetheboss, timing how long the new iPhone 12 and iPhone 12 Pro models last on a single charge compared to older models, with equal brightness, settings, battery health, and usage. All of the devices are running iOS 14 without a SIM card inserted. In the test, the iPhone 11 Pro outlasted both ...

Apple Warns MagSafe Charger Can Leave Circular Imprints on Leather Cases

Friday October 23, 2020 3:23 pm PDT by
If you keep your iPhone in a leather case while charging with Apple's new MagSafe Charger, the case might show circular imprints from contact with the accessory, according to a new Apple support document published today. Apple's leather cases for the iPhone 12 and iPhone 12 Pro are not available until November 6, but a MacRumors reader has already shared a photo of a circular imprint on...