Apple to Integrate Augmented Reality Functionality Into iPhone's Camera App

Apple has a team of people working on integrating augmented reality functionality into the iPhone's camera app, reports Business Insider. Employees from several different augmented reality companies Apple has acquired are said to be involved in the effort, including those from Metaio and Flyby Media.

Citing sources familiar with Apple's plans, Business Insider says Apple's goal is for consumers to be able point the phone at a real-world object and have that object be recognized. Apple is already working on similar technology using machine learning techniques and has built basic object recognition capabilities into the Photos app.

Subscribe to the MacRumors YouTube channel for more videos.

Object recognition in the camera app would potentially allow for the real-time recognition of objects, but it is more complex than what is available in the Photos app because it involves 3D objects. Such a feature could be used for spacial recognition and mapping, among other things.

Facial recognition and manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people. Apple is said to be planning to release an augmented reality SDK for developers following the debut of the rumored camera feature, which would allow third-party apps access the AR camera technology too.

Another early feature for Apple's AR integration into the camera app could be to recognize and manipulate people's faces. Apple integrated facial recognition technology into the photos app in the most recent version of its iOS software, and purchased FaceShift, a company with similar technology in 2015.

Recent rumors have suggested Apple's ultimate augmented reality ambition may be a set of smart glasses, which would connect wirelessly to the iPhone and display "images and other information" to the wearer.

A recent hire, John Border, seems to confirm Apple's rumored interest in smart glasses. Border, who describes himself as a "subject matter expert" in head-worn displays, plastic optics manufacturing, and camera systems and image sensors, recently joined Apple after serving as the chief engineer at Osterhout Design Group, a company that developed a set of smart glasses.

Apple's work on smart glasses is said to be in the early stages, and no product is expected to debut before 2018 should the project continue. It is not clear when Apple might introduce AR functionality in the camera app.

Related Roundup: Apple Glasses

Top Rated Comments

(View all)
Avatar
49 months ago


VR. It's the future! Guys!
Score: 17 Votes (Like | Disagree)
Avatar
49 months ago
I propose Apple create a feature whereby you aim your camera at a new MacBook and nearby device, and the app overlays all the dongles and adaptors you'd need to connect the two devices. You could even buy them right from the camera app!
Score: 13 Votes (Like | Disagree)
Avatar
49 months ago
While object recognition would be of limited use to most users, imagine what it could do for the vision impaired? A set of smart glasses that translated what they were seeing into sound with Voice Over would be a game changer. Kind of like how VO can identify faces in the camera app already, but for a whole library of every day objects. The blind would literally be able to see and navigate the world by sound.
Score: 10 Votes (Like | Disagree)
Avatar
49 months ago
"[facial] manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people."

Oh goody, yet another social media thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event.
Score: 7 Votes (Like | Disagree)
Avatar
49 months ago
The skeptic in me says that this is just the next piece of feces that Apple is throwing against the wall to see if it sticks. The Watch was not the next iPod. The car went down a dead end.
I hope this doesn't distract them from the next revolutionary breakthrough in emoji development.
Score: 5 Votes (Like | Disagree)
Avatar
49 months ago

"[facial] manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people."

Oh goody, yet another Snapchat thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event :oops:

Seems like it would be a fun addition to FaceTime.
Score: 3 Votes (Like | Disagree)

Top Stories

Apple Takes Legal Action Against Small Company With Pear Logo

Saturday August 8, 2020 11:09 am PDT by
Apple is taking legal action against the developers of the app "Prepear" due to its logo, according to iPhone in Canada. Prepear is an app that helps users discover recipes, plan meals, make lists, and arrange grocery deliveries. The app is a spinoff of "Super Healthy Kids," and the founders claim that they are facing litigation from Apple. Apple reportedly takes issue with Prepear's logo, ...

Kuo: Global iPhone Shipments Could Decline Up to 30% If Apple Forced to Remove WeChat From App Store [Updated x2]

Sunday August 9, 2020 10:17 pm PDT by
In a worst-case scenario, Apple's annual global iPhone shipments could decline by 25–30% if it is forced to remove WeChat from its App Stores around the world, according to a new research note from analyst Ming-Chi Kuo viewed by MacRumors. The removal could occur due to a recent executive order aiming to ban U.S. transactions with WeChat and its parent company Tencent. Kuo lays out...

Google Maps Debuts New Apple Watch App and CarPlay Features

Monday August 10, 2020 9:16 am PDT by
Google today announced the launch of several features for Google Maps on Apple products, including new CarPlay functionality and a new Google Maps app that works on Apple Watch. The new Google Maps app for Apple Watch works similarly to the iOS app, allowing Apple Watch owners to get directions for a car, bike, public transit, or on foot. The app supports estimated arrival times and...

2020 iMac Teardown Reveals Internal Changes and Similarities

Saturday August 8, 2020 12:44 pm PDT by
A teardown video, shared by OWC, reveals the internal changes in the new 2020 27-inch iMac. The 2020 27-inch iMac was announced earlier this week with 10th-generation Intel Core processors, AMD Radeon Pro 5000 series graphics, up to 128GB of RAM, up to 8TB of storage, a 1080p front-facing FaceTime camera, a True Tone display with a nano-texture glass option, higher fidelity speakers, and...

Apple Seeds iOS 14 and iPadOS 14 Public Beta 4 to Testers

Thursday August 6, 2020 10:05 am PDT by
Apple today seeded new public betas of upcoming iOS 14 and iPadOS 14 updates to its public beta testing group. Today's software releases, which Apple labels as fourth betas to keep them in line with developer betas, are actually the third betas that Apple has provided and they come two weeks after the prior beta releases. Public beta testers who have signed up for Apple's beta testing...

Apple Believes This German Cycling Path Logo Infringes on Its Own Logo

Wednesday May 1, 2019 9:51 am PDT by
Apple recently objected to the logo of a new German cycling path in an appeal filed with the German Patent and Trademark Office, according to German outlets General-Anzeiger Bonn and Westdeutscher Rundfunk. Apple reportedly takes issue with the logo's green leaf and supposed "bitten" right side, attributes the company believes are too similar to its own logo. The logo, registered with the ...

Supposed iPhone 12 Display Unit Leaks

Thursday August 6, 2020 8:13 am PDT by
An image supposedly of an iPhone 12 display unit has been shared online by leaker "Twitter user Mr. White". Compared to images of an iPhone 11 Pro display piece, this new unit has a reoriented display connector, reaching up from the bottom of the display, rather than from the left-hand side on iPhone 11 Pro. This may be due to the logic board moving to the other side of the device. A...

Foxconn Reportedly Begins Seasonal Hiring Spree for iPhone 12 Production

Monday August 10, 2020 7:03 am PDT by
Apple's largest manufacturing partner Foxconn has begun its seasonal hiring spree to assist with iPhone 12 production, offering employees who recruit qualified applicants up to a 9,000 yuan bonus, according to Chinese media reports. As usual, Foxconn needs as many hands on deck as possible at its factory in Zhengzhou, China to assist with mass production of the upcoming iPhones. Apple is...

8 Third-Party Home Screen Widgets That You Can Try Out Now on iOS 14

Wednesday August 5, 2020 12:56 pm PDT by
One of the biggest new features of iOS 14 is Home Screen widgets, which provide information from apps at a glance. The widgets can be pinned to the Home Screen in various spots and sizes, allowing for many different layouts. When the iOS 14 beta was first released in June, widgets were limited to Apple's own apps like Calendar and Weather, but several third-party developers have begun to test ...

New 27-Inch iMac's Storage Affixed to Logic Board, 4TB and 8TB Configurations Have Expansion Connector

Friday August 7, 2020 7:46 am PDT by
Following a report from German blog iFun.de that claimed the new 27-inch iMac's flash storage is soldered to the logic board, MacRumors has obtained additional information in an internal document for Apple technicians. In the document, Apple says that the flash storage is indeed affixed to the logic board and cannot be removed. However, for the 4TB and 8TB configurations, Apple says that a...