Apple to Integrate Augmented Reality Functionality Into iPhone's Camera App

Apple has a team of people working on integrating augmented reality functionality into the iPhone's camera app, reports Business Insider. Employees from several different augmented reality companies Apple has acquired are said to be involved in the effort, including those from Metaio and Flyby Media.

Citing sources familiar with Apple's plans, Business Insider says Apple's goal is for consumers to be able point the phone at a real-world object and have that object be recognized. Apple is already working on similar technology using machine learning techniques and has built basic object recognition capabilities into the Photos app.


Object recognition in the camera app would potentially allow for the real-time recognition of objects, but it is more complex than what is available in the Photos app because it involves 3D objects. Such a feature could be used for spacial recognition and mapping, among other things.

Facial recognition and manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people. Apple is said to be planning to release an augmented reality SDK for developers following the debut of the rumored camera feature, which would allow third-party apps access the AR camera technology too.

Another early feature for Apple's AR integration into the camera app could be to recognize and manipulate people's faces. Apple integrated facial recognition technology into the photos app in the most recent version of its iOS software, and purchased FaceShift, a company with similar technology in 2015.

Recent rumors have suggested Apple's ultimate augmented reality ambition may be a set of smart glasses, which would connect wirelessly to the iPhone and display "images and other information" to the wearer.

A recent hire, John Border, seems to confirm Apple's rumored interest in smart glasses. Border, who describes himself as a "subject matter expert" in head-worn displays, plastic optics manufacturing, and camera systems and image sensors, recently joined Apple after serving as the chief engineer at Osterhout Design Group, a company that developed a set of smart glasses.

Apple's work on smart glasses is said to be in the early stages, and no product is expected to debut before 2018 should the project continue. It is not clear when Apple might introduce AR functionality in the camera app.

Related Roundup: Apple Glasses

Top Rated Comments

Zirel Avatar
56 months ago


VR. It's the future! Guys!
Score: 17 Votes (Like | Disagree)
AngerDanger Avatar
56 months ago
I propose Apple create a feature whereby you aim your camera at a new MacBook and nearby device, and the app overlays all the dongles and adaptors you'd need to connect the two devices. You could even buy them right from the camera app!
Score: 13 Votes (Like | Disagree)
ErikGrim Avatar
56 months ago
While object recognition would be of limited use to most users, imagine what it could do for the vision impaired? A set of smart glasses that translated what they were seeing into sound with Voice Over would be a game changer. Kind of like how VO can identify faces in the camera app already, but for a whole library of every day objects. The blind would literally be able to see and navigate the world by sound.
Score: 10 Votes (Like | Disagree)
FatherJack1980 Avatar
56 months ago
"[facial] manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people."

Oh goody, yet another social media thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event.
Score: 7 Votes (Like | Disagree)
62tele Avatar
56 months ago
The skeptic in me says that this is just the next piece of feces that Apple is throwing against the wall to see if it sticks. The Watch was not the next iPod. The car went down a dead end.
I hope this doesn't distract them from the next revolutionary breakthrough in emoji development.
Score: 5 Votes (Like | Disagree)
jclo Avatar
56 months ago
"[facial] manipulation may also be a goal, perhaps allowing the camera app to be used to apply Snapchat-style augmented reality filters to people."

Oh goody, yet another Snapchat thingy! Man, can't wait to emoji-fy my face!! *sigh*

....I can just imagine the presentation for this in the next Apple event :oops:
Seems like it would be a fun addition to FaceTime.
Score: 3 Votes (Like | Disagree)

Top Stories

iphone 6 in hand

Apple Faces Another iPhone Lawsuit Over 'Programmed Obsolescence'

Monday March 1, 2021 6:44 am PST by
Apple faces a new class-action lawsuit that accuses it of deliberately releasing iOS updates that slowly reduce the performance of an iPhone, forcing customers to upgrade their devices. The lawsuit comes from the Portuguese Consumer Protection Agency, Deco Proteste (via Marketeer), which in a statement says that it will proceed with a case against the Cupertino tech giant because it...
PIA23764 RoverNamePlateonMars web

NASA Mars Perseverance Rover Uses Same PowerPC Chipset Found in 1998 G3 iMac

Tuesday March 2, 2021 3:21 am PST by
NASA's Perseverance rover, which recently made history landing on the surface of Mars, is powered by the same processor used in an iMac more than 23 years old. Image Credit: NASA As reported by NewScientist (via Gizmodo), the rover includes the PowerPC 750 processor, the same chip used in the G3 iMac in 1998. The main chipset is the same; however, there are differences between the...
magsafe charging brick feature

Prosser: iPhone 12 MagSafe Battery Pack to Feature Reverse Charging

Monday March 1, 2021 1:18 pm PST by
Apple is working on MagSafe-compatible battery packs that will be compatible with the iPhone 12 models, and leaker Jon Prosser today said on the Genius Bar podcast that one version of the battery pack will feature "reverse charging." According to Prosser, Apple is working on two versions of the battery packs, one that's a standard version and one that's a premium version with reverse...
rosetta 2

Rosetta May Be Removed From M1 Macs in Some Regions on macOS 11.3

Tuesday March 2, 2021 5:20 pm PST by
Installing the upcoming macOS 11.3 software update on an M1 Mac may result in Rosetta 2 being removed in one or more regions around the world. In the third beta of macOS 11.3 seeded to developers for testing today, MacRumors contributor Steve Moser uncovered new strings in the beta's code indicating that "Rosetta will be removed upon installing this update." Another new string reads "Rosetta ...
iphone 12 pro display video

iPhone 13 to Include 1TB Storage Option and LiDAR Across the Board, Says Wedbush Analyst

Monday March 1, 2021 4:00 am PST by
Apple's forthcoming iPhone 13 could include a 1TB storage option for some models and LiDAR Scanners across the entire lineup, according to a report from Wedbush analysts. In a new note to investors, seen by MacRumors, Wedbush analyst Daniel Ives said that initial Asian supply chain checks gave the firm "increased confidence" that Apple's 5G-driven product cycle would extend well into 2022,...
Apple Prefer Lightning Over USB C Feature

iPhone Sticking With Lightning Port Over USB-C for 'Foreseeable Future'

Tuesday March 2, 2021 9:32 am PST by
Apple will retain the Lightning connector on the iPhone for the "foreseeable future," with no intention of switching to USB-C, according to reliable analyst Ming-Chi Kuo. In spite of much of the industry moving toward USB-C, Apple will not be using it to replace the Lightning connector on the iPhone 13, or indeed on any iPhone model for the time being. In a note seen by MacRumors yesterday,...
iphone 12 120hz thumbnail feature

Kuo: iPhone 13 Lineup to Feature Smaller Notch and Larger Batteries, 120Hz Display for Pro Models, and More

Monday March 1, 2021 7:50 am PST by
iPhone 13 models will all feature a smaller notch, while the two Pro models will be equipped with low-power LTPO display technology for a 120Hz refresh rate, analyst Ming-Chi Kuo said today in a research note obtained by MacRumors. Subscribe to the MacRumors YouTube channel for more videos. Several other sources have previously claimed that some iPhone 13 models will support a 120Hz refresh...
Screen Shot 2021 03 03 at 11

MagSafe Charging Port for iPhone Appears in Apple Patent

Wednesday March 3, 2021 1:30 am PST by
In a newly granted patent, Apple envisions a type of connectivity port using an iteration of its magnetic MagSafe charger to charge an iPhone, potentially paving the way for a future without Lightning. Submitted to the United States Patent and Trademark Office on Tuesday (via Patently Apple), the patent outlines a three-pinned MagSafe charger, similar to the ones found in older MacBook...
apple transfer google photos 1

Apple Launches Service for Transferring iCloud Photos and Videos to Google Photos

Wednesday March 3, 2021 12:04 pm PST by
Apple this week introduced a new service that's designed to make it quick and easy for iCloud users to transfer their stored photos and videos to Google Photos. As outlined in an Apple support document, you can go to Apple's privacy website and sign in to see the "Transfer a copy of your data" option. If you select this and go through all the steps, Apple will transfer your iCloud photos and ...
galaxy s21 iphone 12 pro max front feature2

Kuo: Some 2022 iPhones to Abandon Notch in Favor of 'Punch-Hole' Display

Monday March 1, 2021 8:05 am PST by
At least some 2022 iPhone models will abandon the notch and switch to a "punch-hole display design" instead, similar to Samsung's recent high-end Galaxy smartphones, analyst Ming-Chi Kuo said today in a research note obtained by MacRumors. Kuo said the hole-punch display design will be adopted by high-end 2022 iPhones at a minimum, and if production yields are good enough, all 2022 iPhone...