Apple's AI Team Publishes First Research Paper Focused on Advanced Image Recognition

photos-iconEarlier in December, Apple announced that it would begin allowing its artificial intelligence and machine learning researchers to publish and share their work in papers, slightly pulling back the curtain on the company's famously secretive creation processes. Now, just a few weeks later, the first of those papers has been published, focusing on Apple's work in the intelligent image recognition field.

Titled "Learning from Simulated and Unsupervised Images through Adversarial Training," the paper describes a program that can intelligently decipher and understand digital images in a setting similar to the "Siri Intelligence" and facial recognition features introduced in Photos in iOS 10, but more advanced.

In the research, Apple notes the downsides and upsides of using real images compared with that of "synthetic," or computer images. Annotations must be added to real images, an "expensive and time-consuming task" that requires a human workforce to individually label objects in a picture. On the other hand, computer-generated images help to catalyze this process "because the annotations are automatically available."

Still, fully switching to synthetic images could lead to a dip in the quality of the program in question. This is because "synthetic data is often not realistic enough" and would lead to an end-user experience that only responded well to details present in the computer-generated images, while being unable to generalize well on any real-world objects and pictures it faced.

This leads to the paper's central proposition -- the combination of using both simulated and real images to work together in "adversarial training," creating an advanced AI image program:

In this paper, we propose Simulated+Unsupervised (S+U) learning, where the goal is to improve the realism of synthetic images from a simulator using unlabeled real data. The improved realism enables the training of better machine learning models on large datasets without any data collection or human annotation effort.

We show that this enables generation of highly realistic images, which we demonstrate both qualitatively and with a user study.

The rest of the paper goes into the details of Apple's research on the topic, including experiments that have been run and the math proposed to back up its findings. The paper's research focused solely on single images, but the team at Apple notes towards the end that it hopes to sometime soon "investigate refining videos" as well.

The credits on the paper go to Apple researchers Ashish Shrivastava, Tomas Pfister, Oncel Tuzel, Josh Susskind, Wenda Wang, and Russ Webb. The team's research was first submitted on November 15, but it didn't get published until December 22.

At the AI conference in Barcelona a few weeks ago, Apple head of machine learning Russ Salakhutdinov -- and a few other employees -- discussed topics including health and vital signs, volumetric detection of LiDAR, prediction with structured outputs, image processing and colorization, intelligent assistant and language modeling, and activity recognition. We'll likely see papers on a variety of these topics and more in the near future.

Popular Stories

apple wallet drivers license feature iPhone 15 pro

iPhone Driver's Licenses: These 17 U.S. States Offer Them or Will Later

Thursday June 19, 2025 11:28 am PDT by
In select U.S. states, residents can add their driver's license or state ID to the Wallet app on the iPhone and Apple Watch, providing a convenient and contactless way to display proof of identity or age at select airports and businesses, and in select apps. Unfortunately, this feature continues to roll out very slowly since it was announced in 2021, with only nine U.S. states and Puerto...
ios 26 call holding

iOS 26 Beta is Hiding a New Ringtone — Here's What It Sounds Like

Thursday June 19, 2025 7:25 pm PDT by
Apple is hiding a new ringtone within iOS 26. The new ringtone is an alternative version of the existing Reflection ringtone, which has been the default ringtone since the iPhone X was released in 2017. It was discovered within the code for the first developer beta of iOS 26, but it remains hidden, so you will not find it in the list of ringtones available in the Settings app for now. It...
iPhone 17 Pro Blue Feature Tighter Crop

iPhone 17 Pro Launching in Three Months With These 12 New Features

Saturday June 14, 2025 5:45 pm PDT by
The iPhone 17 Pro and iPhone 17 Pro Max are three months away, and there are plenty of rumors about the devices. Below, we recap key changes rumored for the iPhone 17 Pro models as of June 2025:Aluminum frame: iPhone 17 Pro models are rumored to have an aluminum frame, whereas the iPhone 15 Pro and iPhone 16 Pro models have a titanium frame, and the iPhone X through iPhone 14 Pro have a...
iPhone 16 Battery Life Feature

iOS 26's New Battery Life Mode Available Only on These iPhone Models

Saturday June 21, 2025 9:02 am PDT by
Last week, we reported that iOS 26 introduces an opt-in Adaptive Power Mode on the iPhone, alongside the existing Low Power Mode. Apple says that Adaptive Power Mode can make "small performance adjustments" when necessary to extend an iPhone's battery life, including slightly lowering the display brightness or allowing some activities to "take a little longer." The full description of...
iOS 18

Apple Releases iOS 18.6 Public Beta

Wednesday June 18, 2025 10:24 am PDT by
Apple today seeded the first betas of upcoming iOS 18.6 and iPadOS 18.6 updates to public beta testers, with the betas coming just a few days after Apple provided the betas to developers. Testers who have signed up for beta updates through Apple's beta site can download iOS 18.6 and iPadOS 18.6 from the Settings app on a compatible device by going to General > Software Update. When the...
apple watch ultra 2 new black

Apple Watch Ultra 3 Finally Coming After Two-Year Hiatus

Monday June 16, 2025 8:45 am PDT by
Apple will finally deliver the Apple Watch Ultra 3 sometime this year, according to analyst Jeff Pu of GF Securities Hong Kong (via @jukanlosreve). The analyst expects both the Apple Watch Series 11 and Apple Watch Ultra 3 to arrive this year (likely alongside the new iPhone 17 lineup, if previous launches are anything to go by), according to his latest product roadmap shared with...
General Spotify Feature

Spotify Preparing to Launch Long-Awaited Lossless Audio Tier on iPhone

Thursday June 19, 2025 1:46 pm PDT by
Spotify appears to be gearing up to launch its long-awaited lossless music tier. Chris Messina (via TechCrunch) and Spicetify (via The Verge) spotted new lossless references within the code for Spotify's desktop app and web player. With assistance from Aaron Perris, MacRumors has confirmed that the latest beta of the Spotify app for the iPhone also contains new lossless-related code....
Craig Federighi No

John Gruber Reacts to Apple Declining His Interview After His Criticism

Wednesday June 18, 2025 8:10 pm PDT by
Every year between 2015 and 2024, at least one Apple executive agreed to be interviewed by Daring Fireball's John Gruber for a special WWDC episode of his podcast, The Talk Show. Last year, for example, Apple's software engineering chief Craig Federighi, marketing chief Greg Joswiak, and top AI researcher John Giannandrea joined Gruber on stage at the California Theatre in San Jose to discuss...
iPhone 17 Base Model Rumored to Come in New Green and Purple Colors Feature

iPhone 17 Base Model Rumored to Come in New Purple and Green Colors

Friday June 20, 2025 7:24 am PDT by
Apple is testing new Purple and Green color options for the iPhone 17 base model, according to new information shared by a leaker known as Majin Bu. In a blog post today, Majin Bu said that only one of the two new colors might make the final cut, with Purple apparently being the more likely choice. The base model iPhone 16 is available in five colors: Ultramarine, Teal, Pink, White, and...

Top Rated Comments

A MacBook lover Avatar
111 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
Score: 40 Votes (Like | Disagree)
drewyboy Avatar
111 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
Ok, what? You mean like Apple spent more time on describing the new iMessage at WWDC than any other feature for iOS10? Clearly highlighting emoji as the flagship feature for iOS10? Oh, and lets not forget, they were too busy with emoji to realize they have a horrible battery bug in iOS10? I mean, using Apple's built in flashlight app shouldn't drain 10% of my battery for 4 minutes of use should it? Or shutting down when my phone is at 37% just yesterday only to plug it in and it be back at 37% then drain somewhat normal only to shut down again at 12%? And no, the health of my battery is just fine, or is the Apple Store lying to me when they checked. Or how about how they completely compromised the user experience of the new MBP by sacrificing 25% battery capacity to thin it down and make it lighter for a device that sits on a fixed surface for 99% of the users. Or how about Siri has gotten worse as time as gone on, while competitors get better and better each year?

So please, take your pick and lets have some "quality discussion". All I usually see is people offering validated criticism and then the other half defending apple as if it was their child and blaming the user. You're right, Apple can never do any wrong. They are always right and never wrong. Silly me, my messed up iPhone battery life is a new iOS feature, or is it because I'm using an iPhone 5S and as Phil said, I should be upgrading since it's ancient.

Edit: And as far as Photos go, maybe they should actually do something about families because their current "family share" features are a complete joke.
Score: 22 Votes (Like | Disagree)
samcan Avatar
111 months ago
I'm not sure if it is the competition getting better, but I feel as though Siri is getting dumber by the minute. Context requests are out of the question, it isn't current with sports anymore, and I've found myself being cut off with "sorry I didn't get that" while in a quiet room.

Siri stopped being a useful tool ever since they dropped "raise to speak". I find myself using my Pixel for anything requiring hands free.
Score: 19 Votes (Like | Disagree)
AngerDanger Avatar
111 months ago
If anybody wants to play around with AI image recognition, CloudSight ('http://cloudsight.ai/api') (scroll down to the Try it Out area) allows users to upload an image for recognition. It can be pretty cool to see how accurate its tagging is.



This image was described as "grey jar carton and bottle sketch" after uploading.

Attachment Image
Score: 19 Votes (Like | Disagree)
wigby Avatar
111 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
That's all I see here anymore...a race to critique Apple for making thin devices, requiring dongles and make fun of Siri. Oh and everyone gets bonus points for using the word "courage" in any post. Pathetic commenters.
Score: 18 Votes (Like | Disagree)
and 1989 others Avatar
111 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
When there's something of quality to write about, I'll write about it.

Until then, release joke products, receive joke replies.

Quid, pro, quo.
Score: 12 Votes (Like | Disagree)