Apple's AI Team Publishes First Research Paper Focused on Advanced Image Recognition

photos-iconEarlier in December, Apple announced that it would begin allowing its artificial intelligence and machine learning researchers to publish and share their work in papers, slightly pulling back the curtain on the company's famously secretive creation processes. Now, just a few weeks later, the first of those papers has been published, focusing on Apple's work in the intelligent image recognition field.

Titled "Learning from Simulated and Unsupervised Images through Adversarial Training," the paper describes a program that can intelligently decipher and understand digital images in a setting similar to the "Siri Intelligence" and facial recognition features introduced in Photos in iOS 10, but more advanced.

In the research, Apple notes the downsides and upsides of using real images compared with that of "synthetic," or computer images. Annotations must be added to real images, an "expensive and time-consuming task" that requires a human workforce to individually label objects in a picture. On the other hand, computer-generated images help to catalyze this process "because the annotations are automatically available."

Still, fully switching to synthetic images could lead to a dip in the quality of the program in question. This is because "synthetic data is often not realistic enough" and would lead to an end-user experience that only responded well to details present in the computer-generated images, while being unable to generalize well on any real-world objects and pictures it faced.

This leads to the paper's central proposition -- the combination of using both simulated and real images to work together in "adversarial training," creating an advanced AI image program:

In this paper, we propose Simulated+Unsupervised (S+U) learning, where the goal is to improve the realism of synthetic images from a simulator using unlabeled real data. The improved realism enables the training of better machine learning models on large datasets without any data collection or human annotation effort.

We show that this enables generation of highly realistic images, which we demonstrate both qualitatively and with a user study.

The rest of the paper goes into the details of Apple's research on the topic, including experiments that have been run and the math proposed to back up its findings. The paper's research focused solely on single images, but the team at Apple notes towards the end that it hopes to sometime soon "investigate refining videos" as well.

The credits on the paper go to Apple researchers Ashish Shrivastava, Tomas Pfister, Oncel Tuzel, Josh Susskind, Wenda Wang, and Russ Webb. The team's research was first submitted on November 15, but it didn't get published until December 22.

At the AI conference in Barcelona a few weeks ago, Apple head of machine learning Russ Salakhutdinov -- and a few other employees -- discussed topics including health and vital signs, volumetric detection of LiDAR, prediction with structured outputs, image processing and colorization, intelligent assistant and language modeling, and activity recognition. We'll likely see papers on a variety of these topics and more in the near future.

Popular Stories

apple oct 2024 mac tease

Apple Expected to Announce These Two to Three Products 'This Week'

Sunday October 12, 2025 7:05 am PDT by
Apple plans to announce new products "this week," according to Bloomberg's Mark Gurman. Apple's "Mac Your Calendars" teaser last October In his Power On newsletter today, Gurman said the products set to be updated this week include the iPad Pro, Vision Pro, and "likely" the base 14-inch MacBook Pro, with all three likely to receive a spec bump with Apple's next-generation M5 chip. Gurman...
iOS 26 Feature

Apple Preparing iOS 26.0.2 Update for iPhones

Saturday October 11, 2025 6:59 pm PDT by
Apple's software engineers are internally testing iOS 26.0.2, according to MacRumors logs, which have been a reliable indicator of upcoming iOS versions. iOS 26.0.2 will likely be a minor update that addresses bugs and/or security vulnerabilities, but we do not know any specific details yet. The update will likely be released within the next few weeks. Last month, Apple released iOS...
Apple TV Plus Feature 2 Magenta and Blue

Apple TV+ Being Rebranded as Apple TV

Monday October 13, 2025 8:25 am PDT by
Buried in its announcement about "F1: The Movie" making its streaming debut on December 12, Apple has also announced that Apple TV+ is being rebranded as simply Apple TV. A single line near the end of the press release states "Apple TV+ is now simply Apple TV, with a vibrant new identity," though Apple's website has yet to be updated with any changes, so we're unsure on the details of the...
iPhone 17 Pro Colors

iPhone 18 Pro Already Rumored to Have These 6 New Features

Saturday October 11, 2025 10:10 am PDT by
While the iPhone 18 Pro and iPhone 18 Pro Max are still nearly a year away, a handful of new features and changes have already been rumored for the devices. Below, we have recapped some of the early iPhone 18 Pro rumors so far. Smaller Dynamic Island The standard iPhone 18, iPhone 18 Pro, and iPhone 18 Pro Max will be equipped with a slightly smaller Dynamic Island, but the devices will...
All AirPods 2025

Apple Reportedly Working on New AirPods Pro, AirPods 5, and H3 Chip

Sunday October 12, 2025 9:24 am PDT by
After releasing AirPods Pro 3 last month, Apple is already working on the next AirPods Pro, according to Bloomberg's Mark Gurman. It is unclear if the new AirPods Pro would be branded as AirPods Pro 4, or if they would be considered an updated version of AirPods Pro 3. Gurman did not take a position, opting to describe them as a "new version" of the "high-end in-ear buds." AirPods Pro 2...
joz macbook tease

Apple Teases Upcoming M5 MacBook Pro Launch: 'Something Powerful is Coming'

Tuesday October 14, 2025 11:59 am PDT by
Apple marketing chief Greg Joswiak today teased the launch of an upcoming product, saying "something powerful is coming" on social media. Subscribe to the MacRumors YouTube channel for more videos. A short animation accompanying Joswiak's teaser reveals a brief glimpse of a MacBook Pro along with the words "coming soon." The shape of the MacBook Pro is a V, which is the Roman numeral...
Meta Ray Ban Glasses

Apple's Smart Glasses With In-Lens Display May Feature Two Modes

Sunday October 12, 2025 9:43 am PDT by
Apple's second-generation smart glasses with an in-lens display may have two modes, depending on which device they are connected to. Meta Ray-Bans without an in-lens display In his Power On newsletter today, Bloomberg's Mark Gurman said he was told a future version of Apple's smart glasses may be able to run a full version of the visionOS operating system when they are paired with a Mac, and...
10

Apple to Launch New Products Starting Next Week, Claims Dubious Leak [Updated]

Friday October 10, 2025 5:57 am PDT by
Update: the Naver account appears to be referencing a speculative post on X by Vadim Yuryev, dated October 6. The original article follows. Apple will announce new products through a series of press releases beginning as soon as next week, according to a dubious claim posted on the Korean blog Naver. The Naver blog account yeux1122, which aggregates rather than originates Apple...
macbook pro blue

Apple's M5 MacBook Pro Imminent: What to Expect

Tuesday October 14, 2025 4:35 pm PDT by
Apple is going to launch a new version of the MacBook Pro as soon as tomorrow, so we thought we'd go over what to expect from Apple's upcoming Mac. M5 Chip The MacBook Pro will be one of the first new devices to use the next-generation M5 chip, which will replace the M4 chip. The M5 is built on TSMC's more advanced 3-nanometer process, and it will bring speed and efficiency improvements. ...

Top Rated Comments

A MacBook lover Avatar
115 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
Score: 40 Votes (Like | Disagree)
drewyboy Avatar
115 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
Ok, what? You mean like Apple spent more time on describing the new iMessage at WWDC than any other feature for iOS10? Clearly highlighting emoji as the flagship feature for iOS10? Oh, and lets not forget, they were too busy with emoji to realize they have a horrible battery bug in iOS10? I mean, using Apple's built in flashlight app shouldn't drain 10% of my battery for 4 minutes of use should it? Or shutting down when my phone is at 37% just yesterday only to plug it in and it be back at 37% then drain somewhat normal only to shut down again at 12%? And no, the health of my battery is just fine, or is the Apple Store lying to me when they checked. Or how about how they completely compromised the user experience of the new MBP by sacrificing 25% battery capacity to thin it down and make it lighter for a device that sits on a fixed surface for 99% of the users. Or how about Siri has gotten worse as time as gone on, while competitors get better and better each year?

So please, take your pick and lets have some "quality discussion". All I usually see is people offering validated criticism and then the other half defending apple as if it was their child and blaming the user. You're right, Apple can never do any wrong. They are always right and never wrong. Silly me, my messed up iPhone battery life is a new iOS feature, or is it because I'm using an iPhone 5S and as Phil said, I should be upgrading since it's ancient.

Edit: And as far as Photos go, maybe they should actually do something about families because their current "family share" features are a complete joke.
Score: 22 Votes (Like | Disagree)
samcan Avatar
115 months ago
I'm not sure if it is the competition getting better, but I feel as though Siri is getting dumber by the minute. Context requests are out of the question, it isn't current with sports anymore, and I've found myself being cut off with "sorry I didn't get that" while in a quiet room.

Siri stopped being a useful tool ever since they dropped "raise to speak". I find myself using my Pixel for anything requiring hands free.
Score: 19 Votes (Like | Disagree)
AngerDanger Avatar
115 months ago
If anybody wants to play around with AI image recognition, CloudSight ('http://cloudsight.ai/api') (scroll down to the Try it Out area) allows users to upload an image for recognition. It can be pretty cool to see how accurate its tagging is.



This image was described as "grey jar carton and bottle sketch" after uploading.

Attachment Image
Score: 19 Votes (Like | Disagree)
wigby Avatar
115 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
That's all I see here anymore...a race to critique Apple for making thin devices, requiring dongles and make fun of Siri. Oh and everyone gets bonus points for using the word "courage" in any post. Pathetic commenters.
Score: 18 Votes (Like | Disagree)
and 1989 others Avatar
115 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
When there's something of quality to write about, I'll write about it.

Until then, release joke products, receive joke replies.

Quid, pro, quo.
Score: 12 Votes (Like | Disagree)