Apple's AI Team Publishes First Research Paper Focused on Advanced Image Recognition

photos-iconEarlier in December, Apple announced that it would begin allowing its artificial intelligence and machine learning researchers to publish and share their work in papers, slightly pulling back the curtain on the company's famously secretive creation processes. Now, just a few weeks later, the first of those papers has been published, focusing on Apple's work in the intelligent image recognition field.

Titled "Learning from Simulated and Unsupervised Images through Adversarial Training," the paper describes a program that can intelligently decipher and understand digital images in a setting similar to the "Siri Intelligence" and facial recognition features introduced in Photos in iOS 10, but more advanced.

In the research, Apple notes the downsides and upsides of using real images compared with that of "synthetic," or computer images. Annotations must be added to real images, an "expensive and time-consuming task" that requires a human workforce to individually label objects in a picture. On the other hand, computer-generated images help to catalyze this process "because the annotations are automatically available."

Still, fully switching to synthetic images could lead to a dip in the quality of the program in question. This is because "synthetic data is often not realistic enough" and would lead to an end-user experience that only responded well to details present in the computer-generated images, while being unable to generalize well on any real-world objects and pictures it faced.

This leads to the paper's central proposition -- the combination of using both simulated and real images to work together in "adversarial training," creating an advanced AI image program:

In this paper, we propose Simulated+Unsupervised (S+U) learning, where the goal is to improve the realism of synthetic images from a simulator using unlabeled real data. The improved realism enables the training of better machine learning models on large datasets without any data collection or human annotation effort.

We show that this enables generation of highly realistic images, which we demonstrate both qualitatively and with a user study.

The rest of the paper goes into the details of Apple's research on the topic, including experiments that have been run and the math proposed to back up its findings. The paper's research focused solely on single images, but the team at Apple notes towards the end that it hopes to sometime soon "investigate refining videos" as well.

The credits on the paper go to Apple researchers Ashish Shrivastava, Tomas Pfister, Oncel Tuzel, Josh Susskind, Wenda Wang, and Russ Webb. The team's research was first submitted on November 15, but it didn't get published until December 22.

At the AI conference in Barcelona a few weeks ago, Apple head of machine learning Russ Salakhutdinov -- and a few other employees -- discussed topics including health and vital signs, volumetric detection of LiDAR, prediction with structured outputs, image processing and colorization, intelligent assistant and language modeling, and activity recognition. We'll likely see papers on a variety of these topics and more in the near future.

Popular Stories

Beyond iPhone 13 Better Blue Face ID Single Camera Hole

10 Reasons to Wait for Next Year's iPhone 17

Monday July 8, 2024 5:00 am PDT by
Apple's iPhone development roadmap runs several years into the future and the company is continually working with suppliers on several successive iPhone models simultaneously, which is why we sometimes get rumored feature leaks so far ahead of launch. The iPhone 17 series is no different – already we have some idea of what to expect from Apple's 2025 smartphone lineup. If you plan to skip...
iPhone 15 Pro Cameras

iPhone 17 Pro Max Will Be First Model to Feature Three 48MP Cameras

Thursday July 11, 2024 12:20 am PDT by
Next year's iPhone 17 Pro Max will feature an upgraded 48-megapixel Tetraprism camera for enhanced photo quality and zoom functionality, according to Apple analyst Ming-Chi Kuo. In his n-iphone-tetraprism-upgrade-ca62dd37e364">latest investor note published to Medium, Kuo said the key specification change would be a 1/2.6" 48MP CIS sensor, up from the 1/3.1" 12MP sensor expected to be used...
iOS 18 on iPhone Feature

Everything New in iOS 18 Beta 3

Monday July 8, 2024 12:54 pm PDT by
With the third betas of iOS 18 and iPadOS 18 that Apple released today, there are additional tweaks to a number of features like Dark Mode icons, the Photos app, emoji in Messages, and more, with Apple appearing to be preparing for the launch of Apple Intelligence. Apple will continue updating iOS 18‌ over the course of the next couple of months, refining the beta prior to its launch....
iPhone 17 Plus Feature

iPhone 17 'Slim': Everything We Know So Far

Friday July 5, 2024 5:13 am PDT by
In 2025, Apple is expected to discontinue the iPhone "Plus" device in its iPhone 17 lineup to make way for an iPhone "Slim" – although it may not actually be called this when it debuts in the fall of next year. Even though the iPhone 16 series launch is still over two months away, when you consider that we learned about larger displays on the iPhone 16 Pro models way back in May 2023, rumors...
Beyond iPhone 13 Better Blue Face ID

iPhone 16 Models Rumored to Have Face ID-Related Design Changes

Tuesday July 9, 2024 9:15 am PDT by
iPhone 16 models coming later this year could have some Face ID-related "design changes," supply chain publication DigiTimes said this week. The original source of this information is British newspaper The Telegraph, which six weeks ago reported that Face ID component supplier Coherent was considering selling or repurposing a manufacturing facility in Newton Aycliffe, a small town in...
AirPods Pro Beta Firmware

Apple Releases New AirPods Pro 2 Beta Firmware With Support for iOS 18 Features

Tuesday July 9, 2024 11:46 am PDT by
Apple today released a second beta firmware for the AirPods Pro 2, including both the Lightning and USB-C versions. The updated firmware has a build number 7A5244b and it is available to developers at the current time. This is the second firmware update that Apple has released since announcing new AirPods Pro 2 features in June. There are several new features that are coming to the AirPods...
iPhone 16 Pro Front Update Blue

iPhone 16 Pro Rumored to Support 40W Fast Charging and 20W MagSafe

Wednesday July 10, 2024 3:57 am PDT by
Apple's forthcoming iPhone 16 Pro and iPhone 16 Pro Max will support 40W wired fast charging and 20W MagSafe charging, claims a rumor currently swirling around China. Right now, iPhone 15 and iPhone 15 Pro models are capable of up to 27W peak charging speeds with an appropriate USB-C power adapter, while official MagSafe chargers from Apple and authorized third parties can wirelessly charge...

Top Rated Comments

A MacBook lover Avatar
99 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
Score: 40 Votes (Like | Disagree)
drewyboy Avatar
99 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
Ok, what? You mean like Apple spent more time on describing the new iMessage at WWDC than any other feature for iOS10? Clearly highlighting emoji as the flagship feature for iOS10? Oh, and lets not forget, they were too busy with emoji to realize they have a horrible battery bug in iOS10? I mean, using Apple's built in flashlight app shouldn't drain 10% of my battery for 4 minutes of use should it? Or shutting down when my phone is at 37% just yesterday only to plug it in and it be back at 37% then drain somewhat normal only to shut down again at 12%? And no, the health of my battery is just fine, or is the Apple Store lying to me when they checked. Or how about how they completely compromised the user experience of the new MBP by sacrificing 25% battery capacity to thin it down and make it lighter for a device that sits on a fixed surface for 99% of the users. Or how about Siri has gotten worse as time as gone on, while competitors get better and better each year?

So please, take your pick and lets have some "quality discussion". All I usually see is people offering validated criticism and then the other half defending apple as if it was their child and blaming the user. You're right, Apple can never do any wrong. They are always right and never wrong. Silly me, my messed up iPhone battery life is a new iOS feature, or is it because I'm using an iPhone 5S and as Phil said, I should be upgrading since it's ancient.

Edit: And as far as Photos go, maybe they should actually do something about families because their current "family share" features are a complete joke.
Score: 22 Votes (Like | Disagree)
samcan Avatar
99 months ago
I'm not sure if it is the competition getting better, but I feel as though Siri is getting dumber by the minute. Context requests are out of the question, it isn't current with sports anymore, and I've found myself being cut off with "sorry I didn't get that" while in a quiet room.

Siri stopped being a useful tool ever since they dropped "raise to speak". I find myself using my Pixel for anything requiring hands free.
Score: 19 Votes (Like | Disagree)
AngerDanger Avatar
99 months ago
If anybody wants to play around with AI image recognition, CloudSight ('http://cloudsight.ai/api') (scroll down to the Try it Out area) allows users to upload an image for recognition. It can be pretty cool to see how accurate its tagging is.



This image was described as "grey jar carton and bottle sketch" after uploading.

Attachment Image
Score: 19 Votes (Like | Disagree)
wigby Avatar
99 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
That's all I see here anymore...a race to critique Apple for making thin devices, requiring dongles and make fun of Siri. Oh and everyone gets bonus points for using the word "courage" in any post. Pathetic commenters.
Score: 18 Votes (Like | Disagree)
and 1989 others Avatar
99 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
When there's something of quality to write about, I'll write about it.

Until then, release joke products, receive joke replies.

Quid, pro, quo.
Score: 12 Votes (Like | Disagree)