Apple's AI Team Publishes First Research Paper Focused on Advanced Image Recognition

photos-iconEarlier in December, Apple announced that it would begin allowing its artificial intelligence and machine learning researchers to publish and share their work in papers, slightly pulling back the curtain on the company's famously secretive creation processes. Now, just a few weeks later, the first of those papers has been published, focusing on Apple's work in the intelligent image recognition field.

Titled "Learning from Simulated and Unsupervised Images through Adversarial Training," the paper describes a program that can intelligently decipher and understand digital images in a setting similar to the "Siri Intelligence" and facial recognition features introduced in Photos in iOS 10, but more advanced.

In the research, Apple notes the downsides and upsides of using real images compared with that of "synthetic," or computer images. Annotations must be added to real images, an "expensive and time-consuming task" that requires a human workforce to individually label objects in a picture. On the other hand, computer-generated images help to catalyze this process "because the annotations are automatically available."

Still, fully switching to synthetic images could lead to a dip in the quality of the program in question. This is because "synthetic data is often not realistic enough" and would lead to an end-user experience that only responded well to details present in the computer-generated images, while being unable to generalize well on any real-world objects and pictures it faced.

This leads to the paper's central proposition -- the combination of using both simulated and real images to work together in "adversarial training," creating an advanced AI image program:

In this paper, we propose Simulated+Unsupervised (S+U) learning, where the goal is to improve the realism of synthetic images from a simulator using unlabeled real data. The improved realism enables the training of better machine learning models on large datasets without any data collection or human annotation effort.

We show that this enables generation of highly realistic images, which we demonstrate both qualitatively and with a user study.

The rest of the paper goes into the details of Apple's research on the topic, including experiments that have been run and the math proposed to back up its findings. The paper's research focused solely on single images, but the team at Apple notes towards the end that it hopes to sometime soon "investigate refining videos" as well.

The credits on the paper go to Apple researchers Ashish Shrivastava, Tomas Pfister, Oncel Tuzel, Josh Susskind, Wenda Wang, and Russ Webb. The team's research was first submitted on November 15, but it didn't get published until December 22.

At the AI conference in Barcelona a few weeks ago, Apple head of machine learning Russ Salakhutdinov -- and a few other employees -- discussed topics including health and vital signs, volumetric detection of LiDAR, prediction with structured outputs, image processing and colorization, intelligent assistant and language modeling, and activity recognition. We'll likely see papers on a variety of these topics and more in the near future.

Top Rated Comments

Amacfa Avatar
51 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
Score: 40 Votes (Like | Disagree)
drewyboy Avatar
51 months ago

Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys

Ok, what? You mean like Apple spent more time on describing the new iMessage at WWDC than any other feature for iOS10? Clearly highlighting emoji as the flagship feature for iOS10? Oh, and lets not forget, they were too busy with emoji to realize they have a horrible battery bug in iOS10? I mean, using Apple's built in flashlight app shouldn't drain 10% of my battery for 4 minutes of use should it? Or shutting down when my phone is at 37% just yesterday only to plug it in and it be back at 37% then drain somewhat normal only to shut down again at 12%? And no, the health of my battery is just fine, or is the Apple Store lying to me when they checked. Or how about how they completely compromised the user experience of the new MBP by sacrificing 25% battery capacity to thin it down and make it lighter for a device that sits on a fixed surface for 99% of the users. Or how about Siri has gotten worse as time as gone on, while competitors get better and better each year?

So please, take your pick and lets have some "quality discussion". All I usually see is people offering validated criticism and then the other half defending apple as if it was their child and blaming the user. You're right, Apple can never do any wrong. They are always right and never wrong. Silly me, my messed up iPhone battery life is a new iOS feature, or is it because I'm using an iPhone 5S and as Phil said, I should be upgrading since it's ancient.

Edit: And as far as Photos go, maybe they should actually do something about families because their current "family share" features are a complete joke.
Score: 22 Votes (Like | Disagree)
samcan Avatar
51 months ago
I'm not sure if it is the competition getting better, but I feel as though Siri is getting dumber by the minute. Context requests are out of the question, it isn't current with sports anymore, and I've found myself being cut off with "sorry I didn't get that" while in a quiet room.

Siri stopped being a useful tool ever since they dropped "raise to speak". I find myself using my Pixel for anything requiring hands free.
Score: 19 Votes (Like | Disagree)
AngerDanger Avatar
51 months ago
If anybody wants to play around with AI image recognition, CloudSight ('http://cloudsight.ai/api') (scroll down to the Try it Out area) allows users to upload an image for recognition. It can be pretty cool to see how accurate its tagging is.



This image was described as "grey jar carton and bottle sketch" after uploading.

Attachment Image
Score: 19 Votes (Like | Disagree)
wigby Avatar
51 months ago

Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys

That's all I see here anymore...a race to critique Apple for making thin devices, requiring dongles and make fun of Siri. Oh and everyone gets bonus points for using the word "courage" in any post. Pathetic commenters.
Score: 18 Votes (Like | Disagree)
and 1989 others Avatar
51 months ago

Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys

When there's something of quality to write about, I'll write about it.

Until then, release joke products, receive joke replies.

Quid, pro, quo.
Score: 12 Votes (Like | Disagree)

Top Stories

0 Deals Hero

Black Friday 2020: Best Apple Deals to Plan For

Saturday November 21, 2020 10:00 am PST by
In the lead-up to Black Friday next week, we've been putting a spotlight on the best deals coming from various retailers like Best Buy and Walmart. In an effort to further prepare our readers for the best Black Friday deals, we're breaking down what we think should be on your radar for Black Friday in 2020. Note: MacRumors is an affiliate partner with some of these vendors. When you click a...
2020 apple shopping event

Apple Offering Up to $150 Gift Card With Select Products on Black Friday Through Cyber Monday

Monday November 23, 2020 2:53 am PST by
Apple has announced its annual four-day shopping event, offering customers up to a $150 Apple Store gift card with the purchase of select products between Black Friday and Cyber Monday in the United States. The gift card values in the United States are as follows: $150 for 16-inch MacBook Pro $150 for 21.5-inch iMac $50 for 13-inch MacBook Pro $50 for MacBook Air $50 for iPhone SE,...
m1 mac mini vignette

Apple Lists M1-Based Mac Mini Logic Boards With 10 Gigabit Ethernet in Internal Parts Ordering System

Friday November 20, 2020 9:32 am PST by
While the new Mac mini with the M1 chip is only available with Gigabit Ethernet, Apple has listed multiple M1-based Mac mini logic boards with 10 Gigabit Ethernet in an internal parts list for Apple Authorized Service Providers. For every Mac mini logic board with Gigabit Ethernet in the parts list, obtained by MacRumors, there is a corresponding logic board with 10 Gigabit Ethernet:...
ipad pro 2020 display

Black Friday Week Kicks Off With Up to $150 Savings on 2020 iPad Pro

Sunday November 22, 2020 2:37 pm PST by
As we head into Black Friday week, we're seeing some of the best deals of the season so far, with Amazon and Best Buy today discounting the latest iPad Pro models by up to $150 at the lowest prices we've ever tracked on these models. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a small payment, which helps us keep ...
macos big sur m1 macs restore issue

Apple Provides Instructions to Fix macOS Reinstallation Errors on M1 Macs

Sunday November 22, 2020 3:30 pm PST by
Shortly after the launch of Apple's new M1 Macs, we saw reports that attempts to restore and reinstall macOS on those machines right away could result in an installation error that would leave your Mac non-functional. Specifically, the error message would read: "An error occurred preparing the update. Failed to personalize the software update. Please try again." Over the weekend, Apple p...
iPhone 6s main

Rumor Claims iOS 15 to Drop Support for iPhone 6s and Original iPhone SE

Sunday November 22, 2020 9:25 am PST by
Apple will drop support for the iPhone SE, iPhone 6s, and iPhone 6s Plus in next year's release of iOS 15, according to a rumor shared today by Israeli site The Verifier. If the rumor is accurate, that would mean iOS 15 will be compatible with the following Apple devices: 2021 iPhone series iPhone 12 Pro Max iPhone 12 Pro iPhone 12 mini iPhone 12 iPhone 11 iPhone 11 Pro iPhone 11 Pro ...
new m1 chip

Craig Federighi: Native Windows on M1 Macs is 'Really up to Microsoft'

Friday November 20, 2020 11:57 am PST by
Following the release of the M1 Macs Apple executives have been doing interviews with a range of publications, and today, Ars Technica published another interview with software engineering chief Craig Federighi, hardware technologies lead Johny Srouji, and marketing VP Greg Joswiak. Much of the interview focuses on topics that the three have already covered in prior discussions, but there is ...
Walmart November Deals Hero

Black Friday Spotlight: Walmart Will Have AirPods Pro Down to Lowest Price of $169, and More Apple Deals

Thursday November 19, 2020 8:05 am PST by
We've been tracking early Black Friday deals in our dedicated Black Friday Roundup, and in an effort to prepare our readers for the big shopping event we're highlighting sales store-by-store in the lead-up to November 27. Note: MacRumors is an affiliate partner with Walmart. When you click a link and make a purchase, we may receive a small payment, which helps us keep the site running. Next ...
xlK6MS4MLqVFaskl

iFixit Shares iPhone 12 Pro Max Teardown Revealing L-Shaped Battery and Bigger Camera Module

Sunday November 22, 2020 2:53 am PST by
In the last of its iPhone 12 series teardowns, iFixit has published its iPhone 12 Pro Max disassembly, which reveals some unique differences in the design of the internals, including the full extent of the larger camera system responsible for the improved low-light performance on Apple's largest iPhone to date. On opening the iPhone 12 Pro Max, the first obvious difference compared to the...
apple leather sleeve

Leather Sleeve for iPhone 12 Models Now Available From Apple

Friday November 20, 2020 12:16 pm PST by
Apple today began selling the Leather Sleeve for the new iPhone 12 models, with the accessory having first been announced alongside the updated iPhones in October. Priced at $129, the Leather Sleeve is not a case and is designed to be removed when the iPhone is in use. It features a cutout at the front that displays the time, and it comes with a matching leather strap. According to Apple, it ...