Apple's AI Team Publishes First Research Paper Focused on Advanced Image Recognition

photos-iconEarlier in December, Apple announced that it would begin allowing its artificial intelligence and machine learning researchers to publish and share their work in papers, slightly pulling back the curtain on the company's famously secretive creation processes. Now, just a few weeks later, the first of those papers has been published, focusing on Apple's work in the intelligent image recognition field.

Titled "Learning from Simulated and Unsupervised Images through Adversarial Training," the paper describes a program that can intelligently decipher and understand digital images in a setting similar to the "Siri Intelligence" and facial recognition features introduced in Photos in iOS 10, but more advanced.

In the research, Apple notes the downsides and upsides of using real images compared with that of "synthetic," or computer images. Annotations must be added to real images, an "expensive and time-consuming task" that requires a human workforce to individually label objects in a picture. On the other hand, computer-generated images help to catalyze this process "because the annotations are automatically available."

Still, fully switching to synthetic images could lead to a dip in the quality of the program in question. This is because "synthetic data is often not realistic enough" and would lead to an end-user experience that only responded well to details present in the computer-generated images, while being unable to generalize well on any real-world objects and pictures it faced.

This leads to the paper's central proposition -- the combination of using both simulated and real images to work together in "adversarial training," creating an advanced AI image program:

In this paper, we propose Simulated+Unsupervised (S+U) learning, where the goal is to improve the realism of synthetic images from a simulator using unlabeled real data. The improved realism enables the training of better machine learning models on large datasets without any data collection or human annotation effort.

We show that this enables generation of highly realistic images, which we demonstrate both qualitatively and with a user study.

The rest of the paper goes into the details of Apple's research on the topic, including experiments that have been run and the math proposed to back up its findings. The paper's research focused solely on single images, but the team at Apple notes towards the end that it hopes to sometime soon "investigate refining videos" as well.

The credits on the paper go to Apple researchers Ashish Shrivastava, Tomas Pfister, Oncel Tuzel, Josh Susskind, Wenda Wang, and Russ Webb. The team's research was first submitted on November 15, but it didn't get published until December 22.

At the AI conference in Barcelona a few weeks ago, Apple head of machine learning Russ Salakhutdinov -- and a few other employees -- discussed topics including health and vital signs, volumetric detection of LiDAR, prediction with structured outputs, image processing and colorization, intelligent assistant and language modeling, and activity recognition. We'll likely see papers on a variety of these topics and more in the near future.

Top Rated Comments

Amacfa Avatar
62 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
Score: 40 Votes (Like | Disagree)
drewyboy Avatar
62 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
Ok, what? You mean like Apple spent more time on describing the new iMessage at WWDC than any other feature for iOS10? Clearly highlighting emoji as the flagship feature for iOS10? Oh, and lets not forget, they were too busy with emoji to realize they have a horrible battery bug in iOS10? I mean, using Apple's built in flashlight app shouldn't drain 10% of my battery for 4 minutes of use should it? Or shutting down when my phone is at 37% just yesterday only to plug it in and it be back at 37% then drain somewhat normal only to shut down again at 12%? And no, the health of my battery is just fine, or is the Apple Store lying to me when they checked. Or how about how they completely compromised the user experience of the new MBP by sacrificing 25% battery capacity to thin it down and make it lighter for a device that sits on a fixed surface for 99% of the users. Or how about Siri has gotten worse as time as gone on, while competitors get better and better each year?

So please, take your pick and lets have some "quality discussion". All I usually see is people offering validated criticism and then the other half defending apple as if it was their child and blaming the user. You're right, Apple can never do any wrong. They are always right and never wrong. Silly me, my messed up iPhone battery life is a new iOS feature, or is it because I'm using an iPhone 5S and as Phil said, I should be upgrading since it's ancient.

Edit: And as far as Photos go, maybe they should actually do something about families because their current "family share" features are a complete joke.
Score: 22 Votes (Like | Disagree)
samcan Avatar
62 months ago
I'm not sure if it is the competition getting better, but I feel as though Siri is getting dumber by the minute. Context requests are out of the question, it isn't current with sports anymore, and I've found myself being cut off with "sorry I didn't get that" while in a quiet room.

Siri stopped being a useful tool ever since they dropped "raise to speak". I find myself using my Pixel for anything requiring hands free.
Score: 19 Votes (Like | Disagree)
AngerDanger Avatar
62 months ago
If anybody wants to play around with AI image recognition, CloudSight ('http://cloudsight.ai/api') (scroll down to the Try it Out area) allows users to upload an image for recognition. It can be pretty cool to see how accurate its tagging is.



This image was described as "grey jar carton and bottle sketch" after uploading.

Attachment Image
Score: 19 Votes (Like | Disagree)
wigby Avatar
62 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
That's all I see here anymore...a race to critique Apple for making thin devices, requiring dongles and make fun of Siri. Oh and everyone gets bonus points for using the word "courage" in any post. Pathetic commenters.
Score: 18 Votes (Like | Disagree)
and 1989 others Avatar
62 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
When there's something of quality to write about, I'll write about it.

Until then, release joke products, receive joke replies.

Quid, pro, quo.
Score: 12 Votes (Like | Disagree)

Top Stories

calculatorapp

iOS 11 Bug: Typing 1+2+3 Quickly in the Calculator App Won't Get You 6

Tuesday October 24, 2017 2:03 pm PDT by
A bug in the built-in Calculator app in iOS 11 is getting some major attention this week, despite the fact that it's been around since iOS 11 was in beta testing. At issue is a calculator animation that causes some symbols to be ignored when calculations are entered in rapid succession. You can try it for yourself: Type 1+2+3 and then the equals sign into the Calculator app quickly. Due to...
gradiente iphone white

Brazilian Electronics Company Revives Long-Running iPhone Trademark Dispute

Tuesday May 19, 2020 1:06 pm PDT by
Apple has been involved in a long-running iPhone trademark dispute in Brazil, which was revived today by IGB Electronica, a Brazilian consumer electronics company that originally registered the "iPhone" name in 2000. IGB Electronica fought a multi-year battle with Apple in an attempt to get exclusive rights to the "iPhone" trademark, but ultimately lost, and now the case has been brought to...
original iphone

Phil Schiller Says iPhone Was 'Earth-Shattering' Ten Years Ago and Remains 'Unmatched' Today

Monday January 9, 2017 7:15 am PST by
To commemorate the tenth anniversary of the iPhone, Apple marketing chief Phil Schiller sat down with tech journalist Steven Levy for a wide-ranging interview about the smartphone's past, present, and future. The report first reflects upon the iPhone's lack of support for third-party apps in its first year. The argument inside Apple was split between whether the iPhone should be a closed...
iphone 12 colors 2021

iPhone 12 Colors: Deciding on The Right Color

Thursday November 5, 2020 8:35 am PST by
The iPhone 12 and iPhone 12 Pro arrived last October in a range of color options, with entirely new hues available on both devices, as well as some popular classics. The 12 and 12 Pro have different color choices, so if you have your heart set on a particular shade, you might not be able to get your preferred model in that color. iPhone 12 mini and iPhone 12 The iPhone 12 mini and iPhone 12...
iPhone 13 Dummy Thumbnail 2

Full iPhone 13 Feature Breakdown: Everything Rumors Say We Can Expect

Tuesday August 31, 2021 7:50 am PDT by
With the launch of Apple's iPhone 13 lineup believed to be just a few weeks away, we have compiled all of the coherent rumors from our coverage over the past year to build a full picture of the features and upgrades coming to the company's new smartphones. For clarity, only explicit improvements, upgrades, and new features compared to the iPhone 12 lineup are listed. It is worth noting that...
omg lightning cable comparison

Security Researcher Develops Lightning Cable With Hidden Chip to Steal Passwords

Thursday September 2, 2021 6:59 am PDT by
A normal-looking Lightning cable that can used to steal data like passwords and send it to a hacker has been developed, Vice reports. The "OMG Cable" compared to Apple's Lightning to USB cable. The "OMG Cable" works exactly like a normal Lightning to USB cable and can log keystrokes from connected Mac keyboards, iPads, and iPhones, and then send this data to a bad actor who could be over a...
macbook air deals

Deals: Amazon Drops Price of 256GB M1 MacBook Air to New Low of $849.99 ($149 Off)

Friday August 27, 2021 6:16 am PDT by
Amazon today introduced new low prices on the M1 MacBook Air for both 256GB and 512GB storage options. To start, you can get the 256GB model for $849.99, down from an original price of $999.00. Note: MacRumors is an affiliate partner with Amazon. When you click a link and make a purchase, we may receive a small payment, which helps us keep the site running. Only Silver and Gold are...
maxresdefault

New MacBook Pro Models Coming at WWDC, Suggests Leaker

Monday May 24, 2021 1:27 pm PDT by
New MacBook Pro models are coming at WWDC, according to leaker Jon Prosser who has a mixed track record when it comes to predicting Apple's plans. Subscribe to the MacRumors YouTube channel for more videos. Prosser provided no additional information, but there are new 14 and 16-inch MacBook Pro models in the works. The new MacBook Pros will feature the most radical redesign to the MacBook Pro ...
M1X MBP Feature

Leaker: Upcoming MacBook Pro to See Price Hike Over Current Model, Equal Performance Across 14 and 16-Inch Sizes

Tuesday August 24, 2021 5:28 am PDT by
The upcoming 14-inch MacBook Pro is set to be more expensive than the current 13-inch MacBook Pro and both the 14 and 16-inch models will offer the same performance, according to the leaker known as "Dylandkt." The leaker shared the information on Twitter, explaining that both of the upcoming MacBook Pro models, expected to come in 14 and 16-inch sizes, will feature the same performance due...
iPhone 13 Dummy Thumbnail 2

Kuo: iPhone 13 to Feature LEO Satellite Communications to Make Calls and Texts Without Cellular Coverage

Sunday August 29, 2021 7:39 am PDT by
The iPhone 13 will feature low earth orbit (LEO) satellite communication connectivity to allow users to make calls and send messages in areas without 4G or 5G coverage, according to the reliable analyst Ming-Chi Kuo. In a note to investors, seen by MacRumors, Kuo explained that the iPhone 13 lineup will feature hardware that is able to connect to LEO satellites. If enabled with the relevant...