Apple's AI Team Publishes First Research Paper Focused on Advanced Image Recognition

photos-iconEarlier in December, Apple announced that it would begin allowing its artificial intelligence and machine learning researchers to publish and share their work in papers, slightly pulling back the curtain on the company's famously secretive creation processes. Now, just a few weeks later, the first of those papers has been published, focusing on Apple's work in the intelligent image recognition field.

Titled "Learning from Simulated and Unsupervised Images through Adversarial Training," the paper describes a program that can intelligently decipher and understand digital images in a setting similar to the "Siri Intelligence" and facial recognition features introduced in Photos in iOS 10, but more advanced.

In the research, Apple notes the downsides and upsides of using real images compared with that of "synthetic," or computer images. Annotations must be added to real images, an "expensive and time-consuming task" that requires a human workforce to individually label objects in a picture. On the other hand, computer-generated images help to catalyze this process "because the annotations are automatically available."

Still, fully switching to synthetic images could lead to a dip in the quality of the program in question. This is because "synthetic data is often not realistic enough" and would lead to an end-user experience that only responded well to details present in the computer-generated images, while being unable to generalize well on any real-world objects and pictures it faced.

This leads to the paper's central proposition -- the combination of using both simulated and real images to work together in "adversarial training," creating an advanced AI image program:

In this paper, we propose Simulated+Unsupervised (S+U) learning, where the goal is to improve the realism of synthetic images from a simulator using unlabeled real data. The improved realism enables the training of better machine learning models on large datasets without any data collection or human annotation effort.

We show that this enables generation of highly realistic images, which we demonstrate both qualitatively and with a user study.

The rest of the paper goes into the details of Apple's research on the topic, including experiments that have been run and the math proposed to back up its findings. The paper's research focused solely on single images, but the team at Apple notes towards the end that it hopes to sometime soon "investigate refining videos" as well.

The credits on the paper go to Apple researchers Ashish Shrivastava, Tomas Pfister, Oncel Tuzel, Josh Susskind, Wenda Wang, and Russ Webb. The team's research was first submitted on November 15, but it didn't get published until December 22.

At the AI conference in Barcelona a few weeks ago, Apple head of machine learning Russ Salakhutdinov -- and a few other employees -- discussed topics including health and vital signs, volumetric detection of LiDAR, prediction with structured outputs, image processing and colorization, intelligent assistant and language modeling, and activity recognition. We'll likely see papers on a variety of these topics and more in the near future.

Top Rated Comments

Amacfa Avatar
77 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
Score: 40 Votes (Like | Disagree)
drewyboy Avatar
77 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
Ok, what? You mean like Apple spent more time on describing the new iMessage at WWDC than any other feature for iOS10? Clearly highlighting emoji as the flagship feature for iOS10? Oh, and lets not forget, they were too busy with emoji to realize they have a horrible battery bug in iOS10? I mean, using Apple's built in flashlight app shouldn't drain 10% of my battery for 4 minutes of use should it? Or shutting down when my phone is at 37% just yesterday only to plug it in and it be back at 37% then drain somewhat normal only to shut down again at 12%? And no, the health of my battery is just fine, or is the Apple Store lying to me when they checked. Or how about how they completely compromised the user experience of the new MBP by sacrificing 25% battery capacity to thin it down and make it lighter for a device that sits on a fixed surface for 99% of the users. Or how about Siri has gotten worse as time as gone on, while competitors get better and better each year?

So please, take your pick and lets have some "quality discussion". All I usually see is people offering validated criticism and then the other half defending apple as if it was their child and blaming the user. You're right, Apple can never do any wrong. They are always right and never wrong. Silly me, my messed up iPhone battery life is a new iOS feature, or is it because I'm using an iPhone 5S and as Phil said, I should be upgrading since it's ancient.

Edit: And as far as Photos go, maybe they should actually do something about families because their current "family share" features are a complete joke.
Score: 22 Votes (Like | Disagree)
samcan Avatar
77 months ago
I'm not sure if it is the competition getting better, but I feel as though Siri is getting dumber by the minute. Context requests are out of the question, it isn't current with sports anymore, and I've found myself being cut off with "sorry I didn't get that" while in a quiet room.

Siri stopped being a useful tool ever since they dropped "raise to speak". I find myself using my Pixel for anything requiring hands free.
Score: 19 Votes (Like | Disagree)
AngerDanger Avatar
77 months ago
If anybody wants to play around with AI image recognition, CloudSight ('http://cloudsight.ai/api') (scroll down to the Try it Out area) allows users to upload an image for recognition. It can be pretty cool to see how accurate its tagging is.



This image was described as "grey jar carton and bottle sketch" after uploading.

Attachment Image
Score: 19 Votes (Like | Disagree)
wigby Avatar
77 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
That's all I see here anymore...a race to critique Apple for making thin devices, requiring dongles and make fun of Siri. Oh and everyone gets bonus points for using the word "courage" in any post. Pathetic commenters.
Score: 18 Votes (Like | Disagree)
and 1989 others Avatar
77 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
When there's something of quality to write about, I'll write about it.

Until then, release joke products, receive joke replies.

Quid, pro, quo.
Score: 12 Votes (Like | Disagree)

Popular Stories

General Black Friday Deals 2022 Green

All the Apple Black Friday Deals You Can Still Get

Friday November 25, 2022 4:40 am PST by
Although Black Friday is now technically over, many Apple products are still seeing major discounts through the weekend as we head into Cyber Monday. In this article, you'll find every Apple device with a notable Black Friday sale that's still available. We'll be updating as prices change and new deals arrive, so be sure to keep an eye out if you don't see the sale you're looking for yet. Note:...
iphone 14 pro hands snowflakes 1

Best Cyber Monday iPhone Deals Available Today

Wednesday November 23, 2022 1:55 pm PST by
Cellular carriers have always offered big savings on the newest iPhone models during the holidays, and Cyber Monday is no different. We're tracking notable offers on the iPhone 14 and iPhone 14 Pro devices from AT&T, Verizon, and T-Mobile. For even more savings, keep an eye on older models like the iPhone 13. Note: MacRumors is an affiliate partner with some of these vendors. When you click a...
maxresdefault

Nothing Phone 1 Displays AirPods Battery Level After Latest OS Update

Friday November 25, 2022 3:33 am PST by
Nothing Phone 1 users today began receiving the Nothing OS 1.1.7 update, which adds support for displaying the battery percentage of connected AirPods, amongst other improvements and bug fixes. If you own a Nothing Phone 1, you can check for the OTA update by going to Settings -> System -> System updates. Bear in mind that as support for displaying AirPods battery level is still an...
ipad holiday bulbs

Best Cyber Monday iPad Deals Available Today

Thursday November 24, 2022 12:25 pm PST by
Cyber Monday deals have been in full swing since Black Friday deals ended, and we're seeing solid discounts on Apple devices. We're highlighting the best sales for all of Apple's product lines, and in this article you'll find the best Cyber Monday sales on iPad, iPad Pro, iPad Air, and iPad mini. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make ...
airpods pro 2

Apple Engineer Addresses Lack of Lossless Support on New AirPods Pro

Friday November 25, 2022 2:58 am PST by
An Apple engineer has addressed the lack of lossless audio support in the second-generation AirPods Pro in a new interview. Current Bluetooth technology in the AirPods lineup means that Apple's audio products do not support Apple Music Lossless audio. Apple has previously hinted that it may develop its own codec and connectivity standard that builds on AirPlay and supports higher quality...
Cyber Monday Deals Feature 2022

Best Cyber Monday Apple Deals Still Available for AirPods, Apple TV, iPad, and More

Monday November 28, 2022 5:24 am PST by
The Black Friday and Cyber Monday holiday shopping rush is drawing to a close, but there are still some good deals to be had out there. For Apple products, many of the deals you've seen since last week are still available, though some have expired. So for anyone who missed out on Black Friday deals, there's still an opportunity to get some of the year's best prices on many Apple devices. Note: ...
Apple Watch Ultra Oceanic Plus App

Apple Announces Oceanic+ App Now Available for Apple Watch Ultra

Monday November 28, 2022 6:11 am PST by
Apple today announced that the Oceanic+ app is available for the Apple Watch Ultra starting today. Designed by Huish Outdoors in collaboration with Apple, the app serves as a dive computer for recreational scuba diving at depths up to 40 meters/130 feet. Apple already offers a basic Depth app on the Apple Watch Ultra for viewing your current depth, maximum depth reached, water temperature,...
Three Biggest iPhone SE 4 Questions Feature

Three Biggest Questions About the iPhone SE 4

Saturday November 26, 2022 12:00 am PST by
While we already have some clear indications about what to expect from the fourth-generation iPhone SE, there are three major questions looming over the device at the current time. Chinese site MyDrivers and and leaker Jon Prosser believe that the iPhone SE is set to move to an iPhone XR-like design in its next incarnation, which would involve eliminating the Home button and adding a "notch" ...