Apple's AI Team Publishes First Research Paper Focused on Advanced Image Recognition

photos-iconEarlier in December, Apple announced that it would begin allowing its artificial intelligence and machine learning researchers to publish and share their work in papers, slightly pulling back the curtain on the company's famously secretive creation processes. Now, just a few weeks later, the first of those papers has been published, focusing on Apple's work in the intelligent image recognition field.

Titled "Learning from Simulated and Unsupervised Images through Adversarial Training," the paper describes a program that can intelligently decipher and understand digital images in a setting similar to the "Siri Intelligence" and facial recognition features introduced in Photos in iOS 10, but more advanced.

In the research, Apple notes the downsides and upsides of using real images compared with that of "synthetic," or computer images. Annotations must be added to real images, an "expensive and time-consuming task" that requires a human workforce to individually label objects in a picture. On the other hand, computer-generated images help to catalyze this process "because the annotations are automatically available."

Still, fully switching to synthetic images could lead to a dip in the quality of the program in question. This is because "synthetic data is often not realistic enough" and would lead to an end-user experience that only responded well to details present in the computer-generated images, while being unable to generalize well on any real-world objects and pictures it faced.

This leads to the paper's central proposition -- the combination of using both simulated and real images to work together in "adversarial training," creating an advanced AI image program:

In this paper, we propose Simulated+Unsupervised (S+U) learning, where the goal is to improve the realism of synthetic images from a simulator using unlabeled real data. The improved realism enables the training of better machine learning models on large datasets without any data collection or human annotation effort.

We show that this enables generation of highly realistic images, which we demonstrate both qualitatively and with a user study.

The rest of the paper goes into the details of Apple's research on the topic, including experiments that have been run and the math proposed to back up its findings. The paper's research focused solely on single images, but the team at Apple notes towards the end that it hopes to sometime soon "investigate refining videos" as well.

The credits on the paper go to Apple researchers Ashish Shrivastava, Tomas Pfister, Oncel Tuzel, Josh Susskind, Wenda Wang, and Russ Webb. The team's research was first submitted on November 15, but it didn't get published until December 22.

At the AI conference in Barcelona a few weeks ago, Apple head of machine learning Russ Salakhutdinov -- and a few other employees -- discussed topics including health and vital signs, volumetric detection of LiDAR, prediction with structured outputs, image processing and colorization, intelligent assistant and language modeling, and activity recognition. We'll likely see papers on a variety of these topics and more in the near future.

Top Rated Comments

A MacBook lover Avatar
95 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
Score: 40 Votes (Like | Disagree)
drewyboy Avatar
95 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
Ok, what? You mean like Apple spent more time on describing the new iMessage at WWDC than any other feature for iOS10? Clearly highlighting emoji as the flagship feature for iOS10? Oh, and lets not forget, they were too busy with emoji to realize they have a horrible battery bug in iOS10? I mean, using Apple's built in flashlight app shouldn't drain 10% of my battery for 4 minutes of use should it? Or shutting down when my phone is at 37% just yesterday only to plug it in and it be back at 37% then drain somewhat normal only to shut down again at 12%? And no, the health of my battery is just fine, or is the Apple Store lying to me when they checked. Or how about how they completely compromised the user experience of the new MBP by sacrificing 25% battery capacity to thin it down and make it lighter for a device that sits on a fixed surface for 99% of the users. Or how about Siri has gotten worse as time as gone on, while competitors get better and better each year?

So please, take your pick and lets have some "quality discussion". All I usually see is people offering validated criticism and then the other half defending apple as if it was their child and blaming the user. You're right, Apple can never do any wrong. They are always right and never wrong. Silly me, my messed up iPhone battery life is a new iOS feature, or is it because I'm using an iPhone 5S and as Phil said, I should be upgrading since it's ancient.

Edit: And as far as Photos go, maybe they should actually do something about families because their current "family share" features are a complete joke.
Score: 22 Votes (Like | Disagree)
samcan Avatar
95 months ago
I'm not sure if it is the competition getting better, but I feel as though Siri is getting dumber by the minute. Context requests are out of the question, it isn't current with sports anymore, and I've found myself being cut off with "sorry I didn't get that" while in a quiet room.

Siri stopped being a useful tool ever since they dropped "raise to speak". I find myself using my Pixel for anything requiring hands free.
Score: 19 Votes (Like | Disagree)
AngerDanger Avatar
95 months ago
If anybody wants to play around with AI image recognition, CloudSight ('http://cloudsight.ai/api') (scroll down to the Try it Out area) allows users to upload an image for recognition. It can be pretty cool to see how accurate its tagging is.



This image was described as "grey jar carton and bottle sketch" after uploading.

Attachment Image
Score: 19 Votes (Like | Disagree)
wigby Avatar
95 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
That's all I see here anymore...a race to critique Apple for making thin devices, requiring dongles and make fun of Siri. Oh and everyone gets bonus points for using the word "courage" in any post. Pathetic commenters.
Score: 18 Votes (Like | Disagree)
and 1989 others Avatar
95 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
When there's something of quality to write about, I'll write about it.

Until then, release joke products, receive joke replies.

Quid, pro, quo.
Score: 12 Votes (Like | Disagree)

Popular Stories

reset password request iphone

Warning: Apple Users Targeted in Phishing Attack Involving Rapid Password Reset Requests

Tuesday March 26, 2024 4:34 pm PDT by
Phishing attacks taking advantage of Apple's password reset feature have become increasingly common, according to a report from KrebsOnSecurity. Multiple Apple users have been targeted in an attack that bombards them with an endless stream of notifications or multi-factor authentication (MFA) messages in an attempt to cause panic so they'll respond favorably to social engineering. An...
Generic iOS 18 Feature Purple

iOS 18: What to Expect From 'Biggest' Update in iPhone's History

Wednesday March 27, 2024 11:10 am PDT by
At least some Apple software engineers continue to believe that iOS 18 will be the "biggest" update in the iPhone's history, according to Bloomberg's Mark Gurman. Below, we recap rumored features and changes for the iPhone. "The iOS 18 update is expected to be the most ambitious overhaul of the iPhone's software in its history, according to people working on the upgrade," wrote Gurman, in a r...
maxresdefault

Apple Announces WWDC 2024 Event for June 10 to 14

Tuesday March 26, 2024 10:02 am PDT by
Apple today announced that its 35th annual Worldwide Developers Conference is set to take place from Monday, June 10 to Friday, June 14. As with WWDC events since 2020, WWDC 2024 will be an online event that is open to all developers at no cost. Subscribe to the MacRumors YouTube channel for more videos. WWDC 2024 will include online sessions and labs so that developers can learn about new...
apple maps 3d feature

Apple Maps May Gain Custom Routes With iOS 18

Tuesday March 26, 2024 3:10 pm PDT by
Apple may be planning to add support for "custom routes" in Apple Maps in iOS 18, according to code reviewed by MacRumors. Apple Maps does not currently offer a way to input self-selected routes, with Maps users limited to Apple's pre-selected options, but that may change in iOS 18. Apple has pushed an iOS 18 file to its maps backend labeled "CustomRouteCreation." While not much is revealed...
General iOS 17 Feature Orange Purple

Apple Releases Revised Versions of iOS 17.4.1 and iPadOS 17.4.1 With Updated Build Number

Wednesday March 27, 2024 5:59 am PDT by
Apple on late Tuesday released revised versions of iOS 17.4.1 and iPadOS 17.4.1 with an updated build number of 21E237, according to MacRumors contributor Aaron Perris. The updates previously had a build number of 21E236. The revised updates are available for all iPhone and iPad models that are compatible with iOS 17 and iPadOS 17, but they can only be installed via the Finder app on macOS...
maxresdefault

Apple to Launch New iPad Pro and iPad Air Models in May

Thursday March 28, 2024 11:07 am PDT by
Apple will introduce new iPad Pro and iPad Air models in early May, according to Bloomberg's Mark Gurman. Gurman previously suggested the new iPads would come out in March, and then April, but the timeline has been pushed back once again. Subscribe to the MacRumors YouTube channel for more videos. Apple is working on updates to both the iPad Pro and iPad Air models. The iPad Pro models will...
applephilschiller

Apple's Phil Schiller Works 80 Hours a Week Overseeing App Store

Wednesday March 27, 2024 2:03 pm PDT by
With the App Store and app ecosystem undergoing major changes in the European Union, The Wall Street Journal today shared a profile on App Store chief Phil Schiller, who is responsible for the App Store. Though Schiller transitioned from marketing chief to "Apple Fellow" in 2020 to take a step back from Apple and spend more time on personal projects and friends, he is reportedly working...