Apple's AI Team Publishes First Research Paper Focused on Advanced Image Recognition

photos-iconEarlier in December, Apple announced that it would begin allowing its artificial intelligence and machine learning researchers to publish and share their work in papers, slightly pulling back the curtain on the company's famously secretive creation processes. Now, just a few weeks later, the first of those papers has been published, focusing on Apple's work in the intelligent image recognition field.

Titled "Learning from Simulated and Unsupervised Images through Adversarial Training," the paper describes a program that can intelligently decipher and understand digital images in a setting similar to the "Siri Intelligence" and facial recognition features introduced in Photos in iOS 10, but more advanced.

In the research, Apple notes the downsides and upsides of using real images compared with that of "synthetic," or computer images. Annotations must be added to real images, an "expensive and time-consuming task" that requires a human workforce to individually label objects in a picture. On the other hand, computer-generated images help to catalyze this process "because the annotations are automatically available."

Still, fully switching to synthetic images could lead to a dip in the quality of the program in question. This is because "synthetic data is often not realistic enough" and would lead to an end-user experience that only responded well to details present in the computer-generated images, while being unable to generalize well on any real-world objects and pictures it faced.

This leads to the paper's central proposition -- the combination of using both simulated and real images to work together in "adversarial training," creating an advanced AI image program:

In this paper, we propose Simulated+Unsupervised (S+U) learning, where the goal is to improve the realism of synthetic images from a simulator using unlabeled real data. The improved realism enables the training of better machine learning models on large datasets without any data collection or human annotation effort.

We show that this enables generation of highly realistic images, which we demonstrate both qualitatively and with a user study.

The rest of the paper goes into the details of Apple's research on the topic, including experiments that have been run and the math proposed to back up its findings. The paper's research focused solely on single images, but the team at Apple notes towards the end that it hopes to sometime soon "investigate refining videos" as well.

The credits on the paper go to Apple researchers Ashish Shrivastava, Tomas Pfister, Oncel Tuzel, Josh Susskind, Wenda Wang, and Russ Webb. The team's research was first submitted on November 15, but it didn't get published until December 22.

At the AI conference in Barcelona a few weeks ago, Apple head of machine learning Russ Salakhutdinov -- and a few other employees -- discussed topics including health and vital signs, volumetric detection of LiDAR, prediction with structured outputs, image processing and colorization, intelligent assistant and language modeling, and activity recognition. We'll likely see papers on a variety of these topics and more in the near future.

Top Rated Comments

Amacfa Avatar
57 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
Score: 40 Votes (Like | Disagree)
drewyboy Avatar
57 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
Ok, what? You mean like Apple spent more time on describing the new iMessage at WWDC than any other feature for iOS10? Clearly highlighting emoji as the flagship feature for iOS10? Oh, and lets not forget, they were too busy with emoji to realize they have a horrible battery bug in iOS10? I mean, using Apple's built in flashlight app shouldn't drain 10% of my battery for 4 minutes of use should it? Or shutting down when my phone is at 37% just yesterday only to plug it in and it be back at 37% then drain somewhat normal only to shut down again at 12%? And no, the health of my battery is just fine, or is the Apple Store lying to me when they checked. Or how about how they completely compromised the user experience of the new MBP by sacrificing 25% battery capacity to thin it down and make it lighter for a device that sits on a fixed surface for 99% of the users. Or how about Siri has gotten worse as time as gone on, while competitors get better and better each year?

So please, take your pick and lets have some "quality discussion". All I usually see is people offering validated criticism and then the other half defending apple as if it was their child and blaming the user. You're right, Apple can never do any wrong. They are always right and never wrong. Silly me, my messed up iPhone battery life is a new iOS feature, or is it because I'm using an iPhone 5S and as Phil said, I should be upgrading since it's ancient.

Edit: And as far as Photos go, maybe they should actually do something about families because their current "family share" features are a complete joke.
Score: 22 Votes (Like | Disagree)
samcan Avatar
57 months ago
I'm not sure if it is the competition getting better, but I feel as though Siri is getting dumber by the minute. Context requests are out of the question, it isn't current with sports anymore, and I've found myself being cut off with "sorry I didn't get that" while in a quiet room.

Siri stopped being a useful tool ever since they dropped "raise to speak". I find myself using my Pixel for anything requiring hands free.
Score: 19 Votes (Like | Disagree)
AngerDanger Avatar
57 months ago
If anybody wants to play around with AI image recognition, CloudSight ('http://cloudsight.ai/api') (scroll down to the Try it Out area) allows users to upload an image for recognition. It can be pretty cool to see how accurate its tagging is.



This image was described as "grey jar carton and bottle sketch" after uploading.

Attachment Image
Score: 19 Votes (Like | Disagree)
wigby Avatar
57 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
That's all I see here anymore...a race to critique Apple for making thin devices, requiring dongles and make fun of Siri. Oh and everyone gets bonus points for using the word "courage" in any post. Pathetic commenters.
Score: 18 Votes (Like | Disagree)
and 1989 others Avatar
57 months ago
Bunch of sad half witty replies fishing for likes. Try to post some quality discussion next time guys
When there's something of quality to write about, I'll write about it.

Until then, release joke products, receive joke replies.

Quid, pro, quo.
Score: 12 Votes (Like | Disagree)

Top Stories

tracking disabled ios 14 5

Analytics Suggest 96% of Users Leave App Tracking Disabled in iOS 14.5

Friday May 7, 2021 1:51 am PDT by
An early look at an ongoing analysis of Apple's App Tracking Transparency suggests that the vast majority of iPhone users are leaving app tracking disabled since the feature went live on April 26 with the release of iOS 14.5. According to the latest data from analytics firm Flurry, just 4% of iPhone users in the U.S. have actively chosen to opt into app tracking after updating their device...
tile amazon sidewalk integration

Tile to Leverage Amazon Echo and Ring Devices to Better Compete With AirTags

Friday May 7, 2021 2:07 pm PDT by
Amazon today announced that it is teaming up with Tile to add Amazon Sidewalk integration to Tile's Bluetooth trackers. Amazon Sidewalk, for those unfamiliar, is a network of Amazon Bluetooth devices that's designed to improve the connectivity of devices like the Ring and Amazon Echo. Tile will now be joining Amazon Sidewalk, and through this integration, Amazon Echo and Ring devices will be ...
Top Stories 59 Feature

Top Stories: Epic Games vs. Apple, Hidden AirTag Mode, Apple Music Hi-Fi, Colorful MacBook Air?

Saturday May 8, 2021 6:00 am PDT by
While we wait for the newly introduced iMac, iPad Pro, and Apple TV models to launch later this month, this week saw the kickoff of the big Epic Games v. Apple trial, with lots of juicy tidbits coming out as the two sides make their arguments. This week also saw some rumors about a Hi-Fi tier for Apple Music, more biometric sensing capabilities for Apple Watch, and timing for the...
airtag hacked

AirTag Successfully Hacked to Show Custom URL in Lost Mode

Monday May 10, 2021 1:52 am PDT by
The inevitable race to hack Apple's AirTag item tracker has reportedly been won by a German security researcher, who managed to break into the device's microcontroller and successfully modify its firmware. Yesss!!! After hours of trying (and bricking 2 AirTags) I managed to break into the microcontroller of the AirTag! 🥳🥳🥳/cc @colinoflynn @LennertWo pic.twitter.com/zGALc2S2Ph— stacksmashing ...
macbook colors 3d black bezels

Prosser: Next MacBook Air Could Come in Colors Similar to iMac

Friday May 7, 2021 6:55 am PDT by
According to Apple leaker Jon Prosser, Apple's upcoming release of the MacBook Air will feature various colors, similar to the colors in the newly released 24-inch iMac. In the latest video of his YouTube channel Front Page Tech, Prosser says the same source who accurately provided him information on the first Apple silicon iMac coming in colors has told him that he recently saw a prototype...
zoom app icon

Apple Gave Zoom Access to Special API to Use iPad Camera During Split View Multitasking

Sunday May 9, 2021 2:00 am PDT by
Zoom, a hallmark platform used by millions during the global health crisis, has been given access to a special iPadOS API that allows the app to use the iPad camera while the app is in use in Split View multitasking mode. This case of special treatment was first brought to attention by app developer Jeremy Provost, who, in a blog post, explains that Zoom uses a special API that allows the...
Apple 5G Modem Feature

Kuo: Apple-Designed 5G Modem May Debut in iPhones as Early as 2023

Sunday May 9, 2021 10:02 pm PDT by
Apple plans to adopt its own custom-designed 5G baseband chip starting with the 2023 iPhones, meaning it'll no longer need to rely on Qualcomm to supply the 5G cellular modem for the iPhone, Apple analyst Ming-Chi Kuo said today in an investors note obtained by MacRumors. According to Kuo, Apple plans to include its own custom-designed 5G baseband chip starting with the launch of the 2023...
3d printed airtag case siri remote

3D-Printed Case Lets You Attach an AirTag to Your Apple TV Remote

Monday May 10, 2021 8:11 am PDT by
Apple recently released a redesigned Siri Remote with a physical clickpad, but if you have an original Siri Remote laying around that you still plan on using, you may be interested in getting an AirTag case for the remote. Etsy user PrintSpiredDesigns has capitalized on the opportunity with a new 3D printed, made-to-order AirTag case for the original Siri Remote. The remote slides into the...
iPhone 13 Camera Backs

iPhone 13 Models Will Be Slightly Thicker and Will Have Larger Camera Bumps

Monday May 10, 2021 10:41 am PDT by
Apple's upcoming iPhone 13 models will be slightly thicker than the iPhone 12 models and will also feature larger, thicker camera bumps with lenses that protrude less, according to iPhone 13 schematics seen by MacRumors. The new iPhone 13 and 13 Pro models are expected to feature a thickness of 7.57mm, up from 7.4mm in the iPhone 12 models. That's an increase of 0.17mm, which won't be hugely ...
ifixit m1 macbook teardown

Leaked MacBook Schematics Have Aided Independent Technicians Performing Complicated Repairs

Monday May 10, 2021 8:51 am PDT by
Independent technicians have reportedly been able to use leaked MacBook schematics to aid repairs (via Vice). Last month, Apple supplier Quanta Computer was hit with an attack by the ransomware group known as "REvil," leading to a large quantity of internal Apple documents about both released and unreleased devices being exposed. The leaked PDFs, available on a number of forums and online...