Facebook iOS App Will Soon Gain Prisma-Like Art Filters for Photos and Videos

by

In a lengthy blog post about Facebook's ten-year plan to "accelerate innovation and power new experiences with AI," chief technology officer Mike Schroepfer mentioned that one update coming to the company's mobile apps will be a "style transfer" tool that turns normal photos and videos into works of art using "high efficiency neural networks," all running directly on iOS and Android smartphones.

Facebook's impending update will, according to Schroepfer, run entirely on the user's smartphone and not be dependent on the content to be sent to servers, creating long load times and frustrating users. The CTO described this as the most demanding and "technically difficult" hurdle to clear in the process of adding the feature to the company's mobile apps, but Schroepfer said the company has done just that, and the result is a deep learning platform called "Caffe2Go."

facebook-art-filters

Just three months ago we set out to do something nobody else had done before: ship AI-based style transfer running live, in real time, on mobile devices. This was a major engineering challenge, as we needed to design software that could run high-powered computing operations on a device with unique resource constraints in areas like power, memory and compute capability. The result is Caffe2Go, a new deep learning platform that can capture, analyze and process pixels in real time on a mobile device.

We found that by condensing the size of the AI model used to process images and videos by 100x, we’re able to run deep neural networks with high efficiency on both iOS and Android. This is all happening in the palm of your hand, so you can apply styles to videos as you’re taking them.

Schroepfer said that the alternative of sending the content to data centers to be analyzed and filtered was "not ideal for letting people share fun content in the moment." In addition to basic image and video filtering capabilities, the new deep-learning platform will also feasibly be able to understand gesture controls when taking a selfie, for instance. In today's blog post, an example is given of a user swiping right and left between various artistic filters for a selfie (all running live, in real-time) and even snapping a picture when the user smiles.

As a comparison, the update sounds largely similar to Prisma, an app that launched over the summer and impressed many with its ability to turn photos, and eventually videos, into stylized images. Originally, Prisma used a server-side combination of neural networks and artificial intelligence to apply the different filters to user photos, but an update also introduced offline image processing, meaning users could use some of the app's filters to alter their images right from their smartphones.

For Facebook, the announcement follows a year of video-first announcements from the company, most recently embodied in CEO and co-founder Mark Zuckerberg's plan to make the camera more prominent in the app. No specific timeline was given for a possible launch of the new features on Facebook's mobile apps, but the company is clearly looking to lay the groundwork for its future, calling its new AI initiative, along with virtual reality, "new technologies that will shape the next decade."

Top Rated Comments

(View all)
Avatar
52 months ago
Facebook - the startup eater
Score: 3 Votes (Like | Disagree)
Avatar
52 months ago
Wow prisma filters! Sounds like a cool feature no one asked for. When will it stop looking like mud on the iPad Pro that's now a year old?
Score: 2 Votes (Like | Disagree)
Avatar
52 months ago
Yeh man keep beefing and slowing that sluggish app up. Why not a build in a music player?
Score: 1 Votes (Like | Disagree)
Avatar
52 months ago
Prisma just added the ability to apply filters to Facebook live video on the iPhone 7 and 6S today.
Score: 1 Votes (Like | Disagree)
Avatar
52 months ago
But I deleted the official Facebook app and installed Friendly for iOS when the official app lost messenger feature. Mark Zuckerberg is dumb and arrogant as hell.
Score: 1 Votes (Like | Disagree)
Avatar
52 months ago
Facebook is out to lunch. Their product is dated, the design is 2009 with a side of stale fries.
Score: 1 Votes (Like | Disagree)

Top Stories

First Impressions From New iPhone 12 and 12 Pro Owners

Thursday October 22, 2020 4:20 pm PDT by
It's already Friday, October 23, in Australia and New Zealand, which means some customers who purchased an iPhone 12 or 12 Pro already have their new devices in hand. We've seen dozens of reviews of the iPhone 12 and iPhone 12 Pro from media sites, but now first impressions from regular Apple customers are available. Image via MacRumors reader Boardiesboi New iPhone 12 and 12 Pro owners are...

Early iPhone 12 Tests Show Ceramic Shield is Stronger and More Scratch Resistant Than iPhone 11 Glass

Friday October 23, 2020 1:21 pm PDT by
Apple's new iPhone 12 models are protected by a Ceramic Shield cover glass that has nano-ceramic crystals infused right into the glass to improve durability. According to Apple, Ceramic Shield offers four times better drop protection than the glass used for the iPhone 11 models. YouTube channel MobileReviewsEh conducted some tests on the iPhone 12 using a force meter to compare its performance ...

Apple Distributing New Heated Display Removal Machine for iPhone 12 Repairs

Thursday October 22, 2020 6:20 pm PDT by
Apple is providing Genius Bars and Apple Authorized Service Providers with a new heated display removal fixture for iPhone 12 and iPhone 12 Pro repairs, according to information obtained by MacRumors from a reliable source. To open iPhone 12 models, technicians will be required to slide the device into a specialized tray, and then place the tray into the high-temperature fixture for two...

Apple VP Kaiann Drance Interview Addresses Battery Life, MagSafe, and Power Adapter Concerns

Friday October 23, 2020 3:37 am PDT by
Apple's Vice President of iPhone Marketing, Kaiann Drance, has provided a new interview to Rich DeMuro on the Rich on Tech Podcast, to discuss the iPhone 12 and iPhone 12 Pro. Although much of the interview repeated points from Apple's "Hi, Speed" event, there were a number of interesting tidbits regarding the affect of 5G on battery life, MagSafe concerns, and the lack of a power adapter in...

Teardown Video Confirms iPhone 12 and iPhone 12 Pro Use Same 2,815mAh Battery

Thursday October 22, 2020 9:47 am PDT by
With the iPhone 12 launching on Friday and in just a few hours to Australia and New Zealand, hands-on videos, teardowns, reviews, and other iPhone-related content has been coming out. A new teardown video delves into both the iPhone 12 and the 12 Pro, confirming battery life for both models and giving us a closer look at their internals. The video from Io Technology is in Chinese, but ...

Images of Supposed AirPods Diagnosis Tool Shared Online

Thursday October 22, 2020 5:24 am PDT by
Apple is reportedly rolling out a new tool to Apple service providers for testing AirPods, according to leaker known as "Fudge," who shared images of the tool on Twitter. Apple appears to be seeking to reduce unnecessary AirPods services by more accurately diagnosing the cause of a fault. Instances of a dirt-blockage, which may be difficult to ascertain visually, can apparently be...

New Photos Offer Better Look at iPhone 12 Color Options

Tuesday October 20, 2020 2:34 am PDT by
As we wait for the iPhone 12 review embargo to lift later today, more pictures are circulating of the devices in real-world lighting conditions, providing a better look at the different colors available. Leaker DuanRui has shared images on Twitter of the iPhone 12 in white, black, blue, green, and (PRODUCT)RED. The black and white colors are similar to the iPhone 11 colors, but the other...

Apple's AirTags Revealed in Newly Published Patent Applications

Thursday October 22, 2020 9:13 am PDT by
Two patent applications filed by Apple appear to depict the company's widely expected AirTags item trackers (via Patently Apple). The filings, which include a large number of images, are titled "Mounting Base for a Wirelessly Locatable Tag" and "Fastener with a Constrained Retention Ring," and describe a wirelessly locatable tag that can be used to determine the absolute location of an...

iPhone 11 Pro Outlasts iPhone 12 and 12 Pro in Extensive Battery Life Test

Friday October 23, 2020 8:36 am PDT by
Arun Maini today shared a new side-by-side iPhone battery life video test on his YouTube channel Mrwhosetheboss, timing how long the new iPhone 12 and iPhone 12 Pro models last on a single charge compared to older models, with equal brightness, settings, battery health, and usage. All of the devices are running iOS 14 without a SIM card inserted. In the test, the iPhone 11 Pro outlasted both ...

Apple Warns MagSafe Charger Can Leave Circular Imprints on Leather Cases

Friday October 23, 2020 3:23 pm PDT by
If you keep your iPhone in a leather case while charging with Apple's new MagSafe Charger, the case might show circular imprints from contact with the accessory, according to a new Apple support document published today. Apple's leather cases for the iPhone 12 and iPhone 12 Pro are not available until November 6, but a MacRumors reader has already shared a photo of a circular imprint on...