In a lengthy blog post about Facebook's ten-year plan to "accelerate innovation and power new experiences with AI," chief technology officer Mike Schroepfer mentioned that one update coming to the company's mobile apps will be a "style transfer" tool that turns normal photos and videos into works of art using "high efficiency neural networks," all running directly on iOS and Android smartphones.

Facebook's impending update will, according to Schroepfer, run entirely on the user's smartphone and not be dependent on the content to be sent to servers, creating long load times and frustrating users. The CTO described this as the most demanding and "technically difficult" hurdle to clear in the process of adding the feature to the company's mobile apps, but Schroepfer said the company has done just that, and the result is a deep learning platform called "Caffe2Go."

facebook-art-filters

Just three months ago we set out to do something nobody else had done before: ship AI-based style transfer running live, in real time, on mobile devices. This was a major engineering challenge, as we needed to design software that could run high-powered computing operations on a device with unique resource constraints in areas like power, memory and compute capability. The result is Caffe2Go, a new deep learning platform that can capture, analyze and process pixels in real time on a mobile device.

We found that by condensing the size of the AI model used to process images and videos by 100x, we’re able to run deep neural networks with high efficiency on both iOS and Android. This is all happening in the palm of your hand, so you can apply styles to videos as you’re taking them.

Schroepfer said that the alternative of sending the content to data centers to be analyzed and filtered was "not ideal for letting people share fun content in the moment." In addition to basic image and video filtering capabilities, the new deep-learning platform will also feasibly be able to understand gesture controls when taking a selfie, for instance. In today's blog post, an example is given of a user swiping right and left between various artistic filters for a selfie (all running live, in real-time) and even snapping a picture when the user smiles.

As a comparison, the update sounds largely similar to Prisma, an app that launched over the summer and impressed many with its ability to turn photos, and eventually videos, into stylized images. Originally, Prisma used a server-side combination of neural networks and artificial intelligence to apply the different filters to user photos, but an update also introduced offline image processing, meaning users could use some of the app's filters to alter their images right from their smartphones.

For Facebook, the announcement follows a year of video-first announcements from the company, most recently embodied in CEO and co-founder Mark Zuckerberg's plan to make the camera more prominent in the app. No specific timeline was given for a possible launch of the new features on Facebook's mobile apps, but the company is clearly looking to lay the groundwork for its future, calling its new AI initiative, along with virtual reality, "new technologies that will shape the next decade."

Top Rated Comments

jmh600cbr Avatar
58 months ago
Facebook - the startup eater
Score: 3 Votes (Like | Disagree)
Zorn Avatar
58 months ago
Wow prisma filters! Sounds like a cool feature no one asked for. When will it stop looking like mud on the iPad Pro that's now a year old?
Score: 2 Votes (Like | Disagree)
-BigMac- Avatar
58 months ago
Yeh man keep beefing and slowing that sluggish app up. Why not a build in a music player?
Score: 1 Votes (Like | Disagree)
OldSchoolMacGuy Avatar
58 months ago
Prisma just added the ability to apply filters to Facebook live video on the iPhone 7 and 6S today.
Score: 1 Votes (Like | Disagree)
Hiran Avatar
58 months ago
But I deleted the official Facebook app and installed Friendly for iOS when the official app lost messenger feature. Mark Zuckerberg is dumb and arrogant as hell.
Score: 1 Votes (Like | Disagree)
smacrumon Avatar
58 months ago
Facebook is out to lunch. Their product is dated, the design is 2009 with a side of stale fries.
Score: 1 Votes (Like | Disagree)

Top Stories

april 2021 event coverage feature

Apple Event Live Coverage: New iPads, AirTags, and More Expected [Event Over]

Tuesday April 20, 2021 9:07 am PDT by
Apple's virtual "Spring Loaded" event kicks off today at 10:00 a.m. Pacific Time, with Apple expected to debut updated iPad models and perhaps some other hardware such as AirTags or iMac models based on Apple silicon. Apple is providing a live video stream on its website, on YouTube, and in the company's TV app across its platforms. We will also be updating this article with live blog...
m1 imac colors

Apple Announces Redesigned iMac With M1 Chip and Seven Color Options

Tuesday April 20, 2021 10:22 am PDT by
Apple has announced a new, redesigned 24-inch iMac, featuring an M1 chip, a 4.5K display, and a range of color options, as well as an improved cooling system, front-facing camera, speaker system, microphones, power connector, and peripherals. The new iMac features a completely new compact design, and comes in a range of seven striking colors, including green, yellow, orange, pink, purple,...
duan rui iphone 12 13 notch

New Images Show Smaller iPhone 13 Notch Compared to iPhone 12

Saturday April 17, 2021 11:38 pm PDT by
Leaker known as "DuanRui" has shared more images that could give us our best look yet at Apple's redesigned notch for the iPhone 13. The new pictures follow similar images shared by the leaker last week, but the latest shots include a comparison with the existing iPhone 12 notch. DuanRui posted three images on Twitter that apparently originate from Weibo, although source details remain...
Top Stories 57 Feature

Top Stories: Apple Event Next Tuesday, Mini-LED iPad Pro, iPhone Rumors

Saturday April 17, 2021 6:00 am PDT by
It feels like we've been waiting forever for new Apple products, but the wait is almost over as Apple has announced a media event for next Tuesday, so make sure to tune into MacRumors for full coverage of everything Apple announces. While that was the big news this week, we also got some new details on Apple's iPhone plans for 2022 and 2023 courtesy of analyst Ming-Chi Kuo, and we also saw...
iphone 12 preorder purple

Apple Launching iPhone 12 and 12 Mini in New Purple Color on April 30

Tuesday April 20, 2021 10:08 am PDT by
Apple today announced that the iPhone 12 and iPhone 12 mini will be available in a new purple color starting April 30, with pre-orders starting this Friday. Apple is also releasing a new MagSafe Leather Case and Leather Sleeve in Deep Violet, a Silicone Case in Capri Blue, Pistachio, Cantaloupe, or Amethyst, and a Leather Wallet in Arizona, all available to order beginning today. iPhone...
f1618938547

Apple Announces AirTag Tracking Devices Starting At $29 Each

Tuesday April 20, 2021 10:10 am PDT by
Apple today announced AirTag, a Tile-like Bluetooth tracking device that's designed to be attached to items like keys and wallets for tracking purposes, letting you find them right in the Find My app. AirTags are accessories for attaching to backpacks, luggage, and other items. Any U1 device like the iPhone 12 can be used for precision finding to guide you right to the item you're looking...
iPad Pro

New 12.9-Inch iPad Pro Will Be 0.5mm Thicker to Accommodate Mini-LED Display

Monday April 19, 2021 11:30 am PDT by
The upcoming 12.9-inch iPad Pro will be thicker than the previous-generation version, likely due to the inclusion of the mini-LED display. We've heard several rumors about the change in thickness, and now leaked design images have confirmed it. A source that designs accessories for Apple devices sent MacRumors a series of photos that feature exact dimensions for the new iPad Pro models, and...
ipad pro with m1 chip

Apple Introduces Next-Generation iPad Pro With M1 Chip, Thunderbolt, 5G, XDR Display, and More

Tuesday April 20, 2021 10:40 am PDT by
Apple today announced the next-generation iPad Pro with the same M1 chip found in the latest Macs, Thunderbolt and USB4 support, 5G connectivity on cellular models with mmWave support in the United States, and more. With an 8-core CPU and 8-core GPU, Apple says the M1 chip in the new iPad Pro provides up to 50% faster performance and up to 40% faster graphics compared to the A12Z Bionic chip ...
iPad Pro Feature Orange

Wedbush Analysts Say 'Spring Loaded' Event Will Debut New iPads With 'Modest Price Increase,' Along With 'a Few Surprises'

Monday April 19, 2021 6:37 am PDT by
Apple is planning to launch a new entry-level iPad, iPad mini, and iPad Pro at its "Spring Loaded" event tomorrow, along with "a few surprises," according to Wedbush analysts. In a new note to investors, seen by MacRumors, Wedbush analysts Daniel Ives and Strecker Backe explained that the iPad will be the main focus of Apple's "Spring Loaded" event, with new entry-level iPad, iPad mini, and ...
flat imac 3d 3 teal

Reliable Leaker Hints Redesigned Colorful iMac to Debut at 'Spring Loaded' Event

Saturday April 17, 2021 4:43 am PDT by
Reliable leaker known as l0vetodream has hinted that Apple may debut its rumored redesigned and colorful iMac at its "Spring Loaded" event on Tuesday, April 20. In a tweet, the leaker posted an image of Apple's logo used for marketing the upcoming event and an image of the retro rainbow Apple logo alongside the colorful lineup of G3 iMacs. Apple leaker Jon Prosser previously reported that...