In a lengthy blog post about Facebook's ten-year plan to "accelerate innovation and power new experiences with AI," chief technology officer Mike Schroepfer mentioned that one update coming to the company's mobile apps will be a "style transfer" tool that turns normal photos and videos into works of art using "high efficiency neural networks," all running directly on iOS and Android smartphones.

Facebook's impending update will, according to Schroepfer, run entirely on the user's smartphone and not be dependent on the content to be sent to servers, creating long load times and frustrating users. The CTO described this as the most demanding and "technically difficult" hurdle to clear in the process of adding the feature to the company's mobile apps, but Schroepfer said the company has done just that, and the result is a deep learning platform called "Caffe2Go."

facebook-art-filters

Just three months ago we set out to do something nobody else had done before: ship AI-based style transfer running live, in real time, on mobile devices. This was a major engineering challenge, as we needed to design software that could run high-powered computing operations on a device with unique resource constraints in areas like power, memory and compute capability. The result is Caffe2Go, a new deep learning platform that can capture, analyze and process pixels in real time on a mobile device.

We found that by condensing the size of the AI model used to process images and videos by 100x, we’re able to run deep neural networks with high efficiency on both iOS and Android. This is all happening in the palm of your hand, so you can apply styles to videos as you’re taking them.

Schroepfer said that the alternative of sending the content to data centers to be analyzed and filtered was "not ideal for letting people share fun content in the moment." In addition to basic image and video filtering capabilities, the new deep-learning platform will also feasibly be able to understand gesture controls when taking a selfie, for instance. In today's blog post, an example is given of a user swiping right and left between various artistic filters for a selfie (all running live, in real-time) and even snapping a picture when the user smiles.

As a comparison, the update sounds largely similar to Prisma, an app that launched over the summer and impressed many with its ability to turn photos, and eventually videos, into stylized images. Originally, Prisma used a server-side combination of neural networks and artificial intelligence to apply the different filters to user photos, but an update also introduced offline image processing, meaning users could use some of the app's filters to alter their images right from their smartphones.

For Facebook, the announcement follows a year of video-first announcements from the company, most recently embodied in CEO and co-founder Mark Zuckerberg's plan to make the camera more prominent in the app. No specific timeline was given for a possible launch of the new features on Facebook's mobile apps, but the company is clearly looking to lay the groundwork for its future, calling its new AI initiative, along with virtual reality, "new technologies that will shape the next decade."

Top Rated Comments

jmh600cbr Avatar
112 months ago
Facebook - the startup eater
Score: 3 Votes (Like | Disagree)
Zorn Avatar
112 months ago
Wow prisma filters! Sounds like a cool feature no one asked for. When will it stop looking like mud on the iPad Pro that's now a year old?
Score: 2 Votes (Like | Disagree)
-BigMac- Avatar
112 months ago
Yeh man keep beefing and slowing that sluggish app up. Why not a build in a music player?
Score: 1 Votes (Like | Disagree)
OldSchoolMacGuy Avatar
112 months ago
Prisma just added the ability to apply filters to Facebook live video on the iPhone 7 and 6S today.
Score: 1 Votes (Like | Disagree)
Hiran Avatar
112 months ago
But I deleted the official Facebook app and installed Friendly for iOS when the official app lost messenger feature. Mark Zuckerberg is dumb and arrogant as hell.
Score: 1 Votes (Like | Disagree)
smacrumon Avatar
112 months ago
Facebook is out to lunch. Their product is dated, the design is 2009 with a side of stale fries.
Score: 1 Votes (Like | Disagree)

Popular Stories

iPhone 17 Pro Blue Feature Tighter Crop

iPhone 17 Pro Launching in Three Months With These 12 New Features

Saturday June 14, 2025 5:45 pm PDT by
The iPhone 17 Pro and iPhone 17 Pro Max are three months away, and there are plenty of rumors about the devices. Below, we recap key changes rumored for the iPhone 17 Pro models as of June 2025:Aluminum frame: iPhone 17 Pro models are rumored to have an aluminum frame, whereas the iPhone 15 Pro and iPhone 16 Pro models have a titanium frame, and the iPhone X through iPhone 14 Pro have a...
apple watch ultra 2 new black

Apple Watch Ultra 3 Finally Coming After Two-Year Hiatus

Monday June 16, 2025 8:45 am PDT by
Apple will finally deliver the Apple Watch Ultra 3 sometime this year, according to analyst Jeff Pu of GF Securities Hong Kong (via @jukanlosreve). The analyst expects both the Apple Watch Series 11 and Apple Watch Ultra 3 to arrive this year (likely alongside the new iPhone 17 lineup, if previous launches are anything to go by), according to his latest product roadmap shared with...
apple watch ultra snow

6 Features Coming to the Apple Watch Ultra 3

Tuesday February 25, 2025 9:00 am PST by
The Apple Watch Ultra 3 is expected to launch later this year, arriving two years after the previous model with a series of improvements. While no noticeable design changes are expected for the third generation since the company tends to stick with the same Apple Watch design through three generations before changing it, there are a series of internal upgrades on the way. By the time the ...
terminal macos tahoe

Apple's Terminal App Gets Colorful Redesign in macOS Tahoe

Monday June 16, 2025 4:12 am PDT by
Apple's Terminal app is getting a visual refresh in macOS Tahoe, and it's the first notable design update since the command-line tool debuted. The updated Terminal will support 24-bit color and Powerline fonts, according to Apple's State of the Platforms presentation at WWDC25. The app will also adopt the new Liquid Glass aesthetic with redesigned themes that align with macOS 26's broader...
Logitech Logo Feature

Logitech Announces Two New Accessories for WWDC

Friday June 13, 2025 7:22 am PDT by
Alongside WWDC this week, Logitech announced notable new accessories for the iPad and Apple Vision Pro. The Logitech Muse is a spatially-tracked stylus developed for use with the Apple Vision Pro. Introduced during the WWDC 2025 keynote address, Muse is intended to support the next generation of spatial computing workflows enabled by visionOS 26. The device incorporates six degrees of...
iPadOS 26 App Windowing

Apple Explains Why iPads Don't Just Run macOS

Friday June 13, 2025 7:46 am PDT by
iPadOS 26 allows iPads to function much more like Macs, with a new app windowing system, a swipe-down menu bar at the top of the screen, and more. However, Apple has stopped short of allowing iPads to run macOS, and it has now explained why. In an interview this week with Swiss tech journalist Rafael Zeier, Apple's software engineering chief Craig Federighi said that iPadOS 26's new Mac-like ...
iphone 16 pro models 1

17 Reasons to Wait for the iPhone 17

Thursday June 12, 2025 8:58 am PDT by
Apple's iPhone development roadmap runs several years into the future and the company is continually working with suppliers on several successive iPhone models simultaneously, which is why we often get rumored features months ahead of launch. The iPhone 17 series is no different, and we already have a good idea of what to expect from Apple's 2025 smartphone lineup. If you skipped the iPhone...