In a lengthy blog post about Facebook's ten-year plan to "accelerate innovation and power new experiences with AI," chief technology officer Mike Schroepfer mentioned that one update coming to the company's mobile apps will be a "style transfer" tool that turns normal photos and videos into works of art using "high efficiency neural networks," all running directly on iOS and Android smartphones.

Facebook's impending update will, according to Schroepfer, run entirely on the user's smartphone and not be dependent on the content to be sent to servers, creating long load times and frustrating users. The CTO described this as the most demanding and "technically difficult" hurdle to clear in the process of adding the feature to the company's mobile apps, but Schroepfer said the company has done just that, and the result is a deep learning platform called "Caffe2Go."

facebook-art-filters

Just three months ago we set out to do something nobody else had done before: ship AI-based style transfer running live, in real time, on mobile devices. This was a major engineering challenge, as we needed to design software that could run high-powered computing operations on a device with unique resource constraints in areas like power, memory and compute capability. The result is Caffe2Go, a new deep learning platform that can capture, analyze and process pixels in real time on a mobile device.

We found that by condensing the size of the AI model used to process images and videos by 100x, we’re able to run deep neural networks with high efficiency on both iOS and Android. This is all happening in the palm of your hand, so you can apply styles to videos as you’re taking them.

Schroepfer said that the alternative of sending the content to data centers to be analyzed and filtered was "not ideal for letting people share fun content in the moment." In addition to basic image and video filtering capabilities, the new deep-learning platform will also feasibly be able to understand gesture controls when taking a selfie, for instance. In today's blog post, an example is given of a user swiping right and left between various artistic filters for a selfie (all running live, in real-time) and even snapping a picture when the user smiles.

As a comparison, the update sounds largely similar to Prisma, an app that launched over the summer and impressed many with its ability to turn photos, and eventually videos, into stylized images. Originally, Prisma used a server-side combination of neural networks and artificial intelligence to apply the different filters to user photos, but an update also introduced offline image processing, meaning users could use some of the app's filters to alter their images right from their smartphones.

For Facebook, the announcement follows a year of video-first announcements from the company, most recently embodied in CEO and co-founder Mark Zuckerberg's plan to make the camera more prominent in the app. No specific timeline was given for a possible launch of the new features on Facebook's mobile apps, but the company is clearly looking to lay the groundwork for its future, calling its new AI initiative, along with virtual reality, "new technologies that will shape the next decade."

Top Rated Comments

jmh600cbr Avatar
62 months ago
Facebook - the startup eater
Score: 3 Votes (Like | Disagree)
Zorn Avatar
62 months ago
Wow prisma filters! Sounds like a cool feature no one asked for. When will it stop looking like mud on the iPad Pro that's now a year old?
Score: 2 Votes (Like | Disagree)
-BigMac- Avatar
62 months ago
Yeh man keep beefing and slowing that sluggish app up. Why not a build in a music player?
Score: 1 Votes (Like | Disagree)
OldSchoolMacGuy Avatar
62 months ago
Prisma just added the ability to apply filters to Facebook live video on the iPhone 7 and 6S today.
Score: 1 Votes (Like | Disagree)
Hiran Avatar
62 months ago
But I deleted the official Facebook app and installed Friendly for iOS when the official app lost messenger feature. Mark Zuckerberg is dumb and arrogant as hell.
Score: 1 Votes (Like | Disagree)
smacrumon Avatar
62 months ago
Facebook is out to lunch. Their product is dated, the design is 2009 with a side of stale fries.
Score: 1 Votes (Like | Disagree)

Top Stories

REC ASA CODE2016 20160601 205816 2745

Elon Musk Reportedly Demanded to Become Apple CEO as Part of Potential Tesla Acquisition [Update: Musk Denies]

Friday July 30, 2021 9:04 am PDT by
Tesla CEO Elon Musk reportedly once demanded that he be made Apple CEO in a brief discussion of a potential acquisition with Apple's current CEO, Tim Cook. The claim comes in a new book titled "Power Play: Tesla, Elon Musk and the Bet of the Century," as reviewed by The Los Angeles Times. According to the book, during a 2016 phone call between Musk and Cook that touched on the possibility of ...
General Apps Messages

Android iMessage Competitor Puts Pressure on Apple

Friday July 30, 2021 3:15 am PDT by
Google and the three major U.S. carriers, including Verizon, AT&T, and T-Mobile, will all support a new communications protocol on Android smartphones starting in 2022, a move that puts pressure on Apple to adopt a new cross-platform messaging standard and may present a challenge to iMessage. Verizon recently announced that it is planning to adopt Messages by Google as its default messaging...
a15 chip

iPhone 13 and Redesigned MacBook Pro Chip Production Hit With Gas Contamination

Friday July 30, 2021 5:44 am PDT by
The most important TSMC factory that manufactures Apple's chips destined for next-generation iPhone and Mac models has been hit by a gas contamination, according to Nikkei Asia. The factory, known as "Fab 18," is TSMC's most advanced chipmaking facility. TSMC is Apple's sole chip supplier, making all of the processors used in every Apple device with a custom silicon chip. Industry...
Apple watch series 5 new case material made of titanium 091019

Titanium Apple Watch Series 6 Models Currently Widely Unavailable

Sunday August 1, 2021 6:21 am PDT by
Models of the Apple Watch Series 6 with titanium cases part of the "Apple Watch Edition" collection is currently widely unavailable for pick-up in several of Apple's retail stores in the United States and is unavailable entirely for delivery in major markets. Noted by Bloomberg's Mark Gurman in the latest edition of his "Power On" newsletter, titanium models of the Apple Watch Series 6,...
iPhone 13 Always On Feature

iPhone 13 to Bring Over a Major Feature From the Apple Watch

Wednesday July 28, 2021 2:21 am PDT by
Apple's upcoming iPhone 13 lineup will feature an always-on display akin to the Apple Watch Series 5 and Series 6, according to recent reports. In his weekly Power On newsletter, Bloomberg journalist Mark Gurman, who often reveals accurate insights into Apple's plans, said that the iPhone 13 may feature an Apple Watch-inspired always-on mode. The Apple Watch Series 5 and Apple Watch...
apple rtp land

Apple Preparing to Occupy 200,000 Square Feet of Temporary Space Ahead of New $1 Billion North Carolina Campus

Thursday July 29, 2021 9:14 am PDT by
Back in April, Apple announced a $430 billion investment over the next five years to create more than 20,000 new jobs as the company continues to expand. One significant piece of that plan is a new engineering and research center in North Carolina where Apple will be investing over $1 billion and hiring at least 3,000 employees. Assemblage of seven properties in Research Triangle Park owned by ...
telegram

Bumper Telegram Update Enables Video Calls With Up to 1,000 Viewers

Saturday July 31, 2021 12:13 am PDT by
Telegram Messenger has received a major update to its video capabilities, including support for video calls with up to 1,000 viewers. Group video calls in Telegram allow up to 30 users to stream video from both their camera and their screen, and now a maximum of 1,000 people can tune into the broadcast. Telegram says it intends to continue increasing this limit "until all humans on Earth...
app store blue banner

Elon Musk: Apple's App Store Fees Are a 'De Facto Global Tax on the Internet'

Friday July 30, 2021 10:04 am PDT by
Tesla CEO Elon Musk took to Twitter today to criticize Apple's App Store fees in a tweet that sides with Epic in the ongoing Epic v. Apple dispute. "Epic is right," wrote Musk, before going on to call Apple's App Store fees a "de facto global tax on the Internet." Musk earlier this week made veiled comments about App Store fees, but today's statement is a much more direct criticism....
FaceID iMac REREREREMIX

Top Stories: Face ID on Future Macs, Elon Musk Criticizes Apple, and More

Saturday July 31, 2021 6:00 am PDT by
This week saw an interesting range of Apple news and rumors, including a blockbuster earnings report, rumors about next year's "iPhone 14" and Face ID coming to Macs, and more. Subscribe to the MacRumors YouTube channel for more videos. Other popular topics included Apple's crackdown on leaks, changes in the latest round of betas for iOS 15, iPadOS 15, and macOS Monterey, and several stories...
duracell battery bitter coating

Apple Says Don't Buy AirTag Replacement Batteries With Bitter Coating

Wednesday July 28, 2021 11:08 am PDT by
Since AirTags were just released earlier this year and are expected to have a year-long battery life, it may be some time yet before AirTag users need a replacement battery, but when the time comes for a refresh, Apple is warning customers not to buy batteries with a bitter coating. AirTags use coin-shaped CR2032 batteries, which happen to be a size that's easy to swallow. Some battery...