University Researchers Who Built a CSAM Scanning System Urge Apple to Not Use the 'Dangerous' Technology

Respected university researchers are sounding the alarm bells over the technology behind Apple's plans to scan iPhone users' photo libraries for CSAM, or child sexual abuse material, calling the technology "dangerous."

apple privacy
Jonanath Mayer, an assistant professor of computer science and public affairs at Princeton University, as well as Anunay Kulshrestha, a researcher at Princeton University Center for Information Technology Policy, both penned an op-ed for The Washington Post, outlining their experiences with building image detection technology.

The researchers started a project two years ago to identity CSAM in end-to-end encrypted online services. The researchers note that given their field, they "know the value of end-to-end encryption, which protects data from third-party access." That concern, they say, is what horrifies them over CSAM "proliferating on encrypted platforms."

Mayer and Kulshrestha said they wanted to find a middle ground for the situation: build a system that online platforms could use to find CSAM and protect end-to-end encryption. The researchers note that experts in the field doubted the prospect of such a system, but they did manage to build it and in the process noticed a significant problem.

We sought to explore a possible middle ground, where online services could identify harmful content while otherwise preserving end-to-end encryption. The concept was straightforward: If someone shared material that matched a database of known harmful content, the service would be alerted. If a person shared innocent content, the service would learn nothing. People couldn't read the database or learn whether content matched, since that information could reveal law enforcement methods and help criminals evade detection.

Knowledgeable observers argued a system like ours was far from feasible. After many false starts, we built a working prototype. But we encountered a glaring problem.

Since Apple's announcement of the feature, the company has been bombarded with concerns that the system behind detecting CSAM could be used to detect other forms of photos at the request of oppressive governments. Apple has strongly pushed back against such a possibility, saying it will refuse any such request from governments.

Nonetheless, concerns over the future implications of the technology being used for CSAM detection are widespread. Mayer and Kulshrestha said that their concerns over how governments could use the system to detect content other than CSAM had them "disturbed."

A foreign government could, for example, compel a service to out people sharing disfavored political speech. That's no hypothetical: WeChat, the popular Chinese messaging app, already uses content matching to identify dissident material. India enacted rules this year that could require pre-screening content critical of government policy. Russia recently fined Google, Facebook and Twitter for not removing pro-democracy protest materials.

We spotted other shortcomings. The content-matching process could have false positives, and malicious users could game the system to subject innocent users to scrutiny.

We were so disturbed that we took a step we hadn't seen before in computer science literature: We warned against our own system design, urging further research on how to mitigate the serious downsides....

Apple has continued to address user concerns over its plans, publishing additional documents and an FAQ page. Apple continues to believe that its CSAM detection system, which will occur on a user's device, aligns with its long-standing privacy values.

Top Rated Comments

Geert76 Avatar
19 months ago
Apple, just cancel this on device surveillance software
Score: 166 Votes (Like | Disagree)
nvmls Avatar
19 months ago
They clearly don't know how the technology works.. oh wait
Score: 125 Votes (Like | Disagree)
nexu Avatar
19 months ago
This is getting more and more disturbing
Score: 117 Votes (Like | Disagree)
TheRealNick Avatar
19 months ago
If they don’t cancel this I’m seriously going to have to look at alternative products, which saddens me.
Score: 108 Votes (Like | Disagree)
dwsolberg Avatar
19 months ago
It's time for Apple to admit it's made a mistake. It's seriously bizarre they ever thought this was a good idea.
Score: 95 Votes (Like | Disagree)
VulchR Avatar
19 months ago
Waiting for the surveillance apologists to argue these two researchers "don't understand" the process...
Score: 93 Votes (Like | Disagree)

Popular Stories

Multi Display CarPlay 1

Apple Launching All-New CarPlay Experience Later This Year With These 5 Features

Sunday January 29, 2023 10:15 am PST by
In June 2022, Apple previewed the next generation of CarPlay, promising deeper integration with vehicle functions like A/C and FM radio, support for multiple displays across the dashboard, personalization options, and more. Apple says the first vehicles with support for the next-generation CarPlay experience will be announced in late 2023, with committed automakers including Acura, Audi,...
iphone 15 pro wifi 6e

Internal Apple Document From Leaker 'Unknownz21' Confirms Wi-Fi 6E Will Be Limited to iPhone 15 Pro Models

Friday January 27, 2023 10:01 am PST by
Multiple rumors have suggested that the next-generation iPhone 15 models will adopt the Wi-Fi 6E standard that Apple has already introduced in the iPad Pro and MacBook Pro, and now a leaked document appears to confirm Apple's plans. Sourced from researcher and Apple leaker Unknownz21 (@URedditor), the document features diagrams of the iPhone 15's antenna architecture. D8x refers to the...
iPhone 14 Pro Purple Side Perspective Feature Purple

iPhone 15 Pro Rumored to Have These 8 Features

Friday January 27, 2023 2:11 pm PST by
Apple's next-generation iPhone 15 Pro and iPhone 15 Pro Max are expected to be announced in September as usual. Already, rumors suggest the devices will have at least eight exclusive features not available on the standard iPhone 15 and iPhone 15 Plus. An overview of the eight features rumored to be exclusive to iPhone 15 Pro models:A17 chip: iPhone 15 Pro models will be equipped with an A17...
top stories 28jan2023

Top Stories: iOS 16.3 Released, iPhone 15 Pro Rumors, macOS Tips and Tricks, and More

Saturday January 28, 2023 6:00 am PST by
Following last week's hardware announcements, this week saw the actual release of several of the new products as well as operating system updates bringing new features and bug fixes across Apple's platforms. This week also saw some fresh rumors about the iPhone 15 lineup and Apple's upcoming AR/VR headset, while we shared some tips to help you get the most of your macOS experience, so read...
Foldable iPhone 2023 Feature Blue

Kuo: Apple to Release Foldable iPad With Carbon Fiber Kickstand in 2024

Monday January 30, 2023 12:55 am PST by
Apple will launch a foldable iPad with a carbon fiber kickstand sometime next year, according to analyst Ming-Chi Kuo. In a series of tweets, Kuo said he expects an "all-new design foldable iPad" to be the next big product launch in the iPad lineup, with no other major iPad releases in the next nine to 12 months. The analyst said he was "positive" that the foldable device would arrive in...
Hero0013

Best Apple Deals of the Week: 2021 MacBook Pro Gets Massive $500 Markdowns as Samsung Discounts TVs and Memory Accessories

Friday January 27, 2023 11:07 am PST by
As we near the end of January, this week we saw a collection of deals on Apple's iPad Pro, iMac, and the 2021 MacBook Pro. Additionally, we're tracking new sales on TVs and memory accessories from Samsung. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a small payment, which helps us keep the site running. Samsung...