University Researchers Who Built a CSAM Scanning System Urge Apple to Not Use the 'Dangerous' Technology

Respected university researchers are sounding the alarm bells over the technology behind Apple's plans to scan iPhone users' photo libraries for CSAM, or child sexual abuse material, calling the technology "dangerous."

apple privacy
Jonanath Mayer, an assistant professor of computer science and public affairs at Princeton University, as well as Anunay Kulshrestha, a researcher at Princeton University Center for Information Technology Policy, both penned an op-ed for The Washington Post, outlining their experiences with building image detection technology.

The researchers started a project two years ago to identity CSAM in end-to-end encrypted online services. The researchers note that given their field, they "know the value of end-to-end encryption, which protects data from third-party access." That concern, they say, is what horrifies them over CSAM "proliferating on encrypted platforms."

Mayer and Kulshrestha said they wanted to find a middle ground for the situation: build a system that online platforms could use to find CSAM and protect end-to-end encryption. The researchers note that experts in the field doubted the prospect of such a system, but they did manage to build it and in the process noticed a significant problem.

We sought to explore a possible middle ground, where online services could identify harmful content while otherwise preserving end-to-end encryption. The concept was straightforward: If someone shared material that matched a database of known harmful content, the service would be alerted. If a person shared innocent content, the service would learn nothing. People couldn't read the database or learn whether content matched, since that information could reveal law enforcement methods and help criminals evade detection.

Knowledgeable observers argued a system like ours was far from feasible. After many false starts, we built a working prototype. But we encountered a glaring problem.

Since Apple's announcement of the feature, the company has been bombarded with concerns that the system behind detecting CSAM could be used to detect other forms of photos at the request of oppressive governments. Apple has strongly pushed back against such a possibility, saying it will refuse any such request from governments.

Nonetheless, concerns over the future implications of the technology being used for CSAM detection are widespread. Mayer and Kulshrestha said that their concerns over how governments could use the system to detect content other than CSAM had them "disturbed."

A foreign government could, for example, compel a service to out people sharing disfavored political speech. That's no hypothetical: WeChat, the popular Chinese messaging app, already uses content matching to identify dissident material. India enacted rules this year that could require pre-screening content critical of government policy. Russia recently fined Google, Facebook and Twitter for not removing pro-democracy protest materials.

We spotted other shortcomings. The content-matching process could have false positives, and malicious users could game the system to subject innocent users to scrutiny.

We were so disturbed that we took a step we hadn't seen before in computer science literature: We warned against our own system design, urging further research on how to mitigate the serious downsides....

Apple has continued to address user concerns over its plans, publishing additional documents and an FAQ page. Apple continues to believe that its CSAM detection system, which will occur on a user's device, aligns with its long-standing privacy values.

Popular Stories

iPhone 17 Pro and Air Feature

Two iPhone 17 Pro and iPhone Air Colors Appear to Scratch More Easily

Friday September 19, 2025 10:02 am PDT by
As reported by Bloomberg today, some of the new iPhone 17 Pro and iPhone Air models on display at Apple Stores today are already scratched and scuffed. French blog Consomac also reported on this topic. The scratches appear to be most prominent on models with darker finishes, including the iPhone 17 Pro and Pro Max in Deep Blue, and the iPhone Air in Space Black. Images Credit: Consoma ...
iOS 26

iOS 26.0.1 Coming Soon, Likely With iPhone Air and iPhone 17 Pro Fix

Thursday September 18, 2025 9:17 am PDT by
Apple is preparing to release iOS 26.0.1, according to a private account on X with a proven track record of sharing information about future iOS versions. The update will have a build number of 23A350, or similar, the account said. It is likely that iOS 26.0.1 will fix a camera-related bug on the new iPhone Air and iPhone 17 Pro models. In his iPhone Air review, CNN Underscored's Henry T. ...
iPhone 17 Pro Colors

iPhone 17 Pro Max Teardown Reveals Qualcomm's Snapdragon X80 Modem for 5G

Friday September 19, 2025 7:39 am PDT by
While the iPhone Air is equipped with Apple's custom C1X modem for cellular connectivity, all of the iPhone 17 models are outfitted with Qualcomm modems still. A teardown video shared on Chinese platform Bilibili today (via Reddit) appears to confirm the iPhone 17 Pro Max is equipped with Qualcomm's Snapdragon X80 modem in particular. The same modem is likely used in the iPhone 17 and iPhone ...
iphone 17 pro max techwoven

Here Are The Best Cases You Can Buy for Your New iPhone 17 and iPhone Air

Friday September 19, 2025 6:46 am PDT by
Apple's new iPhones launch today, and there are plenty of options to choose from when it comes to protecting your new device from drops and scratches. In this article, we're taking a look at some of the best options for iPhone 17, iPhone 17 Pro, and iPhone 17 Air cases, as well as a few charging accessories. Note: MacRumors is an affiliate partner with some of these vendors. When you click a...
iOS 26 on Three iPhones

iOS 26's Liquid Glass Design Draws Criticism From Users

Wednesday September 17, 2025 2:56 pm PDT by
It's been two days since iOS 26 was released, and Apple's new Liquid Glass design is even more divisive than expected. Any major design change can create controversy as people get used to the new look, but the MacRumors forums, Reddit, Apple Support Communities, and social media sites seem to feature more criticism than praise as people discuss the update. Complaints There are a long...
iphone 17 pro inside

iPhone 17 Teardowns Confirm SIM and eSIM-Only Battery Capacities

Friday September 19, 2025 8:39 am PDT by
YouTube channel REWA Technology today shared an iPhone 17 Pro teardown video, offering a closer look inside the model with a SIM card tray. We are still waiting for repair website iFixit to share a more comprehensive teardown of the latest iPhone models, but this video provides a good look in the meantime. The device features various internal design changes, including larger rear camera...

Top Rated Comments

Geert76 Avatar
53 months ago
Apple, just cancel this on device surveillance software
Score: 166 Votes (Like | Disagree)
nvmls Avatar
53 months ago
They clearly don't know how the technology works.. oh wait
Score: 125 Votes (Like | Disagree)
nexu Avatar
53 months ago
This is getting more and more disturbing
Score: 117 Votes (Like | Disagree)
TheRealNick Avatar
53 months ago
If they don’t cancel this I’m seriously going to have to look at alternative products, which saddens me.
Score: 108 Votes (Like | Disagree)
dwsolberg Avatar
53 months ago
It's time for Apple to admit it's made a mistake. It's seriously bizarre they ever thought this was a good idea.
Score: 95 Votes (Like | Disagree)
VulchR Avatar
53 months ago
Waiting for the surveillance apologists to argue these two researchers "don't understand" the process...
Score: 93 Votes (Like | Disagree)