University Researchers Who Built a CSAM Scanning System Urge Apple to Not Use the 'Dangerous' Technology

Respected university researchers are sounding the alarm bells over the technology behind Apple's plans to scan iPhone users' photo libraries for CSAM, or child sexual abuse material, calling the technology "dangerous."

apple privacy
Jonanath Mayer, an assistant professor of computer science and public affairs at Princeton University, as well as Anunay Kulshrestha, a researcher at Princeton University Center for Information Technology Policy, both penned an op-ed for The Washington Post, outlining their experiences with building image detection technology.

The researchers started a project two years ago to identity CSAM in end-to-end encrypted online services. The researchers note that given their field, they "know the value of end-to-end encryption, which protects data from third-party access." That concern, they say, is what horrifies them over CSAM "proliferating on encrypted platforms."

Mayer and Kulshrestha said they wanted to find a middle ground for the situation: build a system that online platforms could use to find CSAM and protect end-to-end encryption. The researchers note that experts in the field doubted the prospect of such a system, but they did manage to build it and in the process noticed a significant problem.

We sought to explore a possible middle ground, where online services could identify harmful content while otherwise preserving end-to-end encryption. The concept was straightforward: If someone shared material that matched a database of known harmful content, the service would be alerted. If a person shared innocent content, the service would learn nothing. People couldn't read the database or learn whether content matched, since that information could reveal law enforcement methods and help criminals evade detection.

Knowledgeable observers argued a system like ours was far from feasible. After many false starts, we built a working prototype. But we encountered a glaring problem.

Since Apple's announcement of the feature, the company has been bombarded with concerns that the system behind detecting CSAM could be used to detect other forms of photos at the request of oppressive governments. Apple has strongly pushed back against such a possibility, saying it will refuse any such request from governments.

Nonetheless, concerns over the future implications of the technology being used for CSAM detection are widespread. Mayer and Kulshrestha said that their concerns over how governments could use the system to detect content other than CSAM had them "disturbed."

A foreign government could, for example, compel a service to out people sharing disfavored political speech. That's no hypothetical: WeChat, the popular Chinese messaging app, already uses content matching to identify dissident material. India enacted rules this year that could require pre-screening content critical of government policy. Russia recently fined Google, Facebook and Twitter for not removing pro-democracy protest materials.

We spotted other shortcomings. The content-matching process could have false positives, and malicious users could game the system to subject innocent users to scrutiny.

We were so disturbed that we took a step we hadn't seen before in computer science literature: We warned against our own system design, urging further research on how to mitigate the serious downsides....

Apple has continued to address user concerns over its plans, publishing additional documents and an FAQ page. Apple continues to believe that its CSAM detection system, which will occur on a user's device, aligns with its long-standing privacy values.

Popular Stories

iphone 16 pro ghost hand

5 Reasons to Skip This Year's iPhone 17 Pro

Thursday July 10, 2025 4:54 am PDT by
Apple will launch its new iPhone 17 series in two months, and the iPhone 17 Pro models are expected to get a new design for the rear casing and the camera area. But more significant changes to the lineup are not expected until next year, when the iPhone 18 models arrive. If you're thinking of trading in your iPhone for this year's latest, consider the following features rumored to be coming...
apple wallet drivers license feature iPhone 15 pro

Apple Says iPhone Driver's Licenses Will Expand to These 8 U.S. States

Tuesday July 8, 2025 11:26 am PDT by
In select U.S. states, residents can add their driver's license or state ID to the Wallet app on the iPhone and Apple Watch, providing a convenient and contactless way to display proof of identity or age at select airports and businesses, and in select apps. Unfortunately, this feature continues to roll out very slowly since it was announced in 2021, with only nine U.S. states, Puerto Rico,...
iPhone 17 Pro in Hand Feature Lowgo

iPhone 17 Pro to Reverse iPhone X Design Decision

Monday July 7, 2025 9:46 am PDT by
Since the iPhone X in 2017, all of Apple's highest-end iPhone models have featured either stainless steel or titanium frames, but it has now been rumored that this design decision will be coming to an end with the iPhone 17 Pro models later this year. In a post on Chinese social media platform Weibo today, the account Instant Digital said that the iPhone 17 Pro models will have an aluminum...
iPhone 17 Pro in Hand Feature Lowgo

Leaker Reveals Amount of RAM in iPhone 17 Through iPhone 17 Pro Max

Wednesday July 9, 2025 8:08 am PDT by
Three out of four iPhone 17 models will feature more RAM than the equivalent iPhone 16 models, according to a new leak that aligns with previous rumors. The all-new iPhone 17 Air, the iPhone 17 Pro, and the iPhone 17 Pro Max will each be equipped with 12GB of RAM, according to Fixed Focus Digital, an account with more than two million followers on Chinese social media platform Weibo. The...
apple account card feature

Apple Account Card Expanding to More Countries

Tuesday July 8, 2025 7:34 pm PDT by
Apple is expanding the ability to add an Apple Account Card to the Wallet app to more countries, according to backend Apple Pay changes. With iOS 15.5, Apple updated the Wallet app to allow users to add an Apple Account Card, which displays the Apple credit balance associated with an Apple ID. If you receive an Apple gift card, for example, it is added to an Apple Account that is also...
macbook pro blue green

M5 MacBook Pro No Longer Coming in 2025

Thursday July 10, 2025 12:38 pm PDT by
Apple does not plan to refresh any Macs with updated M5 chips in 2025, according to Bloomberg's Mark Gurman. Updated MacBook Air and MacBook Pro models are now planned for the first half of 2026. Gurman previously said that Apple would debut the M5 MacBook Pro models in late 2025, but his newest report suggests that Apple is "considering" pushing them back to 2026. Apple is now said to be...
iOS 26 Feature

Everything New in iOS 26 Beta 3

Monday July 7, 2025 1:20 pm PDT by
Apple is continuing to refine and update iOS 26, and beta three features smaller changes than we saw in beta 2, plus further tweaks to the Liquid Glass design. Apple is gearing up for the next phase of beta testing, and the company has promised that a public beta is set to come out in July. Transparency In some apps like Apple Music, Podcasts, and the App Store, Apple has toned down the...
iCloud General Feature Redux

iPhone Users Who Pay for iCloud Storage Receive These Five Perks

Wednesday July 9, 2025 9:20 am PDT by
If you pay for iCloud storage on your iPhone, did you know that Apple offers you five perks beyond the extra storage space, at no additional cost? Here are the perks included with all iCloud+ plans:Private Relay keeps your Safari browsing history entirely private from network providers, websites, and even Apple. Hide My Email generates unique, random email addresses whenever needed. Hom...

Top Rated Comments

Geert76 Avatar
51 months ago
Apple, just cancel this on device surveillance software
Score: 166 Votes (Like | Disagree)
nvmls Avatar
51 months ago
They clearly don't know how the technology works.. oh wait
Score: 125 Votes (Like | Disagree)
nexu Avatar
51 months ago
This is getting more and more disturbing
Score: 117 Votes (Like | Disagree)
TheRealNick Avatar
51 months ago
If they don’t cancel this I’m seriously going to have to look at alternative products, which saddens me.
Score: 108 Votes (Like | Disagree)
dwsolberg Avatar
51 months ago
It's time for Apple to admit it's made a mistake. It's seriously bizarre they ever thought this was a good idea.
Score: 95 Votes (Like | Disagree)
VulchR Avatar
51 months ago
Waiting for the surveillance apologists to argue these two researchers "don't understand" the process...
Score: 93 Votes (Like | Disagree)