University Researchers Who Built a CSAM Scanning System Urge Apple to Not Use the 'Dangerous' Technology

Respected university researchers are sounding the alarm bells over the technology behind Apple's plans to scan iPhone users' photo libraries for CSAM, or child sexual abuse material, calling the technology "dangerous."

apple privacy
Jonanath Mayer, an assistant professor of computer science and public affairs at Princeton University, as well as Anunay Kulshrestha, a researcher at Princeton University Center for Information Technology Policy, both penned an op-ed for The Washington Post, outlining their experiences with building image detection technology.

The researchers started a project two years ago to identity CSAM in end-to-end encrypted online services. The researchers note that given their field, they "know the value of end-to-end encryption, which protects data from third-party access." That concern, they say, is what horrifies them over CSAM "proliferating on encrypted platforms."

Mayer and Kulshrestha said they wanted to find a middle ground for the situation: build a system that online platforms could use to find CSAM and protect end-to-end encryption. The researchers note that experts in the field doubted the prospect of such a system, but they did manage to build it and in the process noticed a significant problem.

We sought to explore a possible middle ground, where online services could identify harmful content while otherwise preserving end-to-end encryption. The concept was straightforward: If someone shared material that matched a database of known harmful content, the service would be alerted. If a person shared innocent content, the service would learn nothing. People couldn't read the database or learn whether content matched, since that information could reveal law enforcement methods and help criminals evade detection.

Knowledgeable observers argued a system like ours was far from feasible. After many false starts, we built a working prototype. But we encountered a glaring problem.

Since Apple's announcement of the feature, the company has been bombarded with concerns that the system behind detecting CSAM could be used to detect other forms of photos at the request of oppressive governments. Apple has strongly pushed back against such a possibility, saying it will refuse any such request from governments.

Nonetheless, concerns over the future implications of the technology being used for CSAM detection are widespread. Mayer and Kulshrestha said that their concerns over how governments could use the system to detect content other than CSAM had them "disturbed."

A foreign government could, for example, compel a service to out people sharing disfavored political speech. That's no hypothetical: WeChat, the popular Chinese messaging app, already uses content matching to identify dissident material. India enacted rules this year that could require pre-screening content critical of government policy. Russia recently fined Google, Facebook and Twitter for not removing pro-democracy protest materials.

We spotted other shortcomings. The content-matching process could have false positives, and malicious users could game the system to subject innocent users to scrutiny.

We were so disturbed that we took a step we hadn't seen before in computer science literature: We warned against our own system design, urging further research on how to mitigate the serious downsides....

Apple has continued to address user concerns over its plans, publishing additional documents and an FAQ page. Apple continues to believe that its CSAM detection system, which will occur on a user's device, aligns with its long-standing privacy values.

Popular Stories

Apple Watch Ultra Night Mode Screen

Apple Watch Ultra 3 Launching Later This Year With Two Key Upgrades

Wednesday July 2, 2025 1:13 pm PDT by
The long wait for an Apple Watch Ultra 3 appears to be nearly over, and it is rumored to feature both satellite connectivity and 5G support. Apple Watch Ultra's existing Night Mode In his latest Power On newsletter, Bloomberg's Mark Gurman said that the Apple Watch Ultra 3 is on track to launch this year with "significant" new features, including satellite connectivity, which would let you...
iPhone 17 Pro in Hand Feature Lowgo

iPhone 17 Pro Coming Soon With These 14 New Features

Friday July 4, 2025 1:05 pm PDT by
Apple's next-generation iPhone 17 Pro and iPhone 17 Pro Max are just over two months away, and there are plenty of rumors about the devices. Below, we recap key changes rumored for the iPhone 17 Pro models. Latest Rumors These rumors surfaced in June and July:Apple logo repositioned: Apple's logo may have a lower position on the back of the iPhone 17 Pro models, compared to previous...
iPhone 17 Pro in Hand Feature Lowgo

iPhone 17 Pro Max Battery Capacity Leaked

Thursday July 3, 2025 5:40 am PDT by
The iPhone 17 Pro Max will feature the biggest ever battery in an iPhone, according to the Weibo leaker known as "Instant Digital." In a new post, the leaker listed the battery capacities of the iPhone 11 Pro Max through to the iPhone 16 Pro Max, and added that the iPhone 17 Pro Max will feature a battery capacity of 5,000mAh: iPhone 11 Pro Max: 3,969mAh iPhone 12 Pro Max: 3,687mAh...
airpods pro 2

AirPods Pro 3 to Help Maintain Apple's Place in Earbud Market Amid Increasing Low-Cost Competition

Thursday July 3, 2025 7:25 am PDT by
Apple's position as the dominant force in the global true wireless stereo (TWS) earbud market is expected to continue through 2025, according to Counterpoint Research. The forecast outlines a 3% year-over-year increase in global TWS unit shipments for 2025, signaling a transition from rapid growth to a more mature phase for the category. While Apple is set to remain the leading brand by...
apple silicon mac lineup 2024 feature purple m5

Apple's Upcoming Macs Listed in New Report

Thursday July 3, 2025 9:09 am PDT by
AppleInsider's Marko Zivkovic today shared a list of alleged identifiers for future Mac models, which should roll out over the next year or so. The report does not reveal anything too surprising, but it does serve as further evidence that Apple is seemingly working on new models of every Mac, including the MacBook Air, MacBook Pro, iMac, Mac mini, Mac Studio, and Mac Pro. Apple is...
iPhone 17 Pro Lower Logo Magsafe

iPhone 17 Pro's New MagSafe Design Revealed in Leaked Photo

Wednesday July 2, 2025 8:37 am PDT by
The upcoming iPhone 17 Pro and iPhone 17 Pro Max are rumored to have a slightly different MagSafe magnet layout compared to existing iPhone models, and a leaked photo has offered a closer look at the supposed new design. The leaker Majin Bu today shared a photo of alleged MagSafe magnet arrays for third-party iPhone 17 Pro cases. On existing iPhone models with MagSafe, the magnets form a...

Top Rated Comments

Geert76 Avatar
51 months ago
Apple, just cancel this on device surveillance software
Score: 166 Votes (Like | Disagree)
nvmls Avatar
51 months ago
They clearly don't know how the technology works.. oh wait
Score: 125 Votes (Like | Disagree)
nexu Avatar
51 months ago
This is getting more and more disturbing
Score: 117 Votes (Like | Disagree)
TheRealNick Avatar
51 months ago
If they don’t cancel this I’m seriously going to have to look at alternative products, which saddens me.
Score: 108 Votes (Like | Disagree)
dwsolberg Avatar
51 months ago
It's time for Apple to admit it's made a mistake. It's seriously bizarre they ever thought this was a good idea.
Score: 95 Votes (Like | Disagree)
VulchR Avatar
51 months ago
Waiting for the surveillance apologists to argue these two researchers "don't understand" the process...
Score: 93 Votes (Like | Disagree)