University Researchers Who Built a CSAM Scanning System Urge Apple to Not Use the 'Dangerous' Technology

Respected university researchers are sounding the alarm bells over the technology behind Apple's plans to scan iPhone users' photo libraries for CSAM, or child sexual abuse material, calling the technology "dangerous."

apple privacy
Jonanath Mayer, an assistant professor of computer science and public affairs at Princeton University, as well as Anunay Kulshrestha, a researcher at Princeton University Center for Information Technology Policy, both penned an op-ed for The Washington Post, outlining their experiences with building image detection technology.

The researchers started a project two years ago to identity CSAM in end-to-end encrypted online services. The researchers note that given their field, they "know the value of end-to-end encryption, which protects data from third-party access." That concern, they say, is what horrifies them over CSAM "proliferating on encrypted platforms."

Mayer and Kulshrestha said they wanted to find a middle ground for the situation: build a system that online platforms could use to find CSAM and protect end-to-end encryption. The researchers note that experts in the field doubted the prospect of such a system, but they did manage to build it and in the process noticed a significant problem.

We sought to explore a possible middle ground, where online services could identify harmful content while otherwise preserving end-to-end encryption. The concept was straightforward: If someone shared material that matched a database of known harmful content, the service would be alerted. If a person shared innocent content, the service would learn nothing. People couldn't read the database or learn whether content matched, since that information could reveal law enforcement methods and help criminals evade detection.

Knowledgeable observers argued a system like ours was far from feasible. After many false starts, we built a working prototype. But we encountered a glaring problem.

Since Apple's announcement of the feature, the company has been bombarded with concerns that the system behind detecting CSAM could be used to detect other forms of photos at the request of oppressive governments. Apple has strongly pushed back against such a possibility, saying it will refuse any such request from governments.

Nonetheless, concerns over the future implications of the technology being used for CSAM detection are widespread. Mayer and Kulshrestha said that their concerns over how governments could use the system to detect content other than CSAM had them "disturbed."

A foreign government could, for example, compel a service to out people sharing disfavored political speech. That's no hypothetical: WeChat, the popular Chinese messaging app, already uses content matching to identify dissident material. India enacted rules this year that could require pre-screening content critical of government policy. Russia recently fined Google, Facebook and Twitter for not removing pro-democracy protest materials.

We spotted other shortcomings. The content-matching process could have false positives, and malicious users could game the system to subject innocent users to scrutiny.

We were so disturbed that we took a step we hadn't seen before in computer science literature: We warned against our own system design, urging further research on how to mitigate the serious downsides....

Apple has continued to address user concerns over its plans, publishing additional documents and an FAQ page. Apple continues to believe that its CSAM detection system, which will occur on a user's device, aligns with its long-standing privacy values.

Popular Stories

ios 26 1 liquid glass opaque

iOS 26.1 Beta 4 Lets Users Control Liquid Glass Transparency with New Toggle

Monday October 20, 2025 10:57 am PDT by
With the fourth betas of iOS 26.1, iPadOS 26.1, and macOS 26.1, Apple has introduced a new setting that's designed to allow users to customize the look of Liquid Glass. The toggle lets users select from a clear look for Liquid Glass, or a tinted look. Clear is the current Liquid Glass design, which is more transparent and shows the background underneath buttons, bars, and menus, while tinted ...
iPhone Air

Report: 'Virtually No Demand' for iPhone Air

Wednesday October 22, 2025 3:22 am PDT by
Apple is "drastically" cutting production of the iPhone Air and shifting focus toward the iPhone 17 and iPhone 17 Pro models, Nikkei Asia reports. The business publication claims to have learned of a major cut to iPhone Air production motivated by weaker-than-expected consumer interest, nearly to "end of production levels." Despite early reports of the iPhone Air selling out within hours of...
maxresdefault

Here's How the iOS 26.1 Transparency Toggle Changes Liquid Glass

Monday October 20, 2025 1:55 pm PDT by
With the fourth beta of iOS 26.1, Apple added a toggle that makes Liquid Glass more opaque and reduces transparency. We tested the beta to see where the toggle works and what it looks like. Subscribe to the MacRumors YouTube channel for more videos. If you have the latest iOS 26.1 beta, you can go to Settings > Display and Brightness to get to the new option. Tap on Liquid Glass, then...
All Screen iPhone 2027 Feature 1

Apple's Plan to Launch Three New iPhone Designs Allegedly Revealed

Wednesday October 22, 2025 6:24 am PDT by
Apple plans to launch a new type of iPhone every year for the foreseeable future, according to an Asia-based source. The detailed information was shared by the account "yeux1122" in a blog post on the Korean platform Naver, citing domestic trend and component research companies. Corroborating other reports, Apple will apparently launch its first foldable iPhone in 2026, featuring a...
iOS 26 Feature

iOS 26.1 to iOS 26.4 Will Add These New Features to Your iPhone

Saturday October 18, 2025 11:00 am PDT by
iOS 26 was released last month, but the software train never stops, and iOS 26.1 beta testing is already underway. So far, iOS 26.1 makes both Apple Intelligence and Live Translation on compatible AirPods available in additional languages, and it includes some other minor changes across the Apple Music, Calendar, Photos, Clock, and Safari apps. More features and changes will follow in future ...
iOS 26

What's New in iOS 26.1 Beta 4

Monday October 20, 2025 1:02 pm PDT by
Even though we're at the fourth beta of iOS 26.1, Apple is continuing to add new features. In fact, the fourth beta has some of the biggest changes that we'll get when iOS 26.1 releases to the public later this month. We've rounded up what's new below. Liquid Glass Transparency Toggle Apple added a toggle for customizing the look of Liquid Glass. In Settings > Display and Brightness,...
cadillac lyric infotainment

GM to Remove CarPlay from All Future Vehicles, Including Gas Cars

Wednesday October 22, 2025 11:34 am PDT by
General Motors began phasing out support for CarPlay in its electric vehicles back in 2023, leading to complaints from iPhone users, but the company has no plans to back down. In fact, GM is going further and plans to remove CarPlay from all future gas vehicles, too. In an interview with The Verge, GM CEO Mary Barra said that the company opted to prioritize its platform for EVs, but the...
kohler toilet sensor

Kohler Launches $600 iPhone-Connected Toilet Camera That Monitors Your Health Through Waste Analysis

Monday October 20, 2025 1:42 pm PDT by
Kohler is expanding its line of bathroom products with Dekoda, an iPhone-connected device that's designed to be attached to a toilet rim (via The Verge). The device's included "sensors" point into the toilet bowl, allowing it to analyze what goes on in the bathroom. According to Kohler, Dekoda is a health tracker that can monitor gut health and hydration, as well as detect the presence of...
iOS 26

iOS 26.0.2 Update for iPhones Coming Soon

Friday October 17, 2025 7:35 am PDT by
Apple's software engineers continue to internally test iOS 26.0.2, according to MacRumors logs, which have been a reliable indicator of upcoming iOS versions. iOS 26.0.2 will be a minor update that addresses bugs and/or security vulnerabilities, but we do not know any specific details yet. The update will likely be released by the end of next week. Last month, Apple released iOS 26.0.1,...

Top Rated Comments

Geert76 Avatar
55 months ago
Apple, just cancel this on device surveillance software
Score: 166 Votes (Like | Disagree)
nvmls Avatar
55 months ago
They clearly don't know how the technology works.. oh wait
Score: 125 Votes (Like | Disagree)
nexu Avatar
55 months ago
This is getting more and more disturbing
Score: 117 Votes (Like | Disagree)
TheRealNick Avatar
55 months ago
If they don’t cancel this I’m seriously going to have to look at alternative products, which saddens me.
Score: 108 Votes (Like | Disagree)
dwsolberg Avatar
55 months ago
It's time for Apple to admit it's made a mistake. It's seriously bizarre they ever thought this was a good idea.
Score: 95 Votes (Like | Disagree)
VulchR Avatar
55 months ago
Waiting for the surveillance apologists to argue these two researchers "don't understand" the process...
Score: 93 Votes (Like | Disagree)