Apple Introducing New Child Safety Features, Including Scanning Users' Photo Libraries for Known Sexual Abuse Material

Apple today previewed new child safety features that will be coming to its platforms with software updates later this year. The company said the features will be available in the U.S. only at launch and will be expanded to other regions over time.

iphone communication safety feature

Communication Safety

First, the Messages app on the iPhone, iPad, and Mac will be getting a new Communication Safety feature to warn children and their parents when receiving or sending sexually explicit photos. Apple said the Messages app will use on-device machine learning to analyze image attachments, and if a photo is determined to be sexually explicit, the photo will be automatically blurred and the child will be warned.

When a child attempts to view a photo flagged as sensitive in the Messages app, they will be alerted that the photo may contain private body parts, and that the photo may be hurtful. Depending on the age of the child, there will also be an option for parents to receive a notification if their child proceeds to view the sensitive photo or if they choose to send a sexually explicit photo to another contact after being warned.

Apple said the new Communication Safety feature will be coming in updates to iOS 15, iPadOS 15 and macOS Monterey later this year for accounts set up as families in iCloud. Apple ensured that iMessage conversations will remain protected with end-to-end encryption, making private communications unreadable by Apple.

Scanning Photos for Child Sexual Abuse Material (CSAM)

Second, starting this year with iOS 15 and iPadOS 15, Apple will be able to detect known Child Sexual Abuse Material (CSAM) images stored in iCloud Photos, enabling Apple to report these instances to the National Center for Missing and Exploited Children (NCMEC), a non-profit organization that works in collaboration with U.S. law enforcement agencies.

Apple said its method of detecting known CSAM is designed with user privacy in mind. Instead of scanning images in the cloud, Apple said the system will perform on-device matching against a database of known CSAM image hashes provided by the NCMEC and other child safety organizations. Apple said it will further transform this database into an unreadable set of hashes that is securely stored on users' devices.

The hashing technology, called NeuralHash, analyzes an image and converts it to a unique number specific to that image, according to Apple.

"The main purpose of the hash is to ensure that identical and visually similar images result in the same hash, while images that are different from one another result in different hashes," said Apple in a new "Expanded Protections for Children" white paper. "For example, an image that has been slightly cropped, resized or converted from color to black and white is treated identical to its original, and has the same hash."

apple csam flow chart
Before an image is stored in iCloud Photos, Apple said an on-device matching process is performed for that image against the unreadable set of known CSAM hashes. If there is a match, the device creates a cryptographic safety voucher. This voucher is uploaded to iCloud Photos along with the image, and once an undisclosed threshold of matches is exceeded, Apple is able to interpret the contents of the vouchers for CSAM matches. Apple then manually reviews each report to confirm there is a match, disables the user's iCloud account, and sends a report to NCMEC. Apple is not sharing what its exact threshold is, but ensures an "extremely high level of accuracy" that accounts are not incorrectly flagged.

Apple said its method of detecting known CSAM provides "significant privacy benefits" over existing techniques:

• This system is an effective way to identify known CSAM stored in iCloud Photos accounts while protecting user privacy.
• As part of the process, users also can't learn anything about the set of known CSAM images that is used for matching. This protects the contents of the database from malicious use.
• The system is very accurate, with an extremely low error rate of less than one in one trillion account per year.
• The system is significantly more privacy-preserving than cloud-based scanning, as it only reports users who have a collection of known CSAM stored in iCloud Photos.

The underlying technology behind Apple's system is quite complex and it has published a technical summary with more details.

"Apple's expanded protection for children is a game changer. With so many people using Apple products, these new safety measures have lifesaving potential for children who are being enticed online and whose horrific images are being circulated in child sexual abuse material," said John Clark, the President and CEO of the National Center for Missing & Exploited Children. "At the National Center for Missing & Exploited Children we know this crime can only be combated if we are steadfast in our dedication to protecting children. We can only do this because technology partners, like Apple, step up and make their dedication known. The reality is that privacy and child protection can co-exist. We applaud Apple and look forward to working together to make this world a safer place for children."

Expanded CSAM Guidance in Siri and Search

iphone csam siri
Third, Apple said it will be expanding guidance in Siri and Spotlight Search across devices by providing additional resources to help children and parents stay safe online and get help with unsafe situations. For example, users who ask Siri how they can report CSAM or child exploitation will be pointed to resources for where and how to file a report.

The updates to Siri and Search are coming later this year in an update to iOS 15, iPadOS 15, watchOS 8, and macOS Monterey, according to Apple.

Note: Due to the political or social nature of the discussion regarding this topic, the discussion thread is located in our Political News forum. All forum members and site visitors are welcome to read and follow the thread, but posting is limited to forum members with at least 100 posts.

Top Rated Comments

levitynyc Avatar
30 months ago
Not gonna lie...that's kinda creepy.
Score: 71 Votes (Like | Disagree)
Exponent Avatar
30 months ago
No, too far, Apple.

What is going to keep you from scanning my library for NeuralHash matches against politics you don’t like? Or criticism of mainland dictatorial China?

if that doesn’t happen in the US, what will keep other countries (read above) from doing just that to their citizens?
Score: 58 Votes (Like | Disagree)
gaximus Avatar
30 months ago
What about photos of "Baby's first bath" will those users get treated as child exploitation?
Score: 56 Votes (Like | Disagree)
arn Avatar
30 months ago

Yeh good luck if say you have small young kids who don't keep their clothes on. Like what,t every baby?

This is also creepy asf sorry. Child predators are bad, obviously, but this isn't the way.
The CSAM thing doesn't detect/determine content of images. It checks photos against a database of specific (actively circulating) child abuse images.

Not to say there aren't legitimate concerns, but worrying that it is going to somehow flag your own kid's photos is not one of them.

(The child safety thing does detect, but seems the worst that does is through up a warning/blurring if you have it on)
Score: 44 Votes (Like | Disagree)
fenderbass146 Avatar
30 months ago
every year we go to 1984 a little more.
Score: 37 Votes (Like | Disagree)
Apple_Robert Avatar
30 months ago
I am against child abuse and child porn. However, I don't agree with what Apple is going to be doing with user phones.
Score: 34 Votes (Like | Disagree)

Popular Stories

apple card 1

Apple Ending Apple Card Partnership With Goldman Sachs

Tuesday November 28, 2023 3:09 pm PST by
Apple is ending its credit card partnership with Goldman Sachs, according to The Wall Street Journal. Apple plans to stop working with Goldman Sachs in the next 12 to 15 months, and it is not yet clear if Apple has established a new partnership for the Apple Card. Apple and Goldman Sachs will dissolve their entire consumer partnership, including the Apple Card and the Apple Savings account....
iOS 17

Everything New in iOS 17.2 Beta 4

Tuesday November 28, 2023 12:18 pm PST by
Apple is wrapping up development on iOS 17.2, with the update expected to come out in December. While we're getting to the end of the beta testing period, Apple is still tweaking features and adding new functionality. We've rounded up everything new in the fourth beta of iOS 17.2. Default Notification Sound Under Sounds & Haptics, there's a new "Default Alerts" section that allows you to ...
ios 17 namedrop

Police Departments and News Sites Spreading Misinformation About How iOS 17 NameDrop Feature Works

Monday November 27, 2023 5:11 pm PST by
Apple with iOS 17.1 and watchOS 10.1 introduced a new NameDrop feature that is designed to allow users to place Apple devices near one another to quickly exchange contact information. Sharing contact information is done with explicit user permission, but some news organizations and police departments have been spreading misinformation about how NameDrop functions. As noted by The Washington...
All New CarPlay Five New Features Article 2

What to Expect From All-New CarPlay, Still Listed as Coming 'Late 2023'

Tuesday November 28, 2023 7:44 am PST by
At WWDC in June 2022, Apple previewed the next generation of CarPlay, promising deeper integration with vehicle functions like A/C and FM radio, support for multiple displays across the dashboard, increased personalization, and more. Apple's website still says the first vehicles with support for the next-generation CarPlay experience will be announced in "late 2023," but it has not shared...
iOS 17

iOS 17.1.2 Update for iPhone Likely to Be Released This Week

Monday November 27, 2023 8:24 am PST by
Apple will likely release iOS 17.1.2 this week, based on mounting evidence of the software in our website's analytics logs in recent days. As a minor update, iOS 17.1.2 should be focused on bug fixes, but it's unclear exactly which issues might be addressed. Some users have continued to experience Wi-Fi issues on iOS 17.1.1, so perhaps iOS 17.1.2 will include the same fix for Wi-Fi...
Apple 5G Modem Feature Triad

Apple to Discontinue Custom 5G Modem Development, Claim Reports

Wednesday November 29, 2023 4:19 am PST by
Apple is discontinuing in-house modem development after several unsuccessful attempts to perfect its own custom 5G modem chip, according to unconfirmed reports coming out of Asia. According to the operator of news aggregator account "yeux1122" on the Naver blog, supply chain sources related to Apple's 5G modem departments claim that the company's attempts to develop its own modem have...
Apple Logo

Apple Discontinued These 5 Products This Year

Monday November 27, 2023 7:03 am PST by
As the end of 2023 nears, now is a good opportunity to look back at some of the devices and accessories that Apple discontinued throughout the year. Apple products discontinued in 2023 include the iPhone 13 mini, 13-inch MacBook Pro, MagSafe Battery Pack, MagSafe Duo Charger, and leather accessories. Also check out our lists of Apple products discontinued in 2022 and 2021. iPhone Mini ...
Google Drive

Some Google Drive Users' Files Have Mysteriously Vanished

Tuesday November 28, 2023 3:55 am PST by
Google Drive users have been warned not to disconnect their account within the Google Drive for desktop app, after a spate of reports of files going missing from the cloud service. Alarm bells began ringing last week on Google's community support site when some users reported files mysteriously disappearing from Google Drive, with some posters claiming six or more months of data had...