Apple Introducing New Child Safety Features, Including Scanning Users' Photo Libraries for Known Sexual Abuse Material

Apple today previewed new child safety features that will be coming to its platforms with software updates later this year. The company said the features will be available in the U.S. only at launch and will be expanded to other regions over time.

iphone communication safety feature

Communication Safety

First, the Messages app on the iPhone, iPad, and Mac will be getting a new Communication Safety feature to warn children and their parents when receiving or sending sexually explicit photos. Apple said the Messages app will use on-device machine learning to analyze image attachments, and if a photo is determined to be sexually explicit, the photo will be automatically blurred and the child will be warned.

When a child attempts to view a photo flagged as sensitive in the Messages app, they will be alerted that the photo may contain private body parts, and that the photo may be hurtful. Depending on the age of the child, there will also be an option for parents to receive a notification if their child proceeds to view the sensitive photo or if they choose to send a sexually explicit photo to another contact after being warned.

Apple said the new Communication Safety feature will be coming in updates to iOS 15, iPadOS 15 and macOS Monterey later this year for accounts set up as families in iCloud. Apple ensured that iMessage conversations will remain protected with end-to-end encryption, making private communications unreadable by Apple.

Scanning Photos for Child Sexual Abuse Material (CSAM)

Second, starting this year with iOS 15 and iPadOS 15, Apple will be able to detect known Child Sexual Abuse Material (CSAM) images stored in iCloud Photos, enabling Apple to report these instances to the National Center for Missing and Exploited Children (NCMEC), a non-profit organization that works in collaboration with U.S. law enforcement agencies.

Apple said its method of detecting known CSAM is designed with user privacy in mind. Instead of scanning images in the cloud, Apple said the system will perform on-device matching against a database of known CSAM image hashes provided by the NCMEC and other child safety organizations. Apple said it will further transform this database into an unreadable set of hashes that is securely stored on users' devices.

The hashing technology, called NeuralHash, analyzes an image and converts it to a unique number specific to that image, according to Apple.

"The main purpose of the hash is to ensure that identical and visually similar images result in the same hash, while images that are different from one another result in different hashes," said Apple in a new "Expanded Protections for Children" white paper. "For example, an image that has been slightly cropped, resized or converted from color to black and white is treated identical to its original, and has the same hash."

apple csam flow chart
Before an image is stored in iCloud Photos, Apple said an on-device matching process is performed for that image against the unreadable set of known CSAM hashes. If there is a match, the device creates a cryptographic safety voucher. This voucher is uploaded to iCloud Photos along with the image, and once an undisclosed threshold of matches is exceeded, Apple is able to interpret the contents of the vouchers for CSAM matches. Apple then manually reviews each report to confirm there is a match, disables the user's iCloud account, and sends a report to NCMEC. Apple is not sharing what its exact threshold is, but ensures an "extremely high level of accuracy" that accounts are not incorrectly flagged.

Apple said its method of detecting known CSAM provides "significant privacy benefits" over existing techniques:

• This system is an effective way to identify known CSAM stored in iCloud Photos accounts while protecting user privacy.
• As part of the process, users also can't learn anything about the set of known CSAM images that is used for matching. This protects the contents of the database from malicious use.
• The system is very accurate, with an extremely low error rate of less than one in one trillion account per year.
• The system is significantly more privacy-preserving than cloud-based scanning, as it only reports users who have a collection of known CSAM stored in iCloud Photos.

The underlying technology behind Apple's system is quite complex and it has published a technical summary with more details.

"Apple's expanded protection for children is a game changer. With so many people using Apple products, these new safety measures have lifesaving potential for children who are being enticed online and whose horrific images are being circulated in child sexual abuse material," said John Clark, the President and CEO of the National Center for Missing & Exploited Children. "At the National Center for Missing & Exploited Children we know this crime can only be combated if we are steadfast in our dedication to protecting children. We can only do this because technology partners, like Apple, step up and make their dedication known. The reality is that privacy and child protection can co-exist. We applaud Apple and look forward to working together to make this world a safer place for children."

Expanded CSAM Guidance in Siri and Search

iphone csam siri
Third, Apple said it will be expanding guidance in Siri and Spotlight Search across devices by providing additional resources to help children and parents stay safe online and get help with unsafe situations. For example, users who ask Siri how they can report CSAM or child exploitation will be pointed to resources for where and how to file a report.

The updates to Siri and Search are coming later this year in an update to iOS 15, iPadOS 15, watchOS 8, and macOS Monterey, according to Apple.

Note: Due to the political or social nature of the discussion regarding this topic, the discussion thread is located in our Political News forum. All forum members and site visitors are welcome to read and follow the thread, but posting is limited to forum members with at least 100 posts.

Top Rated Comments

levitynyc Avatar
24 weeks ago
Not gonna lie...that's kinda creepy.
Score: 71 Votes (Like | Disagree)
Exponent Avatar
24 weeks ago
No, too far, Apple.

What is going to keep you from scanning my library for NeuralHash matches against politics you don’t like? Or criticism of mainland dictatorial China?

if that doesn’t happen in the US, what will keep other countries (read above) from doing just that to their citizens?
Score: 58 Votes (Like | Disagree)
gaximus Avatar
24 weeks ago
What about photos of "Baby's first bath" will those users get treated as child exploitation?
Score: 56 Votes (Like | Disagree)
arn Avatar
24 weeks ago

Yeh good luck if say you have small young kids who don't keep their clothes on. Like what,t every baby?

This is also creepy asf sorry. Child predators are bad, obviously, but this isn't the way.
The CSAM thing doesn't detect/determine content of images. It checks photos against a database of specific (actively circulating) child abuse images.

Not to say there aren't legitimate concerns, but worrying that it is going to somehow flag your own kid's photos is not one of them.

(The child safety thing does detect, but seems the worst that does is through up a warning/blurring if you have it on)
Score: 44 Votes (Like | Disagree)
fenderbass146 Avatar
24 weeks ago
every year we go to 1984 a little more.
Score: 37 Votes (Like | Disagree)
Apple_Robert Avatar
24 weeks ago
I am against child abuse and child porn. However, I don't agree with what Apple is going to be doing with user phones.
Score: 34 Votes (Like | Disagree)

Related Stories

apple privacy

Apple Publishes FAQ to Address Concerns About CSAM Detection and Messages Scanning

Monday August 9, 2021 1:50 am PDT by
Apple has published a FAQ titled "Expanded Protections for Children" which aims to allay users' privacy concerns about the new CSAM detection in iCloud Photos and communication safety for Messages features that the company announced last week. "Since we announced these features, many stakeholders including privacy organizations and child safety organizations have expressed their support of...
iphone communication safety feature

Apple Remains Committed to Launching New Child Safety Features Later This Year

Tuesday August 10, 2021 10:58 am PDT by
Last week, Apple previewed new child safety features that it said will be coming to the iPhone, iPad, and Mac with software updates later this year. The company said the features will be available in the U.S. only at launch. A refresher on Apple's new child safety features from our previous coverage:First, an optional Communication Safety feature in the Messages app on iPhone, iPad, and Mac...
craig wwdc 2021 privacy

Craig Federighi Acknowledges Confusion Around Apple Child Safety Features and Explains New Details About Safeguards

Friday August 13, 2021 6:33 am PDT by
Apple's senior vice president of software engineering, Craig Federighi, has today defended the company's controversial planned child safety features in a significant interview with The Wall Street Journal, revealing a number of new details about the safeguards built into Apple's system for scanning users' photos libraries for Child Sexual Abuse Material (CSAM). Federighi admitted that Apple...
iCloud General Feature

Apple Confirms Detection of Child Sexual Abuse Material is Disabled When iCloud Photos is Turned Off

Thursday August 5, 2021 2:16 pm PDT by
Apple today announced that iOS 15 and iPadOS 15 will see the introduction of a new method for detecting child sexual abuse material (CSAM) on iPhones and iPads in the United States. User devices will download an unreadable database of known CSAM image hashes and will do an on-device comparison to the user's own photos, flagging them for known CSAM material before they're uploaded to iCloud...
communication safety 1

iOS 15.2 Beta Adds Messages Communication Safety Feature for Kids

Tuesday November 9, 2021 10:07 am PST by
Apple over the summer announced new Child Safety features that are aimed at keeping children safer online. Apple has confirmed that one of those features, Communication Safety in Messages, has been enabled in the second beta of iOS 15.2 that was released today, after hints of it appeared in the first beta. Note that Communication Safety is not the same as the controversial anti-CSAM feature that...
iphone communication safety feature

Apple Open to Expanding New Child Safety Features to Third-Party Apps

Monday August 9, 2021 11:00 am PDT by
Apple today held a questions-and-answers session with reporters regarding its new child safety features, and during the briefing, Apple confirmed that it would be open to expanding the features to third-party apps in the future. As a refresher, Apple unveiled three new child safety features coming to future versions of iOS 15, iPadOS 15, macOS Monterey, and/or watchOS 8. Apple's New Child ...
iphone communication safety feature arned

Apple's New Feature That Scans Messages for Nude Photos is Only for Children, Parental Notifications Limited to Kids Under 13

Thursday August 5, 2021 2:50 pm PDT by
Apple today announced a series of new child safety initiatives that are coming alongside the latest iOS 15, iPadOS 15, and macOS Monterey updates and that are aimed at keeping children safer online. One of the new features, Communication Safety, has raised privacy concerns because it allows Apple to scan images sent and received by the Messages app for sexually explicit content, but Apple...
iphone communication safety feature arned

Code for Apple's Communication Safety Feature for Kids Found in iOS 15.2 Beta [Updated]

Wednesday October 27, 2021 1:04 pm PDT by
Update: We've learned from Apple that the Communication Safety code found in the first iOS 15.2 beta is not a feature in that update and Apple does not plan to release the feature as it is described in the article. Apple this summer announced new Child Safety Features that are designed to keep children safer online. One of those features, Communication Safety, appears to be included in the...

Popular Stories

iPhone 14 Mock pill and hole thumb

ProMotion Now Expected to Remain Exclusive to iPhone 14 Pro Models, Not Expand to Entire Lineup

Sunday January 16, 2022 8:56 am PST by
Continuing the tradition set with the iPhone 13 Pro, only the highest-end iPhone 14 models will feature Apple's ProMotion display technology, according to a respected display analyst. Ross Young, who on multiple occasions has detailed accurate information about Apple's future products, said in a tweet that ProMotion will not be expanded to the entire iPhone 14 lineup and will remain...
safari icon blue banner

Safari Bug Allows Websites to Track Your Recent Browsing Activity in Real Time [Updated]

Sunday January 16, 2022 3:37 pm PST by
A bug in WebKit's implementation of a JavaScript API called IndexedDB can reveal your recent browsing history and even your identity, according to a blog post shared on Friday by browser fingerprinting service FingerprintJS. In a nutshell, the bug allows any website that uses IndexedDB to access the names of IndexedDB databases generated by other websites during a user's browsing session....
ipad air 4 video

New iPad Air Rumored to Launch This Spring With A15 Chip, 5G, Center Stage Camera, and More

Saturday January 15, 2022 8:05 pm PST by
Apple is planning to release a fifth-generation iPad Air with similar features as the sixth-generation iPad mini, including an A15 Bionic chip, 12-megapixel Ultra Wide front camera with Center Stage support, 5G for cellular models, and Quad-LED True Tone flash, according to Japanese blog Mac Otakara. Citing reliables sources in China, the report claims that the new iPad Air could be...
Unlikely Products 2022 Feature

Six Rumored Apple Products You're Unlikely to See This Year

Saturday January 15, 2022 2:06 pm PST by
Much has been said about what consumers could see from Apple in 2022, but the company is also working on a handful of rumored products that aren't expected to be unveiled for at least another 12 months, and in some cases a lot longer. Of course, that's assuming they get released at all. Apple works on many potential products some of which ultimately never see the light of day. With that in...
AirPods Pro Gen 3 Mock Feature Red

AirPods Pro 2 Could Start a New Accessory Ecosystem

Friday January 14, 2022 2:34 am PST by
Apple's second-generation AirPods Pro could arrive alongside a new series of accessories, recent leaked images suggest. Alleged leaked photos of the next-generation AirPods Pro obtained by MacRumors showed a charging case with a metal loop on the side for attaching a strap. Apple has not used this design for any of its other AirPod models and it is unclear why it would be added in this...
netflix2

Netflix Again Raises Prices for All Plans, 4K Streaming Now $20 Per Month

Friday January 14, 2022 12:46 pm PST by
Netflix today updated the prices for its streaming plans, and all of its offerings are now more expensive. The Basic plan is now priced at $9.99 per month, the Standard plan is priced at $15.49 per month, and the Premium plan is priced at $19.99 per month. The Basic plan is $1 more expensive, up from $8.99 per month. This plan allows users to watch on just one screen at a time, and it limits ...
tesla carplay solution

Developer Showcases Apple CarPlay Workaround for Teslas

Monday January 17, 2022 7:24 am PST by
A Tesla Model 3 owner has resorted to a workaround to implement Apple CarPlay in his vehicle, amid no sign of official support from Tesla (via Tesla North). Apple CarPlay and Apple Music support are among the most-requested Tesla features, but with no indication that Tesla is willing to implement Apple CarPlay in its vehicles, Polish developer Michał Gapiński took matters into his own...
top stories 20220115

Top Stories: iPhone 14 Pro Rumors, iCloud Private Relay Controversy, iOS 15.2.1 Released, and More

Saturday January 15, 2022 6:00 am PST by
Hole-punch? Pill? Hole-punch and pill? Rumors about what the front camera system on the iPhone 14 Pro will look like are evolving rapidly, and it now appears we might be getting a novel but potentially controversial design later this year. Other major stories this week included some confusion and controversy about iCloud Private Relay being disabled for some T-Mobile customers, increasing...