Researchers Demonstrate Subliminal Smart Device Commands That Have Potential for Malicious Attacks

Researchers in the United States and China have been performing tests in an effort to demonstrate that "hidden" commands, or those undetectable to human ears, can reach AI assistants like Siri and force them to perform actions their owners never intended. The research was highlighted in a piece today by The New York Times, suggesting that these subliminal commands can dial phone numbers, open websites, and more potentially malicious actions if placed in the wrong hands.

A group of students from the University of California, Berkeley and Georgetown University published a research paper this month, stating that they could embed commands into music recordings or spoken text. When played near an Amazon Echo or Apple iPhone, a person would just hear the song or someone speaking, while Siri and Alexa "might hear an instruction to add something to your shopping list." Or, more dangerous, unlock doors, wire money from your bank, and purchase items online.

siri iphone x
The method by which the students were able to accomplish the hidden commands shouldn't be a concern for the public at large, but one of the paper's authors, Nicholas Carlini, believes malicious parties could already be making inroads with similar technology.

“We wanted to see if we could make it even more stealthy,” said Nicholas Carlini, a fifth-year Ph.D. student in computer security at U.C. Berkeley and one of the paper’s authors.

Mr. Carlini added that while there was no evidence that these techniques have left the lab, it may only be a matter of time before someone starts exploiting them. “My assumption is that the malicious people already employ people to do what I do,” he said.

Last year, researchers based at Princeton University and Zheijiang University in China performed similar tests, demonstrating that AI assistants could be activated through frequencies not heard by humans. In a technique dubbed "DolphinAttack," the researchers built a transmitter to send the hidden command that dialed a specific phone number, while other tests took pictures and sent text messages. DolphinAttack is said to be limited in terms of range, however, since it "must be close to the receiving device."

DolphinAttack could inject covert voice commands at 7 state-of-the-art speech recognition systems (e.g., Siri, Alexa) to activate always-on system and achieve various attacks, which include activating Siri to initiate a FaceTime call on iPhone, activating Google Now to switch the phone to the airplane mode, and even manipulating the navigation system in an Audi automobile.

In yet another set of research, a group at the University of Illinois at Urbana-Champaign proved this range limitation could be increased, showing off commands received from 25 feet away. For the most recent group of researchers from Berkeley, Carlini told The New York Times that he was "confident" his team would soon be able to deliver successful commands "against any smart device system on the market." He said the group wants to prove to companies that this flaw is a potential problem, "and then hope that other people will say, 'O.K. this is possible, now let's try and fix it.'"

For security purposes, Apple is stringent with certain HomeKit-related Siri commands, locking them behind device passcodes whenever users have passcodes enabled. For example, if you want to unlock your front door with a connected smart lock, you can ask Siri to do so, but you'll have to enter your passcode on an iPhone or iPad after issuing the command. The HomePod, on the other hand, purposefully lacks this functionality.

Top Rated Comments

nwcs Avatar
45 months ago
This is really clever. I wouldn’t have thought that the AIs would respond to non-vocal frequencies as they’re intended to listen to humans only. I would think that checking the frequency range of the command would be enough to counteract this problem fairly simply.
Score: 18 Votes (Like | Disagree)
daveschroeder Avatar
45 months ago
That is NOT "subliminal".

I think you're looking for another word.
Score: 12 Votes (Like | Disagree)
bluespark Avatar
45 months ago
In fairness, Apple has made great strides in tackling this issue already through Siri's continued uselessness.
Score: 12 Votes (Like | Disagree)
amaier1986 Avatar
45 months ago
HomePod directs me to use my phone to unlock my front door or open my garage doors. This potential issue seems to be somewhat under control with iOS.
Score: 7 Votes (Like | Disagree)
w5jck Avatar
45 months ago
That is NOT "subliminal".

I think you're looking for another word.
My thought is that the word should be "inaudible" and NOT "subliminal". As in, "The devices can react to inaudible commands."
Score: 7 Votes (Like | Disagree)
jarman92 Avatar
45 months ago
This is really clever. I wouldn’t have thought that the AIs would respond to non-vocal frequencies as they’re intended to listen to humans only. I would think that checking the frequency range of the command would be enough to counteract this problem fairly simply.
Agreed. But why wouldn't Apple have foreseen this and limited the frequency range in the first place? There's literally no need for phone mics to detect anything below/above human voice frequencies.
Score: 6 Votes (Like | Disagree)

Related Stories

studio buds family

Beats Studio Buds Debuting Today With Active Noise Cancellation, Stemless Design, and More for $150

Monday June 14, 2021 8:00 am PDT by
We've seen a lot of teasers about the Beats Studio Buds over the past month since they first showed up in Apple's beta software updates, and today they're finally official. The Beats Studio Buds are available to order today in red, white, and black ahead of a June 24 ship date, and they're priced at $149.99. The Studio Buds are the first Beats-branded earbuds to truly compete with AirPods...
gradiente iphone white

Brazilian Electronics Company Revives Long-Running iPhone Trademark Dispute

Tuesday May 19, 2020 1:06 pm PDT by
Apple has been involved in a long-running iPhone trademark dispute in Brazil, which was revived today by IGB Electronica, a Brazilian consumer electronics company that originally registered the "iPhone" name in 2000. IGB Electronica fought a multi-year battle with Apple in an attempt to get exclusive rights to the "iPhone" trademark, but ultimately lost, and now the case has been brought to...
youtube apple tv

YouTube Discontinuing 3rd-Generation Apple TV App, AirPlay Still Available

Wednesday February 3, 2021 3:09 pm PST by
YouTube is planning to stop supporting its YouTube app on the third-generation Apple TV models, where YouTube has long been available as a channel option. A 9to5Mac reader received a message about the upcoming app discontinuation, which is set to take place in March.Starting early March, the YouTube app will no longer be available on Apple TV (3rd generation). You can still watch YouTube on...
YouTube Picture in Picture Feature

YouTube Premium Subscribers Can Now Use iOS Picture-in-Picture: Here's How

Wednesday August 25, 2021 3:55 am PDT by
Google has rolled out picture-in-picture support as an "experimental" feature for YouTube premium subscribers, allowing them to watch video in a small window when the app is closed. If you're a premium YouTube subscriber looking to try out picture-in-picture, follow these steps: Launch a web browser and sign into your YouTube account at YouTube.com. Navigate to www.youtube.com/new. Scroll...
2012macpro

Apple Outlines Metal-Capable Cards Compatible With macOS Mojave on 2010 and 2012 Mac Pro Models

Monday September 24, 2018 3:26 pm PDT by
Apple's new macOS Mojave update is not compatible with mid-2010 and mid-2012 Mac Pros with stock GPUs, but it is supported on 2010 and 2012 Mac Pro models that have been upgraded with graphics cards that support Metal. Apple today shared a new support document that provides a list of graphics cards that are Metal-capable, which will be useful for 2010 and 2012 Mac Pro owners who want to...
apple privacy

Apple Publishes FAQ to Address Concerns About CSAM Detection and Messages Scanning

Monday August 9, 2021 1:50 am PDT by
Apple has published a FAQ titled "Expanded Protections for Children" which aims to allay users' privacy concerns about the new CSAM detection in iCloud Photos and communication safety for Messages features that the company announced last week. "Since we announced these features, many stakeholders including privacy organizations and child safety organizations have expressed their support of...
iphone 13 teal with text

Apple Expecting iPhone and iPad Supply Constraints in September Quarter

Tuesday July 27, 2021 2:34 pm PDT by
During today's earnings call covering the third fiscal quarter of 2021 (second calendar quarter), Apple CFO Luca Maesteri said that Apple is expecting supply constraints to affect the iPhone and the iPad in the coming quarter. "The supply constraints that we've seen in the June quarter will be higher in the September quarter," said Maestri. The constraints will impact iPhone and iPad sales...
anker lightning cable mfi

Unwrap a New Apple Device? Stock Up on Extra Certified Lightning Cables for as Little as $6

Monday December 25, 2017 5:45 am PST by
If you unwrapped an Apple product today it likely came with one of the company's first-party Lightning cables, but having an extra on hand is always a good idea, so you can place it in other rooms in your house, in your car, or in a bag when you travel. For that reason, now's a good time to shop for third-party Lightning cables that are cheaper than Apple's own accessory, but still Made For...
apple screen time screen icons

Persistent Kids Finding Loopholes in Apple's Screen Time Limits

Tuesday October 15, 2019 9:44 am PDT by
Apple is currently engaged in a cat-and-mouse game with persistent kids looking to circumvent Screen Time restrictions, but the company has been receiving some criticism for not moving quickly enough to lock down some of the loopholes, reports The Washington Post. A few of the loopholes and ways for parents to shut them down are documented on the site Protect Young Eyes, while these and...
os x mountain lion macs 16x9 2

Apple Makes OS X Lion and Mountain Lion Free to Download

Wednesday June 30, 2021 12:19 pm PDT by
Apple recently dropped the $19.99 fee for OS X Lion and Mountain Lion, making the older Mac updates free to download, reports Macworld. Apple has kept OS X 10.7 Lion and OS X 10.8 Mountain Lion available for customers who have machines limited to the older software, but until recently, Apple was charging $19.99 to get download codes for the updates. As of last week, these updates no...