Researchers Demonstrate Subliminal Smart Device Commands That Have Potential for Malicious Attacks

Researchers in the United States and China have been performing tests in an effort to demonstrate that "hidden" commands, or those undetectable to human ears, can reach AI assistants like Siri and force them to perform actions their owners never intended. The research was highlighted in a piece today by The New York Times, suggesting that these subliminal commands can dial phone numbers, open websites, and more potentially malicious actions if placed in the wrong hands.

A group of students from the University of California, Berkeley and Georgetown University published a research paper this month, stating that they could embed commands into music recordings or spoken text. When played near an Amazon Echo or Apple iPhone, a person would just hear the song or someone speaking, while Siri and Alexa "might hear an instruction to add something to your shopping list." Or, more dangerous, unlock doors, wire money from your bank, and purchase items online.

siri iphone x
The method by which the students were able to accomplish the hidden commands shouldn't be a concern for the public at large, but one of the paper's authors, Nicholas Carlini, believes malicious parties could already be making inroads with similar technology.

“We wanted to see if we could make it even more stealthy,” said Nicholas Carlini, a fifth-year Ph.D. student in computer security at U.C. Berkeley and one of the paper’s authors.

Mr. Carlini added that while there was no evidence that these techniques have left the lab, it may only be a matter of time before someone starts exploiting them. “My assumption is that the malicious people already employ people to do what I do,” he said.

Last year, researchers based at Princeton University and Zheijiang University in China performed similar tests, demonstrating that AI assistants could be activated through frequencies not heard by humans. In a technique dubbed "DolphinAttack," the researchers built a transmitter to send the hidden command that dialed a specific phone number, while other tests took pictures and sent text messages. DolphinAttack is said to be limited in terms of range, however, since it "must be close to the receiving device."

DolphinAttack could inject covert voice commands at 7 state-of-the-art speech recognition systems (e.g., Siri, Alexa) to activate always-on system and achieve various attacks, which include activating Siri to initiate a FaceTime call on iPhone, activating Google Now to switch the phone to the airplane mode, and even manipulating the navigation system in an Audi automobile.

In yet another set of research, a group at the University of Illinois at Urbana-Champaign proved this range limitation could be increased, showing off commands received from 25 feet away. For the most recent group of researchers from Berkeley, Carlini told The New York Times that he was "confident" his team would soon be able to deliver successful commands "against any smart device system on the market." He said the group wants to prove to companies that this flaw is a potential problem, "and then hope that other people will say, 'O.K. this is possible, now let's try and fix it.'"

For security purposes, Apple is stringent with certain HomeKit-related Siri commands, locking them behind device passcodes whenever users have passcodes enabled. For example, if you want to unlock your front door with a connected smart lock, you can ask Siri to do so, but you'll have to enter your passcode on an iPhone or iPad after issuing the command. The HomePod, on the other hand, purposefully lacks this functionality.

Top Rated Comments

nwcs Avatar
42 months ago
This is really clever. I wouldn’t have thought that the AIs would respond to non-vocal frequencies as they’re intended to listen to humans only. I would think that checking the frequency range of the command would be enough to counteract this problem fairly simply.
Score: 18 Votes (Like | Disagree)
daveschroeder Avatar
42 months ago
That is NOT "subliminal".

I think you're looking for another word.
Score: 12 Votes (Like | Disagree)
bluespark Avatar
42 months ago
In fairness, Apple has made great strides in tackling this issue already through Siri's continued uselessness.
Score: 12 Votes (Like | Disagree)
amaier1986 Avatar
42 months ago
HomePod directs me to use my phone to unlock my front door or open my garage doors. This potential issue seems to be somewhat under control with iOS.
Score: 7 Votes (Like | Disagree)
w5jck Avatar
42 months ago
That is NOT "subliminal".

I think you're looking for another word.
My thought is that the word should be "inaudible" and NOT "subliminal". As in, "The devices can react to inaudible commands."
Score: 7 Votes (Like | Disagree)
jarman92 Avatar
42 months ago
This is really clever. I wouldn’t have thought that the AIs would respond to non-vocal frequencies as they’re intended to listen to humans only. I would think that checking the frequency range of the command would be enough to counteract this problem fairly simply.
Agreed. But why wouldn't Apple have foreseen this and limited the frequency range in the first place? There's literally no need for phone mics to detect anything below/above human voice frequencies.
Score: 6 Votes (Like | Disagree)

Top Stories

REC ASA CODE2016 20160601 205816 2745

Elon Musk Reportedly Demanded to Become Apple CEO as Part of Potential Tesla Acquisition [Update: Musk Denies]

Friday July 30, 2021 9:04 am PDT by
Tesla CEO Elon Musk reportedly once demanded that he be made Apple CEO in a brief discussion of a potential acquisition with Apple's current CEO, Tim Cook. The claim comes in a new book titled "Power Play: Tesla, Elon Musk and the Bet of the Century," as reviewed by The Los Angeles Times. According to the book, during a 2016 phone call between Musk and Cook that touched on the possibility of ...
General Apps Messages

Android iMessage Competitor Puts Pressure on Apple

Friday July 30, 2021 3:15 am PDT by
Google and the three major U.S. carriers, including Verizon, AT&T, and T-Mobile, will all support a new communications protocol on Android smartphones starting in 2022, a move that puts pressure on Apple to adopt a new cross-platform messaging standard and may present a challenge to iMessage. Verizon recently announced that it is planning to adopt Messages by Google as its default messaging...
Apple watch series 5 new case material made of titanium 091019

Titanium Apple Watch Series 6 Models Currently Widely Unavailable

Sunday August 1, 2021 6:21 am PDT by
Models of the Apple Watch Series 6 with titanium cases part of the "Apple Watch Edition" collection is currently widely unavailable for pick-up in several of Apple's retail stores in the United States and is unavailable entirely for delivery in major markets. Noted by Bloomberg's Mark Gurman in the latest edition of his "Power On" newsletter, titanium models of the Apple Watch Series 6,...
ifixit iphone12 mini

Apple to Make Space for Larger Batteries in iPhones, iPads, and MacBooks By Adopting Slimmer Peripheral Chips

Monday August 2, 2021 2:12 am PDT by
For future iPhones, iPads, and MacBooks, Apple plans to use smaller internal components in an effort to increase the size of the device's battery, according to DigiTimes. Image Credit: iFixit Specifically, Apple plans to "significantly increase the adoption" of IPDs or integrated passive devices for the peripheral chips in its products. These news chips will be slimmer in size and allow for...
Flat 2021 MacBook Pro Mockup Feature

Unreleased Apple Macs and Apple Watches Listed in Eurasian Database Ahead of Fall Product Launches

Monday August 2, 2021 9:34 am PDT by
Apple is preparing for a slew of fall product launches according to new filings that showed up today in the Eurasian Economic Commission database. There are listings for new Mac and Apple Watch models, all of which have previously unknown model identifiers that indicate that they're upcoming devices. There are six new Apple Watch identifiers, including A2473, A2474, A2475, A2476, A2477, and...
a15 chip

iPhone 13 and Redesigned MacBook Pro Chip Production Hit With Gas Contamination

Friday July 30, 2021 5:44 am PDT by
The most important TSMC factory that manufactures Apple's chips destined for next-generation iPhone and Mac models has been hit by a gas contamination, according to Nikkei Asia. The factory, known as "Fab 18," is TSMC's most advanced chipmaking facility. TSMC is Apple's sole chip supplier, making all of the processors used in every Apple device with a custom silicon chip. Industry...
iPhone 13 Wi Fi 6E feature update

Wi-Fi 6E Explained: What It Could Mean for iPhone 13 and Beyond

Monday August 2, 2021 8:00 am PDT by
The iPhone 13 is widely expected to come with Wi-Fi 6E capabilities, and while it may seem rather nuanced to the average consumer, with only improved speeds and being "up to date" in the realm of Wi-Fi technology, it's actually a fairly significant improvement, laying the groundwork for much of what we know the future holds. To truly understand Wi-Fi 6E, MacRumors sat down for an exclusive...
youtube premium lite

YouTube Tests Cheaper 'Premium Lite' Subscription for Ad-Free Viewing

Monday August 2, 2021 3:22 am PDT by
YouTube is piloting a new cheaper subscription tier in Europe called "Premium Lite," which offers ad-free viewing minus YouTube Premium's other features. First spotted by a user on ResetEra and subsequently confirmed by Google, the "Lite" plan means users who aren't interested in offline downloads or background playback can still enjoy YouTube videos on web and mobile app without being...
iPhone 13 Always On Feature

iPhone 13 to Bring Over a Major Feature From the Apple Watch

Wednesday July 28, 2021 2:21 am PDT by
Apple's upcoming iPhone 13 lineup will feature an always-on display akin to the Apple Watch Series 5 and Series 6, according to recent reports. In his weekly Power On newsletter, Bloomberg journalist Mark Gurman, who often reveals accurate insights into Apple's plans, said that the iPhone 13 may feature an Apple Watch-inspired always-on mode. The Apple Watch Series 5 and Apple Watch...
COVID19 Digital Wallet Apple Wallet

Australian Government Now Offering COVID-19 Digital Vaccination Certificates for Apple Wallet

Monday August 2, 2021 12:04 am PDT by
The Australian government has introduced support for adding COVID-19 vaccination digital certificates to Apple Wallet via the Express Plus Medicare app on iOS. Image credit: Tap Down Under As spotted by Tap Down Under, users who have received two doses of either the AstraZeneca or Pfizer vaccine now have access to the digital certificate through their Medicare online account or via the Medica...