Researchers Demonstrate Subliminal Smart Device Commands That Have Potential for Malicious Attacks

Researchers in the United States and China have been performing tests in an effort to demonstrate that "hidden" commands, or those undetectable to human ears, can reach AI assistants like Siri and force them to perform actions their owners never intended. The research was highlighted in a piece today by The New York Times, suggesting that these subliminal commands can dial phone numbers, open websites, and more potentially malicious actions if placed in the wrong hands.

A group of students from the University of California, Berkeley and Georgetown University published a research paper this month, stating that they could embed commands into music recordings or spoken text. When played near an Amazon Echo or Apple iPhone, a person would just hear the song or someone speaking, while Siri and Alexa "might hear an instruction to add something to your shopping list." Or, more dangerous, unlock doors, wire money from your bank, and purchase items online.

siri iphone x
The method by which the students were able to accomplish the hidden commands shouldn't be a concern for the public at large, but one of the paper's authors, Nicholas Carlini, believes malicious parties could already be making inroads with similar technology.

“We wanted to see if we could make it even more stealthy,” said Nicholas Carlini, a fifth-year Ph.D. student in computer security at U.C. Berkeley and one of the paper’s authors.

Mr. Carlini added that while there was no evidence that these techniques have left the lab, it may only be a matter of time before someone starts exploiting them. “My assumption is that the malicious people already employ people to do what I do,” he said.

Last year, researchers based at Princeton University and Zheijiang University in China performed similar tests, demonstrating that AI assistants could be activated through frequencies not heard by humans. In a technique dubbed "DolphinAttack," the researchers built a transmitter to send the hidden command that dialed a specific phone number, while other tests took pictures and sent text messages. DolphinAttack is said to be limited in terms of range, however, since it "must be close to the receiving device."

DolphinAttack could inject covert voice commands at 7 state-of-the-art speech recognition systems (e.g., Siri, Alexa) to activate always-on system and achieve various attacks, which include activating Siri to initiate a FaceTime call on iPhone, activating Google Now to switch the phone to the airplane mode, and even manipulating the navigation system in an Audi automobile.

In yet another set of research, a group at the University of Illinois at Urbana-Champaign proved this range limitation could be increased, showing off commands received from 25 feet away. For the most recent group of researchers from Berkeley, Carlini told The New York Times that he was "confident" his team would soon be able to deliver successful commands "against any smart device system on the market." He said the group wants to prove to companies that this flaw is a potential problem, "and then hope that other people will say, 'O.K. this is possible, now let's try and fix it.'"

For security purposes, Apple is stringent with certain HomeKit-related Siri commands, locking them behind device passcodes whenever users have passcodes enabled. For example, if you want to unlock your front door with a connected smart lock, you can ask Siri to do so, but you'll have to enter your passcode on an iPhone or iPad after issuing the command. The HomePod, on the other hand, purposefully lacks this functionality.

Top Rated Comments

nwcs Avatar
35 months ago
This is really clever. I wouldn’t have thought that the AIs would respond to non-vocal frequencies as they’re intended to listen to humans only. I would think that checking the frequency range of the command would be enough to counteract this problem fairly simply.
Score: 18 Votes (Like | Disagree)
daveschroeder Avatar
35 months ago
That is NOT "subliminal".

I think you're looking for another word.
Score: 12 Votes (Like | Disagree)
bluespark Avatar
35 months ago
In fairness, Apple has made great strides in tackling this issue already through Siri's continued uselessness.
Score: 12 Votes (Like | Disagree)
amaier1986 Avatar
35 months ago
HomePod directs me to use my phone to unlock my front door or open my garage doors. This potential issue seems to be somewhat under control with iOS.
Score: 7 Votes (Like | Disagree)
w5jck Avatar
35 months ago

That is NOT "subliminal".

I think you're looking for another word.

My thought is that the word should be "inaudible" and NOT "subliminal". As in, "The devices can react to inaudible commands."
Score: 7 Votes (Like | Disagree)
jarman92 Avatar
35 months ago

This is really clever. I wouldn’t have thought that the AIs would respond to non-vocal frequencies as they’re intended to listen to humans only. I would think that checking the frequency range of the command would be enough to counteract this problem fairly simply.

Agreed. But why wouldn't Apple have foreseen this and limited the frequency range in the first place? There's literally no need for phone mics to detect anything below/above human voice frequencies.
Score: 6 Votes (Like | Disagree)

Top Stories

16inchmacbookpromain

Kuo: New MacBook Pro Models to Feature Flat-Edged Design, MagSafe, No Touch Bar and More Ports

Thursday January 14, 2021 9:32 pm PST by
Apple is working on two new MacBook Pro models that will feature significant design changes, well-respected Apple analyst Ming-Chi Kuo said today in a note to investors that was obtained by MacRumors. According to Kuo, Apple is developing two models in 14 and 16-inch size options. The new MacBook Pro machines will feature a flat-edged design, which Kuo describes as "similar to the iPhone 12" ...
iphone x camera close

iOS 14.4 Will Introduce Warning on iPhones With Non-Genuine Cameras

Thursday January 14, 2021 8:07 am PST by
In the second beta of iOS 14.4 seeded to developers and public testers this week, MacRumors contributor Steve Moser has discovered code indicating that Apple will be introducing a new warning on iPhones that have had their camera repaired or replaced with aftermarket components rather than genuine Apple components. "Unable to verify this iPhone has a genuine Apple camera," the message will...
prototype iphone 12 pro

Prototype iPhone 12 Pro Shown Off in Photos

Wednesday January 13, 2021 3:39 pm PST by
Developer Giulio Zompetti, who often shows off prototype versions of Apple devices, today highlighted a prototype version of the iPhone 12 Pro. The iPhone 12 Pro is running an operating system called SwitchBoard, a nonUI version of the iOS 14 update that Apple uses internally. We've seen SwitchBoard on prototype devices before, as Apple uses it to test new features. Zompetti's prototype...
foldable iPhone concept feature

Apple Testing In-Display Fingerprint Sensor for iPhone 13, Foldable iPhone Also in the Works

Friday January 15, 2021 1:46 pm PST by
Apple has started "early work" on an iPhone that has a foldable display, according to a new report from Bloomberg's Mark Gurman. Though testing of a foldable iPhone has begun, Apple has not committed to releasing a device that has a foldable display. Development has not yet expanded beyond a display and Apple does not have full foldable iPhone prototypes in its labs. Like foldable...
Apple TV Ray Light 2 Triad

Apple Extends Free Apple TV+ Trials Until July

Friday January 15, 2021 10:50 am PST by
Apple is once again planning to extend its free Apple TV+ trial subscriptions, this time until July. When Apple TV+ launched in November 2019, Apple offered free Apple TV+ subscriptions to those who purchased a new Apple device in or after September 2019. Those free subscriptions were set to expire in November 2020, but in October 2020, Apple announced that it was extending free trials...
macbook pro 16 inch thunderbolt

Bloomberg: Next-Generation MacBook Pro to Offer Improved Displays, Faster Charging Over MagSafe

Thursday January 14, 2021 11:36 pm PST by
Following today's report from analyst Ming-Chi Kuo outlining major changes for the next-generation MacBook Pro models coming in the third quarter of this year, Bloomberg's Mark Gurman has weighed in with his own report corroborating some of the details but seemingly differing a bit on others. First, Gurman shares more details on the return of MagSafe charging to the MacBook Pro, indicating...
Hue module dimmer switch

Philips Hue Announces New Wall Switch Module, Dimmer Switch, and Outdoor Light Bar

Thursday January 14, 2021 3:11 am PST by
Philips Hue has announced a new wireless dimmer switch module that lets Hue bridge owners directly control the smart lighting from their standard wall switches. The new Philips Hue wall switch module is the ideal addition to any Philips Hue set up. Installed behind existing light switches, it allows users to turn their existing switch into a smart switch and ensures their smart lighting is...
pat gelsinger intel

Incoming Intel CEO Derides Company's Inability to 'Deliver Better Products' Than Apple's M1 Chip

Friday January 15, 2021 6:17 am PST by
Incoming Intel CEO Pat Gelsinger has said that the company must "deliver better products" than Apple, which he described as a "lifestyle company," and says that Intel's best days are "in front of it" (via The Oregonian). Speaking at an Intel all-hands meeting yesterday, Gelsinger derisively implied that Apple is merely a "lifestyle company," so Intel must be able to surpass its technology: ...
mac pro mini feature

Apple Working on Two New Mac Pro Desktops, One of Which Will Be Reminiscent of Power Mac G4 Cube

Friday January 15, 2021 10:23 am PST by
Apple is developing two versions of the Mac Pro to succeed the Mac Pro that was first released in December 2019, according to a new report from Bloomberg. The first updated Mac Pro is a direct successor to the current Mac Pro and it will use the same design. It may also be equipped with Intel processors rather than Apple silicon chips, and it could be one of the sole machines in the Mac...
pioneer carplay wc5700nex

The Best Apple-Related Accessories at CES 2021

Wednesday January 13, 2021 1:16 pm PST by
CES 2021 is taking place digitally this year, and it hasn't been as exciting as in past years because many vendors have opted out. That said, some companies are still showing off some interesting Apple-related accessories that are coming out this year and that will be of interest to Mac, iPad, and iPhone users. Subscribe to the MacRumors YouTube channel for more videos. Pioneer Wireless...