Siri on HomePod Correctly Answered 52.3% of Queries in New AI Test

Apple's Mac event will be starting shortly – Follow our Live Coverage.

In a new test shared today by Loup Ventures, Apple's HomePod was put through its paces in categories including Siri, sound quality, and ease of use. For Siri, Loup Ventures' Gene Munster reported that while the AI assistant understood 99.4 percent of queries asked of it, it answered only 52.3 percent of them correctly. Loup Ventures tested three separate HomePods and asked 782 queries total.

Compared to previous tests of rival speakers, HomePod is "at the bottom of the totem pole" in the AI assistant performance category. Google Home answered 81 percent correctly, Amazon's Alexa answered 64 percent correctly, and Microsoft's Cortana answered 57 percent correctly.

mitchs homepod on shelf
Munster broke this information down further, stating that Siri is good for "local" and "commerce" queries, like asking about nearby coffee shops or assisting in buying new shoes. In this area, Siri beats Alexa and Cortana but still falls behind Google Assistant on Google Home.

Despite the low percentage of correctly answering the 782 total queries asked of it, Munster said Siri's overall performance rose above expectations "given the limited scope of HomePod's music focus."

homepod siri loup ventures

Chart via Loup Ventures

The researchers explained that over time HomePod and Siri should grow to match, or surpass, rival assistants by simply adding query domains like calendar, email, calling, and navigation.

Some domains like navigation, calendar, email, and calling are simply not supported. These questions were met with, “I can’t ___ on HomePod.” Also, in any case that iPhone-based Siri would bring up Google search results, HomePod would reply, “I can’t get the answer to that on HomePod,” which forces you to use your phone or give up on the question altogether. Removing navigation, calling, email, and calendar-related queries from our question set yields a 67% correct response, a jump from overall of 52.3% correct.

This means added support for these domains would bring HomePod performance above that of Alexa (64%) and Cortana (57%), though still shy of Google Home (81%). We know Siri has the ability to correctly answer a whole range of queries that HomePod cannot, evidenced by our note here. Apple’s limiting of HomePod’s domains should change over time, at which point we expect the speaker to be vastly more useful and integrated with your other Apple devices.

As discovered in the research, where HomePod excelled was its "superior" listening skills. The HomePod allows users to speak at a normal volume, even when music from the speaker is particularly loud, and Siri will pick up on the voice and hear the user. "This was HomePod's most stellar feature," according to Munster.

Loup Ventures also favored HomePod's sound quality, saying that "it sounds incredible." Like other reviews and impressions of the device, Munster's enjoyment of the HomePod's audio quality was countered with the speaker's lackluster Siri performance, which he thinks will be changing fairly soon.

Don’t be fooled by HomePod’s sound quality-focused first step into smart speakers; Apple has a grander vision than delivering a better sounding Echo. While not present in the first version of HomePod (i.e. you can’t even make a phone call with HomePod), we believe Apple’s goal is to make Siri a ubiquitous, ambient presence that connects and controls all your connected devices and services – and to make a leap forward in the transition to voice-first computing.

Visit Loup Ventures to read more of the information gathered from the HomePod "smart speaker gauntlet," including the publication's prediction for demand and market share of HomePod through the rest of 2018.

Related Roundup: HomePod
Buyer's Guide: HomePod (Don't Buy)

Top Rated Comments

RudySnow Avatar
48 months ago
Me: “Hey Siri, why are you so terrible at answering questions?”

Siri: “I found several articles on the web about crustaceans.”
Score: 132 Votes (Like | Disagree)
Mobster1983 Avatar
48 months ago
That's pretty damn pathetic...
Score: 108 Votes (Like | Disagree)
SeattleMoose Avatar
48 months ago
Siri is Apple's biggest failure. They have had years to make it at least 80% functional and have failed to do so. Just like the ATV, it seems to be just a "hobby" for Apple.
Score: 84 Votes (Like | Disagree)
unobtainium Avatar
48 months ago
Siri sucks. I don’t think I’ve used it in over a year.
Score: 77 Votes (Like | Disagree)
jasvncnt Avatar
48 months ago
What good is "superior" listeng skills if it can't answer many of the questions?
Score: 75 Votes (Like | Disagree)
palmerc2 Avatar
48 months ago
I think Siri is more than adequate for what most need it for the HomePod. Is it better than Google Home? No. Does it need to be better than the competition? Not necessarily. I use Siri all the time and I really don’t have any general complaints, aside from not understanding some of the things I ask it. None of these voice assistants are perfect.
I can only imagine the amount of gloating on these forums if Siri was #1 on the list instead of Google.
Score: 38 Votes (Like | Disagree)

Related Stories

studio buds family

Beats Studio Buds Debuting Today With Active Noise Cancellation, Stemless Design, and More for $150

Monday June 14, 2021 8:00 am PDT by
We've seen a lot of teasers about the Beats Studio Buds over the past month since they first showed up in Apple's beta software updates, and today they're finally official. The Beats Studio Buds are available to order today in red, white, and black ahead of a June 24 ship date, and they're priced at $149.99. The Studio Buds are the first Beats-branded earbuds to truly compete with AirPods...
YouTube Picture in Picture Feature

YouTube Premium Subscribers Can Now Use iOS Picture-in-Picture: Here's How

Wednesday August 25, 2021 3:55 am PDT by
Google has rolled out picture-in-picture support as an "experimental" feature for YouTube premium subscribers, allowing them to watch video in a small window when the app is closed. If you're a premium YouTube subscriber looking to try out picture-in-picture, follow these steps: Launch a web browser and sign into your YouTube account at YouTube.com. Navigate to www.youtube.com/new. Scroll...
M1X MBP Feature

Leaker: Upcoming MacBook Pro to See Price Hike Over Current Model, Equal Performance Across 14 and 16-Inch Sizes

Tuesday August 24, 2021 5:28 am PDT by
The upcoming 14-inch MacBook Pro is set to be more expensive than the current 13-inch MacBook Pro and both the 14 and 16-inch models will offer the same performance, according to the leaker known as "Dylandkt." The leaker shared the information on Twitter, explaining that both of the upcoming MacBook Pro models, expected to come in 14 and 16-inch sizes, will feature the same performance due...
youtube apple tv

YouTube Discontinuing 3rd-Generation Apple TV App, AirPlay Still Available

Wednesday February 3, 2021 3:09 pm PST by
YouTube is planning to stop supporting its YouTube app on the third-generation Apple TV models, where YouTube has long been available as a channel option. A 9to5Mac reader received a message about the upcoming app discontinuation, which is set to take place in March.Starting early March, the YouTube app will no longer be available on Apple TV (3rd generation). You can still watch YouTube on...
ryanscoolios13thumbnail

Apple Releases iOS 13.1.3 With Bug Fixes for Phone, Mail, Health, and More

Tuesday October 15, 2019 10:09 am PDT by
Apple today released iOS 13.1.3 and iPadOS 13.1.3, minor updates to the iOS 13.1.2 software that was released two weeks ago. This is the fourth update to the iOS 13 operating system that came out in September. The iOS and iPadOS 13.1.3 updates are available on all eligible devices over-the-air in the Settings app. To access the updates, go to Settings > General > Software Update....
Top Stories 75 Thumbnail

Top Stories: Last-Minute iPhone 13 Rumors, Apple Announces App Store Changes, and More

Saturday September 4, 2021 6:00 am PDT by
The finish line is in sight! Apple's annual iPhone event is likely just a week or so away and all eyes will be on the company as it unveils the next version of its most popular product line. With any luck, we'll also see the next-generation Apple Watch and perhaps even some new AirPods. Other news this week saw Apple making some more changes to its App Store policies in response to a...
affinity designer contour tool

Serif Updates Affinity Photo, Designer, and Publisher With New Tools and Functions

Thursday February 4, 2021 1:58 am PST by
Serif today announced across-the-board updates for its popular suite of Affinity creative apps, including Affinity Photo, Affinity Designer, and the Apple award-winning Affinity Publisher for Mac, all of which were among the first professional creative suites to be optimized for Apple's new M1 chip. "After another year which saw record numbers of people switching to Affinity, it's exciting to...
iPhone 13 Dummy Thumbnail 2

Kuo: iPhone 13 to Feature LEO Satellite Communications to Make Calls and Texts Without Cellular Coverage

Sunday August 29, 2021 7:39 am PDT by
The iPhone 13 will feature low earth orbit (LEO) satellite communication connectivity to allow users to make calls and send messages in areas without 4G or 5G coverage, according to the reliable analyst Ming-Chi Kuo. In a note to investors, seen by MacRumors, Kuo explained that the iPhone 13 lineup will feature hardware that is able to connect to LEO satellites. If enabled with the relevant...
maxresdefault

Craig Federighi and Greg Joswiak Discuss iPadOS 15, macOS Monterey, Privacy, Shortcuts on Mac, and More

Saturday June 12, 2021 6:12 am PDT by
As is tradition, Apple executives Craig Federighi and Greg Joswiak joined Daring Fireball's John Gruber in an episode of The Talk Show to discuss several announcements that Apple made over this weeks WWDC, including iPadOS 15, macOS Monterey, and a large focus around privacy. Federighi kicks off the conversation discussing the common architecture, now thanks to Apple silicon, across all of...
iOS 14 on iPhone feature emergency

Apple Seeds Third Betas of iOS and iPadOS 14.6 to Developers [Update: Public Beta Available]

Monday May 10, 2021 10:13 am PDT by
Apple today seeded the third betas of new iOS and iPadOS 14.6 updates to developers for testing purposes, one week after seeding the second iOS and iPadOS 14.6 updates. iOS and iPadOS 14.6 can be downloaded through the Apple Developer Center or over the air after the proper profile has been installed on an iPhone or iPad. iOS and iPadOS 14.6 appear to be more minor changes focusing on...