Siri on HomePod Correctly Answered 52.3% of Queries in New AI Test

In a new test shared today by Loup Ventures, Apple's HomePod was put through its paces in categories including Siri, sound quality, and ease of use. For Siri, Loup Ventures' Gene Munster reported that while the AI assistant understood 99.4 percent of queries asked of it, it answered only 52.3 percent of them correctly. Loup Ventures tested three separate HomePods and asked 782 queries total.

Compared to previous tests of rival speakers, HomePod is "at the bottom of the totem pole" in the AI assistant performance category. Google Home answered 81 percent correctly, Amazon's Alexa answered 64 percent correctly, and Microsoft's Cortana answered 57 percent correctly.

mitchs homepod on shelf
Munster broke this information down further, stating that Siri is good for "local" and "commerce" queries, like asking about nearby coffee shops or assisting in buying new shoes. In this area, Siri beats Alexa and Cortana but still falls behind Google Assistant on Google Home.

Despite the low percentage of correctly answering the 782 total queries asked of it, Munster said Siri's overall performance rose above expectations "given the limited scope of HomePod's music focus."

homepod siri loup ventures

Chart via Loup Ventures

The researchers explained that over time HomePod and Siri should grow to match, or surpass, rival assistants by simply adding query domains like calendar, email, calling, and navigation.

Some domains like navigation, calendar, email, and calling are simply not supported. These questions were met with, “I can’t ___ on HomePod.” Also, in any case that iPhone-based Siri would bring up Google search results, HomePod would reply, “I can’t get the answer to that on HomePod,” which forces you to use your phone or give up on the question altogether. Removing navigation, calling, email, and calendar-related queries from our question set yields a 67% correct response, a jump from overall of 52.3% correct.

This means added support for these domains would bring HomePod performance above that of Alexa (64%) and Cortana (57%), though still shy of Google Home (81%). We know Siri has the ability to correctly answer a whole range of queries that HomePod cannot, evidenced by our note here. Apple’s limiting of HomePod’s domains should change over time, at which point we expect the speaker to be vastly more useful and integrated with your other Apple devices.

As discovered in the research, where HomePod excelled was its "superior" listening skills. The HomePod allows users to speak at a normal volume, even when music from the speaker is particularly loud, and Siri will pick up on the voice and hear the user. "This was HomePod's most stellar feature," according to Munster.

Loup Ventures also favored HomePod's sound quality, saying that "it sounds incredible." Like other reviews and impressions of the device, Munster's enjoyment of the HomePod's audio quality was countered with the speaker's lackluster Siri performance, which he thinks will be changing fairly soon.

Don’t be fooled by HomePod’s sound quality-focused first step into smart speakers; Apple has a grander vision than delivering a better sounding Echo. While not present in the first version of HomePod (i.e. you can’t even make a phone call with HomePod), we believe Apple’s goal is to make Siri a ubiquitous, ambient presence that connects and controls all your connected devices and services – and to make a leap forward in the transition to voice-first computing.

Visit Loup Ventures to read more of the information gathered from the HomePod "smart speaker gauntlet," including the publication's prediction for demand and market share of HomePod through the rest of 2018.

Top Rated Comments

RudySnow Avatar
59 months ago
Me: “Hey Siri, why are you so terrible at answering questions?”

Siri: “I found several articles on the web about crustaceans.”
Score: 132 Votes (Like | Disagree)
Mobster1983 Avatar
59 months ago
That's pretty damn pathetic...
Score: 108 Votes (Like | Disagree)
SeattleMoose Avatar
59 months ago
Siri is Apple's biggest failure. They have had years to make it at least 80% functional and have failed to do so. Just like the ATV, it seems to be just a "hobby" for Apple.
Score: 84 Votes (Like | Disagree)
unobtainium Avatar
59 months ago
Siri sucks. I don’t think I’ve used it in over a year.
Score: 77 Votes (Like | Disagree)
jasvncnt Avatar
59 months ago
What good is "superior" listeng skills if it can't answer many of the questions?
Score: 75 Votes (Like | Disagree)
palmerc2 Avatar
59 months ago
I think Siri is more than adequate for what most need it for the HomePod. Is it better than Google Home? No. Does it need to be better than the competition? Not necessarily. I use Siri all the time and I really don’t have any general complaints, aside from not understanding some of the things I ask it. None of these voice assistants are perfect.
I can only imagine the amount of gloating on these forums if Siri was #1 on the list instead of Google.
Score: 38 Votes (Like | Disagree)

Popular Stories

airpods pro black background

AirPods Pro 2: Five New Features and Improvements to Expect

Sunday August 14, 2022 3:28 pm PDT by
Apple's second-generation AirPods Pro are finally nearing launch, with a release expected later this year. If you are considering upgrading to the new AirPods Pro once they are released, keep reading for a list of five new features to expect. In addition to all-new features, the second-generation AirPods Pro will likely adopt some features added to the standard AirPods last year. H2 Chip ...
iPhone 14 Dummies 1 Feature

Everything Rumored for Apple's September Event: iPhone 14, Apple Watch Pro and More

Friday August 12, 2022 2:34 pm PDT by
There's just about a month to go until Apple holds its annual September event focusing on new iPhone and Apple Watch models. We thought we'd take a quick look at everything that's rumored for Apple's September event to give MacRumors readers a rundown on what to expect when the first fall event rolls around. iPhone 14 The iPhone 14 can probably be described more as an "iPhone 13S" because...
10th Generation iPad Render

10th-Generation iPad With Major Design Changes Reportedly in Production Ahead of September Launch

Monday August 15, 2022 8:02 pm PDT by
Apple's rumored 10th-generation iPad is currently in production and will feature "major" design changes, according to a report from Taiwanese website DigiTimes. A mockup of the potential 10th-generation iPad design by Renders By Shailesh The report did not provide any specific details about the 10th-generation iPad's new design, but rumors suggest the device will feature a larger 10.5-inch...
ios 16 lock screen feature2

Apple Seeds Sixth Betas of iOS 16 and iPadOS 16 to Developers [Update: Public Beta Available]

Monday August 15, 2022 10:04 am PDT by
Apple today seeded the sixth betas of upcoming iOS 16 and iPadOS 16 updates to developers for testing purposes, with the updates coming a week after Apple released the fifth developer betas. Registered developers can download the iOS and iPadOS 16 profiles from the Apple Developer Center, and once installed, the betas will be available over the air. iOS 16 introduces a revamped Lock...
apple watch se

Apple Watch SE vs. Apple Watch Series 8: New Features to Expect If You've Waited to Upgrade

Monday August 15, 2022 2:44 am PDT by
The Apple Watch SE was announced in September 2020 and has been a popular Apple Watch model for customers looking for their first smartwatch or an affordable Apple Watch. Apple Watch SE customers may be wondering, however, what's in store for them with the upcoming Apple Watch Series 8 and what new features they can expect. Continue reading to find out. Apple Watch SE vs. Apple Watch Series ...
odyssey ark 1

Samsung Launches $3,500 'Odyssey Ark' 55-Inch Curved Gaming Display

Monday August 15, 2022 7:15 am PDT by
Samsung today announced the launch of the Odyssey Ark, which the company says is the world's first 55-inch 4K curved display with a 165Hz refresh rate for improved gaming performance. Samsung first previewed the monitor at CES, but it is now available to order. Subscribe to the MacRumors YouTube channel for more videos. Priced at $3,500, the Samsung Odyssey Ark features mini-LED technology...
top stories 13aug2022

Top Stories: New Battery Percentage Icon in iOS 16, USB-C AirPods Rumor, and More

Saturday August 13, 2022 6:00 am PDT by
This week brought some shocking news for iOS 16 beta testers, with Apple bringing back the iPhone battery percentage to the status bar after a number of years. Other news and rumors this week included word that Apple has started recording portions of its media event planned for next month to introduce the iPhone 14 and Apple Watch Series 8, claims about iPhone 14 Pro pricing and Apple's...