Siri on HomePod Correctly Answered 52.3% of Queries in New AI Test

In a new test shared today by Loup Ventures, Apple's HomePod was put through its paces in categories including Siri, sound quality, and ease of use. For Siri, Loup Ventures' Gene Munster reported that while the AI assistant understood 99.4 percent of queries asked of it, it answered only 52.3 percent of them correctly. Loup Ventures tested three separate HomePods and asked 782 queries total.

Compared to previous tests of rival speakers, HomePod is "at the bottom of the totem pole" in the AI assistant performance category. Google Home answered 81 percent correctly, Amazon's Alexa answered 64 percent correctly, and Microsoft's Cortana answered 57 percent correctly.

mitchs homepod on shelf
Munster broke this information down further, stating that Siri is good for "local" and "commerce" queries, like asking about nearby coffee shops or assisting in buying new shoes. In this area, Siri beats Alexa and Cortana but still falls behind Google Assistant on Google Home.

Despite the low percentage of correctly answering the 782 total queries asked of it, Munster said Siri's overall performance rose above expectations "given the limited scope of HomePod's music focus."

homepod siri loup ventures

Chart via Loup Ventures

The researchers explained that over time HomePod and Siri should grow to match, or surpass, rival assistants by simply adding query domains like calendar, email, calling, and navigation.

Some domains like navigation, calendar, email, and calling are simply not supported. These questions were met with, “I can’t ___ on HomePod.” Also, in any case that iPhone-based Siri would bring up Google search results, HomePod would reply, “I can’t get the answer to that on HomePod,” which forces you to use your phone or give up on the question altogether. Removing navigation, calling, email, and calendar-related queries from our question set yields a 67% correct response, a jump from overall of 52.3% correct.

This means added support for these domains would bring HomePod performance above that of Alexa (64%) and Cortana (57%), though still shy of Google Home (81%). We know Siri has the ability to correctly answer a whole range of queries that HomePod cannot, evidenced by our note here. Apple’s limiting of HomePod’s domains should change over time, at which point we expect the speaker to be vastly more useful and integrated with your other Apple devices.

As discovered in the research, where HomePod excelled was its "superior" listening skills. The HomePod allows users to speak at a normal volume, even when music from the speaker is particularly loud, and Siri will pick up on the voice and hear the user. "This was HomePod's most stellar feature," according to Munster.

Loup Ventures also favored HomePod's sound quality, saying that "it sounds incredible." Like other reviews and impressions of the device, Munster's enjoyment of the HomePod's audio quality was countered with the speaker's lackluster Siri performance, which he thinks will be changing fairly soon.

Don’t be fooled by HomePod’s sound quality-focused first step into smart speakers; Apple has a grander vision than delivering a better sounding Echo. While not present in the first version of HomePod (i.e. you can’t even make a phone call with HomePod), we believe Apple’s goal is to make Siri a ubiquitous, ambient presence that connects and controls all your connected devices and services – and to make a leap forward in the transition to voice-first computing.

Visit Loup Ventures to read more of the information gathered from the HomePod "smart speaker gauntlet," including the publication's prediction for demand and market share of HomePod through the rest of 2018.

Related Roundup: HomePod
Buyer's Guide: HomePod (Don't Buy)

Top Rated Comments

RudySnow Avatar
42 months ago
Me: “Hey Siri, why are you so terrible at answering questions?”

Siri: “I found several articles on the web about crustaceans.”
Score: 132 Votes (Like | Disagree)
Mobster1983 Avatar
42 months ago
That's pretty damn pathetic...
Score: 108 Votes (Like | Disagree)
SeattleMoose Avatar
42 months ago
Siri is Apple's biggest failure. They have had years to make it at least 80% functional and have failed to do so. Just like the ATV, it seems to be just a "hobby" for Apple.
Score: 84 Votes (Like | Disagree)
unobtainium Avatar
42 months ago
Siri sucks. I don’t think I’ve used it in over a year.
Score: 77 Votes (Like | Disagree)
jasvncnt Avatar
42 months ago
What good is "superior" listeng skills if it can't answer many of the questions?
Score: 75 Votes (Like | Disagree)
palmerc2 Avatar
42 months ago
I think Siri is more than adequate for what most need it for the HomePod. Is it better than Google Home? No. Does it need to be better than the competition? Not necessarily. I use Siri all the time and I really don’t have any general complaints, aside from not understanding some of the things I ask it. None of these voice assistants are perfect.
I can only imagine the amount of gloating on these forums if Siri was #1 on the list instead of Google.
Score: 38 Votes (Like | Disagree)

Top Stories

siir apple event april 20

Siri Reveals Apple Event Planned for Tuesday, April 20

Tuesday April 13, 2021 12:04 am PDT by
Siri has apparently prematurely revealed that Apple plans to hold an event on Tuesday, April 20, where the company is expected to reveal brand new iPad Pro models and possibly its long-awaited AirTags trackers. Subscribe to the MacRumors YouTube channel for more videos. Upon being asked "When is the next Apple Event," Siri is currently responding with, "The special event is on Tuesday, April...
apple event spring loaded

Apple's 'Spring Loaded' Event Officially Announced for Tuesday, April 20

Tuesday April 13, 2021 9:04 am PDT by
Following an overnight leak by Siri, Apple today officially announced that it will be holding a special "Spring Loaded" event on Tuesday, April 20 at 10:00 a.m. Pacific Time at the Steve Jobs Theater on the Apple Park campus in Cupertino, California. As with all of Apple's 2020 events, the April 2021 event will be a digital-only gathering with no members of the media invited to attend in...
Google maps feaure green

Google Maps App for iOS Finally Updated After Four Months

Monday April 12, 2021 10:03 am PDT by
Following the completed rollout of App Privacy labels for its App Store apps, Google today updated the Google Maps app for the first time in four months. Apple in December began requiring all new app submissions and app updates to include App Privacy labels, detailing the data that is collected by the app so consumers know what they're sharing. Google didn't begin implementing App Privacy ...
pixel watch prosser leak

Google Pixel Watch Allegedly Leaks with Circular Design, Rumored to Launch in October

Monday April 12, 2021 2:49 am PDT by
Renders of Google's first smartwatch, codenamed "Rohan," have been shared by Jon Prosser, showing that Google plans to adopt a circular design for its flagship wearable watch. Prosser shared the renders in an episode of his YouTube show "Front Page Tech," in which he claims they were made based on marketing material he had seen from a source within Google. The renders show that the Pixel...
apple event hashflag

Twitter Hashflag for April 20 Apple Event Goes Live

Tuesday April 13, 2021 2:21 pm PDT by
Following the overnight Siri leak and subsequent announcement that Apple will hold a media event on Tuesday, April 20, a new Twitter hashflag has appeared to help provide visibility for the event on the platform. For the last several recent events, Apple has utilized hashflags, which are little icons next to hashtags on Twitter, as a way to market its events. The company first started the...
macos catalina serial number

Apple Preparing Rollout of New Randomized Product Serial Numbers Ahead of 'Spring Loaded' Event

Wednesday April 14, 2021 2:08 am PDT by
Apple is advising its authorized premium resellers and dealers to prepare for new products with 10 and 12 digital serial numbers, days ahead of when it's expected to reveal a slew of new products. MacRumors previously reported that Apple plans to switch to randomized serial numbers for future products starting in early 2021. The company now seems to be preparing for that roll-out, telling...
iphone12cameras

Kuo: 2022 iPhones to Feature 48-Megapixel Camera, 8K Video, and 6.1 and 6.7" Sizes With No 5.4" Mini Option

Tuesday April 13, 2021 10:45 pm PDT by
The upcoming 2022 iPhone lineup will feature two 6.1-inch devices and two 6.7-inch devices, with no mini-sized 5.4-inch iPhone, well-respected Apple analyst Ming-Chi Kuo said in a note to investors that was seen by MacRumors. Two of the iPhones will be high-end models and two of the iPhones will be lower-end models, similar to the current iPhone 12 lineup. Apple introduced the 5.4-inch...
AppleTV and HomePod Feature

Bloomberg: Apple Working on New Apple TV With Integrated HomePod Speaker and FaceTime Camera

Monday April 12, 2021 3:32 am PDT by
Apple is working on a combined Apple TV with HomePod speaker that has a camera for video calls through a connected television set, according to Bloomberg's Mark Gurman. From the report: The company is working on a product that would combine an Apple TV set-top box with a HomePod speaker and include a camera for video conferencing through a connected TV and other smart-home functions,...
epic iap feature 3

Tim Cook Says App Store Would Become a 'Flea Market' if Third-Party Payment Systems Were Allowed

Monday April 12, 2021 9:41 am PDT by
In a recent interview with the Toronto Star, Apple CEO Tim Cook spoke about a wide variety of topics, ranging from App Tracking Transparency to Apple's ongoing legal battle over App Store policies with Fortnite creator Epic Games. Notably, Cook said that Epic Games' desire for Apple to let developers offer their own payment systems in apps "would make the App Store a flea market":At the...
tim cook toronto star

Tim Cook Says Apple is 'Not Against Digital Advertising' Ahead of iOS 14.5 Launch With App Tracking Transparency

Monday April 12, 2021 8:00 am PDT by
Starting with iOS 14.5, iPadOS 14.5, and tvOS 14.5, Apple will be requiring apps to receive a user's permission to track their activity for targeted advertising purposes, as part of a privacy measure known as App Tracking Transparency. Ahead of App Tracking Transparency being enforced, Apple CEO Tim Cook has participated in a privacy-focused interview with the Toronto Star, telling the...