Siri on HomePod Correctly Answered 52.3% of Queries in New AI Test

In a new test shared today by Loup Ventures, Apple's HomePod was put through its paces in categories including Siri, sound quality, and ease of use. For Siri, Loup Ventures' Gene Munster reported that while the AI assistant understood 99.4 percent of queries asked of it, it answered only 52.3 percent of them correctly. Loup Ventures tested three separate HomePods and asked 782 queries total.

Compared to previous tests of rival speakers, HomePod is "at the bottom of the totem pole" in the AI assistant performance category. Google Home answered 81 percent correctly, Amazon's Alexa answered 64 percent correctly, and Microsoft's Cortana answered 57 percent correctly.

mitchs homepod on shelf
Munster broke this information down further, stating that Siri is good for "local" and "commerce" queries, like asking about nearby coffee shops or assisting in buying new shoes. In this area, Siri beats Alexa and Cortana but still falls behind Google Assistant on Google Home.

Despite the low percentage of correctly answering the 782 total queries asked of it, Munster said Siri's overall performance rose above expectations "given the limited scope of HomePod's music focus."

homepod siri loup ventures

Chart via Loup Ventures

The researchers explained that over time HomePod and Siri should grow to match, or surpass, rival assistants by simply adding query domains like calendar, email, calling, and navigation.

Some domains like navigation, calendar, email, and calling are simply not supported. These questions were met with, “I can’t ___ on HomePod.” Also, in any case that iPhone-based Siri would bring up Google search results, HomePod would reply, “I can’t get the answer to that on HomePod,” which forces you to use your phone or give up on the question altogether. Removing navigation, calling, email, and calendar-related queries from our question set yields a 67% correct response, a jump from overall of 52.3% correct.

This means added support for these domains would bring HomePod performance above that of Alexa (64%) and Cortana (57%), though still shy of Google Home (81%). We know Siri has the ability to correctly answer a whole range of queries that HomePod cannot, evidenced by our note here. Apple’s limiting of HomePod’s domains should change over time, at which point we expect the speaker to be vastly more useful and integrated with your other Apple devices.

As discovered in the research, where HomePod excelled was its "superior" listening skills. The HomePod allows users to speak at a normal volume, even when music from the speaker is particularly loud, and Siri will pick up on the voice and hear the user. "This was HomePod's most stellar feature," according to Munster.

Loup Ventures also favored HomePod's sound quality, saying that "it sounds incredible." Like other reviews and impressions of the device, Munster's enjoyment of the HomePod's audio quality was countered with the speaker's lackluster Siri performance, which he thinks will be changing fairly soon.

Don’t be fooled by HomePod’s sound quality-focused first step into smart speakers; Apple has a grander vision than delivering a better sounding Echo. While not present in the first version of HomePod (i.e. you can’t even make a phone call with HomePod), we believe Apple’s goal is to make Siri a ubiquitous, ambient presence that connects and controls all your connected devices and services – and to make a leap forward in the transition to voice-first computing.

Visit Loup Ventures to read more of the information gathered from the HomePod "smart speaker gauntlet," including the publication's prediction for demand and market share of HomePod through the rest of 2018.

Related Roundup: HomePod
Buyer's Guide: HomePod (Buy Now)

Top Rated Comments

RudySnow Avatar
67 months ago
Me: “Hey Siri, why are you so terrible at answering questions?”

Siri: “I found several articles on the web about crustaceans.”
Score: 132 Votes (Like | Disagree)
Mobster1983 Avatar
67 months ago
That's pretty damn pathetic...
Score: 108 Votes (Like | Disagree)
SeattleMoose Avatar
67 months ago
Siri is Apple's biggest failure. They have had years to make it at least 80% functional and have failed to do so. Just like the ATV, it seems to be just a "hobby" for Apple.
Score: 84 Votes (Like | Disagree)
unobtainium Avatar
67 months ago
Siri sucks. I don’t think I’ve used it in over a year.
Score: 77 Votes (Like | Disagree)
jasvncnt Avatar
67 months ago
What good is "superior" listeng skills if it can't answer many of the questions?
Score: 75 Votes (Like | Disagree)
palmerc2 Avatar
67 months ago
I think Siri is more than adequate for what most need it for the HomePod. Is it better than Google Home? No. Does it need to be better than the competition? Not necessarily. I use Siri all the time and I really don’t have any general complaints, aside from not understanding some of the things I ask it. None of these voice assistants are perfect.
I can only imagine the amount of gloating on these forums if Siri was #1 on the list instead of Google.
Score: 38 Votes (Like | Disagree)

Popular Stories

iOS 16

iOS 16.4 for iPhone Nearing Launch With These 5 New Features

Monday March 20, 2023 11:50 am PDT by
Apple says iOS 16.4 is coming in the spring, which began this week. In his Sunday newsletter, Bloomberg's Mark Gurman said the update should be released "in the next three weeks or so," meaning a public release is likely in late March or early April. iOS 16.4 remains in beta testing and introduces a handful of new features and changes for the iPhone. Below, we have recapped five new features ...
iPhone 15 Pro Buttons CAD Leak

iPhone 15 Pro Leak Reveals Unified Volume Button and Mute Button

Monday March 20, 2023 8:33 am PDT by
As previously rumored, the next-generation iPhone 15 Pro and iPhone 15 Pro Max will feature a unified volume button and a mute button, according to leaked CAD images shared in a video on the Chinese version of TikTok and posted to Twitter by ShrimpApplePro. Instead of separate buttons for volume up and volume down, the iPhone 15 Pro models are expected to have a single elongated button for...
voice isolation

iOS 16.4 Adds Voice Isolation for Cellular Phone Calls

Tuesday March 21, 2023 11:01 am PDT by
The iOS 16.4 update that is set to be released to the public in the near future includes voice isolation for cellular calls, according to notes that Apple shared today. Apple says that Voice Isolation will prioritize your voice and block out the ambient noise around you, making for clearer phone calls where you can better hear the person you're chatting with and vice versa. Voice...
airpodsd 3 purple 4

iOS 16.4 Seemingly References New AirPods and AirPods Case

Tuesday March 21, 2023 11:43 am PDT by
The iOS 16.4 release candidate version that was provided to developers today appears to hint at a new set of AirPods that could be coming in the near future. According to @aaronp613, the beta features references to AirPods that have a model number of A3048 and an AirPods case with a model number of A2968. There have been no rumors that new AirPods are on the horizon, and it is early for...
maxresdefault

Nothing Launches $149 Ear (2) Wireless Earbuds to Compete With AirPods Pro 2

Wednesday March 22, 2023 9:48 am PDT by
Nothing today announced the launch of its second-generation wireless earbuds, the Nothing Ear (2), which offer many of the same features as Apple's AirPods Pro 2 at a lower price point. We went hands-on with the Ear (2) earbuds to see whether they're a viable alternative to the AirPods Pro 2 for those who want to save some cash. The Ear (2) earbuds are the successor to the Nothing Ear (1),...
iOS 16

Apple Seeds Release Candidate Versions of iOS 16.4 and iPadOS 16.4 to Developers [Update: Public RC Available]

Tuesday March 21, 2023 10:25 am PDT by
Apple today seeded the release candidate versions of upcoming iOS 16.4 and iPadOS 16.4 updates to developers for testing purposes, with the software coming a week after the launch of the fourth betas. The RCs mark the final version of the software that will be provided to the public in the near future. Registered developers are able to download the iOS 16.4 and iPadOS 16.4 updates...
apple park at night 1

Apple 'Tracking Employee Attendance' in Crackdown on Remote Working

Thursday March 23, 2023 3:41 am PDT by
Apple is tracking the attendance of its employees at offices using badge records in order to ensure they are coming in at least three times a week, according to Platformer's Zoë Schiffer. Since April 2022, Apple employees have been operating on a hybrid home/office work policy as part of a gradual return strategy following the pandemic, with staff required to work from the office at least...
iphone 14 pro max deep purple feature purple

iPhone 15 Pro Rumor Recap: 10 New Features and Changes to Expect

Thursday March 23, 2023 6:42 am PDT by
While the iPhone 15 series is still around six months away from launching, there have already been plenty of rumors about the devices. Many new features and changes have been rumored for the iPhone 15 Pro and iPhone 15 Pro Max in particular. Below, we have recapped 10 changes rumored for iPhone 15 Pro models that are not expected to be available on the standard iPhone 15 and iPhone 15 Plus:A1...