Siri on HomePod Correctly Answered 52.3% of Queries in New AI Test

In a new test shared today by Loup Ventures, Apple's HomePod was put through its paces in categories including Siri, sound quality, and ease of use. For Siri, Loup Ventures' Gene Munster reported that while the AI assistant understood 99.4 percent of queries asked of it, it answered only 52.3 percent of them correctly. Loup Ventures tested three separate HomePods and asked 782 queries total.

Compared to previous tests of rival speakers, HomePod is "at the bottom of the totem pole" in the AI assistant performance category. Google Home answered 81 percent correctly, Amazon's Alexa answered 64 percent correctly, and Microsoft's Cortana answered 57 percent correctly.

mitchs homepod on shelf
Munster broke this information down further, stating that Siri is good for "local" and "commerce" queries, like asking about nearby coffee shops or assisting in buying new shoes. In this area, Siri beats Alexa and Cortana but still falls behind Google Assistant on Google Home.

Despite the low percentage of correctly answering the 782 total queries asked of it, Munster said Siri's overall performance rose above expectations "given the limited scope of HomePod's music focus."

homepod siri loup ventures

Chart via Loup Ventures

The researchers explained that over time HomePod and Siri should grow to match, or surpass, rival assistants by simply adding query domains like calendar, email, calling, and navigation.

Some domains like navigation, calendar, email, and calling are simply not supported. These questions were met with, “I can’t ___ on HomePod.” Also, in any case that iPhone-based Siri would bring up Google search results, HomePod would reply, “I can’t get the answer to that on HomePod,” which forces you to use your phone or give up on the question altogether. Removing navigation, calling, email, and calendar-related queries from our question set yields a 67% correct response, a jump from overall of 52.3% correct.

This means added support for these domains would bring HomePod performance above that of Alexa (64%) and Cortana (57%), though still shy of Google Home (81%). We know Siri has the ability to correctly answer a whole range of queries that HomePod cannot, evidenced by our note here. Apple’s limiting of HomePod’s domains should change over time, at which point we expect the speaker to be vastly more useful and integrated with your other Apple devices.

As discovered in the research, where HomePod excelled was its "superior" listening skills. The HomePod allows users to speak at a normal volume, even when music from the speaker is particularly loud, and Siri will pick up on the voice and hear the user. "This was HomePod's most stellar feature," according to Munster.

Loup Ventures also favored HomePod's sound quality, saying that "it sounds incredible." Like other reviews and impressions of the device, Munster's enjoyment of the HomePod's audio quality was countered with the speaker's lackluster Siri performance, which he thinks will be changing fairly soon.

Don’t be fooled by HomePod’s sound quality-focused first step into smart speakers; Apple has a grander vision than delivering a better sounding Echo. While not present in the first version of HomePod (i.e. you can’t even make a phone call with HomePod), we believe Apple’s goal is to make Siri a ubiquitous, ambient presence that connects and controls all your connected devices and services – and to make a leap forward in the transition to voice-first computing.

Visit Loup Ventures to read more of the information gathered from the HomePod "smart speaker gauntlet," including the publication's prediction for demand and market share of HomePod through the rest of 2018.

Related Roundup: HomePod
Buyer's Guide: HomePod (Caution)

Top Rated Comments

RudySnow Avatar
39 months ago
Me: “Hey Siri, why are you so terrible at answering questions?”

Siri: “I found several articles on the web about crustaceans.”
Score: 132 Votes (Like | Disagree)
Mobster1983 Avatar
39 months ago
That's pretty damn pathetic...
Score: 108 Votes (Like | Disagree)
SeattleMoose Avatar
39 months ago
Siri is Apple's biggest failure. They have had years to make it at least 80% functional and have failed to do so. Just like the ATV, it seems to be just a "hobby" for Apple.
Score: 84 Votes (Like | Disagree)
unobtainium Avatar
39 months ago
Siri sucks. I don’t think I’ve used it in over a year.
Score: 77 Votes (Like | Disagree)
jasvncnt Avatar
39 months ago
What good is "superior" listeng skills if it can't answer many of the questions?
Score: 75 Votes (Like | Disagree)
palmerc2 Avatar
39 months ago

I think Siri is more than adequate for what most need it for the HomePod. Is it better than Google Home? No. Does it need to be better than the competition? Not necessarily. I use Siri all the time and I really don’t have any general complaints, aside from not understanding some of the things I ask it. None of these voice assistants are perfect.

I can only imagine the amount of gloating on these forums if Siri was #1 on the list instead of Google.
Score: 38 Votes (Like | Disagree)

Top Stories

lg wing

LG Considering Exit From Smartphone Business, Halts LCD Production for iPhone

Wednesday January 20, 2021 5:38 am PST by
LG is considering exiting the smartphone business entirely amid declining shipments and accrued losses of $4.5 billion over the past five years (via The Korea Herald). LG CEO Kwon Bong-Seok cautioned staff earlier today that the company is re-evaluating its presence in the smartphone industry: Since the competition in the global market for mobile devices is getting fiercer, it is about...
2019 mac pro side and front

Tim Cook Gifted Donald Trump 'First' 2019 Mac Pro

Wednesday January 20, 2021 5:45 pm PST by
Apple CEO Tim Cook gifted former United States President Donald Trump with the first 2019 Mac Pro that came off of the assembly line in Austin, Texas, according to a financial disclosure report that was released today (via The Verge). "Mac Pro Computer, the first created at the Flex Factory in Austin, Texas," reads the entry, which values the machine at $5,999, the base price for a Mac Pro....
Apple VR Feature

Bloomberg: Apple's First AR/VR Headset 'Pricey, Niche Precursor' to More Ambitious AR Glasses and Could Launch Next Year

Thursday January 21, 2021 3:27 am PST by
Apple's first virtual reality headset will be a "pricey, niche precursor" to a more ambitious augmented reality product, according to a new report from Bloomberg's Mark Gurman. As a mostly virtual reality device, it will display an all-encompassing 3-D digital environment for gaming, watching video and communicating. AR functionality, the ability to overlay images and information over a view...
Flat MacBook Air Feature

Bloomberg: Apple Working on 'Thinner and Lighter' High-End MacBook Air With MagSafe, Could Launch in Second Half of 2021

Friday January 22, 2021 3:34 am PST by
Apple is working on a "thinner and lighter" version of the MacBook Air that the company plans to release during the second half of this year at the earliest or in 2022, according to a new report by well-connected Bloomberg journalist Mark Gurman. It will include Apple's MagSafe charging technology and a next-generation version of the company's in-house Mac processors. Apple has discussed...
iOS 15 icon mock banner

iOS 15 Rumored to Drop Support for iPhone 6s and 2016 iPhone SE

Thursday January 21, 2021 11:58 am PST by
Apple's upcoming iOS 15 operating system, which we expect to see unveiled in June, is rumored to be dropping support for a few of Apple's older iPhones. According to French site iPhoneSoft, iOS 15 will not be able to be installed on the iPhone 6s, the iPhone 6s Plus, or the 2016 iPhone SE, all of which have an A9 chip. The iPhone 6s and 6s Plus were introduced in 2015 and are now more...
iphone 12 vs iphone 12 mini

Apple Shifting Some Production From iPhone 12 mini to iPhone 12 Pro to Meet Demand

Wednesday January 20, 2021 8:12 am PST by
Apple has reportedly cut production of the iPhone 12 mini by two million units to create more manufacturing capacity for the iPhone 12 Pro, according to a new Morgan Stanley investment note seen by PED30. Apple is believed to have made the switch for the first quarter of 2021 in an effort to combat continuing lead times for the more popular iPhone 12 Pro. iPhone 12 Pro lead times remain ...
iPhone 13 Notch Feature

iPhone 13 Rumored to Feature Smaller Notch, Pro Model Cameras to Use Larger Image Sensor

Thursday January 21, 2021 1:38 am PST by
Apple's iPhone 13 series will feature a redesigned Face ID system that will allow for a smaller notch at the top of the screen, according to a new report today. The rumor comes via hit-and-miss Taiwanese industry publication DigiTimes, whose supply chain sources also claim that the ultra wide-angle lens in Apple's next-generation iPhones is due for an upgrade. The next-generation iPhones'...
maxresdefault

Video Demos macOS Catalina Running on iPad Pro via x86 Emulation

Thursday January 21, 2021 11:36 am PST by
A video demonstrating macOS Catalina running on a current 2020 iPad Pro has been shared on YouTube, giving us a look at an interesting hack that has a Mac OS up and working on one of Apple's iPads. There's limited information about how the process of getting macOS Catalina on an iPad Pro works, but it uses x86 emulation and was done through the UTM software that allows virtual machines to...
iOS 14

Apple Seeds iOS 14.4 and iPadOS 14.4 Release Candidate to Developers and Public Beta Testers

Thursday January 21, 2021 10:14 am PST by
Apple today seeded the RC version of upcoming iOS 14.4 and iPadOS 14.4 updates to developers for testing purposes, with the new betas coming a week after Apple released the second betas. iOS 14.4 and iPadOS 14.4 can be downloaded through the Apple Developer Center or over the air after the proper profile has been installed on an iPhone or iPad. Paired with the HomePod 14.4 beta that is...
iPhone OIS Feature2

Sensor-Shift Camera Stabilization Rumored to Expand to Entire iPhone 13 Lineup

Wednesday January 20, 2021 7:46 am PST by
The entire iPhone 13 lineup will feature sensor-shift optical image stabilization, compared to only the iPhone 12 Pro Max among current models, according to a brief story preview shared today by Taiwanese publication DigiTimes. "Apple's next-generation iPhones slated for launch in the second half of 2021 will all come with sensor-shift stabilization technology, according to industry...