Siri Answers 83% of Questions Correctly in Test, Beating Alexa But Trailing Google Assistant

In an annual test comparing Google Assistant, Siri, and Alexa on smartphones, Loup Ventures' Gene Munster found that ‌Siri‌ was able to correctly answer 83 percent of questions, beating Alexa but trailing behind Google Assistant.

Munster asked each digital assistant 800 questions during the test to compare how each one responded. Alexa answered 79.8 percent of questions correctly, while Google Assistant answered 92.9 percent of questions correctly.

aitestresults
Compared to last year, ‌Siri‌ has seen improvement. During the July 2018 test, ‌Siri‌ answered 79 percent of questions correctly compared to the 83 percent of questions answered right this time around. Alexa last year was at 61 percent while Google Assistant was at 86, so there have been digital voice assistant improvements across the board.

siriimprovementsovertime
This test covered smartphones specifically, comparing iPhones and Android devices. Munster says that smartphones were isolated from smart speakers because while underlying technology is similar, "use cases vary." ‌Siri‌ was tested on an iPhone running iOS 12.4, Google Assistant on a Pixel XL, and Alexa in the iOS app.

Questions were based on five categories and all assistants were asked the same 800 questions. Each question set was designed to "comprehensively test a digital assistant's ability and utility." Some of the sample questions across each of the categories:

  • Local - Where is the nearest coffee shop?
  • Commerce - Order me more paper towels.
  • Navigation - How do I get to Uptown on the bus?
  • Information - Who do the Twins play tonight?
  • Command - Remind me to call Jerome at 2 pm today

‌Siri‌ did best in the command, local, and navigation categories, faring less well in the information and commerce categories. ‌Siri‌ actually won out in the command category, but trailed behind Google Assistant in other categories.

questionsbycategory

Siri continues to prove more useful with phone-related functions like calling, texting, emailing, calendar, and music. Both Siri and Google Assistant, which are baked into the OS of the phone, far outperformed Alexa in the Command section.

Munster says that the continued rate of improvement "continues to surprise" based on the notable improvements that each voice assistant has demonstrated over the course of the last few years.

In the future, Loup Ventures expects to see further improvements from extending the feature sets of each voice assistant.

Top Rated Comments

JetLaw Avatar
41 months ago
I'm not sure about the methodology they used to reach these results, but my anecdotal experience is that Alexa does a MUCH better job than Siri on almost everything I've tried to use them for.
Score: 36 Votes (Like | Disagree)
airwalk331 Avatar
41 months ago
Does anyone else feel like this is crap? 99.8% of the time Siri doesn’t work for me (though my friends’ Alexa does). Period. It’s even worse when I try to use it and there are 2 or more apple devices in the vicinity.
Score: 34 Votes (Like | Disagree)
Blackstick Avatar
41 months ago
Interesting... in my un-scientific testing, Alexa consistently beats Siri at recognizing what I want, and more importantly, the gibberish requests that my 4 year old spouts "Alexa play more Frozen!" (kill me)
Score: 32 Votes (Like | Disagree)
MallardDuck Avatar
41 months ago
* Local - Where is the nearest coffee shop?
* Commerce - Order me more paper towels.
* Navigation - How do I get to Uptown on the bus?
* Information - Who do the Twins play tonight?
* Command - Remind me to call Jerome at 2 pm today

And the responses:

Here's what I found on the web for coffee soup
I can't sort paper trowels in order
Playing Uptown girl by billy joel
I don't know who will win the game tonight
I've updated your reminder to call home


Most of Siri's brain-deadness is in the voice recognition portion of the system.
Score: 32 Votes (Like | Disagree)
diego.caraballo Avatar
41 months ago
Siri better than Alexa? Fake news
Score: 27 Votes (Like | Disagree)
mediotanque Avatar
41 months ago
This is ********. Siri is the dumbest Ai at the moment. Even Cortana works better than Siri.
Score: 14 Votes (Like | Disagree)

Popular Stories

USB C Over Lightning Feature

EU Passes Law to Switch iPhone to USB-C by End of 2024

Tuesday October 4, 2022 3:30 am PDT by
The European Parliament today voted overwhelmingly in favor of enforcing USB-C as a common charging port across a wide range of consumer electronic devices, including the iPhone and AirPods, by the end of 2024. The proposal, known as a directive, forces all consumer electronics manufacturers who sell their products in Europe to ensure that a wide range of devices feature a USB-C port. This...
General iOS 16 Feature Yellow

10 New iOS 16 Features Coming Later This Year

Monday October 3, 2022 2:41 pm PDT by
iOS 16 was released to the public three weeks ago with a customizable Lock Screen, the ability to edit iMessages, improvements to Focus modes, and much more. And in the coming months, iPhone and iPad users have even more new features to look forward to. We've rounded up 10 new features coming to the iPhone and iPad later this year, according to Apple. Many of the features are part of iOS...
ipad pro m1 feature

Gurman: Apple Event This October Remains Unlikely, No Touch ID for iPhone 15

Sunday October 2, 2022 6:41 am PDT by
Apple is developing new iPad Pro, Mac, and Apple TV models, and at least some of these products will be released in October, according to Bloomberg's Mark Gurman. However, Gurman continues to believe that Apple is unlikely to hold an event this month. In the latest edition of his Power On newsletter, Gurman said "the big iPhone 14 unveiling last month was probably it for Apple in 2022 in...
iOS 16

Apple Preparing iOS 16.0.3 With More Bug Fixes Following iPhone 14 Launch

Monday October 3, 2022 7:53 am PDT by
iOS 16.0.2 was released last month with several bug fixes for iPhone 14 issues, excessive copy and paste permission prompts, and more. Now, evidence suggests that Apple is planning to release iOS 16.0.3 with additional bug fixes. Evidence of an upcoming iOS 16.0.3 software update has shown up in MacRumors analytics logs, which have been a reliable indicator in the past. There are several...
Apple SIM Card

Apple SIM No Longer Available for Activating New Cellular Data Plans on iPads

Sunday October 2, 2022 8:04 am PDT by
As of October 1, Apple SIM is no longer available for activating new cellular data plans on supported iPad models, according to an Apple support document. Introduced in 2014, the Apple SIM was designed to allow iPad users to activate cellular data plans from multiple carriers around the world. Initially, the Apple SIM was a physical nano-SIM card, but it was embedded inside later iPad Pro...
maxresdefault

Video: AirPods Pro 2 vs. Bose QuietComfort II

Monday October 3, 2022 12:50 pm PDT by
Apple on September 23 officially launched the second-generation version of the AirPods Pro, introducing updated Active Noise Cancellation, Adaptive Transparency, improved sound, and more. Right around the same time, Bose introduced new QuietComfort II earbuds with many similar features, so we thought we'd compare the two to see which has the edge. Subscribe to the MacRumors YouTube channel for ...
General YouTube Feature 1

You May Soon Need to Be a YouTube Premium Subscriber to Watch 4K Videos

Monday October 3, 2022 4:29 am PDT by
YouTube may make watching videos in 4K quality on the platform exclusive to only YouTube Premium subscribers, according to screenshots posted by users on Twitter and Reddit. On Reddit (1,2) and Twitter, some users have started to recently notice that on iOS, and presumably across other platforms also, YouTube is now saying that in order to watch videos in 4K, the user must be a paying...
dynamic island alan dye

Apple Executives Talk About iPhone 14 Pro's Dynamic Island in New Interview

Sunday October 2, 2022 10:48 am PDT by
In a new interview, Apple's senior vice president of software engineering, Craig Federighi, and Apple's vice president of human interface design, Alan Dye, sat down to discuss the thinking behind the iPhone 14 Pro's Dynamic Island and how it was developed. During the interview with the Japanese magazine Axis, Federighi, who oversees the development of iOS, said Dynamic Island represents the...