New in OS X: Get MacRumors Push Notifications on your Mac

Resubscribe Now Close

Siri Answers 83% of Questions Correctly in Test, Beating Alexa But Trailing Google Assistant

Thursday August 15, 2019 12:05 pm PDT by Juli Clover
In an annual test comparing Google Assistant, Siri, and Alexa on smartphones, Loup Ventures' Gene Munster found that Siri was able to correctly answer 83 percent of questions, beating Alexa but trailing behind Google Assistant.

Munster asked each digital assistant 800 questions during the test to compare how each one responded. Alexa answered 79.8 percent of questions correctly, while Google Assistant answered 92.9 percent of questions correctly.


Compared to last year, Siri has seen improvement. During the July 2018 test, Siri answered 79 percent of questions correctly compared to the 83 percent of questions answered right this time around. Alexa last year was at 61 percent while Google Assistant was at 86, so there have been digital voice assistant improvements across the board.


This test covered smartphones specifically, comparing iPhones and Android devices. Munster says that smartphones were isolated from smart speakers because while underlying technology is similar, "use cases vary." Siri was tested on an iPhone running iOS 12.4, Google Assistant on a Pixel XL, and Alexa in the iOS app.

Questions were based on five categories and all assistants were asked the same 800 questions. Each question set was designed to "comprehensively test a digital assistant's ability and utility." Some of the sample questions across each of the categories:
  • Local - Where is the nearest coffee shop?
  • Commerce - Order me more paper towels.
  • Navigation - How do I get to Uptown on the bus?
  • Information - Who do the Twins play tonight?
  • Command - Remind me to call Jerome at 2 pm today
Siri did best in the command, local, and navigation categories, faring less well in the information and commerce categories. Siri actually won out in the command category, but trailed behind Google Assistant in other categories.

Siri continues to prove more useful with phone-related functions like calling, texting, emailing, calendar, and music. Both Siri and Google Assistant, which are baked into the OS of the phone, far outperformed Alexa in the Command section.
Munster says that the continued rate of improvement "continues to surprise" based on the notable improvements that each voice assistant has demonstrated over the course of the last few years.

In the future, Loup Ventures expects to see further improvements from extending the feature sets of each voice assistant.

Tags: Siri Guide, Loup Ventures
[ 133 comments ]


Top Rated Comments

(View all)

Avatar
JetLaw
3 days ago at 12:09 pm
I'm not sure about the methodology they used to reach these results, but my anecdotal experience is that Alexa does a MUCH better job than Siri on almost everything I've tried to use them for.
Rating: 33 Votes
Avatar
MallardDuck
3 days ago at 12:14 pm
* Local - Where is the nearest coffee shop?
* Commerce - Order me more paper towels.
* Navigation - How do I get to Uptown on the bus?
* Information - Who do the Twins play tonight?
* Command - Remind me to call Jerome at 2 pm today

And the responses:

Here's what I found on the web for coffee soup
I can't sort paper trowels in order
Playing Uptown girl by billy joel
I don't know who will win the game tonight
I've updated your reminder to call home


Most of Siri's brain-deadness is in the voice recognition portion of the system.
Rating: 31 Votes
Avatar
Blackstick
3 days ago at 12:09 pm
Interesting... in my un-scientific testing, Alexa consistently beats Siri at recognizing what I want, and more importantly, the gibberish requests that my 4 year old spouts "Alexa play more Frozen!" (kill me)
Rating: 30 Votes
Avatar
airwalk331
3 days ago at 12:09 pm
Does anyone else feel like this is crap? 99.8% of the time Siri doesn’t work for me (though my friends’ Alexa does). Period. It’s even worse when I try to use it and there are 2 or more apple devices in the vicinity.
Rating: 29 Votes
Avatar
diego.caraballo
3 days ago at 12:10 pm
Siri better than Alexa? Fake news
Rating: 25 Votes
Avatar
spazzcat
3 days ago at 12:18 pm

Does anyone else feel like this is crap? 99.8% of the time Siri doesn’t work for me (though my friends’ Alexa does). Period. It’s even worse when I try to use it and there are 2 or more apple devices in the vicinity.


Nope, never got all the siri hate. I have very few issues with using Siri, I use it daily around my house.
Rating: 14 Votes
Avatar
now i see it
3 days ago at 12:15 pm
I'd like to see a stat of the percentage of iPhone users who've given up on Siri entirely & switched her off forever. Probably mostly everyone.
Rating: 13 Votes
Avatar
MrGimper
3 days ago at 12:23 pm
My experience of Siri is that it’s truly garbage. I’ve resorted to just using it to set timers. I have several Echo’s in the house with a Sonos setup and Hue lights and Alexa is beyond amazing at understanding what we’re saying and controls all that. As a consequence I no longer look for HomeKit compatibility, rather Alexa compatibility.
Rating: 12 Votes
Avatar
mediotanque
3 days ago at 12:13 pm
This is ********. Siri is the dumbest Ai at the moment. Even Cortana works better than Siri.
Rating: 12 Votes
Avatar
ChrisMoBro
3 days ago at 12:17 pm
From my experience of using both, Siri doesn’t even come close to Alexa. Using it is just a painful experience in comparison.
Rating: 12 Votes

[ Read All Comments ]