Siri Answers 83% of Questions Correctly in Test, Beating Alexa But Trailing Google Assistant

In an annual test comparing Google Assistant, Siri, and Alexa on smartphones, Loup Ventures' Gene Munster found that ‌Siri‌ was able to correctly answer 83 percent of questions, beating Alexa but trailing behind Google Assistant.

Munster asked each digital assistant 800 questions during the test to compare how each one responded. Alexa answered 79.8 percent of questions correctly, while Google Assistant answered 92.9 percent of questions correctly.


Compared to last year, ‌Siri‌ has seen improvement. During the July 2018 test, ‌Siri‌ answered 79 percent of questions correctly compared to the 83 percent of questions answered right this time around. Alexa last year was at 61 percent while Google Assistant was at 86, so there have been digital voice assistant improvements across the board.


This test covered smartphones specifically, comparing iPhones and Android devices. Munster says that smartphones were isolated from smart speakers because while underlying technology is similar, "use cases vary." ‌Siri‌ was tested on an iPhone running iOS 12.4, Google Assistant on a Pixel XL, and Alexa in the iOS app.

Questions were based on five categories and all assistants were asked the same 800 questions. Each question set was designed to "comprehensively test a digital assistant's ability and utility." Some of the sample questions across each of the categories:

  • Local - Where is the nearest coffee shop?
  • Commerce - Order me more paper towels.
  • Navigation - How do I get to Uptown on the bus?
  • Information - Who do the Twins play tonight?
  • Command - Remind me to call Jerome at 2 pm today

‌Siri‌ did best in the command, local, and navigation categories, faring less well in the information and commerce categories. ‌Siri‌ actually won out in the command category, but trailed behind Google Assistant in other categories.

‌Siri‌ continues to prove more useful with phone-related functions like calling, texting, emailing, calendar, and music. Both ‌Siri‌ and Google Assistant, which are baked into the OS of the phone, far outperformed Alexa in the Command section.

Munster says that the continued rate of improvement "continues to surprise" based on the notable improvements that each voice assistant has demonstrated over the course of the last few years.

In the future, Loup Ventures expects to see further improvements from extending the feature sets of each voice assistant.

Top Rated Comments

(View all)
Avatar
9 months ago
I'm not sure about the methodology they used to reach these results, but my anecdotal experience is that Alexa does a MUCH better job than Siri on almost everything I've tried to use them for.
Score: 36 Votes (Like | Disagree)
Avatar
9 months ago
Does anyone else feel like this is crap? 99.8% of the time Siri doesn’t work for me (though my friends’ Alexa does). Period. It’s even worse when I try to use it and there are 2 or more apple devices in the vicinity.
Score: 34 Votes (Like | Disagree)
Avatar
9 months ago
Interesting... in my un-scientific testing, Alexa consistently beats Siri at recognizing what I want, and more importantly, the gibberish requests that my 4 year old spouts "Alexa play more Frozen!" (kill me)
Score: 32 Votes (Like | Disagree)
Avatar
9 months ago
* Local - Where is the nearest coffee shop?
* Commerce - Order me more paper towels.
* Navigation - How do I get to Uptown on the bus?
* Information - Who do the Twins play tonight?
* Command - Remind me to call Jerome at 2 pm today

And the responses:

Here's what I found on the web for coffee soup
I can't sort paper trowels in order
Playing Uptown girl by billy joel
I don't know who will win the game tonight
I've updated your reminder to call home


Most of Siri's brain-deadness is in the voice recognition portion of the system.
Score: 32 Votes (Like | Disagree)
Avatar
9 months ago
Siri better than Alexa? Fake news
Score: 27 Votes (Like | Disagree)
Avatar
9 months ago
This is ********. Siri is the dumbest Ai at the moment. Even Cortana works better than Siri.
Score: 14 Votes (Like | Disagree)

Top Stories

Leaker Claims New 13-inch MacBook Pro Coming as Soon as Next Month

Monday April 6, 2020 2:56 am PDT by Tim Hardwick
Apple will announce a new 13-inch MacBook Pro in May with the codename J223, according to a rumor shared by YouTuber and leaker Jon Prosser. Note: it’s a refresh to the current 13” So the bigger 14” display upgrade is a big possibility— Jon Prosser (@jon_prosser) April 4, 2020 Analyst Ming-Chi Kuo has said Apple plans to release new MacBook Pro and MacBook Air models with scissor keyboards ...

iOS 14 Could Offer Home Screen Widgets, Wallpaper Customizations

Saturday April 4, 2020 3:30 pm PDT by Frank McShan
iOS 14 could offer home screen widgets and wallpaper customizations for the first time, according to 9to5Mac and Twitter user DongleBookPro. Apple is reportedly working to implement widgets that can be moved freely around like icons on the iPhone and iPad homescreen for the very first time. The feature is reportedly codenamed "Avocado" and no other details are available. It was also...

Top Stories: Apple Leaks iPhone SE and AirTags, Apple Buys Dark Sky, and More

Saturday April 4, 2020 6:00 am PDT by MacRumors Staff
With the calendar rolling over to April this week, we yet again saw several leaks and rumors, most notably including Apple itself leaking some references to a pair of long-rumored products: a new budget iPhone SE and AirTags item trackers. Subscribe to the MacRumors YouTube channel for more videos. Apple also acquired popular weather app Dark Sky, while Amazon's Prime Video app now allows...

Apple Music Was Second Biggest Global Music Streaming Service in 2019

Friday April 3, 2020 5:38 pm PDT by Juli Clover
Global online music streaming subscriptions were up 32 percent year-over-year in 2019, hitting 358 million subscribers, according to new estimates shared today by Counterpoint Research. Spotify was the market leader with a 31 percent share of total revenue and a 35 percent share of total paid subscriptions, while Apple Music earned the second place slot with a 24 percent share of total...

More References to Apple's Upcoming Low-Cost iPhone Appear Online

Monday April 6, 2020 4:38 am PDT by Tim Hardwick
Further references to Apple's upcoming low-cost iPhone have appeared online, one on a Chinese e-commerce website and another on Verizon's smartphone trade-in page. Spotted by tech blog MySmartPrice, Chinese retailer JD.com has published a placeholder for Apple's so-called "iPhone 9" that includes a teaser image of a veiled smartphone, but other than that it lacks any particularly revealing...

The New York Times, IFTTT, Medium, and Other Apps Adopt Sign in With Apple Ahead of June 30 Deadline

Sunday April 5, 2020 7:08 pm PDT by Frank McShan
Apps with sign-in functionality, including The New York Times, IFTTT, Medium, and more, have continued to adopt Apple's secure Sign in with Apple feature ahead of a deadline of June 30. The deadline for these apps to support the feature was recently extended from April 30. Sign in with Apple, first introduced in iOS 13, allows users to create accounts for apps and websites using an Apple ID. ...

Apple Offers Employees Deep Discounts on HomePod, Beats Headphones and More

Friday April 3, 2020 11:14 am PDT by Juli Clover
Apple is offering employees significant discounts on products that include the HomePod and Beats headphones, according to details on the deals shared by MacGeneration and 9to5Mac. Apple is said to be clearing stock of Beats headphones and HomePods, providing them to employees through an internal deals program. The HomePod is available at a 50 percent discount, dropping the price to $149.50,...

New Low-Cost 'iPhone SE' Could Launch as Soon as Tomorrow

Thursday April 2, 2020 4:06 pm PDT by Juli Clover
Apple's new low-cost iPhone is set to launch as early as Friday, April 3, according to a new report from 9to5Mac that cites a tip from a "highly trusted reader." The site says that while it can't be certain about the launch date, "Apple could reveal and begin taking orders for the new iPhone as soon as tomorrow." The iPhone 8 Apple is said to be planning to call the new iPhone, which is...

EPIX Available for Free Through Apple TV Channels Until May 2, No Subscription Required

Saturday April 4, 2020 9:28 am PDT by Frank McShan
EPIX is currently offering free access to its catalog of movies and TV shows through the Apple TV Channels feature in the TV app, and many additional services are also offering extended free trials. Upon navigating to the Apple TV app, users will notice that EPIX is listed under the "My Channels" section. Rather than offering an extended trial where users must first sign up, EPIX is...

iPad Launched 10 Years Ago Today, App Developer Reflects on Getting Access to a Top-Secret Prototype

Friday April 3, 2020 10:53 am PDT by Joe Rossignol
On the 10th anniversary of the iPad launching in stores, Agile Partners co-founder Jack Ivers has shared an interesting story about how his company managed to get in Apple's good graces and ultimately gain access to prototype iPads. The story begins in 2008, when Agile Partners released GuitarToolkit as one of the first iPhone apps on the App Store. The app used the iPhone's microphone to...