Siri Answers 83% of Questions Correctly in Test, Beating Alexa But Trailing Google Assistant

In an annual test comparing Google Assistant, Siri, and Alexa on smartphones, Loup Ventures' Gene Munster found that ‌Siri‌ was able to correctly answer 83 percent of questions, beating Alexa but trailing behind Google Assistant.

Munster asked each digital assistant 800 questions during the test to compare how each one responded. Alexa answered 79.8 percent of questions correctly, while Google Assistant answered 92.9 percent of questions correctly.

aitestresults
Compared to last year, ‌Siri‌ has seen improvement. During the July 2018 test, ‌Siri‌ answered 79 percent of questions correctly compared to the 83 percent of questions answered right this time around. Alexa last year was at 61 percent while Google Assistant was at 86, so there have been digital voice assistant improvements across the board.

siriimprovementsovertime
This test covered smartphones specifically, comparing iPhones and Android devices. Munster says that smartphones were isolated from smart speakers because while underlying technology is similar, "use cases vary." ‌Siri‌ was tested on an iPhone running iOS 12.4, Google Assistant on a Pixel XL, and Alexa in the iOS app.

Questions were based on five categories and all assistants were asked the same 800 questions. Each question set was designed to "comprehensively test a digital assistant's ability and utility." Some of the sample questions across each of the categories:

  • Local - Where is the nearest coffee shop?
  • Commerce - Order me more paper towels.
  • Navigation - How do I get to Uptown on the bus?
  • Information - Who do the Twins play tonight?
  • Command - Remind me to call Jerome at 2 pm today

‌Siri‌ did best in the command, local, and navigation categories, faring less well in the information and commerce categories. ‌Siri‌ actually won out in the command category, but trailed behind Google Assistant in other categories.

questionsbycategory

‌Siri‌ continues to prove more useful with phone-related functions like calling, texting, emailing, calendar, and music. Both ‌Siri‌ and Google Assistant, which are baked into the OS of the phone, far outperformed Alexa in the Command section.

Munster says that the continued rate of improvement "continues to surprise" based on the notable improvements that each voice assistant has demonstrated over the course of the last few years.

In the future, Loup Ventures expects to see further improvements from extending the feature sets of each voice assistant.

Top Rated Comments

JetLaw Avatar
17 months ago
I'm not sure about the methodology they used to reach these results, but my anecdotal experience is that Alexa does a MUCH better job than Siri on almost everything I've tried to use them for.
Score: 36 Votes (Like | Disagree)
airwalk331 Avatar
17 months ago
Does anyone else feel like this is crap? 99.8% of the time Siri doesn’t work for me (though my friends’ Alexa does). Period. It’s even worse when I try to use it and there are 2 or more apple devices in the vicinity.
Score: 34 Votes (Like | Disagree)
Blackstick Avatar
17 months ago
Interesting... in my un-scientific testing, Alexa consistently beats Siri at recognizing what I want, and more importantly, the gibberish requests that my 4 year old spouts "Alexa play more Frozen!" (kill me)
Score: 32 Votes (Like | Disagree)
MallardDuck Avatar
17 months ago
* Local - Where is the nearest coffee shop?
* Commerce - Order me more paper towels.
* Navigation - How do I get to Uptown on the bus?
* Information - Who do the Twins play tonight?
* Command - Remind me to call Jerome at 2 pm today

And the responses:

Here's what I found on the web for coffee soup
I can't sort paper trowels in order
Playing Uptown girl by billy joel
I don't know who will win the game tonight
I've updated your reminder to call home


Most of Siri's brain-deadness is in the voice recognition portion of the system.
Score: 32 Votes (Like | Disagree)
diego.caraballo Avatar
17 months ago
Siri better than Alexa? Fake news
Score: 27 Votes (Like | Disagree)
mediotanque Avatar
17 months ago
This is ********. Siri is the dumbest Ai at the moment. Even Cortana works better than Siri.
Score: 14 Votes (Like | Disagree)

Top Stories

Apple Watc black friday 20 sale feature

Apple Black Friday 2020: Best Apple Watch Deals [Updated]

Wednesday November 25, 2020 4:01 pm PST by
Black Friday sales have begun on a variety of products, including the Apple Watch. There are quite a few deals across the Apple Watch lineup this year, including one of the lowest price we've ever seen the Apple Watch Series 3. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a small payment, which helps us keep the...
AirPods Pro black friday 20 sale feature 2

Black Friday 2020: AirPods Pro Reach Lowest Price Ever [Updated]

Wednesday November 25, 2020 3:22 pm PST by
Black Friday has kicked off this week, and one of the first major sales for the AirPods Pro is available right now on Walmart. You can find this deal below, along with a few other solid discounts on the regular AirPods. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a small payment, which helps us keep the site...
windows 10

Developer Successfully Virtualizes Windows for Arm on M1 Mac

Friday November 27, 2020 7:16 am PST by
Developer Alexander Graf has successfully virtualized the Arm version of Windows on an M1 Mac, proving that the M1 chip is capable of running Microsoft's operating system (via The 8-Bit). Currently, Macs with the M1 chip do not support Windows and there is no Boot Camp feature as there is on Intel Macs, but support for Windows is a feature that many users would like to see. Using the...
General black friday 20 sale feature

Thanksgiving Day Deals Still Available on AirPods, M1 Macs, Apple Watch Series 6, iPads

Thursday November 26, 2020 10:21 am PST by
Black Friday deals seem to start earlier and earlier every year, so there were already a wide variety of discounts available for Apple products on Thanksgiving Day. Many of the deals remain available even after Thanksgiving, but act fast, as inventory quickly fluctuates. Thanksgiving Day Deals on Apple Products — Still Available:AirPods with a wireless charging case remain available for...
iPhone black friday 20 sale feature

Apple Black Friday 2020: Best iPhone Deals

Friday November 27, 2020 12:56 pm PST by
Black Friday is halfway done, but there are still a few deals to shop for on iPhones at carriers like AT&T, Verizon, and T-Mobile/Sprint. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a small payment, which helps us keep the site running. AT&T Starting with AT&T, you'll find up to $700 off any iPhone 12 when...
13 16 inch macbook pro air trio

Reliable Leaker Suggests Redesigned MacBooks in 2021 Will Include Both Apple Silicon and Intel Models

Wednesday November 25, 2020 9:15 am PST by
Reliable leaker known as "L0vetodream" has today suggested on Twitter that redesigned MacBooks coming in the second half of 2021 will include models with both Apple Silicon chips and Intel processors. The brief Tweet came in response to a MacRumors article from earlier today, which outlined a report from Ming-Chi Kuo claiming that Apple plans to release redesigned MacBook models with Apple ...
iphone trade in store

UK Environmental Committee Says Apple Contributing to 'Throwaway Culture' of 'Short-Lived Products'

Thursday November 26, 2020 7:07 am PST by
Technology companies like Apple are contributing to e-waste by making their products difficult to repair, and charging expensive repair fees, according to a lengthy report published today by the UK Parliament's Environmental Audit Committee. "We were told that Apple glues and solders parts together on their laptops, which makes repairing them very difficult," the Committee wrote in a summary ...
m1 chip macbook air pro

Kuo: Redesigned MacBooks With Apple Silicon to Launch in Second Half of 2021

Tuesday November 24, 2020 7:53 pm PST by
Apple plans to release additional MacBook models with Apple Silicon in the second half of 2021, according to analyst Ming-Chi Kuo, as part of the company's two-year transition away from Intel processors across its Mac lineup. In a research note today, obtained by MacRumors, Kuo said that these MacBook models will feature a new design. Kuo did not specify which models these will be, but he...
mac mini macbook pro macbook air

Apple M1 Hands-On Comparison: MacBook Air vs. MacBook Pro vs. Mac Mini

Monday November 23, 2020 3:40 pm PST by
Apple's M1 Macs are out in the wild now, but ahead of the holidays, you might still be trying to figure out which one to pick up, either for yourself or as a gift for someone else. We've got all three of the new Macs available, so we thought we'd give MacRumors readers a hands-on overview of each machine in our latest YouTube video. Subscribe to the MacRumors YouTube channel for more videos. ...
iPad Pro 5G and Mini LED feature

Rumored 2021 High-End iPad Pro May Feature 5G With mmWave Support

Thursday November 26, 2020 2:14 am PST by
Apple's rumored high-end iPad Pro models to be released next year will be 5G-enabled with mmWave support, according to sources cited by industry publication DigiTimes. Multiple rumors have suggested Apple is planning to release a high-end 12.9-inch iPad Pro with a mini-LED display next year, with the possibility that there will be an 11-inch mini-LED model too, but details beyond that have...