Siri Answers 83% of Questions Correctly in Test, Beating Alexa But Trailing Google Assistant

In an annual test comparing Google Assistant, Siri, and Alexa on smartphones, Loup Ventures' Gene Munster found that ‌Siri‌ was able to correctly answer 83 percent of questions, beating Alexa but trailing behind Google Assistant.

Munster asked each digital assistant 800 questions during the test to compare how each one responded. Alexa answered 79.8 percent of questions correctly, while Google Assistant answered 92.9 percent of questions correctly.


Compared to last year, ‌Siri‌ has seen improvement. During the July 2018 test, ‌Siri‌ answered 79 percent of questions correctly compared to the 83 percent of questions answered right this time around. Alexa last year was at 61 percent while Google Assistant was at 86, so there have been digital voice assistant improvements across the board.


This test covered smartphones specifically, comparing iPhones and Android devices. Munster says that smartphones were isolated from smart speakers because while underlying technology is similar, "use cases vary." ‌Siri‌ was tested on an iPhone running iOS 12.4, Google Assistant on a Pixel XL, and Alexa in the iOS app.

Questions were based on five categories and all assistants were asked the same 800 questions. Each question set was designed to "comprehensively test a digital assistant's ability and utility." Some of the sample questions across each of the categories:

  • Local - Where is the nearest coffee shop?
  • Commerce - Order me more paper towels.
  • Navigation - How do I get to Uptown on the bus?
  • Information - Who do the Twins play tonight?
  • Command - Remind me to call Jerome at 2 pm today

‌Siri‌ did best in the command, local, and navigation categories, faring less well in the information and commerce categories. ‌Siri‌ actually won out in the command category, but trailed behind Google Assistant in other categories.

‌Siri‌ continues to prove more useful with phone-related functions like calling, texting, emailing, calendar, and music. Both ‌Siri‌ and Google Assistant, which are baked into the OS of the phone, far outperformed Alexa in the Command section.

Munster says that the continued rate of improvement "continues to surprise" based on the notable improvements that each voice assistant has demonstrated over the course of the last few years.

In the future, Loup Ventures expects to see further improvements from extending the feature sets of each voice assistant.

Top Rated Comments

(View all)
Avatar
12 months ago
I'm not sure about the methodology they used to reach these results, but my anecdotal experience is that Alexa does a MUCH better job than Siri on almost everything I've tried to use them for.
Score: 36 Votes (Like | Disagree)
Avatar
12 months ago
Does anyone else feel like this is crap? 99.8% of the time Siri doesn’t work for me (though my friends’ Alexa does). Period. It’s even worse when I try to use it and there are 2 or more apple devices in the vicinity.
Score: 34 Votes (Like | Disagree)
Avatar
12 months ago
Interesting... in my un-scientific testing, Alexa consistently beats Siri at recognizing what I want, and more importantly, the gibberish requests that my 4 year old spouts "Alexa play more Frozen!" (kill me)
Score: 32 Votes (Like | Disagree)
Avatar
12 months ago
* Local - Where is the nearest coffee shop?
* Commerce - Order me more paper towels.
* Navigation - How do I get to Uptown on the bus?
* Information - Who do the Twins play tonight?
* Command - Remind me to call Jerome at 2 pm today

And the responses:

Here's what I found on the web for coffee soup
I can't sort paper trowels in order
Playing Uptown girl by billy joel
I don't know who will win the game tonight
I've updated your reminder to call home


Most of Siri's brain-deadness is in the voice recognition portion of the system.
Score: 32 Votes (Like | Disagree)
Avatar
12 months ago
Siri better than Alexa? Fake news
Score: 27 Votes (Like | Disagree)
Avatar
12 months ago
This is ********. Siri is the dumbest Ai at the moment. Even Cortana works better than Siri.
Score: 14 Votes (Like | Disagree)

Top Stories

Apple Warns Against Closing MacBooks With a Cover Over the Camera

Friday July 10, 2020 11:12 am PDT by
Apple this month published a support document that warns customers against closing their Mac notebooks with a cover over the camera as it can lead to display damage. Image via Reddit Apple says that the clearance between the display and the keyboard is designed to very tight tolerances, which can be problematic. Covering the camera can also cause issues with automatic brightness and True Tone....

Leaker: 'iPhone 12 Pro' to Come With 6GB of RAM

Friday July 10, 2020 1:59 am PDT by
Later this year, Apple is expected to release four OLED iPhones in three display sizes, including 5.4, 6.7, and two 6.1-inch models. Rumors suggest the 6.7-inch iPhone and one 6.1-inch model will be higher-end devices, and now leaker @L0vetodream has corroborated previous rumors about the internal specs of Apple's upcoming lineup. Rumors suggest Apple will use 5-nanometer A14 chips in its...

Apple Moving Forward on Semitransparent Lenses for Upcoming AR Headset

Friday July 10, 2020 7:24 am PDT by
Apple and Foxconn have reached a key milestone in the development of Apple's long-rumored augmented reality headset, with the semitransparent lenses for the device moving from prototype to trial production, reports The Information. Apple is developing the lenses on a single production line at a Foxconn factory in Chengdu in southwestern China, where most of Apple’s iPad production is...

Kuo: Apple Silicon Macs to Include 13-inch MacBook Pro and MacBook Air This Year, 14.1-inch and 16-inch MacBook Pro Models Next Year

Friday July 10, 2020 2:58 am PDT by
At last month's WWDC, Apple officially announced that its Mac computers will be transitioned from Intel x86 to homegrown Apple Silicon chips. Apple said it plans to deliver the first Apple Silicon Mac by the end of the year and complete the transition in about two years. According to Apple analyst Ming-Chi Kuo, a 13.3-inch MacBook Pro with a form factor similar to the current 13.3-inch...

Arm-Intel-PowerPC Universal Binaries Are Possible

Saturday July 11, 2020 1:42 pm PDT by
Casual MacRumors visitors may not realize that we have a very active PowerPC forum where users discuss issues related to PowerPC Macs that have not been produced since 2006. Threads range from hardware upgrades and software options to nostalgia: Photo by AphoticD Apple's recently announced transition to Apple Silicon (Arm) based Macs raised some interesting questions about future support...

iPhone 12 Sizes Compared with iPhone SE, 7, 8, SE 2, X, 11, 11 Pro and 11 Pro Max [Update]

Tuesday July 7, 2020 6:49 pm PDT by
Apple is planning on launching the iPhone 12 this fall which is rumored to be coming in 3 different sizes: 5.4", 6.1" and 6.7". The middle size (6.1") matches up with the currently shipping iPhone 11, but the other two sizes will be entirely new. Over the weekend, there was some excitement about how well the new 5.4" iPhone 12 compares to the original iPhone SE. Those who have been hoping...

Tom Hanks WWII Movie 'Greyhound' Debuts on Apple TV+

Friday July 10, 2020 3:33 am PDT by
Apple TV+ today debuted "Greyhound," the highly anticipated Second World War movie starring Tom Hanks as a naval officer given command of Navy destroyer Greyhound in the Battle of the Atlantic. "Greyhound" features Hanks as George Krause, who must fight his own self doubts and personal demons as he leads a convoy of Allied ships against German U-boats to prove that he belongs in command. ...

Top Stories: iOS 14 Public Beta, iPhone 12 Size Comparison, 14-Inch MacBook Pro Rumors

Saturday July 11, 2020 6:00 am PDT by
After one round of developer beta testing, Apple unleashed iOS and iPadOS 14 to a wider audience this week, opening it up to members of the public beta program. There are lots of changes and new features to check out, but as with any beta, be careful about installing it on your main devices. Subscribe to the MacRumors YouTube channel for more videos. Other major stories this week included our ...

Hands-On With tvOS 14: Picture in Picture, 4K YouTube, HomeKit and More

Thursday July 9, 2020 12:48 pm PDT by
Apple at WWDC introduced a new version of tvOS, the software that's designed to run on the fourth and fifth-generation Apple TV models. tvOS updates are often more minor in scale than iOS, watchOS, and macOS updates, but tvOS 14 has some useful new features. Subscribe to the MacRumors YouTube channel for more videos. Apple in tvOS 13 introduced a Picture in Picture option for the Apple TV...

Apple CEO Tim Cook's 2019 Compensation Totaled Over $133 Million

Friday July 10, 2020 12:01 pm PDT by
Apple CEO Tim Cook was the second highest paid CEO in the United States in 2019, according to Bloomberg's list of the highest paid CEOs and executives in 2019. Cook received compensation totaling $133,727,869 by Bloomberg's count. Almost all of Cook's compensation was provided in the form of stock awards and related performance bonuses. As reported by the SEC earlier this year, Cook received ...